Machine Learning #WIP

Gaussian Processes

Core Idea

Rather than sampling parameters, which induces a distribution over functions, we can sample functions directly.

• Place a prior on functions
• Make assumptions about the distribution of functions
• Make assumptions on the distribution of function values

Definition

Place a distribution $p(f)$ on functions $f=[f_1,f_2,f_3,...]$. The Gaussian process is a generalization of a multivariate Gaussian distribution to infinitely many variables.

Defined by a mean function $m(\cdot )$ and a covariance function i.e. kernel $k(\cdot ,\cdot )$.

Key Points

• Define distributions over functions
• Naturally provide uncertainty estimates
• Effective for sparse/limited data
• Combine kernels for complex patterns

References

• Resources on Gaussian Processes
• A Visual Exploration of Gaussian Processes
• Coding GPRs from Scratch
• Gaussian Processes by Mutual Information
• Gaussian Processes by DataMListic