Gaussian Processes

In probability theory and statistics, a Gaussian process is a stochastic process (a collection of random variables indexed by time or space), such that every finite collection of those random variables has a multivariate normal distribution, i.e. every finite linear combination of them is normally distributed. The distribution of a Gaussian process is the joint distribution of all those (infinitely many) random variables, and as such, it is a distribution over functions with a continuous domain, e.g. time or space.

A machine-learning algorithm that involves a Gaussian process uses lazy learning and a measure of the similarity between points (the kernel function) to predict the value for an unseen point from training data. The prediction is not just an estimate for that point, but also has uncertainty information—it is a one-dimensional Gaussian distribution (which is the marginal distribution at that point).

Transcript

1. Gaussian Processes Practical overview https://sciforce.solutions
2. • Rassmussen & Williams, 2006 • Hensman, Matthews, Ghahramani, 2015 • GPy • GPFlow Literature and implementations
3. Weight space view Simple case: Bayesian linear regression
4. Simple case: Bayesian linear regression
5. Simple case: Bayesian linear regression (a) - prior (b) - predictive mean and std (c) - likelihood (d) - posterior Weight space view
6. Adding features How to deal with nonlinearity? Let’s design features! Weight space view
7. Predictive distribution: we have to invert NxN matrix Alternative form: nxn inversion, faster for n<N Weight space view
8. Kernel trick
9. GP is a collection of random variables, any finite number of which have a joint Gaussian distribution. x is not time (in general case)! Function-space view
10. Bayesian linear regression
11. Kernels
12. Algorithm [Rasmussen & Williams, 2006] Making predictions
13. Varying hyperparameters
14. Kernels
15. Classification
16. Thanks for attention Visit our website: https://ai.sciforce.solutions info@sciforce.solutions +380 50 55 21 448