Philip Sterne Stochastic Consulting Assistant Professor at Minerva University Gaussian Processes in Python https://github.com/StoCon/pycon2016 

Gaussian-what? Aka “Normal distribution” 1D 2D Now multiply that by infinity! 

Demo: The only live demo I’m brave enough to run! python sampling.py 

Covariance functions “Sphere of influence” 

Give me cookie data! 

https://cloud.google.com/bigquery/public-data/noaa-ghcn Weather data going as far back as 1850 for SA data (only precipitation) Simple python + bigquery script to download all the data SELECT stn.id,stn.name,stn.latitude,stn.longitude,date, FROM [bigquery-public-data:ghcn_d.ghcnd_stations] as stn ... NOAA Global Historical Climatology 

SA weather stations: 

So what do we get ● Daily maximum temperature ● Daily minimum temperature ● Daily precipitation 

Cape Town Because who cares about the rest of the country? 

Gaussian Processes in Python Two main packages: ● GPy ● sklearn.gaussian_process* (* Only available in scikit version 0.18) 

import GPy (X, Y) = get_data(...) kernel = GPy.kern.RBF(input_dim=1, variance=1., lengthscale=1.) m = GPy.models.GPRegression(X, Y, kernel=kernel) m.optimize_restarts(num_restarts=3) m.plot() (Pronounced “gee-pie”) GPy code 

from sklearn.gaussian_process import GaussianProcessRegressor from sklearn.gaussian_process.kernels import RBF kernel = RBF(length_scale=1.0, length_scale_bounds=(1e-1, 1e3)) (X, Y) = get_data(...) kernel = GPy.kern.RBF(input_dim=1, variance=1., lengthscale=1.) m = GaussianProcessRegressor(kernel=kernel, n_restarts_optimizer=3) m.fit(X, Y) y_pred, sigma = m.predict(x, return_std=True) (Pronounced “sy-kit learn”. sci stands for science!) Sci-kit learn code 

Gaussian Processes for Regression What is the max/min temperature today?* *without looking at a weather forecast, or any current meteorology information 

Daily average min/max temperatures for Cape Town (1973 - 2015) 

GPy: Daily min temperatures for Cape Town (1973 - 2015) 10.2° for Oct 7 Vs 12° for today 

(1973 - 2015) GPy: Daily max temperatures for Cape Town 22.4° for Oct 7 Vs 18° for today 

(1973 - 2015) scikit.learn: Daily min/max temperatures for Cape Town 

Impressions GPy: ● More robust ● More features ● Slower ● Actively developed Scikit.learn: ● Faster ● Actively developed Fairly similar code 

Some stats: Temperature Warmest day Feb 7th 16.2° 28.2° Coldest day July 13th 6.6 18.4 

Gaussian Processes for Classification Will it rain today?* *without looking at a weather forecast, or any current meteorology information 

Did it rain in Cape Town on a given day? (It rained if there was >1mm precipitation) 

Average probability of rain on a given day 

Some more stats Precipitation Driest day Jan 9th 8% Rainiest day June 26th 33% But WTH are “inducing points”? 

Normal Gaussian Processes Suck O(N³) computational complexity Ideally suited for “small data” i.e. get as much insight out of a limited data set 

Sparse Gaussian Processes “Use a small number of points to make predictions, but those points get influenced by the entire dataset” This gets us back to O(N). 

But the code????? num_inducing = 50 kernel = GPy.kern.StdPeriodic( input_dim=1, variance=.1, lengthscale=1., period=366.0) m = GPy.models.SparseGPClassification( X, Y, kernel=kernel, num_inducing=num_inducing) m.optimize_restarts(num_restarts=3) 

The rest of South Africa? If you insist... 

And now for your weather forecast* . (* A random smush of the same week over 8 years - it ran quickly on my laptop, and looked pretty too.) 

GPy 2D code: num_inducing = 30 kernel = GPy.kern.RBF(input_dim=2,variance=1.,lengthscale=1.) m = GPy.models.SparseGPClassification( X, Y, kernel=kernel, num_inducing=num_inducing) m.optimize_restarts(num_restarts=1) m.plot() 

Questions? 

References: 1. Weather data 2. GPy 3. Scikit.learn Thank You! I’m not hiring. But I’m interested in this stuff. Come talk to me!