environmental prediction, rise of the machines

NIWA Auckland Seminar, 28 August 2014

Transcript

1. Environmental Prediction: rise of the machines Nicolas Fauchereau nicolas.Fauchereau@niwa.co.nz NIWA Auckland 28 August 2014
2. Introduction Climate Scientist at NIWA Interested into making sense of data —> solutions to help people anticipate and adapt to climate variability and climate change Nicolas.Fauchereau@gmail.com @NFauchereau https://github.com/nicolasfauchereau/
3. 1. What is Machine Learning (ML) ?! 2. Some ML fundamentals! 3. A brief typology of ML algorithms! 4. ML in industry! 5. ML in the environmental sciences! 6. Development of a ML-based seasonal forecasting scheme for the Pacific Islands ! 7. Lessons learned and conclusions Outline The tools: Python and the scikit-learn ML library https://speakerdeck.com/nicolasf
4. What is Machine Learning ?
5. “[Machine Learning is the] field of study that gives computers the ability to learn without being explicitly programmed.” Arthur Samuel, 1959. “A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.” Tom Mitchell, 1997.
6. 50-70s: Artificial intelligence, pattern recognition (University) 80-90s: Neural Networks, data-mining (applications: science, credit scoring, OCR) mid-90s - today: convergence / explosion (applications: marketing, biotech, retail, telco, govt.) Digital Economy BIG DATA Theoretical Advances (e.g. Deep Learning) Brief history of Machine Learning
7. ML fundamentals
8. how does the learning happen ? • equations with tuneable parameters • performance can be measured • optimisation
9. simplest example: linear regression X Y
10. simplest example: linear regression tuneable parameters errors parameters are tuned as to minimise the errors (sum of square differences between observations and predictions)
11. Typology of ML algorithms
12. supervised learning! • deep learning • semi-supervised learning • reinforcement learning unsupervised learning!
13. supervised learning feature! predictor independent variable labels! target! predictands dependent variable instance, ‘observation’
14. supervised learning
15. classification vs regression supervised learning
16. unsupervised learning No labels !
17. clustering
18. Support Vector Machines for classification
19. feature #1 feature #2 Support Vector Machines ? ? ?
20. feature #1 feature #2 Support Vector Machines
21. feature #1 feature #2 Support Vector Machines
22. feature #1 feature #2 Support Vector Machines
23. what if the classes are not linearly separable ? feature #1 feature #2
24. combine the features ! feature #1 feature #2 feature #3 = feature #1^2 + feature #2^2
25. plane > decision boundary see animation at https://www.youtube.com/watch?v=9NrALgHFwTo
26. machine learning in industry
27. Applications of Machine Learning • search engines • spam filters • recommendations systems • fraud detection • credit rating • targeted marketing • face / object recognition • …
28. Appointed Yann LeCun and Rob Fergus, from NYU to lead new artificial intelligence labs in New York, California and London. Hired Geoff Hinton and Andrew Ng (now joined Baidu)
29. machine learning in the environmental sciences
30. examples of applications Ecology! • species distribution (current, future …) • habitat modelling Climate sciences! • pattern recognition, dimensionality reduction … (PCA, ICA, clustering …) • data mining (Large Ensemble of model outputs) • ‘short-term’ climate forecasting (statistical forecasts)
31. Prediction of seasonal Mean Sea Level Anomalies in the Pacific with: Scott Stephens, Doug Ramsay, Rob Bell (NIWA) John Marra, William Sweet (NOAA) Judith Wells, Rashed Chowduhry (Univ. Hawaii)
32. large impacts from extreme sea-levels in low-lying Islands across the Pacific Extreme sea-levels
33. coastal inundation = combination of: Extreme sea-levels Mean Sea Level Anomaly (MSLA): Climate dependent, ‘slow’ varying • high tides • anomalous Mean Sea Level • weather events (+ wave climate) improved “tide calendar”
34. Hourly tide gauge records from January 1 1979 through December 31 2011. Note that all data are in mm. A: INITIAL DATA MSLA seasonal anomalies for 8 stations data for 1981 - 2010 Target ‘raw’ MSLA anomalies (mm) regression discretized (quintiles) classification
35. Features Seasonal Sea-Surface-Temperatures (SSTs) anomalies the preceding season (e.g. Sep - Nov. MSLA predicted using Jun. - Aug SSTs) Dimensionality reduction via EOF (Empirical Orthogonal Functions) and Independent Components Analysis (ICA, Blind Source Separation) 9 Principal Components = 70 %
36. regression Multiple Linear Regression ! (MLR) Multivariate Adaptative Regression Splines ! (MARS) classification Support Vector Machines ! (SVM) Random Forests (RF) … Algorithms
37. regression Multiple Linear Regression ! (MLR) Multivariate Adaptative Regression Splines ! (MARS) classification Support Vector Machines ! (SVM) Random Forests (RF) … Algorithms Neural Networks! (NARX)
38. results
39. regression records from January 1 1979 through December 31 2011. data are in mm. DATA Cross-validated MARS for Guam
40. Confusion Matrix: Guam, SVMs Ranked Probability Skill Score (RPSS): Guam, RF classification
41. lessons learned and conclusions • pre-processing, models, features, hyper-parameters, metrics, cross-validation procedures, … • testing different models is time-consuming: framework for automation • features choice and feature engineering is key
42. Environmental prediction: rise of the machines ? • HPCs / cloud computing • Satellite remote sensing • In-situ sensors arrays • Model outputs Environmental sciences’s BIG DATA era ? • Mature and stable ML libraries BIG data compute power accessible
43. scikit-learn and Python for Machine Learning
44. Python • dynamic, object- oriented programming language ! • fast growing in the ‘data science’ community ! • huge collection of libraries from linear algebra to bayesian analysis, visualisation etc ! • rapid prototyping to production
45. scikit-learn • State of the art Machine Learning library • open-source (and free) • consistent API (Application Programming Interface) • comprehensive documentation • efficient algorithms • harnesses the power of the ‘Python scientific stack’ • very active development • http://scikit-learn.org/stable/index.html
46. scikit-learn
47. scikit-learn
48. k-means clustering