Bayesian inference and big data: are we there yet? by Jose Luis Hidalgo at Big Data Spain 2017

Bayesian inference has a reputation of being complex and only applicable to fairly small datasets. Very recent developments may be chaining that, and we are starting to see some successful real world application of Bayesian inference to very large datasets, and some tools to make it accessible to many more data science practitioners.

https://www.bigdataspain.org/2017/talk/bayesian-inference-and-big-data-are-we-there-yet

Big Data Spain 2017
November 16th - 17th Kinépolis Madrid

Transcript

2. Bayesian statistics and big data: are we there yet? Jose Luis Hidalgo BigData Spain 2017
3. Clarification of some concepts Bayesian - "Bayes rule" - "Bayesian statistics" (vs. frequentist statistics) - "Reverse probability", Fisher definition - "Bayesian models"!
4. Clarification of some concepts Inference - In classic statistics: "inferential" vs "descriptive" - In machine learning: "inference" vs "training" - In Bayesian statistics: estimation of parameters from data - … to make predictions - … to validate the model
5. Clarification of some concepts Big Data - As many definitions as there are vendors interested in selling you something! - Incremental vs. something new - In our case: "big data" as the fact that we use increasingly larger amounts of data to get to some information/insight (we manage to extract weaker signals from oceans of noise)
6. A bit of history Early Bayesian models - Treated analytically - Limited to what can be treated analytically... duh! Nineties: MCMC - Offers (the promise of) generic inference algorithms - Very hard and computationally expensive - Variational inference as an (even harder) alternative
7. A bit of history Oughties: Probabilistic programming - Standard ways to explain probabilistic models to a computer - Bayesian models are a subset of probabilistic models - JAGS, BUGS, Stan... - Further developments, HMC, Gibbs sampling.. - Becomes quite popular in academic circles
8. A bit of history Tens: “Practical” Probabilistic programming - Further advances in inference: NUTS, ADVI... - New technologies to speedup computations - GPU parallelization - Automatic Differentiation - "Tall" datasets (very large number of cases) - "Wide" datasets (very large number of features)
9. Some sample applications From cognitive science - Exactly the opposite of what our NN friends are trying to do! - Models of human memory, of language understanding, etc. - Bayesian models are very well suited for this kind of studies From fin-tech - Large copula models become tractable using (Bayesian) inference algorithms
10. Some sample applications From AI - Generative image recognition systems From business operations - The inventory information problem - Probabilistic model of inventory - Enables operational optimization
11. Conclusions If you are a data science practitioner - Familiarize yourself with this kind of models - Learn about tools and libraries: Stan, PMC3, Edwars, etc. If you are responsible for technical infrastructure - Leveraging big data will require big compute... -... and not only for neural networks! If you are responsible for a business - Ask for more - Then ask again!
12. Thank you!