Probability For Hackers

Did you just say probability? Mathematics at a tech conference? This mathematical term tends to elicit very strong reactions (either positive or negative, depending on who you ask) since it has a reputation for being difficult to crack/to keep track: it revolves around a seemingly endless jargon, abstract concepts, Greek letters as notations and more. Each concept holds its own subtle assumptions and everything is interlinked with each other. I know you have had enough of mathematics in the university but the way it is being taught is somewhat isolated from its practical application (less of a hands-on experience and more emphasis on theory) even though we encounter probability and statistics in our daily life. So, I am giving a go at ‘hacking the probability using code'.

We live in a world of ‘Big Data’ today. No matter how large or complex a dataset is, it provides only incomplete information about the questions that interest us. This incomplete information leads to uncertainty. This is where ‘probability’ comes into picture. Probability is “The Science of Uncertainty”. It gives us ways to quantify uncertainty and use it as one of the primary methods for designing new algorithms to model complex data. We use a computer to make predictions about new and/or uncertain events. These algorithms are nothing but Machine Learning techniques that provide automated methods of data analysis. In fact, all of us have already been users of these techniques. To name a few – Automated Spam Detection and filtering in e-mails, product/video recommendation (e.g., "customers who bought/watched X are also likely to buy/watch Y"). All of these are applications of probability in computer science. Most of machine learning techniques are rooted in probabilistic methods and we use it in many forms such as to know what is the best prediction about the future given/considering some past data? What is the best model to explain some data? What measurement should I perform next? etc. This is why it is important to learn probability.

In this talk, I will discuss how you can use your coding skills to "hack probability" – to replace some of the theory and jargon with intuitive computational approaches. My intention here is not to explain any concept precisely, but to merely lay down enough of them on the table to emphasize the role of probability in the fast growing areas such as Artificial Intelligence (superset of Machine Learning and Deep Learning), and Data Science.

“Not once, but twice AI was revolutionized by people who understood Probability Theory”

- Stanford University | CS 109: Probability for Computer Scientists

Transcript

1. Probability for Nipun Sadvilkar Pycon India 2017
2. < About me > • Machine Learning Engineer @Juxt-Smart Mandate • AI and ML Enthusiast • Likes to crack puns- Ni..pun (^-^) • @nipunsadvilkar on GitHub • More on website: https://nipunsadvilkar.github.io/
3. #Questions: 1. How many of you are from heavy mathematical background? E.g. Engineering, Physics 2. How many of you have used ML libraries like sklearn in your work? 3. How many of you want to get into following fields? • Artificial Intelligence • Machine Learning • Deep Learning • Data Science
4. Prerequisite for any famous AI and ML course/Book MOTIVATION #1
5. Those who struggle with math be like:
6. Hackers' approach to learn Math • Math is difficult but through coding, we can make it more interactive and intuitive. • I like this quote: “Statistics is Hard. Using programming skills it can be easy” - Jake VanderPlas (Statistics for Hackers - Pycon 2016) [Same for Probability] • Though, I want you to focus more on concepts and not on code (Code is available on GitHub. Have a look at it later)
7. Modern AI "Study and design of any agent that behaves in an intelligent way" MOTIVATION #2
8. Demo Time!⌚ WHO LIKES CHOCOLATES HERE?
9. Human knowledge into program Deep Blue Chess Stanford Self-Driving Car Google Assistant Google Vision API Alpha Go IBM Watson 1950 1997 2005 2011 201X MOTIVATION #2 HISTORY OF AI Uses two revolutionary ideas: 1. Probability with Examples 2. Artificial Neurons
10. “Not once, but twice AI was revolutionized by people who understood Probability Theory” - Stanford University | CS 109: Probability for Computer Scientists Conclusion
11. TARGET To be able to understand following math "Naive" conditional independence assumptions
12. Diving into Probability Obligatory coin toss experiment (interactive way) Using: 1. Virtual Coin with 2. Comparing theoretical to experimental probability with 3. Simulating experiment with Python Link to Ipython Notebook: Introduction to Probability.ipynb
13. #Activity 1 - Virtual Coin with Link to Ipython Notebook: Introduction to Probability.ipynb
14. #Activity 2 - Comparing theoretical to experimental probability with Link to Ipython Notebook: Introduction to Probability.ipynb
15. #Activity 3 - Simulating coin-toss experiment with Python Link to Ipython Notebook: Introduction to Probability.ipynb
16. Ingredients to Modelling Uncertainty 1. Sample space: The set of all possible outcomes for the experiment 2. The probability of each outcome: For each possible outcome, assign a probability that is at least 0 and at most 1. For the fair coin flip:
17. Introduction to Random Variables Link to Ipython Notebook: Introducing Random Variables.ipynb
18. Relation between Random Variables 1. Joint Probability 2. Marginal Probability 3. Conditional Probability 4. Dependence & Independence Link to Ipython Notebook: Relations between Random Variables.ipynb
19. Demystifying Bayes theorem Hello! Posterior Likelihood Prior Evidence
20. Application of Probability Theory learnt so far in Machine learning Naive Bayes Algorithm As a Spam filter
21. Naive Bayes = Bayes theorem + Chain rule + Naive Conditional Independence "Naive" conditional independence assumptions Chain Rule Denominator doesn't matter since it doesn’t depend on the class
22. Take Away: • Mathematics is Hard, using programming skills it can be easy • Intuition of Probability theory behind simple yet fast Naive Bayes algorithm
23. ~ Thank you! ~ https://nipunsadvilkar.github.io https://github.com/nipunsadvilkar https://facebook.com/nipunsadvilkar nipunsadvilkar@gmail.com
24. References: 1. MITx: 6.008.1x Computational Probability and Inference, Edx 2. CS 109: Probability for Computer Scientists, Stanford University 3. Prob140: Probability for Data Science, UC Berkeley 4. Mathsisfun.com 5. https://en.wikipedia.org/wiki/Naive_Bayes_classifier 6. Seeing Theory, Brown University 7. https://github.com/wintersummermint/coin-flip-javascript