Bayesian Inference is known to make machines biased

Humans have many biases. Let's call them priors. They also have their expertises. Let's call them hypothesis spaces. We often make the conclusions we make swiftly, and without much effort and call it intuition. How do we make machines reason intuitively and what it may end with.

Examples will be given in Probabilistic Programming framework called Anglican, successor of the Goodman & Tenenbaum Church language.

Transcript

1. Introduction to Probabilistic Programming
2. (S, P) observations probability distributions on S Statistical Model
3. Bayes Theorem
4. P(A|B) = P(B|A)P(A) P(B)
5. P(̣|⬤) = P(⬤|̣)P(̣) P(⬤)
6. posterior= prior x likelihood evidence
7. Number Guessing
8. guess the arithmetical concept
9. 2, 4, 8
10. Humans are biased towards induction
11. Humans learn from positive examples
12. Empirical Distribution
13. How to bias the machine?
14. Similarity Based Rule Based vs
15. Rule Based Hypothesis Elimination More complex ways to generalise Requires strong priors
16. Similarity Based Require negative examples Require “similarity” measure Improbable features impacting result
17. Priors (biases) Hypothesis spaces
18. odd numbers even numbers primes powers of two ending with n etc... Hypothesis Spaces H:
19. Model Averaging
20. Bayesian Ockham’s Razor
21. Seen: 16
22. Seen: 16, 8, 2, 64
23. Size principle
24. Ruling out unnatural concepts
25. Uniform Prior
26. Seen: 16 prior, likelihood and posterior
27. Seen: 2, 4, 8, 16 prior, likelihood and posterior
28. Data can overwhelm the prior.
29. Hierarchical Bayes Models
30. Seen: 16 Seen: 60 Seen: 2 8 16 64 Seen: 16 23 19 20 Machine
31. Seen: 16 Seen: 60 Seen: 2 8 16 64 Seen: 16 23 19 20 Huma
32. which can be approximated with MCMC To make inference, we need to integrate:
33. Monte Carlo
34. Represents a Probability Distribution by set of samples from it
35. Supported by Law Of Large Numbers™
36. Coin Flips (take 50 (sample (flip 0.5)))
37. Coin Flips (take 50000 (sample (flip 0.4)))
38. Coin Flips (take 5000 (sample (flip 0.5)))
39. MC (Monte Carlo)
40. 1. Define domain of inputs 2. Generate inputs over domain and it’s PD 3. Perform a deterministic computation 4. Aggregate
41. Machine Learning: a Probabilistic Perspective https://www.cs.ubc.ca/~murphyk/MLbook/ Bayesian models of cognition https://cocosci.berkeley.edu/tom/papers/bayeschapter.pdf Bayesian Methods for Hackers https://github.com/CamDavidsonPilon/Probabilistic-Programming-and- Bayesian-Methods-for-Hackers Anglican http://www.robots.ox.ac.uk/~fwood/anglican/index.html Introduction to Markov Chain Monte Carlo http://www.mcmchandbook.net/HandbookChapter1.pdf Reading List