Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bayesian Inference in Scala

Bayesian Inference in Scala

A very short introduction on Bayesian inference with a simple Scala implementation.

Originally presented as a ShiftForward Tech Talk, a series of weekly talks presented by ShiftForward employees.

77db744fe19ef16f523eb97d98d13459?s=128

Rui Gonçalves

June 19, 2015
Tweet

Transcript

  1. Bayesian Inference in Scala Rui Gonçalves ShiftForward 2015-06-19

  2. What is a probability?

  3. Probability Frequentist definition A probability P(A) is the proportion of

    outcomes where the fact A is true after we repeat the experience or observed the situation a near infinite amount of times.
  4. P(A|B) = P(B|A) ⇥ P(A) P(B) Bayes’ Theorem

  5. Web Images Videos News Maps Search tools More SafeSearch conditional

    probabilities Rui
  6. Probability Bayesian definition A probability P(A) is the degree of

    belief in the fact or prediction A.
  7. Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E)

  8. Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

  9. Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

    Prior
  10. Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

    Prior Posterior
  11. Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

    Prior Posterior Likelihood
  12. Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

    Prior Posterior Likelihood Normalizing constant
  13. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  14. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  15. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  16. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  17. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  18. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  19. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 P(H) * P(E|H) 0 1/24 1/32 1/48 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  20. The Dice Problem “Suppose I have a box of dice

    that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 P(H) * P(E|H) 0 1/24 1/32 1/48 Posterior P(H|E) 0 4/9 3/9 2/9 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
  21. The Dice Problem “Now suppose I roll the same die

    from before again, and this time I get an 8. What are the probabilities now?”
  22. The Dice Problem “Now suppose I roll the same die

    from before again, and this time I get an 8. What are the probabilities now?” H Prior P(H) Likelihood P(E|H) P(H) * P(E|H) Posterior P(H|E) 4 0 0 0 0 6 4/9 0 0 0 8 3/9 1/8 1/24 9/13 12 2/9 1/12 1/54 4/13
  23. In Scala: Pmf type Pmf[H] = Map[H, Double] object Pmf

    {
 def apply[H](hypos: Seq[H]): Pmf[H] =
 hypos.map { h => (h, 1.0 / hypos.length) }.toMap
 }
  24. In Scala: Suite trait Suite[H, E] {
 def pmf: Pmf[H]


    def likelihood(ev: E, hypo: H): Double
 
 def observed(ev: E): Suite[H, E] = {
 val newPmf = pmf.map {
 case (h, prob) => (h, prob * likelihood(ev, h))
 }.normalized
 
 Suite(newPmf)(likelihood)
 }
 }
  25. In Scala: Dice case class Dice(hypos: Seq[Int]) extends Suite[Int, Int]

    {
 val pmf = Pmf(hypos)
 
 def likelihood(data: Int, hypo: Int) =
 if (hypo < data) 0 else 1.0 / hypo
 } val prior = Dice(List(4, 6, 8, 12))
 val posterior = prior.observed(6)
  26. The Euro Problem “Lukas was playing with a Belgian one-euro

    coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?”
  27. The Euro Problem “Lukas was playing with a Belgian one-euro

    coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?” “Lukas was playing with a Belgian one-euro coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?”
  28. In Scala: Euro type CoinSide = Boolean
 val Heads =

    true
 val Tails = false
 
 case class Euro(pmf: Pmf[Double]) extends Suite[Double, CoinSide] {
 
 def likelihood(data: CoinSide, hypo: Double) =
 if (data == Heads) hypo else 1.0 - hypo
 } val data1 = Seq.fill(8)(Heads) ++ Seq.fill(2)(Tails) val data2 = Seq.fill(140-8)(Heads) ++ Seq.fill(110-2)(Tails)
 
 val prior = Euro(Pmf(0.0 to 1.0 by 0.001))
 val postData1 = prior.observedSet(data1)
 val postData2 = postData1.observedSet(data2)
  29. Euro Posteriors Prior After 10 spins After 250 spins 0.0

    0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011 0.012 0.013 probability
  30. Choosing a Prior So far, we’ve chosen uninformative priors -

    we started with no knowledge or assumptions about the hypotheses’ probabilities. What if we actually know more or believe in something else about them?
  31. An Informative Prior Uniform Triangle 0.0 0.1 0.2 0.3 0.4

    0.5 0.6 0.7 0.8 0.9 1.0 0.00000 0.00025 0.00050 0.00075 0.00100 0.00125 0.00150 0.00175 0.00200 probability
  32. Uniform Triangle 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

    0.8 0.9 1.0 0.00000 0.00025 0.00050 0.00075 0.00100 0.00125 0.00150 0.00175 0.00200 0.00225 0.00250 0.00275 0.00300 0.00325 probability After 10 Spins
  33. Uniform Triangle 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

    0.8 0.9 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011 0.012 0.013 probability After 250 Spins
  34. Summary • Bayesian inference provides a rational way to combine

    prior beliefs with new evidence • Beliefs can be updated iteratively as evidence arrives • With enough data, different initial beliefs tend to converge on the same posterior
  35. The End Based on Think Bayes, by Allen B. Downey

    http://greenteapress.com/thinkbayes https://github.com/ruippeixotog/think-bayes-scala