1.1k

# Bayesian Inference in Scala

A very short introduction on Bayesian inference with a simple Scala implementation.

Originally presented as a ShiftForward Tech Talk, a series of weekly talks presented by ShiftForward employees.

June 19, 2015

## Transcript

3. ### Probability Frequentist deﬁnition A probability P(A) is the proportion of

outcomes where the fact A is true after we repeat the experience or observed the situation a near inﬁnite amount of times.

5. ### Web Images Videos News Maps Search tools More SafeSearch conditional

probabilities Rui
6. ### Probability Bayesian deﬁnition A probability P(A) is the degree of

belief in the fact or prediction A.

Prior
10. ### Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

Prior Posterior
11. ### Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

Prior Posterior Likelihood
12. ### Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence

Prior Posterior Likelihood Normalizing constant
13. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
14. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
15. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
16. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
17. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
18. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
19. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 P(H) * P(E|H) 0 1/24 1/32 1/48 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
20. ### The Dice Problem “Suppose I have a box of dice

that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 P(H) * P(E|H) 0 1/24 1/32 1/48 Posterior P(H|E) 0 4/9 3/9 2/9 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?”
21. ### The Dice Problem “Now suppose I roll the same die

from before again, and this time I get an 8. What are the probabilities now?”
22. ### The Dice Problem “Now suppose I roll the same die

from before again, and this time I get an 8. What are the probabilities now?” H Prior P(H) Likelihood P(E|H) P(H) * P(E|H) Posterior P(H|E) 4 0 0 0 0 6 4/9 0 0 0 8 3/9 1/8 1/24 9/13 12 2/9 1/12 1/54 4/13
23. ### In Scala: Pmf type Pmf[H] = Map[H, Double] object Pmf

{  def apply[H](hypos: Seq[H]): Pmf[H] =  hypos.map { h => (h, 1.0 / hypos.length) }.toMap  }
24. ### In Scala: Suite trait Suite[H, E] {  def pmf: Pmf[H]

def likelihood(ev: E, hypo: H): Double    def observed(ev: E): Suite[H, E] = {  val newPmf = pmf.map {  case (h, prob) => (h, prob * likelihood(ev, h))  }.normalized    Suite(newPmf)(likelihood)  }  }
25. ### In Scala: Dice case class Dice(hypos: Seq[Int]) extends Suite[Int, Int]

{  val pmf = Pmf(hypos)    def likelihood(data: Int, hypo: Int) =  if (hypo < data) 0 else 1.0 / hypo  } val prior = Dice(List(4, 6, 8, 12))  val posterior = prior.observed(6)
26. ### The Euro Problem “Lukas was playing with a Belgian one-euro

coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?”
27. ### The Euro Problem “Lukas was playing with a Belgian one-euro

coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?” “Lukas was playing with a Belgian one-euro coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?”
28. ### In Scala: Euro type CoinSide = Boolean  val Heads =

true  val Tails = false    case class Euro(pmf: Pmf[Double]) extends Suite[Double, CoinSide] {    def likelihood(data: CoinSide, hypo: Double) =  if (data == Heads) hypo else 1.0 - hypo  } val data1 = Seq.fill(8)(Heads) ++ Seq.fill(2)(Tails) val data2 = Seq.fill(140-8)(Heads) ++ Seq.fill(110-2)(Tails)    val prior = Euro(Pmf(0.0 to 1.0 by 0.001))  val postData1 = prior.observedSet(data1)  val postData2 = postData1.observedSet(data2)
29. ### Euro Posteriors Prior After 10 spins After 250 spins 0.0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011 0.012 0.013 probability
30. ### Choosing a Prior So far, we’ve chosen uninformative priors -

we started with no knowledge or assumptions about the hypotheses’ probabilities. What if we actually know more or believe in something else about them?
31. ### An Informative Prior Uniform Triangle 0.0 0.1 0.2 0.3 0.4

0.5 0.6 0.7 0.8 0.9 1.0 0.00000 0.00025 0.00050 0.00075 0.00100 0.00125 0.00150 0.00175 0.00200 probability
32. ### Uniform Triangle 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

0.8 0.9 1.0 0.00000 0.00025 0.00050 0.00075 0.00100 0.00125 0.00150 0.00175 0.00200 0.00225 0.00250 0.00275 0.00300 0.00325 probability After 10 Spins
33. ### Uniform Triangle 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

0.8 0.9 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011 0.012 0.013 probability After 250 Spins
34. ### Summary • Bayesian inference provides a rational way to combine

prior beliefs with new evidence • Beliefs can be updated iteratively as evidence arrives • With enough data, different initial beliefs tend to converge on the same posterior
35. ### The End Based on Think Bayes, by Allen B. Downey

http://greenteapress.com/thinkbayes https://github.com/ruippeixotog/think-bayes-scala