Bayesian Inference in Scala Rui Gonçalves ShiftForward 2015-06-19 

What is a probability? 

Probability Frequentist definition A probability P(A) is the proportion of outcomes where the fact A is true after we repeat the experience or observed the situation a near infinite amount of times. 

P(A|B) = P(B|A) ⇥ P(A) P(B) Bayes’ Theorem 

Web Images Videos News Maps Search tools More SafeSearch conditional probabilities Rui 

Probability Bayesian definition A probability P(A) is the degree of belief in the fact or prediction A. 

Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) 

Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence 

Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence Prior 

Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence Prior Posterior 

Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence Prior Posterior Likelihood 

Bayes’ Theorem P(H|E) = P(E|H) ⇥ P(H) P(E) Hypothesis Evidence Prior Posterior Likelihood Normalizing constant 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 P(H) * P(E|H) 0 1/24 1/32 1/48 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” H 4 6 8 12 Prior P(H) 1/4 1/4 1/4 1/4 Likelihood P(E|H) 0 1/6 1/8 1/12 P(H) * P(E|H) 0 1/24 1/32 1/48 Posterior P(H|E) 0 4/9 3/9 2/9 “Suppose I have a box of dice that contains a 4-sided die, a 6-sided die, an 8-sided die and a 12-sided die. Suppose I select a die from the box at random, roll it, and get a 6. What is the probability that I rolled each die?” 

The Dice Problem “Now suppose I roll the same die from before again, and this time I get an 8. What are the probabilities now?” 

The Dice Problem “Now suppose I roll the same die from before again, and this time I get an 8. What are the probabilities now?” H Prior P(H) Likelihood P(E|H) P(H) * P(E|H) Posterior P(H|E) 4 0 0 0 0 6 4/9 0 0 0 8 3/9 1/8 1/24 9/13 12 2/9 1/12 1/54 4/13 

In Scala: Pmf type Pmf[H] = Map[H, Double] object Pmf {
 def apply[H](hypos: Seq[H]): Pmf[H] =
 hypos.map { h => (h, 1.0 / hypos.length) }.toMap
 } 

In Scala: Suite trait Suite[H, E] {
 def pmf: Pmf[H]
 def likelihood(ev: E, hypo: H): Double
 
 def observed(ev: E): Suite[H, E] = {
 val newPmf = pmf.map {
 case (h, prob) => (h, prob * likelihood(ev, h))
 }.normalized
 
 Suite(newPmf)(likelihood)
 }
 } 

In Scala: Dice case class Dice(hypos: Seq[Int]) extends Suite[Int, Int] {
 val pmf = Pmf(hypos)
 
 def likelihood(data: Int, hypo: Int) =
 if (hypo < data) 0 else 1.0 / hypo
 } val prior = Dice(List(4, 6, 8, 12))
 val posterior = prior.observed(6) 

The Euro Problem “Lukas was playing with a Belgian one-euro coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?” 

The Euro Problem “Lukas was playing with a Belgian one-euro coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?” “Lukas was playing with a Belgian one-euro coin. He spun it on its edge 10 times and it ended with the heads side up 8 of those times. Intrigued, he proceed to repeat the experience 240 more times. At the end, the coin came up heads 140 times and tails 110. Is this data evidence enough to state that the coin is biased rather than fair?” 

In Scala: Euro type CoinSide = Boolean
 val Heads = true
 val Tails = false
 
 case class Euro(pmf: Pmf[Double]) extends Suite[Double, CoinSide] {
 
 def likelihood(data: CoinSide, hypo: Double) =
 if (data == Heads) hypo else 1.0 - hypo
 } val data1 = Seq.fill(8)(Heads) ++ Seq.fill(2)(Tails) val data2 = Seq.fill(140-8)(Heads) ++ Seq.fill(110-2)(Tails)
 
 val prior = Euro(Pmf(0.0 to 1.0 by 0.001))
 val postData1 = prior.observedSet(data1)
 val postData2 = postData1.observedSet(data2) 

Euro Posteriors Prior After 10 spins After 250 spins 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011 0.012 0.013 probability 

Choosing a Prior So far, we’ve chosen uninformative priors - we started with no knowledge or assumptions about the hypotheses’ probabilities. What if we actually know more or believe in something else about them? 

An Informative Prior Uniform Triangle 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.00000 0.00025 0.00050 0.00075 0.00100 0.00125 0.00150 0.00175 0.00200 probability 

Uniform Triangle 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.00000 0.00025 0.00050 0.00075 0.00100 0.00125 0.00150 0.00175 0.00200 0.00225 0.00250 0.00275 0.00300 0.00325 probability After 10 Spins 

Uniform Triangle 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 0.000 0.001 0.002 0.003 0.004 0.005 0.006 0.007 0.008 0.009 0.010 0.011 0.012 0.013 probability After 250 Spins 

Summary • Bayesian inference provides a rational way to combine prior beliefs with new evidence • Beliefs can be updated iteratively as evidence arrives • With enough data, different initial beliefs tend to converge on the same posterior 

The End Based on Think Bayes, by Allen B. Downey http://greenteapress.com/thinkbayes https://github.com/ruippeixotog/think-bayes-scala