1.2k

# Bayesian Inference in Scala

A very short introduction on Bayesian inference with a simple Scala implementation.

Originally presented as a ShiftForward Tech Talk, a series of weekly talks presented by ShiftForward employees. June 19, 2015

## Transcript

1. Bayesian Inference
in Scala
Rui Gonçalves
ShiftForward
2015-06-19

2. What is a probability?

3. Probability
Frequentist deﬁnition
A probability P(A) is the proportion of outcomes
where the fact A is true after we repeat the
experience or observed the situation a near inﬁnite
amount of times.

4. P(A|B) =
P(B|A) ⇥ P(A)
P(B)
Bayes’ Theorem

5. Web Images Videos News Maps Search tools
More SafeSearch
conditional probabilities Rui

6. Probability
Bayesian deﬁnition
A probability P(A) is the degree of belief in the fact or
prediction A.

7. Bayes’ Theorem
P(H|E) =
P(E|H) ⇥ P(H)
P(E)

8. Bayes’ Theorem
P(H|E) =
P(E|H) ⇥ P(H)
P(E)
Hypothesis Evidence

9. Bayes’ Theorem
P(H|E) =
P(E|H) ⇥ P(H)
P(E)
Hypothesis Evidence
Prior

10. Bayes’ Theorem
P(H|E) =
P(E|H) ⇥ P(H)
P(E)
Hypothesis Evidence
Prior
Posterior

11. Bayes’ Theorem
P(H|E) =
P(E|H) ⇥ P(H)
P(E)
Hypothesis Evidence
Prior
Posterior Likelihood

12. Bayes’ Theorem
P(H|E) =
P(E|H) ⇥ P(H)
P(E)
Hypothesis Evidence
Prior
Posterior Likelihood
Normalizing constant

13. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

14. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

15. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

16. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”
H
4
6
8
12
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

17. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”
H
4
6
8
12
Prior
P(H)
1/4
1/4
1/4
1/4
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

18. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”
H
4
6
8
12
Prior
P(H)
1/4
1/4
1/4
1/4
Likelihood
P(E|H)
0
1/6
1/8
1/12
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

19. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”
H
4
6
8
12
Prior
P(H)
1/4
1/4
1/4
1/4
Likelihood
P(E|H)
0
1/6
1/8
1/12
P(H) * P(E|H)
0
1/24
1/32
1/48
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

20. The Dice Problem
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”
H
4
6
8
12
Prior
P(H)
1/4
1/4
1/4
1/4
Likelihood
P(E|H)
0
1/6
1/8
1/12
P(H) * P(E|H)
0
1/24
1/32
1/48
Posterior
P(H|E)
0
4/9
3/9
2/9
“Suppose I have a box of dice that contains a 4-sided
die, a 6-sided die, an 8-sided die and a 12-sided die.
Suppose I select a die from the box at random, roll it,
and get a 6. What is the probability that I rolled each
die?”

21. The Dice Problem
“Now suppose I roll the same die from before again,
and this time I get an 8. What are the probabilities
now?”

22. The Dice Problem
“Now suppose I roll the same die from before again,
and this time I get an 8. What are the probabilities
now?”
H Prior
P(H)
Likelihood
P(E|H) P(H) * P(E|H)
Posterior
P(H|E)
4 0 0 0 0
6 4/9 0 0 0
8 3/9 1/8 1/24 9/13
12 2/9 1/12 1/54 4/13

23. In Scala: Pmf
type Pmf[H] = Map[H, Double]
object Pmf {
def apply[H](hypos: Seq[H]): Pmf[H] =
hypos.map { h => (h, 1.0 / hypos.length) }.toMap
}

24. In Scala: Suite
trait Suite[H, E] {
def pmf: Pmf[H]
def likelihood(ev: E, hypo: H): Double

def observed(ev: E): Suite[H, E] = {
val newPmf = pmf.map {
case (h, prob) => (h, prob * likelihood(ev, h))
}.normalized

Suite(newPmf)(likelihood)
}
}

25. In Scala: Dice
case class Dice(hypos: Seq[Int]) extends Suite[Int, Int] {
val pmf = Pmf(hypos)

def likelihood(data: Int, hypo: Int) =
if (hypo < data) 0 else 1.0 / hypo
}
val prior = Dice(List(4, 6, 8, 12))
val posterior = prior.observed(6)

26. The Euro Problem
“Lukas was playing with a Belgian one-euro coin. He
spun it on its edge 10 times and it ended with the
heads side up 8 of those times.
Intrigued, he proceed to repeat the experience 240
more times. At the end, the coin came up heads 140
times and tails 110.
Is this data evidence enough to state that the coin is
biased rather than fair?”

27. The Euro Problem
“Lukas was playing with a Belgian one-euro coin. He
spun it on its edge 10 times and it ended with the
heads side up 8 of those times.
Intrigued, he proceed to repeat the experience 240
more times. At the end, the coin came up heads 140
times and tails 110.
Is this data evidence enough to state that the coin is
biased rather than fair?”
“Lukas was playing with a Belgian one-euro coin. He
spun it on its edge 10 times and it ended with the
heads side up 8 of those times.
Intrigued, he proceed to repeat the experience 240
more times. At the end, the coin came up heads 140
times and tails 110.
Is this data evidence enough to state that the coin is
biased rather than fair?”

28. In Scala: Euro
type CoinSide = Boolean
val Tails = false

case class Euro(pmf: Pmf[Double])
extends Suite[Double, CoinSide] {

def likelihood(data: CoinSide, hypo: Double) =
if (data == Heads) hypo else 1.0 - hypo
}
val data1 = Seq.fill(8)(Heads) ++ Seq.fill(2)(Tails)
val data2 = Seq.fill(140-8)(Heads) ++ Seq.fill(110-2)(Tails)

val prior = Euro(Pmf(0.0 to 1.0 by 0.001))
val postData1 = prior.observedSet(data1)
val postData2 = postData1.observedSet(data2)

29. Euro Posteriors
Prior After 10 spins After 250 spins
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0.000
0.001
0.002
0.003
0.004
0.005
0.006
0.007
0.008
0.009
0.010
0.011
0.012
0.013
probability

30. Choosing a Prior
So far, we’ve chosen uninformative priors - we started
with no knowledge or assumptions about the
hypotheses’ probabilities.
What if we actually know more or believe in

31. An Informative Prior
Uniform Triangle
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0.00000
0.00025
0.00050
0.00075
0.00100
0.00125
0.00150
0.00175
0.00200
probability

32. Uniform Triangle
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0.00000
0.00025
0.00050
0.00075
0.00100
0.00125
0.00150
0.00175
0.00200
0.00225
0.00250
0.00275
0.00300
0.00325
probability
After 10 Spins

33. Uniform Triangle
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
0.000
0.001
0.002
0.003
0.004
0.005
0.006
0.007
0.008
0.009
0.010
0.011
0.012
0.013
probability
After 250 Spins

34. Summary
• Bayesian inference provides a rational way to
combine prior beliefs with new evidence
• Beliefs can be updated iteratively as evidence
arrives
• With enough data, different initial beliefs tend to
converge on the same posterior

35. The End
Based on Think Bayes, by Allen B. Downey
http://greenteapress.com/thinkbayes
https://github.com/ruippeixotog/think-bayes-scala