Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Bayesian Statistical Analysis: A Gentle Introdu...

Bayesian Statistical Analysis: A Gentle Introduction

Get to know the Reverend Bayes.Reverend

Chris Fonnesbeck

December 05, 2011
Tweet

More Decks by Chris Fonnesbeck

Other Decks in Research

Transcript

  1. Bayesian Statistical Analysis A Gentle Introduction Center for Quantitative Sciences

    Workshop 18 November 2011 Christopher J. Fonnesbeck Monday, December 5, 11
  2. Practical methods for making inferences from data using probability models

    for quantities we observe and about which we wish to learn. Gelman et al., 2004 Monday, December 5, 11
  3. Choose an estimator ˆ µ = P xi n based

    on frequentist (asymptotic) criteria Monday, December 5, 11
  4. “... the Bayesian approach is attractive because it is useful.

    Its usefulness derives in large measure from its simplicity. Its simplicity allows the investigation of far more complex models than can be handled by the tools in the classical toolbox.” Link and Barker (2010) Monday, December 5, 11
  5. Pr( ¯ Y 1.96 ⇥ ⇥ n < µ <

    ¯ Y + 1.96 ⇥ ⇥ n ) = 0.95 Confidence Interval Pr(a(Y ) < ✓ < b(Y )|✓) = 0.95 Monday, December 5, 11
  6. C alpha N z b_psi beta a_psi pi mu psi

    Ntotal occupied a b Ndist psi z alpha pi N beta mu occupied N alpha beta N alpha beta Complex Models Monday, December 5, 11
  7. Pr(A) = m n A = an event of interest

    m = no. of favourable outcomes n = total no. of possible outcomes (1) classical Monday, December 5, 11
  8. Pr(A) = lim n→∞ m n n = no. of

    identical and independent trials m = no. of times A has occurred (2) frequentist Monday, December 5, 11
  9. Between 1745 and 1770 there were 241,945 girls and 251,527

    boys born in Paris Monday, December 5, 11
  10. Pr( A|H ) = ⇢ 0 . 4 if raining

    in Nashville 0 . 25 otherwise Monday, December 5, 11
  11. S A Pr(A) = area of A area of S

    Monday, December 5, 11
  12. S A B A ∩ B Pr(A ⇥ B) =

    Pr(A) + Pr(B) Pr(A ⇤ B) Monday, December 5, 11
  13. S A B A ∩ B Pr(B|A) = Pr(A B)

    Pr(A) Monday, December 5, 11
  14. S A B A ∩ B Pr(A|B) = Pr(A B)

    Pr(B) Pr(B|A) = Pr(A B) Pr(A) Monday, December 5, 11
  15. Bayes Theorem Pr( |y) = Pr(y| )Pr( ) Pr(y) Posterior

    Probability Prior Probability Likelihood of Observations Normalizing Constant Monday, December 5, 11
  16. Bayes Theorem Pr( |y) = Pr(y| )Pr( ) R Pr(y|

    )Pr( )d Monday, December 5, 11
  17. “Following observation of , the likelihood contains all experimental information

    from about the unknown .” θ y y L(✓|y) Monday, December 5, 11
  18. binomial model data parameter sampling distribution of X p(X|✓) =

    ✓ N n ◆ ✓x (1 ✓)N x Monday, December 5, 11
  19. binomial model likelihood function for θ L(✓|X) = ✓ N

    n ◆ ✓x (1 ✓)N x Monday, December 5, 11
  20. Between 1745 and 1770 there were 241,945 girls and 251,527

    boys born in Paris Monday, December 5, 11
  21. “... all forms of statistical inference make assumptions, assumptions which

    can only be tested very crudely and can almost never be verified.” - Robert E. Kass Monday, December 5, 11
  22. 1.5 2.0 2.5 0.0 0.2 0.4 0.6 0.8 1.0 x

    p(x) separation Monday, December 5, 11
  23. weakly-informative prior -4 -2 0 2 4 0.0 0.1 0.2

    0.3 0.4 xrange Pr(x) Monday, December 5, 11
  24. affected carrier no gene unknown Woman Husband Brother Mother is

    the woman a carrier? Monday, December 5, 11
  25. Pr(θ = 1) = Pr(θ = 0) = 1 2

    Pr(θ = 1) Pr(θ = 0) = 1 prior odds Monday, December 5, 11
  26. Pr(y1 = 0, y2 = 0|θ = 1) = (0.5)(0.5)

    = 0.25 Monday, December 5, 11
  27. Pr(y1 = 0, y2 = 0|θ = 1) = (0.5)(0.5)

    = 0.25 Pr(y1 = 0, y2 = 0|θ = 0) = 1 Monday, December 5, 11
  28. Pr(y1 = 0, y2 = 0|θ = 1) = (0.5)(0.5)

    = 0.25 Pr(y1 = 0, y2 = 0|θ = 0) = 1 “likelihood ratio” p(y1 = 0, y2 = 0|θ = 1) p(y1 = 0, y2 = 0|θ = 0) = 0.25 1 = 1/4 Monday, December 5, 11
  29. what about Mom? y = {y1 = 0, y2 =

    0} Pr( = 1|y) = Pr(y| = 1)Pr( = 1) Pr(y) = Pr(y| = 1)Pr( = 1) P ✓ Pr(y| )Pr( ) Monday, December 5, 11
  30. Pr( = 1|y) = p(y| = 1)Pr( = 1) p(y|

    = 1)Pr( = 1) + p(y| = 0)Pr( = 0) y = {y1 = 0, y2 = 0} Monday, December 5, 11
  31. Pr( = 1|y) = p(y| = 1)Pr( = 1) p(y|

    = 1)Pr( = 1) + p(y| = 0)Pr( = 0) = (0.25)(0.5) (0.25)(0.5) + (1.0)(0.5) = 0.125 0.625 = 0.2 y = {y1 = 0, y2 = 0} Monday, December 5, 11
  32. 3rd unaffected son? Pr( = 1|y3 ) = (0.5)(0.2) (0.5)(0.2)

    + (1)(0.8) = 0.111 posterior from previous Monday, December 5, 11
  33. Hospital Operations Deaths A 47 0 B 148 18 C

    119 8 D 810 46 E 211 8 F 196 13 G 148 9 H 215 31 I 207 14 J 97 8 K 256 29 L 360 24 Monday, December 5, 11
  34. θ1 θ2 θk y1 y2 yk ... ... deaths parameters

    µ, σ2 hyperparameters Monday, December 5, 11
  35. , ϕµ ϕσ θ1 θ2 θk y1 y2 yk ...

    ... deaths parameters µ, σ2 hyperparameters Monday, December 5, 11
  36. “experiments” j = 1, . . . , J likelihood

    ∼ Binomial( , ) deaths j operations j θj logit( ) ∼ N(µ, ) θi σ2 population model µ ∼ , ∼ Pµ σ2 Pσ priors Monday, December 5, 11
  37. 0/47 = 0 18/148 = 0.12 8/119 = 0.07 46/810

    = 0.06 Monday, December 5, 11