L11 Statistical Rethinking Winter 2019

Lecture 11 of the Dec 2018 through March 2019 edition of Statistical Rethinking. Covers Chapters 10 and 11, maximum entropy, generalized linear models, binomial GLMs.

Transcript

1. Big Entropy & The Generalized Linear Model Statistical Rethinking Winter 2019 Lecture 11 / Week 6
2. 1 2 3 4 5 3 4 5 6 7 8 9 10 11 1 2 100 pebbles
3. 1 2 3 4 5
4. 1 2 3 4 5 100
5. 1 2 3 4 5 100
6. 1 2 3 4 5 5 22 12 37 24
7. 1 2 3 4 5 n1 n2 n3 n4 n5
8. 1 2 3 4 5 n1 n2 n3 n4 n5  = ! (g!(h!(i!(j!(k! &** = MPH PG QSPEVDU PG BWFSBHF MJLFMJIPPET = TVN PG MPHT PG BWFSBHF MJLFMJIPPET &** =  #=g &)! ,(3#|θ) ,(θ)θ Number of ways:
9. Suppose only 10 pebbles.... 1 way 1 2 3 4 5 bucket pebbles 0 5 10 10
10. Suppose only 10 pebbles.... 1 way 1 2 3 4 5 bucket pebbles 0 5 10 10 1 2 3 4 5 bucket pebbles 0 5 10 1 8 1
11. Suppose only 10 pebbles.... 1 way 90 ways 1 2 3 4 5 bucket pebbles 0 5 10 10 1 2 3 4 5 bucket pebbles 0 5 10 1 8 1
12. Suppose only 10 pebbles.... 1 way 90 ways 1 2 3 4 5 bucket pebbles 0 5 10 10 1 2 3 4 5 bucket pebbles 0 5 10 1 8 1 1 2 3 4 5 bucket pebbles 0 5 10 2 6 2
13. Suppose only 10 pebbles.... 1 way 90 ways 1 2 3 4 5 bucket pebbles 0 5 10 10 1 2 3 4 5 bucket pebbles 0 5 10 1 8 1 1 2 3 4 5 bucket pebbles 0 5 10 2 6 2 1260 ways
14. Suppose only 10 pebbles.... 1 way 90 ways 1 2 3 4 5 bucket pebbles 0 5 10 10 1 2 3 4 5 bucket pebbles 0 5 10 1 8 1 1 2 3 4 5 bucket pebbles 0 5 10 2 6 2 1260 ways 1 2 3 4 5 bucket pebbles 0 5 10 1 2 4 2 1
15. Suppose only 10 pebbles.... 1 way 90 ways 1 2 3 4 5 bucket pebbles 0 5 10 10 1 2 3 4 5 bucket pebbles 0 5 10 1 8 1 1 2 3 4 5 bucket pebbles 0 5 10 2 6 2 1260 ways 1 2 3 4 5 bucket pebbles 0 5 10 1 2 4 2 1 37800 ways
16. 1 2 3 4 5 bucket pebbles 0 5 10 2 6 2 1260 ways 1 2 3 4 5 bucket pebbles 0 5 10 1 2 4 2 1 37800 ways 1 2 3 4 5 bucket pebbles 0 5 10 2 2 2 2 2 113400 ways
17. 1 2 3 4 5 n1 n2 n3 n4 n5 For large N:  = ! (g!(h!(i!(j!(k! g  &)!  ≈ − # (#  &)! (#  = − # *# &)! *# &** = &)! ) *,)/. ) 0,! &#%&#"))- = -/' ) &)!- ) 0,! &#%&#"))- &** =  #=g &)! ,(3#|θ) ,(θ)θ =  &)!  θ ,(3#|θ)
18. 1 2 3 4 5 n1 n2 n3 n4 n5 For large N:  = ! (g!(h!(i!(j!(k! g  &)!  ≈ − # (#  &)! (#  = − # *# &)! *# &** = &)! ) *,)/. ) 0,! &#%&#"))- = -/' ) &)!- ) 0,! &#%&#"))- &** =  #=g &)! ,(3#|θ) ,(θ)θ =  &)!  θ ,(3#|θ)
19. Maximum entropy • Due to Edwin T. Jaynes (1922–1998) • The maxent principle: • Distribution with largest entropy is distribution most consistent with stated assumptions • Can happen the largest number of ways • For parameters, provides way to construct priors • For observations, way to construct likelihood • Also reproduces Bayesian updating as special case (minimum cross-entropy) E. T. Jaynes (1922–1998)
20. Maximum entropy • Due to Edwin T. Jaynes (1922–1998) • The maxent principle: • Distribution with largest entropy is distribution most consistent with stated assumptions • Can happen the largest number of ways • For parameters, provides way to understand priors • For observations, way to understand likelihood • Also reproduces Bayesian updating as special case (minimum cross-entropy) E. T. Jaynes (1922–1998)
21. Maximum entropy • Ye olde information entropy: • Q: What kind of distribution maximizes this quantity? • A: Flattest distribution still consistent with constraints. This is the distribution that can happen the most unique ways. • Whatever does happen, bound to be one of those ways.  .BYJNVN FOUSPQZ $IBQUFS  ZPV NFU UIF CBTJDT PG JOGPSNBUJPO UIFPSZ *O CSJFG XF TFFL B NFBT UBJOUZ UIBU TBUJTĕFT UISFF DSJUFSJB  UIF NFBTVSF TIPVME CF DPOUJOVPVT  JU T TF BT UIF OVNCFS PG QPTTJCMF FWFOUT JODSFBTFT BOE  JU TIPVME CF BEEJUJWF ć H VOJRVF NFBTVSF PG UIF VODFSUBJOUZ PG B QSPCBCJMJUZ EJTUSJCVUJPO Q XJUI QSPCB FBDI QPTTJCMF FWFOU J UVSOT PVU UP CF KVTU UIF BWFSBHF MPHQSPCBCJMJUZ )(Q) = − J QJ MPH QJ VODUJPO JT LOPXO BT JOGPSNBUJPO FOUSPQZ
22. Conjecture: Data: Counting paths produces flattest distribution consistent with stated constraints.
23. Maximum entropy Constraints Maxent distribution Real value in interval Uniform Real value, finite variance Gaussian Binary events, fixed probability Binomial Non-negative real, has mean Exponential
24. Generalized Linear Models • Goal: Connect linear model to outcome variable • Still geocentric! • Strategy: 1. Pick an outcome distribution 2. Model its parameters using links to linear models 3. Compute posterior • Can model multivariate relationships and non- linear responses • Building blocks of multilevel models
25. Generalized Linear Models • How to pick a data distribution • Mostly exponential family • Arise from natural processes • Maximum entropy interpretations • Select from first principles • Resist histomancy: Superstitious practice of picking likelihoods by gazing at a histogram
26. sum large mean count events low rate count events low probability many trials dnorm dgamma dpois dbinom dexp Z ∼ /PSNBM(µ, σ) Z ∼ #JOPNJBM(O, Q) Z ∼ 1PJTTPO(λ) Z ∼ (BNNB(λ, L) Z ∼ &YQPOFOUJBM(λ) Figure 9.5
27. sum large mean count events low rate count events low probability many trials dnorm dgamma dpois dbinom dexp Z ∼ /PSNBM(µ, σ) Z ∼ #JOPNJBM(O, Q) Z ∼ 1PJTTPO(λ) Z ∼ (BNNB(λ, L) Z ∼ &YQPOFOUJBM(λ) Figure 9.5
28. sum large mean count events low rate count events low probability many trials dnorm dgamma dpois dbinom dexp Z ∼ /PSNBM(µ, σ) Z ∼ #JOPNJBM(O, Q) Z ∼ 1PJTTPO(λ) Z ∼ (BNNB(λ, L) Z ∼ &YQPOFOUJBM(λ) Figure 9.5
29. sum large mean count events low rate count events low probability many trials dnorm dgamma dpois dbinom dexp Z ∼ /PSNBM(µ, σ) Z ∼ #JOPNJBM(O, Q) Z ∼ 1PJTTPO(λ) Z ∼ (BNNB(λ, L) Z ∼ &YQPOFOUJBM(λ) Figure 9.5
30. sum large mean count events low rate count events low probability many trials dnorm dgamma dpois dbinom dexp Z ∼ /PSNBM(µ, σ) Z ∼ #JOPNJBM(O, Q) Z ∼ 1PJTTPO(λ) Z ∼ (BNNB(λ, L) Z ∼ &YQPOFOUJBM(λ) Figure 9.5
31. Tide prediction machine, 1879 design, by William Thomson (Lord Kelvin, 1824–1907)
32. Generalized Linear Models • (1) Pick an outcome distribution • Distances and durations: exponential, gamma (survival or event history) • Counts: Poisson, binomial, multinomial, geometric • Monsters: Ranks and ordered categories • Mixtures: Beta-binomial, gamma-Poisson, zero- inflated processes, occupancy models
33. Generalized Linear Models • (2) Model parameters with a link γSJ|"J= ≈ −. + .() = . Z J ∼ /PSNBM(µJ, σ), µJ = α + β YJ + β YJ + β YJ + β YJ YJ + β YJ YJ + β YJ YJ + β YJ YJ YJ. Z J ∼ /PSNBM(µJ, σ), µJ = α + βYJ. Z J ∼ #JOPNJBM(Q J, O), G(Q J) = α + βYJ. same units
34. Generalized Linear Models µJ = α + β YJ + β YJ + β YJ + β YJ YJ + β YJ YJ + β YJ YJ + β YJ YJ YJ. Z J ∼ /PSNBM(µJ, σ), µJ = α + βYJ. Z J ∼ #JOPNJBM(Q J, O), G(Q J) = α + βYJ. same units count probability ZJ ∼ #JOPNJBM(OJ, QJ) QJ ? α + βYJ -3 -2 -1 0 1 2 3 -0.5 0.0 0.5 1.0 1.5 x a+b*x
35. Generalized Linear Models µJ = α + β YJ + β YJ + β YJ + β YJ YJ + β YJ YJ + β YJ YJ + β YJ YJ YJ. Z J ∼ /PSNBM(µJ, σ), µJ = α + βYJ. Z J ∼ #JOPNJBM(Q J, O), G(Q J) = α + βYJ. same units count link function ZJ ∼ #JOPNJBM(OJ, QJ) G(QJ) = α + βYJ -3 -2 -1 0 1 2 3 -0.5 0.0 0.5 1.0 1.5 x f^-1(a+b*x)
36. Generalized Linear Models • (3) Compute posterior • Search is harder • Interpretation is harder • Links matter • Quadratic approximation often works, but not always • Safer to rely on MCMC
37. • There are floor and ceiling effects floor ceiling -3 -2 -1 0 1 2 3 0.00 0.50 1.00 temperature prob survival Everything interacts
38. • Linear regression: • Logistic regression: Everything interacts FST JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPNF GVSUIFS DMBSJUZ PO U WFSZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ NBUIFNBUJDBMMZ DPNQVUJ PVUDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFEJDUPS 'JSTU SFDBMM UIBU UIF NFBO JT NPEFMFE MJLF µ = α + βY µ XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT DPOTUBOU *U EPFTOU NBUU X DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCBCJMJUZ Q XJUI SFTQFDU UP Q = FYQ(α + βY)  + FYQ(α + βY) WBUJWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) OLJOH 1BSBNFUFST JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPNF GVSUIFS D BU (-.T GPSDF FWFSZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ NBUIFNBUJDBM PG DIBOHF JO UIF PVUDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFEJDUPS 'JSTU (BVTTJBO NPEFM UIF NFBO JT NPEFMFE MJLF µ = α + βY UF PG DIBOHF JO µ XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT DPOTUBOU *U E VF Y IBT #VU OPX DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCBCJMJUZ Q XJUI S Y Q = FYQ(α + βY)  + FYQ(α + βY) X UBLJOH UIF EFSJWBUJWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) QQFBST JO UIJT BOTXFS UIF JNQBDU PG B DIBOHF JO Y EFQFOET VQPO Y ćBUT BO JOU BOE UIF POF UIBU BUUBJOT TUBUJTUJDBM TJHOJĕDBODF JT SFQPSUFE *O TFOTJUJWJUZ OBMZTFT BSF USJFE BOE BMM PG UIFN BSF EFTDSJCFE JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPNF GVSUIFS DMBSJUZ PO UIF SZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ NBUIFNBUJDBMMZ DPNQVUJOH UDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFEJDUPS 'JSTU SFDBMM UIBU JO F NFBO JT NPEFMFE MJLF µ = α + βY XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT DPOTUBOU *U EPFTOU NBUUFS DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCBCJMJUZ Q XJUI SFTQFDU UP B Q = FYQ(α + βY)  + FYQ(α + βY) JWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) 0WFSUIJOLJOH 1BSBNFUFST JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPN DMBJN UIBU (-.T GPSDF FWFSZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ N UIF SBUF PG DIBOHF JO UIF PVUDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFE B DMBTTJD (BVTTJBO NPEFM UIF NFBO JT NPEFMFE MJLF µ = α + βY 4P UIF SBUF PG DIBOHF JO µ XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT D XIBU WBMVF Y IBT #VU OPX DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCB QSFEJDUPS Y Q = FYQ(α + βY)  + FYQ(α + βY) "OE OPX UBLJOH UIF EFSJWBUJWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) 4JODF Y BQQFBST JO UIJT BOTXFS UIF JNQBDU PG B DIBOHF JO Y EFQFOET VQPO Y JUTFMG
39. • Counts of a specific event out of n possibilities • Constant expected value • Maxent: Binomial Binomial distribution count “successes” number of trials probability of success Z ∼ #JOPNJBM(O, Q) 0 2 4 6 8 10 0 500 1500 2500 Count Frequency lambda=0.5
40. Binomial distribution Z ∼ #JOPNJBM(O, Q) &(Z) = OQ WBS(Z) = OQ( − Q) Mean and variance not independent • Counts of a specific event out of n possibilities • Constant expected value • Maxent: Binomial 0 2 4 6 8 10 0 500 1500 2500 Count Frequency lambda=0.5
41. Need a link • y and p on different scales • y: count • p: probability • Want to model p as function of predictor variables • Must bound it to [0,1] interval Z ∼ #JOPNJBM(O, Q)
42. Logit link • Goal: map linear model to [0,1]  (&/&3"-*;&% -*/&"3 .0%&-4  -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI
43. Logit link • Goal: map linear model to [0,1]  (&/&3"-*;&% -*/&"3 .0%&-4  -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI
44. -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę NFBOT MFTT BOE MFTT DIBOHF PO UIF QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [FSP UIF MPHPEET BSF 'PS FYBNQMF JG 'ĶĴ łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF PG [FSP PO UIF MPHPEET TDBMF " IBMGVOJU JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QSPCBCJMJUZ #VU FBDI BEEJUJPO IBMGVOJU XJMM QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJUZ VOUJM BOZ JODSFBTF JT WBOJTIJOHMZ TNBMM ZJ ∼ ;BQIPE(θJ, φ) G (θJ) = α + βYJ G JT B MJOL GVODUJPO VU XIBU GVODUJPO TIPVME G CF " MJOL GVODUJPOT KPC JT UP NBQ UIF MJOFBS TQBDF PG B N + βYJ POUP UIF OPOMJOFBS TQBDF PG B QBSBNFUFS MJLF θ 4P G JT DIPTFO XJUI UIBU H .PTU PG UIF UJNF GPS NPTU (-.T ZPV DBO VTF POF PG UXP FYDFFEJOHMZ DPNNPO MJOL PS B MPH MJOL -FUT JOUSPEVDF FBDI BOE ZPVMM XPSL XJUI CPUI JO MBUFS DIBQUF ćF ĹļĴĶŁ ĹĶĻĸ NBQT B QBSBNFUFS UIBU JT EFĕOFE BT B QSPCBCJMJUZ NBTT BOE UIF BJOFE UP MJF CFUXFFO [FSP BOE POF POUP B MJOFBS NPEFM UIBU DBO UBLF PO BOZ SFBM OL JT FYUSFNFMZ DPNNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFY EFĕOJUJPO JU MPPLT MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ IF MPHJU GVODUJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ
45. -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę NFBOT MFTT BOE MFTT DIBOHF PO UIF QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [FSP UIF MPHPEET BSF 'PS FYBNQMF JG 'ĶĴ łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF PG [FSP PO UIF MPHPEET TDBMF " IBMGVOJU JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QSPCBCJMJUZ #VU FBDI BEEJUJPO IBMGVOJU XJMM QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJUZ VOUJM BOZ JODSFBTF JT WBOJTIJOHMZ TNBMM ZJ ∼ ;BQIPE(θJ, φ) G (θJ) = α + βYJ G JT B MJOL GVODUJPO VU XIBU GVODUJPO TIPVME G CF " MJOL GVODUJPOT KPC JT UP NBQ UIF MJOFBS TQBDF PG B N + βYJ POUP UIF OPOMJOFBS TQBDF PG B QBSBNFUFS MJLF θ 4P G JT DIPTFO XJUI UIBU H .PTU PG UIF UJNF GPS NPTU (-.T ZPV DBO VTF POF PG UXP FYDFFEJOHMZ DPNNPO MJOL PS B MPH MJOL -FUT JOUSPEVDF FBDI BOE ZPVMM XPSL XJUI CPUI JO MBUFS DIBQUF ćF ĹļĴĶŁ ĹĶĻĸ NBQT B QBSBNFUFS UIBU JT EFĕOFE BT B QSPCBCJMJUZ NBTT BOE UIF BJOFE UP MJF CFUXFFO [FSP BOE POF POUP B MJOFBS NPEFM UIBU DBO UBLF PO BOZ SFBM OL JT FYUSFNFMZ DPNNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFY EFĕOJUJPO JU MPPLT MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ IF MPHJU GVODUJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ
46. -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę NFBOT MFTT BOE MFTT DIBOHF PO UIF QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [FSP UIF MPHPEET BSF 'PS FYBNQMF JG 'ĶĴ łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF PG [FSP PO UIF MPHPEET TDBMF " IBMGVOJU JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QSPCBCJMJUZ #VU FBDI BEEJUJPO IBMGVOJU XJMM QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJUZ VOUJM BOZ JODSFBTF JT WBOJTIJOHMZ TNBMM ZJ ∼ ;BQIPE(θJ, φ) G (θJ) = α + βYJ G JT B MJOL GVODUJPO VU XIBU GVODUJPO TIPVME G CF " MJOL GVODUJPOT KPC JT UP NBQ UIF MJOFBS TQBDF PG B N + βYJ POUP UIF OPOMJOFBS TQBDF PG B QBSBNFUFS MJLF θ 4P G JT DIPTFO XJUI UIBU H .PTU PG UIF UJNF GPS NPTU (-.T ZPV DBO VTF POF PG UXP FYDFFEJOHMZ DPNNPO MJOL PS B MPH MJOL -FUT JOUSPEVDF FBDI BOE ZPVMM XPSL XJUI CPUI JO MBUFS DIBQUF ćF ĹļĴĶŁ ĹĶĻĸ NBQT B QBSBNFUFS UIBU JT EFĕOFE BT B QSPCBCJMJUZ NBTT BOE UIF BJOFE UP MJF CFUXFFO [FSP BOE POF POUP B MJOFBS NPEFM UIBU DBO UBLF PO BOZ SFBM OL JT FYUSFNFMZ DPNNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFY EFĕOJUJPO JU MPPLT MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ IF MPHJU GVODUJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ
47. NNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU PG B MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ FMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ F KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT IBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ UJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) BMMZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [F łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QS QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJU "OE JG ZPV UIJOL BCPVU JU B HPPE NPEFM PG QSPCB FWFOU JT BMNPTU HVBSBOUFFE UP IBQQFO JUT QSPCBCJ IPX JNQPSUBOU UIF QSFEJDUPS NBZ CF  (&/&3"-*;&% -*/&"3 .0%&-4 -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability
48. NNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU PG B MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ FMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ F KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT IBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ UJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) BMMZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE NPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU PG B F UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ VTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ O PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE UJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ WFOU BSF KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT BMMZ BMM UIBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ F EFĕOJUJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) O JT VTVBMMZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE ĶŁ CFDBVTF JU JOWFSUT UIF MPHJU USBOTGPSN T NFBOT JT UIBU XIFO ZPV VTF B MPHJU MJOL GPS B QBSBNFUFS ZPV BSF EFĕOJOH MVF UP CF UIF MPHJTUJD USBOTGPSN PG UIF MJOFBS NPEFM 'ĶĴłĿIJ ƑƎ JMMVTUSBUFT O UIBU UBLFT QMBDF XIFO VTJOH B MPHJU MJOL 0O UIF MFę UIF HFPNFUSZ PG UIF PXO XJUI IPSJ[POUBM MJOFT JOEJDBUJOH VOJU DIBOHFT JO UIF WBMVF PG UIF MJO -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [F łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QS QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJU "OE JG ZPV UIJOL BCPVU JU B HPPE NPEFM PG QSPCB FWFOU JT BMNPTU HVBSBOUFFE UP IBQQFO JUT QSPCBCJ IPX JNQPSUBOU UIF QSFEJDUPS NBZ CF  (&/&3"-*;&% -*/&"3 .0%&-4 -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability
49. NNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU PG B MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ FMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ F KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT IBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ UJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) BMMZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE NPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU PG B F UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ VTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ O PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE UJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ WFOU BSF KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT BMMZ BMM UIBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ F EFĕOJUJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) O JT VTVBMMZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE ĶŁ CFDBVTF JU JOWFSUT UIF MPHJU USBOTGPSN T NFBOT JT UIBU XIFO ZPV VTF B MPHJU MJOL GPS B QBSBNFUFS ZPV BSF EFĕOJOH MVF UP CF UIF MPHJTUJD USBOTGPSN PG UIF MJOFBS NPEFM 'ĶĴłĿIJ ƑƎ JMMVTUSBUFT O UIBU UBLFT QMBDF XIFO VTJOH B MPHJU MJOL 0O UIF MFę UIF HFPNFUSZ PG UIF PXO XJUI IPSJ[POUBM MJOFT JOEJDBUJOH VOJU DIBOHFT JO UIF WBMVF PG UIF MJO Solve for pi : MPHJU(QJ) = α + βYJ MG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT BU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ JPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) MZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE TF JU JOWFSUT UIF MPHJU USBOTGPSN JT UIBU XIFO ZPV VTF B MPHJU MJOL GPS B QBSBNFUFS ZPV BSF EFĕOJOH UIF MPHJTUJD USBOTGPSN PG UIF MJOFBS NPEFM 'ĶĴłĿIJ ƑƎ JMMVTUSBUFT LFT QMBDF XIFO VTJOH B MPHJU MJOL 0O UIF MFę UIF HFPNFUSZ PG UIF -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [F łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QS QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJU "OE JG ZPV UIJOL BCPVU JU B HPPE NPEFM PG QSPCB FWFOU JT BMNPTU HVBSBOUFFE UP IBQQFO JUT QSPCBCJ IPX JNQPSUBOU UIF QSFEJDUPS NBZ CF  (&/&3"-*;&% -*/&"3 .0%&-4 -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability inverse-link is logistic
50. 0.00 0.25 0.50 0.75 1.00 -4 -2 0 2 4 probability log-odds 0.00 0 0 20 40 60 80 100 odds 'JHVSF  -PHPEET WFSTVT PSEJOBSZ PEE log-odds = 1 log-odds = 3 p = 0.73 p = 0.95
51. Logit link • Where does this thing come from? • Several good answers: • “Natural” link inside probability formula • log-odds is fundamental parameter • See Overthinking box, pages 313–314 • Other links sometimes justified • Probit (common in economics) • Complementary-log-log (cloglog) • If you have a real scientific model, link is automatic F ĹļĴĶŁ ĹĶĻĸ NBQT B QBSBNFUFS UIBU JT EFĕOFE BT B QSPCBCJMJUZ NBTT BOE UIFSF JOFE UP MJF CFUXFFO [FSP BOE POF POUP B MJOFBS NPEFM UIBU DBO UBLF PO BOZ SFBM W L JT FYUSFNFMZ DPNNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU EFĕOJUJPO JU MPPLT MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ F MPHJU GVODUJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ ETw PG BO FWFOU BSF KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU QFO 4P SFBMMZ BMM UIBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ HVSF PVU UIF EFĕOJUJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BC O GPS QJ  QJ = FYQ(α + βYJ)  + FYQ(α + βYJ)
52. Prosocial chimpanzees partner focal  #*/0.*"- 3&(3&44*0/ 'ĶĴłĿIJ ƉƈƉ $IJNQBO[FF QSPTPD FYQFSJNFOU BT TFFO GSPN UIF QFSTQF PG UIF GPDBM BOJNBM ćF MFę BOE MFWFST BSF JOEJDBUFE JO UIF GPSFHSP 1VMMJOH FJUIFS FYQBOET BO BDDPSEJPO WJDF JO UIF DFOUFS QVTIJOH UIF GPPE UPXBSET CPUI FOET PG UIF UBCMF #PUI USBZT DMPTF UP UIF GPDBM BOJNBM IBWF JO UIFN 0OMZ POF PG UIF GPPE USBZ UIF PUIFS TJEF DPOUBJOT GPPE ćF QBS DPOEJUJPO NFBOT BOPUIFS BOJNBM BT UVSFE TJUT PO UIF PUIFS FOE PG UIF U 0UIFSXJTF UIF PUIFS FOE XBT FNQUZ
53. Prosocial chimpanzees • Two conditions: (1) partner, (2) alone • Two options: (1) prosocial, (2) asocial • Two outcomes: (1) left lever, (2) right lever • Want to predict outcome as function of condition and which side option is on • Do chimps prefer left lever when partner present and prosocial on left? => interaction!  #*/0.*"- 8IFO IVNBO TUVEFOUT QBSUJDJQBUF JO BO FY UIF MFWFS MJOLFE UP UXP QJFDFT PG GPPE UIF QSPT
54. Prosocial chimpanzees • Coding treatments: • (1) right/no-partner • (2) left/no-partner • (3) right/partner • (4) left/partner  #*/0.*"- 3&(3&44*0/  XO ćFSF BSF NBOZ XBZT UP DPOTUSVDU OFX WBSJBCMFT MJLF UIJT JODMVEJOH POT #VU PęFO BMM ZPV OFFE JT B MJUUMF BSJUINFUJD HFU NPEFM 4JODF UIJT JT BO FYQFSJNFOU UIF TUSVDUVSF UFMMT VT UIF NPEFM  ćF NPEFM JNQMJFE CZ UIF SFTFBSDI RVFTUJPO JT JO NBUIFNBUJDBM GPSN -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = αĮİŁļĿ[J] + βŁĿIJĮŁĺIJĻŁ[J] αK ∼ UP CF EFUFSNJOFE βL ∼ UP CF EFUFSNJOFE  WBSJBCMF +0'' Ǿ' !/ 4JODF UIF PVUDPNF DPVOUT BSF KVTU  PS  NF UZQF PG NPEFM EFĕOFE VTJOH B #FSOPVMMJ EJTUSJCVUJPO  #*/0.*"- 8IFO IVNBO TUVEFOUT QBSUJDJQBUF JO BO FY UIF MFWFS MJOLFE UP UXP QJFDFT PG GPPE UIF QSPT TJUT PO UIF PQQPTJUF TJEF PG UIF UBCMF ćF NPUJW CFIBWFT TJNJMBSMZ DIPPTJOH UIF QSPTPDJBM PQUJP *O UFSNT PG MJOFBS NPEFMT XF XBOU UP FTUJNBUF PS BCTFODF PG BOPUIFS BOJNBM BOE PQUJPO XIJ -PBE UIF EBUB GSPN UIF - /#$)&$)" QBDLB '$--4ǿ- /#$)&$)"Ȁ /ǿ#$(+)5 .Ȁ  ʚǶ #$(+)5 . Binomial(1,p) often called logistic regression Same as Bernoulli(p)
55. Logit link priors • Prior on logit scale not same shape as prior on probability scale • Use prior simulation to understand U BOPUIFS XBZ PG TBZJOH #JOPNJBM(, QJ) &JUIFS XBZ UIF NPEFM BCPWF JNQMJF FST POF GPS FBDI DIJNQBO[FF BOE  USFBUNFOU QBSBNFUFST POF GPS FBDI VOJR PO PG UIF QPTJUJPO PG UIF QSPTPDJBM PQUJPO BOE UIF QSFTFODF PG B QBSUOFS *O Q PVME TQFDJGZ B NPEFM UIBU BMMPXT FWFSZ DIJNQBO[FF UP IBWF UIFJS PXO  VOJR QBSBNFUFST *G UIBU TPVOET GVO UP ZPV * IBWF HPPE OFXT 8FMM EP FYBDUMZ UIBU QUFS ę UIF QSJPST BCPWF iUP CF EFUFSNJOFEw -FUT EFUFSNJOF UIFN * XBT USZJOH UP XB QSJPS QSFEJDUJWF TJNVMBUJPO FBSMJFS JO UIF CPPL /PX XJUI (-.T JU JT SFBMMZ HP -FUT DPOTJEFS B SVOU PG B MPHJTUJD SFHSFTTJPO XJUI KVTU B TJOHMF α QBSBNFUFS JO EFM -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = α α ∼ /PSNBM(, ω) P QJDL B WBMVF GPS ω 5P FNQIBTJ[F UIF NBEOFTT PG DPOWFOUJPOBM ĘBU QSJPST M TPNFUIJOH SBUIFS ĘBU MJLF ω =  ,0+ǿ
56. Logit link priors ę UIF QSJPST BCPWF iUP CF EFUFSNJOFEw -FUT EFUFSNJOF UIFN * XBT USZJOH UP XB QSJPS QSFEJDUJWF TJNVMBUJPO FBSMJFS JO UIF CPPL /PX XJUI (-.T JU JT SFBMMZ HP -FUT DPOTJEFS B SVOU PG B MPHJTUJD SFHSFTTJPO XJUI KVTU B TJOHMF α QBSBNFUFS JO EFM -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = α α ∼ /PSNBM(, ω) P QJDL B WBMVF GPS ω 5P FNQIBTJ[F UIF NBEOFTT PG DPOWFOUJPOBM ĘBU QSJPST M TPNFUIJOH SBUIFS ĘBU MJLF ω =  ,0+ǿ ǿ 0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ *"$/ǿ+Ȁ ʚǶ  Ǣ ʡ )*-(ǿ Ǎ Ǣ ǎǍ Ȁ /ʙ Ȁ BNQMF GSPN UIF QSJPS ZPV VQ GPS QSJPS QSFEJDUJWF TJNVMBUJPO FBSMJFS JO UIF CPPL /PX XJUI (-.T JU JT S UP QBZ PČ -FUT DPOTJEFS B SVOU PG B MPHJTUJD SFHSFTTJPO XJUI KVTU B TJOHMF α QBSBN MJOFBS NPEFM -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = α α ∼ /PSNBM(, ω) 8F OFFE UP QJDL B WBMVF GPS ω 5P FNQIBTJ[F UIF NBEOFTT PG DPOWFOUJPOBM ĘBU TUBSU XJUI TPNFUIJOH SBUIFS ĘBU MJLF ω =  (ǎǎǡǎ ʚǶ ,0+ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ  Ǣ  ʡ )*-(ǿ Ǎ Ǣ ǎǍ Ȁ Ȁ Ǣ /ʙ Ȁ /PX MFUT TBNQMF GSPN UIF QSJPS . /ǡ. ǿǎǖǖǖȀ +-$*- ʚǶ 3/-/ǡ+-$*-ǿ (ǎǎǡǎ Ǣ )ʙǎ Ǒ Ȁ
57. 0.0 0.2 0.4 0.6 0.8 1.0 0 5 10 15 prior prob pull left Density a ~ dnorm(0,10) a ~ dnorm(0,1.5) 0.0 0.2 0.4 0.6 0.8 1.0 0 2 4 6 8 10 12 14 prior diff between treatments Density b ~ dnorm(0,10) b ~ dnorm(0,0.5) 'ĶĴłĿIJ ƉƉƋ 1SJPS QSFEJDUJWF TJNVMBUJPOT GPS UIF NPTU CBTJD MPHJTUJD SFHSFT TJPO #MBDL EFOTJUZ " ĘBU /PSNBM   QSJPS PO UIF JOUFSDFQU QSPEVDFT B WFSZ OPOĘBU QSJPS EJTUSJCVUJPO PO UIF PVUDPNF TDBMF #MVF EFOTJUZ " NPSF DPODFOUSBUFE /PSNBM   QSJPS QSPEVDFT TPNFUIJOH NPSF SFBTPOBCMF Figure 11.3
58. Logit link priors UIF QSJPS QSPCBCJMJUZ PO UIF PVUDPNF TDBMF JT SBUIFS ĘBU ćJT JT QSPCBCMZ JT PQUJNBM TJODF QSPCBCJMJUJFT OFBS UIF DFOUFS BSF NPSF QMBVTJCMF #VU UIJ EFGBVMU QSJPST NPTU QFPQMF VTF NPTU PG UIF UJNF 8FMM VTF JU /PX XF OFFE UP EFUFSNJOF B QSJPS GPS UIF USFBUNFOU FČFDUT UIF β QBS EFGBVMU UP VTJOH UIF TBNF /PSNBM   QSJPS GPS UIF USFBUNFOU FČFDUT UIBU UIFZ BSF BMTP KVTU JOUFSDFQUT POF JOUFSDFQU GPS FBDI USFBUNFOU #VU XFJSEOFTT PG DPOWFOUJPOBMMZ ĘBU QSJPST MFUT TFF XIBU /PSNBM   MPP NPEFM 3 DPEF  (ǎǎǡǏ ʚǶ ,0+ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ  ʔ ȁ/- /( )/Ȃ Ǣ  ʡ )*-(ǿ Ǎ Ǣ ǎǡǒ ȀǢ ȁ/- /( )/Ȃ ʡ )*-(ǿ Ǎ Ǣ ǎǍ Ȁ Ȁ Ǣ /ʙ Ȁ • What about slopes?
59. 0.0 0.2 0.4 0.6 0.8 1.0 0 5 10 15 prior prob pull left Density a ~ dnorm(0,10) a ~ dnorm(0,1.5) 0.0 0.2 0.4 0.6 0.8 1.0 0 2 4 6 8 10 12 14 prior diff between treatments Density b ~ dnorm(0,10) b ~ dnorm(0,0.5) 'ĶĴłĿIJ ƉƉƋ 1SJPS QSFEJDUJWF TJNVMBUJPOT GPS UIF NPTU CBTJD MPHJTUJD SFHSFT TJPO #MBDL EFOTJUZ " ĘBU /PSNBM   QSJPS PO UIF JOUFSDFQU QSPEVDFT B WFSZ OPOĘBU QSJPS EJTUSJCVUJPO PO UIF PVUDPNF TDBMF #MVF EFOTJUZ " NPSF DPODFOUSBUFE /PSNBM   QSJPS QSPEVDFT TPNFUIJOH NPSF SFBTPOBCMF Figure 11.3
60. ȕ +-/$' . $) ǎǎǶ$( ).$*)' .+ (ǎǎǡǑ ʚǶ 0'(ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ ȁ/*-Ȃ ʔ ȁ/- /( )/Ȃ Ǣ ȁ/*-Ȃ ʡ )*-(ǿ Ǎ Ǣ ǎǡǒ ȀǢ ȁ/- /( )/Ȃ ʡ )*-(ǿ Ǎ Ǣ Ǎǡǒ Ȁ Ȁ Ǣ /ʙ/Ǿ'$./ Ǣ #$).ʙǑ Ȁ +- $.ǿ (ǎǎǡǑ Ǣ  +/#ʙǏ Ȁ ( ) . ǒǡǒʉ ǖǑǡǒʉ )Ǿ !! #/ ȁǎȂ ǶǍǡǑǑ ǍǡǐǑ ǶǍǡǖǔ Ǎǡǎǎ ǔǐǓ ǎ ȁǏȂ ǐǡǖǍ Ǎǡǔǔ ǏǡǔǕ ǒǡǏǏ ǖǏǎ ǎ ȁǐȂ ǶǍǡǔǒ ǍǡǐǑ ǶǎǡǏǖ ǶǍǡǏǍ ǕǕǓ ǎ ȁǑȂ ǶǍǡǔǑ ǍǡǐǑ ǶǎǡǏǕ ǶǍǡǎǖ ǔǔǍ ǎ ȁǒȂ ǶǍǡǑǑ ǍǡǐǑ ǶǍǡǖǕ ǍǡǍǕ ǕǐǏ ǎ ȁǓȂ ǍǡǑǕ ǍǡǐǑ ǶǍǡǍǕ ǎǡǍǏ ǕǒǑ ǎ ȁǔȂ ǎǡǖǓ ǍǡǑǎ ǎǡǏǖ ǏǡǓǎ ǕǑǔ ǎ ȁǎȂ ǶǍǡǍǒ ǍǡǏǖ ǶǍǡǒǎ ǍǡǑǏ ǔǕǎ ǎ ȁǏȂ ǍǡǑǕ ǍǡǏǖ ǍǡǍǐ ǍǡǖǑ Ǔǒǔ ǎ ȁǐȂ ǶǍǡǐǖ ǍǡǏǕ ǶǍǡǕǐ ǍǡǍǔ ǓǓǖ ǎ ȁǑȂ ǍǡǐǓ ǍǡǏǖ ǶǍǡǎǎ ǍǡǕǎ ǔǐǏ ǎ Chimpanzees { Treatments RN LN RP LP
61. Individual differences ȁǑȂ ǍǡǐǓ ǍǡǏǖ ǶǍǡǎǎ ǍǡǕǎ ǔǐǏ ǎ ćJT JT UIF HVUT PG UIF UJEF QSFEJDUJPO FOHJOF 8FMM OFFE UP EP B MJUUMF XP ĕSTU  QBSBNFUFST BSF UIF JOUFSDFQUT VOJRVF UP FBDI DIJNQBO[FF &BDI P UFOEFODZ PG FBDI JOEJWJEVBM UP QVMM UIF MFę MFWFS -FUT MPPL BU UIFTF PO 3 DPEF  +*./ ʚǶ 3/-/ǡ.(+' .ǿ(ǎǎǡǑȀ +Ǿ' !/ ʚǶ $)1Ǿ'*"$/ǿ +*./ɶ Ȁ +'*/ǿ +- $.ǿ .ǡ/ǡ!-( ǿ+Ǿ' !/Ȁ Ȁ Ǣ 3'$(ʙǿǍǢǎȀ Ȁ V7 V6 V5 V4 V3 V2 V1 0.0 0.2 0.4 0.6 0.8 Value &BDI SPX JT B DIJNQBO[FF UIF OVNCFST DPSSFTQPOEJOH UP UIF WBMVFT JO JOEJWJEVBMT‰OVNCFST    BOE ‰TIPXB QSFGFSFODF GPSUIFSJHIU MFWF JT UIF HVUT PG UIF UJEF QSFEJDUJPO FOHJOF 8FMM OFFE UP EP B MJUUMF XPSL UP JOUFSQSFU JU  QBSBNFUFST BSF UIF JOUFSDFQUT VOJRVF UP FBDI DIJNQBO[FF &BDI PG UIFTF FYQSFTTFT FODZ PG FBDI JOEJWJEVBM UP QVMM UIF MFę MFWFS -FUT MPPL BU UIFTF PO UIF PVUDPNF TDBM ʚǶ 3/-/ǡ.(+' .ǿ(ǎǎǡǑȀ !/ ʚǶ $)1Ǿ'*"$/ǿ +*./ɶ Ȁ ǿ +- $.ǿ .ǡ/ǡ!-( ǿ+Ǿ' !/Ȁ Ȁ Ǣ 3'$(ʙǿǍǢǎȀ Ȁ V7 V6 V5 V4 V3 V2 V1 0.0 0.2 0.4 0.6 0.8 1.0 Value SPX JT B DIJNQBO[FF UIF OVNCFST DPSSFTQPOEJOH UP UIF WBMVFT JO /*- 'PVS PG WJEVBMT‰OVNCFST    BOE ‰TIPXB QSFGFSFODF GPSUIFSJHIU MFWFS 5XPJOEJWJEVB “Lefty”
62. Treatments UFOEFODJFT ćJT JT FYBDUMZ UIF LJOE PG FČFDU UIBU NBLFT QVSF FYQFSJNFOUT EJďD CFIBWJPSBM TDJFODFT )BWJOH SFQFBU NFBTVSFNFOUT MJLF JO UIJT FYQFSJNFOU BOE N UIFN JT WFSZ VTFGVM /PX MFUT DPOTJEFS UIF USFBUNFOU FČFDUT IPQFGVMMZ FTUJNBUFE NPSF QSFDJTFMZ C NPEFM DPVME TVCUSBDU PVU UIF IBOEFEOFTT WBSJBUJPO BNPOH BDUPST 0O UIF MPHJU TD '. ʚǶ ǿǫȅǫǢǫ ȅǫǢǫȅǫǢǫ ȅǫȀ +'*/ǿ +- $.ǿ (ǎǎǡǑ Ǣ  +/#ʙǏ Ǣ +-.ʙǫǫ Ȁ Ǣ ' '.ʙ'. Ȁ L/P R/P L/N R/N -0.5 0.0 0.5 1.0 Value *WF BEEFE USFBUNFOU MBCFMT JO QMBDF PG UIF QBSBNFUFS OBNFT -/ NFBOT iQSPTPDJ OP QBSUOFSw 31 NFBOT wQSPTPDJBM PO SJHIU  QBSUOFSw 5P VOEFSTUBOE UIFTF EJTUSJCV IFMQ UP DPOTJEFS PVS FYQFDUBUJPOT 8IBU XF BSF MPPLJOH GPS JT FWJEFODF UIBU UIF DIJ DIPPTF UIF QSPTPDJBM PQUJPO NPSF XIFO B QBSUOFS JT QSFTFOU ćJT JNQMJFT DPNQ ĕSTU SPX XJUI UIF UIJSE SPX BOE UIF TFDPOE SPX XJUI UIF GPVSUI SPX :PV DBO QSP BMSFBEZ UIBU UIFSF JTOU NVDI FWJEFODF PG QSPTPDJBM JOUFOUJPO JO UIFTF EBUB #VU MFU OEFODJFT ćJT JT FYBDUMZ UIF LJOE PG FČFDU UIBU NBLFT QVSF FYQFSJNFOUT EJďDVMU JO U IBWJPSBM TDJFODFT )BWJOH SFQFBU NFBTVSFNFOUT MJLF JO UIJT FYQFSJNFOU BOE NFBTVSJ FN JT WFSZ VTFGVM /PX MFUT DPOTJEFS UIF USFBUNFOU FČFDUT IPQFGVMMZ FTUJNBUFE NPSF QSFDJTFMZ CFDBVTF U PEFM DPVME TVCUSBDU PVU UIF IBOEFEOFTT WBSJBUJPO BNPOH BDUPST 0O UIF MPHJU TDBMF . ʚǶ ǿǫȅǫǢǫ ȅǫǢǫȅǫǢǫ ȅǫȀ */ǿ +- $.ǿ (ǎǎǡǑ Ǣ  +/#ʙǏ Ǣ +-.ʙǫǫ Ȁ Ǣ ' '.ʙ'. Ȁ L/P R/P L/N R/N -0.5 0.0 0.5 1.0 Value F BEEFE USFBUNFOU MBCFMT JO QMBDF PG UIF QBSBNFUFS OBNFT -/ NFBOT iQSPTPDJBM PO MF QBSUOFSw 31 NFBOT wQSPTPDJBM PO SJHIU  QBSUOFSw 5P VOEFSTUBOE UIFTF EJTUSJCVUJPOT J Q UP DPOTJEFS PVS FYQFDUBUJPOT 8IBU XF BSF MPPLJOH GPS JT FWJEFODF UIBU UIF DIJNQBO[F PPTF UIF QSPTPDJBM PQUJPO NPSF XIFO B QBSUOFS JT QSFTFOU ćJT JNQMJFT DPNQBSJOH U
63. proportion left lever 0 0.5 1 actor 1 actor 2 actor 3 actor 4 actor 5 actor 6 actor 7 R/N L/N R/P L/P observed proportions proportion left lever 0 0.5 1 actor 1 actor 2 actor 3 actor 4 actor 5 actor 6 actor 7 posterior predictions Figure 11.4
64. proportion left lever 0 0.5 1 actor 1 actor 2 actor 3 actor 4 actor 5 actor 6 actor 7 R/N L/N R/P L/P observed proportions proportion left lever 0 0.5 1 actor 1 actor 2 actor 3 actor 4 actor 5 actor 6 actor 7 posterior predictions Figure 11.4
65. Comparing no-interaction .$ ʙ ɶ.$ Ǣ *) ʙ ɶ*) Ȁ (ǎǎǡǒ ʚǶ 0'(ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ ȁ/*-Ȃ ʔ .ȁ.$ Ȃ ʔ ȁ*)Ȃ Ǣ ȁ/*-Ȃ ʡ )*-(ǿ Ǎ Ǣ ǎǡǒ ȀǢ .ȁ.$ Ȃ ʡ )*-(ǿ Ǎ Ǣ Ǎǡǒ ȀǢ ȁ*)Ȃ ʡ )*-(ǿ Ǎ Ǣ Ǎǡǒ Ȁ Ȁ Ǣ /ʙ/Ǿ'$./Ǐ Ǣ #$).ʙǑ Ǣ '*"Ǿ'$&ʙ Ȁ (P CBDL UP NPEFM (ǎǎǡǑ BOE BEE '*"Ǿ'$&ʙ ćFO XF DBO DPNQ IFSF VTJOH -00*4 3 DPEF  *(+- ǿ (ǎǎǡǒ Ǣ (ǎǎǡǑ Ǣ !0)ʙ  Ȁ  +    2 $"#/   (ǎǎǡǒ ǒǐǎǡǏ ǔǡǖ ǍǡǍ ǍǡǓǓ ǎǖǡǎǔ  (ǎǎǡǑ ǒǐǏǡǓ Ǖǡǔ ǎǡǑ ǍǡǐǑ ǎǖǡǍǎ ǎǡǏǕ 8"*$ QSPEVDFT JEFOUJDBM SFTVMUT "T XF HVFTTFE UIF NPEFM XJUIPVU UIF OP XPSTF JO FYQFDUFE QSFEJDUJWF BDDVSBDZ UIBO UIF NPEFM XJUI JU :P 3 DPEF  /Ǿ'$./Ǐ ʚǶ '$./ǿ +0'' Ǿ' !/ ʙ ɶ+0'' Ǿ' !/Ǣ /*- ʙ ɶ/*-Ǣ .$ ʙ ɶ.$ Ǣ *) ʙ ɶ*) Ȁ (ǎǎǡǒ ʚǶ 0'(ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ ȁ/*-Ȃ ʔ .ȁ.$ Ȃ ʔ ȁ*)Ȃ Ǣ ȁ/*-Ȃ ʡ )*-(ǿ Ǎ Ǣ ǎǡǒ ȀǢ .ȁ.$ Ȃ ʡ )*-(ǿ Ǎ Ǣ Ǎǡǒ ȀǢ ȁ*)Ȃ ʡ )*-(ǿ Ǎ Ǣ Ǎǡǒ Ȁ Ȁ Ǣ /ʙ/Ǿ'$./Ǐ Ǣ #$).ʙǑ Ǣ '*"Ǿ'$&ʙ Ȁ (P CBDL UP NPEFM (ǎǎǡǑ BOE BEE '*"Ǿ'$&ʙ ćFO XF DBO DPNQBSF UIF U IFSF VTJOH -00*4 3 DPEF  *(+- ǿ (ǎǎǡǒ Ǣ (ǎǎǡǑ Ǣ !0)ʙ  Ȁ  +    2 $"#/   (ǎǎǡǒ ǒǐǎǡǏ ǔǡǖ ǍǡǍ ǍǡǓǓ ǎǖǡǎǔ  (ǎǎǡǑ ǒǐǏǡǓ Ǖǡǔ ǎǡǑ ǍǡǐǑ ǎǖǡǍǎ ǎǡǏǕ
66. Relative and absolute effects • Parameters on relative effect scale • Predictions on absolute effect scale • Proportional odds: Relative effect measure • Good for scaring people, getting published • Not so good for public health, scientific progress • But needed for causal inference  3FMBUJWF TIBSL BOE BCTPMVUF QFOHVJO *O UIF BOBMZTJT BCPWF * NPTUMZ GP DIBOHFT JO QSFEJDUJPOT PO UIF PVUDPNF TDBMF‰IPX NVDI EJČFSFODF EPFT UIF USFBUN JO UIF QSPCBCJMJUZ PG QVMMJOH B MFWFS ćJT WJFX PG QPTUFSJPS QSFEJDUJPO GPDVTFT PO Į IJijijIJİŁŀ UIF EJČFSFODF B DPVOUFSGBDUVBM DIBOHF JO B WBSJBCMF NJHIU NBLF PO BO TDBMF PG NFBTVSFNFOU MJLF UIF QSPCBCJMJUZ PG BO FWFOU *U JT NPSF DPNNPO UP TFF MPHJTUJD SFHSFTTJPOT JOUFSQSFUFE UISPVHI ĿIJĹĮŁĶŃIJ 3FMBUJWF FČFDUT BSF QSPQPSUJPOBM DIBOHFT JO UIF PEET PG BO PVUDPNF *G XF DIBOHF BOE TBZ UIF PEET PG BO PVUDPNF EPVCMF UIFO XF BSF EJTDVTTJOH SFMBUJWF FČFDUT :PV MBUF UIFTF ĽĿļĽļĿŁĶļĻĮĹ ļııŀ SFMBUJWF FČFDU TJ[FT CZ TJNQMZ FYQPOFOUJBUJOH UIF Q PG JOUFSFTU 'PS FYBNQMF UP DBMDVMBUF UIF QSPQPSUJPOBM PEET PG TXJUDIJOH GSPN USFBU USFBUNFOU  BEEJOH B QBSUOFS  3 DPEF  +*./ ʚǶ 3/-/ǡ.(+' .ǿ(ǎǎǡǑȀ ( )ǿ 3+ǿ+*./ɶȁǢǑȂǶ+*./ɶȁǢǏȂȀ Ȁ ȁǎȂ ǍǡǖǏǍǓǑǔǖ 0O BWFSBHF UIF TXJUDI NVMUJQMFT UIF PEET PG QVMMJOH UIF MFę MFWFS CZ  BO  S JO PEET ćJT JT XIBU JT NFBOU CZ QSPQPSUJPOBM PEET ćF OFX PEET BSF DBMDVMBUFE UIF PME PEET BOE NVMUJQMZJOH UIFN CZ UIF QSPQPSUJPOBM PEET XIJDI JT  JO UIJT ćF SJTL PG GPDVTJOH PO SFMBUJWF FČFDUT TVDI BT QSPQPSUJPOBM PEET JT UIBU UI FOPVHI UP UFMM VT XIFUIFS B WBSJBCMF JT JNQPSUBOU PS OPU *G UIF PUIFS QBSBNFUFST JO U NBLF UIF PVUDPNF WFSZ VOMJLFMZ UIFO FWFO B MBSHF QSPQPSUJPOBM PEET MJLF  XPVME UIF PVUDPNF GSFRVFOU $POTJEFS GPS FYBNQMF B SBSF EJTFBTF XIJDI PDDVST JO  QFS PO QFPQMF 4VQQPTF BMTP UIBU SFBEJOH UIJT UFYUCPPL JODSFBTFE UIF PEET PG UIF EJTFBTF G
67. Relative and absolute effects • Parameters on relative effect scale • Predictions on absolute effect scale • Using relative effects may exaggerate importance of predictor • Good for scaring people, getting published • Not so good for public health, scientific progress • But needed for causal inference relative shark absolute penguin
69. Risk communication • Many people mistake relative risk for absolute risk • Example: • 1/1000 women develop blood clots • 3/1000 women on birth control develop blood clots • => 200% increase in blood clots! • Change in probability is only 0.002 • Pregnancy much more dangerous than blood clots