Statistical Rethinking - Lecture 13

Lecture 13 - Generalized Linear Models - Statistical Rethinking: A Bayesian Course with R Examples

Transcript

1. Week 7: Big Entropy & The Generalized Linear Model Richard McElreath Statistical Rethinking
2. Installing Stan • Two common issues: • Windows: • Problem: RTools doesn’t set PATH correctly, so R can’t find compiler. • Solution: Run RTools installer as administrator. Right- click and choose run as admin. Be sure to check box at end of install. • Mac OS X: • Problem: R keeps trying to use llvm-g++-4.2 instead of clang++ • Solution: Enter each line into Terminal: cd /usr/bin sudo ln -fs clang llvm-gcc-4.2 sudo ln -fs clang++ llvm-g++-4.2
3. Update rethinking package • Version 1.48 up • Fixes: • Weird ensemble issue with zero-weight models • map rejecting linear models like: mu <- a • see notes on github for more
4. Tide prediction machine, 1879 design, by William Thomson (Lord Kelvin, 1824–1907)
5. Tyranny of Gauss • Gaussian fine, as long as only care about mean, stddev • But can often do better, use more information • Abuse of the Gaussian as an outcome distribution • Coercion • Surrender
6. Generalized Linear Models • Freedom from tyranny of Gaussian likelihoods • Freedom from tyranny of randomization tests • But more choices and responsibilities
7. Generalized Linear Models • Goal: Connect linear model to outcome variable • Would be better to ditch linear model, too • Strategy: 1. Pick an outcome distribution 2. Model its parameters using links to linear models 3. Compute posterior • Can model multivariate relationships and non- linear responses • Building blocks of multilevel models
8. Generalized Linear Models • (1) Pick an outcome distribution • Mostly exponential family distributions • Members arise from natural processes; Occur frequently in nature • Have maximum entropy interpretations • Select from first principles • Resist histomancy: Superstitious practice of picking likelihood functions by gazing at a histogram 1 2 3 4 5
9. Maximum entropy • Due to Edwin T. Jaynes (1922–1998) • The maxent principle: • Distribution with largest entropy is distribution most consistent with stated assumptions • Can happen the largest number of ways • For observations, way to construct likelihood • Also reproduces Bayesian updating as special case (minimum cross-entropy) • Posterior least divergence to prior while still consistent with data E. T. Jaynes (1922–1998)
10. Maximum entropy Constraints Maxent distribution Real value in interval Uniform Real value, finite variance Gaussian Binary events, fixed probability Binomial Non-negative real, has mean Exponential
11. exponential Exponential • Machine/organism, n components • Any component breaks, machine/organism breaks • Assume constant chance fail on any day of year PS PSHBOJTN XJMM EJF 3BUIFS * NFBO UIBU UIF EJTUSJCVUJPO PG UIFTF GBJMVSFT UFOET UP UBLF B DIBSBDUFSJTUJD GPSN PS POF WFSZ DMPTF UP JU 4VQQPTF UIFSF JT B NBDIJOF XJUI B OVNCFS PG DPNQPOFOUT OFDFTTBSZ GPS JUT GVODUJPO 8IFO BOZ POF PG UIFTF DPNQPOFOUT GBJMT UIF NBDIJOF GBJMT /PX BMTP TVQQPTF FBDI DPN QPOFOU XJMM GBJM TPNFUJNF CFUXFFO UPEBZ BOE POF ZFBS GSPN OPX &BDI DPNQPOFOU IPX FWFS IBT FRVBM QSPCBCJMJUZ PG GBJMJOH PO BOZ EBZ PG UIF ZFBS BOE FBDI DPNQPOFOU GBJMT JOEFQFOEFOUMZ PG UIF PUIFST 8IBU XJMM UIF EJTUSJCVUJPO PG UJNFT UP GBJMVSF MPPL MJLF ćF BOTXFS EFQFOET DSJUJDBMMZ PO IPX NBOZ DPNQPOFOUT UIFSF BSF JO UIF NBDIJOF 'ĶĴłĿIJ ƉƈƉ  'PS POF DPNQPOFOU 'ĶĴłĿIJ ƉƈƉ MFę UIF EJTUSJCVUJPO JT OBUVSBMMZ KVTU VOJGPSN‰BOZ EBZ CFUXFFO UPNPSSPX BOE POF ZFBS GSPN OPX JT FRVBMMZ MJLFMZ :PV DBO TJNVMBUF UIF EJTUSJCVUJPO GPS ZPVSTFMG XJUI UIJT POF MJOF PG 3 DPEF 4 ʄǤ - +'$/ ǭ Ƽƻƻƻƻ ǐ ($)ǭ -0)$!ǭ)ʃƼǐ($)ʃƻǐ(3ʃƾǁǀǮ Ǯ Ǯ "T ZPV BEE DPNQPOFOUT UP UIF NBDIJOF JODSFBTF )ʃƼ UP IJHIFS JOUFHFST UIF EJTUSJCVUJPO RVJDLMZ DIBOHFT 'PS UISFF DPNQPOFOUT 'ĶĴłĿIJ ƉƈƉ NJEEMF UIF EJTUSJCVUJPO BMSFBEZ QJMFT VQ BU MPX UJNFT UP GBJMVSF XJUI GFXFS BOE GFXFS NBDIJOFT TVSWJWJOH BOZXIFSF DMPTF UP B GVMM ZFBS #Z UIF UJNF UIFSF BSF  DPNQPOFOUT UP UIF NBDIJOF 'ĶĴłĿIJ ƉƈƉ SJHIU UIF EJTUSJCVUJPO PG GBJMVSF UJNFT JT OFBSMZ JEFOUJDBM UP BOPUIFS JNQPSUBOU JEFBM FYQPOFOUJBM GBNJMZ EJTUSJCVUJPO UIF FYQPOFOUJBM ćF CMBDL DVSWF TVQFSJNQPTFE JO UIF UIJSE QMPU JO 'ĶĴłĿIJ ƉƈƉ JT BO JEFBMJ[FE FYQPOFOUJBM EJTUSJCVUJPO XJUI UIF TBNF NFBO BT UIF FNQJSJDBM EJTUSJCVUJPO TIPXO JO CMVF 'BJMVSF UJNFT IBWF DPOWFSHFE UP BO FYQPOFOUJBM EFOTJUZ JO XIJDI UIF QSPCBCJMJUZ PG GBJMJOH BU UJNF 5 JT 1S(5|λ) = λ FYQ(−λ5). ćF QBSBNFUFS λ HPWFSOT UIF SBUF PG GBJMVSF BOE JO UIFTF FYBNQMFT JU JT BQQSPYJNBUFMZ   %*45"/$& "/% %63"5*0/ 0 100 200 300 days until failure 1 component 0 100 200 300 days until failure 3 components 0 50 100 150 200 days until failure 10 components
12. sum large mean count events low rate count events low probability many trials dnorm dgamma dpois dbinom dexp Z ∼ /PSNBM(µ, σ) Z ∼ #JOPNJBM(O, Q) Z ∼ 1PJTTPO(λ) Z ∼ (BNNB(λ, L) Z ∼ &YQPOFOUJBM(λ) Figure 9.5
13. Generalized Linear Models • (1) Pick an outcome distribution • Distances and durations: exponential, gamma (survival or event history) • Counts: Poisson, binomial, multinomial, geometric • Monsters: Ranks and ordered categories • Mixtures: Beta-binomial, gamma-Poisson, zero- inflated processes
14. Generalized Linear Models • (2) Model parameters with a link γSJ|"J= ≈ −. + .() = . Z J ∼ /PSNBM(µJ, σ), µJ = α + β YJ + β YJ + β YJ + β YJ YJ + β YJ YJ + β YJ YJ + β YJ YJ YJ. Z J ∼ /PSNBM(µJ, σ), µJ = α + βYJ. Z J ∼ #JOPNJBM(Q J, O), G(Q J) = α + βYJ. same units
15. Generalized Linear Models µJ = α + β YJ + β YJ + β YJ + β YJ YJ + β YJ YJ + β YJ YJ + β YJ YJ YJ. Z J ∼ /PSNBM(µJ, σ), µJ = α + βYJ. Z J ∼ #JOPNJBM(Q J, O), G(Q J) = α + βYJ. same units count probability ZJ ∼ #JOPNJBM(OJ, QJ) QJ ? α + βYJ -3 -2 -1 0 1 2 3 -0.5 0.0 0.5 1.0 1.5 x a+b*x
16. Generalized Linear Models µJ = α + β YJ + β YJ + β YJ + β YJ YJ + β YJ YJ + β YJ YJ + β YJ YJ YJ. Z J ∼ /PSNBM(µJ, σ), µJ = α + βYJ. Z J ∼ #JOPNJBM(Q J, O), G(Q J) = α + βYJ. same units count link function ZJ ∼ #JOPNJBM(OJ, QJ) G(QJ) = α + βYJ -3 -2 -1 0 1 2 3 -0.5 0.0 0.5 1.0 1.5 x f^-1(a+b*x)
17. Generalized Linear Models • How to choose a link? • Can use canonical or natural link • But natural links not always best (esp. for exponential, gamma) • Imagination and pragmatism • Most common • Constrain to [0,1]: logit • Constrain to +reals: log
18. Logit link • Goal: map linear model to [0,1]  (&/&3"-*;&% -*/&"3 .0%&-4  -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI
19. -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę NFBOT MFTT BOE MFTT DIBOHF PO UIF QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [FSP UIF MPHPEET BSF 'PS FYBNQMF JG 'ĶĴ łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF PG [FSP PO UIF MPHPEET TDBMF " IBMGVOJU JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QSPCBCJMJUZ #VU FBDI BEEJUJPO IBMGVOJU XJMM QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJUZ VOUJM BOZ JODSFBTF JT WBOJTIJOHMZ TNBMM ZJ ∼ ;BQIPE(θJ, φ) G (θJ) = α + βYJ G JT B MJOL GVODUJPO VU XIBU GVODUJPO TIPVME G CF " MJOL GVODUJPOT KPC JT UP NBQ UIF MJOFBS TQBDF PG B N + βYJ POUP UIF OPOMJOFBS TQBDF PG B QBSBNFUFS MJLF θ 4P G JT DIPTFO XJUI UIBU H .PTU PG UIF UJNF GPS NPTU (-.T ZPV DBO VTF POF PG UXP FYDFFEJOHMZ DPNNPO MJOL PS B MPH MJOL -FUT JOUSPEVDF FBDI BOE ZPVMM XPSL XJUI CPUI JO MBUFS DIBQUF ćF ĹļĴĶŁ ĹĶĻĸ NBQT B QBSBNFUFS UIBU JT EFĕOFE BT B QSPCBCJMJUZ NBTT BOE UIF BJOFE UP MJF CFUXFFO [FSP BOE POF POUP B MJOFBS NPEFM UIBU DBO UBLF PO BOZ SFBM OL JT FYUSFNFMZ DPNNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFY EFĕOJUJPO JU MPPLT MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ IF MPHJU GVODUJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ
20. -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B QSPCBCJMJUZ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIF HFPNFUSZ GBS GSPN [FSP TVDI UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę NFBOT MFTT BOE MFTT DIBOHF PO UIF QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [FSP UIF MPHPEET BSF 'PS FYBNQMF JG 'ĶĴ łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF PG [FSP PO UIF MPHPEET TDBMF " IBMGVOJU JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QSPCBCJMJUZ #VU FBDI BEEJUJPO IBMGVOJU XJMM QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJUZ VOUJM BOZ JODSFBTF JT WBOJTIJOHMZ TNBMM ZJ ∼ ;BQIPE(θJ, φ) G (θJ) = α + βYJ G JT B MJOL GVODUJPO VU XIBU GVODUJPO TIPVME G CF " MJOL GVODUJPOT KPC JT UP NBQ UIF MJOFBS TQBDF PG B N + βYJ POUP UIF OPOMJOFBS TQBDF PG B QBSBNFUFS MJLF θ 4P G JT DIPTFO XJUI UIBU H .PTU PG UIF UJNF GPS NPTU (-.T ZPV DBO VTF POF PG UXP FYDFFEJOHMZ DPNNPO MJOL PS B MPH MJOL -FUT JOUSPEVDF FBDI BOE ZPVMM XPSL XJUI CPUI JO MBUFS DIBQUF ćF ĹļĴĶŁ ĹĶĻĸ NBQT B QBSBNFUFS UIBU JT EFĕOFE BT B QSPCBCJMJUZ NBTT BOE UIF BJOFE UP MJF CFUXFFO [FSP BOE POF POUP B MJOFBS NPEFM UIBU DBO UBLF PO BOZ SFBM OL JT FYUSFNFMZ DPNNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFY EFĕOJUJPO JU MPPLT MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ IF MPHJU GVODUJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ
21. NNPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU PG B MJLF UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ FMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ F KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT IBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ UJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) BMMZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE NPO XIFO XPSLJOH XJUI CJOPNJBM (-.T *O UIF DPOUFYU PG B F UIJT ZJ ∼ #JOPNJBM(O, QJ) MPHJU(QJ) = α + βYJ JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ VTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ O PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE UJPO JUTFMG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ WFOU BSF KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT BMMZ BMM UIBU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ F EFĕOJUJPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) O JT VTVBMMZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE ĶŁ CFDBVTF JU JOWFSUT UIF MPHJU USBOTGPSN T NFBOT JT UIBU XIFO ZPV VTF B MPHJU MJOL GPS B QBSBNFUFS ZPV BSF EFĕOJOH MVF UP CF UIF MPHJTUJD USBOTGPSN PG UIF MJOFBS NPEFM 'ĶĴłĿIJ ƑƎ JMMVTUSBUFT O UIBU UBLFT QMBDF XIFO VTJOH B MPHJU MJOL 0O UIF MFę UIF HFPNFUSZ PG UIF PXO XJUI IPSJ[POUBM MJOFT JOEJDBUJOH VOJU DIBOHFT JO UIF WBMVF PG UIF MJO Solve for pi : MPHJU(QJ) = α + βYJ MG JT EFĕOFE BT UIF MPHPEET MPHJU(QJ) = MPH QJ  − QJ KVTU UIF QSPCBCJMJUZ JU IBQQFOT EJWJEFE CZ UIF QSPCBCJMJUZ JU EPFT BU JT CFJOH TUBUFE IFSF JT MPH QJ  − QJ = α + βYJ JPO PG QJ JNQMJFE IFSF KVTU EP B MJUUMF BMHFCSB BOE TPMWF UIF BCPWF QJ = FYQ(α + βYJ)  + FYQ(α + βYJ) MZ DBMMFE UIF ĹļĴĶŀŁĶİ *O UIJT DPOUFYU JU JT BMTP DPNNPOMZ DBMMFE TF JU JOWFSUT UIF MPHJU USBOTGPSN JT UIBU XIFO ZPV VTF B MPHJU MJOL GPS B QBSBNFUFS ZPV BSF EFĕOJOH UIF MPHJTUJD USBOTGPSN PG UIF MJOFBS NPEFM 'ĶĴłĿIJ ƑƎ JMMVTUSBUFT LFT QMBDF XIFO VTJOH B MPHJU MJOL 0O UIF MFę UIF HFPNFUSZ PG UIF -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1 'ĶĴłĿIJ ƑƎ ćF MPHJU MJOL USBOTGPSNT B MJ SJHIU  ćJT USBOTGPSNBUJPO DPNQSFTTFT UIBU B VOJU DIBOHF PO UIF MJOFBS TDBMF MFę QSPCBCJMJUZ TDBMF SJHIU  QSPCBCJMJUZ QJ EFQFOEJOH VQPO IPX GBS GSPN [F łĿIJ ƑƎ XIFO Y =  UIF MJOFBS NPEFM IBT B WBMVF JODSFBTF JO Y SFTVMUT JO BCPVU B  JODSFBTF JO QS QSPEVDF MFTT BOE MFTT PG BO JODSFBTF JO QSPCBCJMJU "OE JG ZPV UIJOL BCPVU JU B HPPE NPEFM PG QSPCB FWFOU JT BMNPTU HVBSBOUFFE UP IBQQFO JUT QSPCBCJ IPX JNQPSUBOU UIF QSFEJDUPS NBZ CF  (&/&3"-*;&% -*/&"3 .0%&-4 -1.0 -0.5 0.0 0.5 1.0 -4 -2 0 2 4 x log-odds -1.0 -0.5 0.0 0.5 1.0 x 0.0 0.5 1.0 probability inverse-link is logistic
22. 0.00 0.25 0.50 0.75 1.00 -4 -2 0 2 4 probability log-odds 0.00 0 0 20 40 60 80 100 odds 'JHVSF  -PHPEET WFSTVT PSEJOBSZ PEE log-odds = 1 log-odds = 3 p = 0.73 p = 0.95
23. Log link • Goal: Map linear model to positive reals   #*( &/5301: "/% 5)& (&/&3"-*;&% -*/&"3 .0%&- -1.0 -0.5 0.0 0.5 1.0 -3 -2 -1 0 1 2 3 x log measurement -1.0 -0.5 0.0 0.5 1.0 x 0 2 4 6 8 10 original measurement 'ĶĴłĿIJ ƑƏ ćF MPH MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B TUSJDUMZ QPT
24. -1.0 -0.5 0.0 0.5 1.0 -3 -2 -1 0 1 2 3 x log measurement -1.0 -0.5 0.0 0.5 1.0 x 0 2 4 6 8 10 original measurement 'ĶĴłĿIJ ƑƏ ćF MPH MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B TUSJDUMZ QPT JUJWF NFBTVSFNFOU SJHIU  ćJT USBOTGPSN SFTVMUT JO BO FYQPOFOUJBM TDBMJOH PG UIF MJOFBS NPEFM XJUI B VOJU DIBOHF PO UIF MJOFBS TDBMF NBQQJOH POUP JODSFBTJOHMZ MBSHFS DIBOHFT PO UIF PVUDPNF TDBMF 8IBU UIF MPH MJOL FČFDUJWFMZ BTTVNFT JT UIBU UIF QBSBNFUFST WBMVF JT UIF FYQPOFOUJBUJPO PG UIF MJOFBS NPEFM 4PMWJOH MPH(σJ) = α + βYJ GPS σJ ZJFMET UIF JOWFSTF MJOL σJ = FYQ(α + βYJ) HF PO UIF PVUDPNF TDBMF 3FDBMM UIBU XF EFĕOFE JOUFSBDUJPO $IBQUFS  BT B TJUVB JDI UIF FČFDU PG B QSFEJDUPS EFQFOET VQPO UIF WBMVF PG BOPUIFS QSFEJDUPS 8FMM QSFEJDUPS FTTFOUJBMMZ JOUFSBDUT XJUI JUTFMG CFDBVTF UIF JNQBDU PG B DIBOHF JO B QSFEJ OET VQPO UIF WBMVF PG UIF QSFEJDUPS CFGPSF UIF DIBOHF .PSF HFOFSBMMZ FWFSZ QSF SJBCMF FČFDUJWFMZ JOUFSBDUT XJUI FWFSZ PUIFS QSFEJDUPS WBSJBCMF XIFUIFS ZPV FYQMJ M UIFN BT JOUFSBDUJPOT $IBQUFS  PS OPU ćJT GBDU NBLFT UIF WJTVBMJ[BUJPO PG DPVO BM QSFEJDUJPOT FWFO NPSF JNQPSUBOU GPS VOEFSTUBOEJOH XIBU UIF NPEFM JT UFMMJOH ZP ćF TFDPOE WFSZ DPNNPO MJOL GVODUJPO JT UIF ĹļĴ ĹĶĻĸ ćJT MJOL GVODUJPO NB NFUFS UIBU JT EFĕOFE PWFS POMZ QPTJUJWF SFBM WBMVFT POUP B MJOFBS NPEFM 'PS FYBN PTF XF XBOU UP NPEFM UIF TUBOEBSE EFWJBUJPO σ PG B (BVTTJBO EJTUSJCVUJPO TP JU JPO PG B QSFEJDUPS WBSJBCMF Y ćF QBSBNFUFS σ NVTU CF QPTJUJWF CFDBVTF B TUBOE UJPO DBOOPU CF OFHBUJWF OPS DBO JU CF [FSP ćF NPEFM NJHIU MPPL MJLF ZJ ∼ /PSNBM(µ, σJ) MPH(σJ) = α + βYJ T NPEFM UIF NFBO µ JT DPOTUBOU CVU UIF TUBOEBSE EFWJBUJPO TDBMFT XJUI UIF WBMV MJOL JT CPUI DPOWFOUJPOBM BOE VTFGVM JO UIJT TJUVBUJPO *U QSFWFOUT σ GSPN UBLJOH JWF WBMVF -1.0 -0.5 0.0 0.5 1.0 -3 -2 -1 x log mea -1.0 -0.5 0.0 0.5 1.0 x 'ĶĴłĿIJ ƑƏ ćF MPH MJOL USBOTGPSNT B MJOFBS NPEFM MFę JOUP B TUSJDUMZ QPT JUJWF NFBTVSFNFOU SJHIU  ćJT USBOTGPSN SFTVMUT JO BO FYQPOFOUJBM TDBMJOH PG UIF MJOFBS NPEFM XJUI B VOJU DIBOHF PO UIF MJOFBS TDBMF NBQQJOH POUP JODSFBTJOHMZ MBSHFS DIBOHFT PO UIF PVUDPNF TDBMF 8IBU UIF MPH MJOL FČFDUJWFMZ BTTVNFT JT UIBU UIF QBSBNFUFST WBMVF JT UIF FYQPO MJOFBS NPEFM 4PMWJOH MPH(σJ) = α + βYJ GPS σJ ZJFMET UIF JOWFSTF MJOL σJ = FYQ(α + βYJ) NQBDU PG UIJT BTTVNQUJPO DBO CF TFFO JO 'ĶĴłĿIJ ƑƏ 6TJOH B MPH MJOL GPS B MJOF NQMJFT BO FYQPOFOUJBM TDBMJOH PG UIF PVUDPNF XJUI UIF QSFEJDUPS WBSJBCMF SJH Solve for sigma:
25. Generalized Linear Models • (3) Compute posterior • Search is harder • Interpretation is harder • Links matter • Quadratic approximation often works, but not always well
26. • There are floor and ceiling effects floor ceiling -3 -2 -1 0 1 2 3 0.00 0.50 1.00 temperature prob survival Everything interacts
27. • Linear regression: • Logistic regression: Everything interacts FST JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPNF GVSUIFS DMBSJUZ PO U WFSZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ NBUIFNBUJDBMMZ DPNQVUJ PVUDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFEJDUPS 'JSTU SFDBMM UIBU UIF NFBO JT NPEFMFE MJLF µ = α + βY µ XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT DPOTUBOU *U EPFTOU NBUU X DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCBCJMJUZ Q XJUI SFTQFDU UP Q = FYQ(α + βY)  + FYQ(α + βY) WBUJWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) OLJOH 1BSBNFUFST JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPNF GVSUIFS D BU (-.T GPSDF FWFSZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ NBUIFNBUJDBM PG DIBOHF JO UIF PVUDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFEJDUPS 'JSTU (BVTTJBO NPEFM UIF NFBO JT NPEFMFE MJLF µ = α + βY UF PG DIBOHF JO µ XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT DPOTUBOU *U E VF Y IBT #VU OPX DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCBCJMJUZ Q XJUI S Y Q = FYQ(α + βY)  + FYQ(α + βY) X UBLJOH UIF EFSJWBUJWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) QQFBST JO UIJT BOTXFS UIF JNQBDU PG B DIBOHF JO Y EFQFOET VQPO Y ćBUT BO JOU BOE UIF POF UIBU BUUBJOT TUBUJTUJDBM TJHOJĕDBODF JT SFQPSUFE *O TFOTJUJWJUZ OBMZTFT BSF USJFE BOE BMM PG UIFN BSF EFTDSJCFE JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPNF GVSUIFS DMBSJUZ PO UIF SZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ NBUIFNBUJDBMMZ DPNQVUJOH UDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFEJDUPS 'JSTU SFDBMM UIBU JO F NFBO JT NPEFMFE MJLF µ = α + βY XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT DPOTUBOU *U EPFTOU NBUUFS DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCBCJMJUZ Q XJUI SFTQFDU UP B Q = FYQ(α + βY)  + FYQ(α + βY) JWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) 0WFSUIJOLJOH 1BSBNFUFST JOUFSBDUJOH XJUI UIFNTFMWFT 8F DBO ĕOE TPN DMBJN UIBU (-.T GPSDF FWFSZ QSFEJDUPS WBSJBCMF UP JOUFSBDU XJUI JUTFMG CZ N UIF SBUF PG DIBOHF JO UIF PVUDPNF GPS B HJWFO DIBOHF JO UIF WBMVF PG UIF QSFE B DMBTTJD (BVTTJBO NPEFM UIF NFBO JT NPEFMFE MJLF µ = α + βY 4P UIF SBUF PG DIBOHF JO µ XJUI SFTQFDU UP Y JT KVTU ∂µ/∂Y = β "OE UIBUT D XIBU WBMVF Y IBT #VU OPX DPOTJEFS UIF SBUF PG DIBOHF JO B CJOPNJBM QSPCB QSFEJDUPS Y Q = FYQ(α + βY)  + FYQ(α + βY) "OE OPX UBLJOH UIF EFSJWBUJWF XJUI SFTQFDU UP Y ZJFMET ∂Q ∂Y = β   + DPTI(α + βY) 4JODF Y BQQFBST JO UIJT BOTXFS UIF JNQBDU PG B DIBOHF JO Y EFQFOET VQPO Y JUTFMG
28. The road ahead • This week: Count models • Week 8: Monsters and mixtures • Week 9: Multilevel models • Week 10: Measurement error, missing data, Gaussian processes
29. sum large mean count events low rate count events low probability many trials dnorm dgamma dpois dbinom dexp Z ∼ /PSNBM(µ, σ) Z ∼ #JOPNJBM(O, Q) Z ∼ 1PJTTPO(λ) Z ∼ (BNNB(λ, L) Z ∼ &YQPOFOUJBM(λ)
30. 0 2 4 6 8 0.0 0.1 0.2 0.3 0.4 0.5 duration Density 0 2 4 6 8 0.0 0.1 0.2 0.3 0.4 0.5 duration Density
31. 0 2 4 6 8 0.0 0.1 0.2 0.3 0.4 0.5 duration Density 0 2 4 6 8 0.0 0.1 0.2 0.3 0.4 0.5 duration Density 0 2 4 6 8 0.0 0.1 0.2 0.3 0.4 0.5 duration Density 0 2 4 6 8 0.0 0.1 0.2 0.3 0.4 0.5 duration Density 0 2 4 6 8 10 0 500 1500 2500 Count Frequency lambda=0.5
32. 0 2 4 6 8 10 0 500 1500 2500 Count Frequency lambda=0.5 0 2 4 6 8 0.0 0.1 0.2 0.3 0.4 0.5 duration Density 0 2 4 6 8 0.05 0.10 0.15 0.20 duration Density 0 2 4 6 8 0.0 0.4 0.8 1.2 Density 0 2 4 6 8 10 0 500 1500 2500 Count Frequency 0 2 4 6 8 10 0 500 1500 2500 Frequency
33. • Counts of a specific event out of n possibilities • Goal is to model probability as a function of predictors • If n = 1, called logistic regression Binomial distribution count “successes” number of trials probability of success Z ∼ #JOPNJBM(O, Q) 0 2 4 6 8 10 0 500 1500 2500 Count Frequency lambda=0.5
34. Binomial distribution • Counts of a specific event out of n possibilities • Goal is to model probability as a function of predictors • If n = 1, called logistic regression Z ∼ #JOPNJBM(O, Q) &(Z) = OQ WBS(Z) = OQ( − Q) Mean and variance not independent
35. 0 2 4 6 8 10 0 500 1500 2500 Count Frequency lambda=0.5 Binomial distribution • Counts of a specific event out of n possibilities • Goal is to model probability as a function of predictors • If n = 1, called logistic regression Z ∼ #JOPNJBM(O, Q)
36. Binomial examples • Logistic regression: • 0/1 outcomes, logit link • Aggregated binomial: • [0,n] outcomes, logit link • Along the way • Cope with relative versus absolute effects • Cope with ceiling/floor effects   #*( &/5301: "/% 5 ww bw wb bb ww bw wb bb ww bw wb bb ww bw wb bb A B C D ȕ *(+0/ 3+ /  1'0 *! # .++'4ǿ + Ǣ !0)/$*)ǿ+Ȁ .0(ǿ+ȉǿǍǢǎ
37. Prosocial chimpanzees partner focal  #*/0.*"- 3&(3&44*0/ 'ĶĴłĿIJ ƉƈƉ $IJNQBO[FF QSPTPD FYQFSJNFOU BT TFFO GSPN UIF QFSTQF PG UIF GPDBM BOJNBM ćF MFę BOE MFWFST BSF JOEJDBUFE JO UIF GPSFHSP 1VMMJOH FJUIFS FYQBOET BO BDDPSEJPO WJDF JO UIF DFOUFS QVTIJOH UIF GPPE UPXBSET CPUI FOET PG UIF UBCMF #PUI USBZT DMPTF UP UIF GPDBM BOJNBM IBWF JO UIFN 0OMZ POF PG UIF GPPE USBZ UIF PUIFS TJEF DPOUBJOT GPPE ćF QBS DPOEJUJPO NFBOT BOPUIFS BOJNBM BT UVSFE TJUT PO UIF PUIFS FOE PG UIF U 0UIFSXJTF UIF PUIFS FOE XBT FNQUZ
38. Prosocial chimpanzees • Two conditions: (1) partner, (2) alone • Two options: (1) prosocial, (2) asocial • Two outcomes: (1) left lever, (2) right lever • Want to predict outcome as function of condition and which side option is on • Do chimps prefer left lever when partner present and prosocial on left? => interaction!  #*/0.*"- 8IFO IVNBO TUVEFOUT QBSUJDJQBUF JO BO FY UIF MFWFS MJOLFE UP UXP QJFDFT PG GPPE UIF QSPT
39. )$/$*) BT QSFEJDUPS WBSJBCMFT ćF PVUDPNF +0'' Ǿ' !/ JT B  PS  JOEJDBUPS UIBU BM BOJNBM QVMMFE UIF MFęIBOE MFWFS ćF QSFEJDUPS +-*.*Ǿ' !/ JT B  JOEJDBUPS MFęIBOE MFWFS XBT  PS XBT OPU  BUUBDIFE UP UIF QSPTPDJBM PQUJPO UIF TJEF XJUI DFT PG GPPE ćF *)$/$*) QSFEJDUPS JT BOPUIFS  JOEJDBUPS XJUI WBMVF  GPS UIF QBSU EJUJPO BOE WBMVF  GPS UIF DPOUSPM DPOEJUJPO /PX XFSF SFBEZ UP ĕU B NPEFM ćF NPEFM JNQMJFE CZ UIF SFTFBSDI RVFTUJPO JT JO NB BUJDBM GPSN ĕSTU -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = α + (β1 + β1$ $J)1J α ∼ /PSNBM(, ) β1 ∼ /PSNBM(, ) β1$ ∼ /PSNBM(, ) SF - JOEJDBUFT +0'' Ǿ' !/ 1 JOEJDBUFT +-*.*Ǿ' !/ BOE $ JOEJDBUFT *)$/$*) LZ QBSU PG UIF NPEFM BCPWF JT UIF MJOFBS NPEFM GPS MPHJU(QJ) *U JT BO JOUFSBDUJPO NPEF JDI UIF BTTPDJBUJPO CFUXFFO 1J BOE UIF MPHPEET UIBU -J =  EFQFOET VQPO UIF WBMV #VU OPUF UIBU UIFSF JT OP NBJO FČFDU PG $J JUTFMG OP QMBJO CFUBDPFďDJFOU GPS DPOEJU Do chimps prefer left lever (L) when partner present (C) and prosocial on left (P)?
40. MPPLT B MPU MJLF UIF NBUIFNBUJDBM TQFDJĕDBUJPO 3 DPEF  (ǎǍǡǎ ʚǶ (+ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ  Ǣ  ʡ )*-(ǿǍǢǎǍȀ Ȁ Ǣ /ʙ Ȁ +- $.ǿ(ǎǍǡǎȀ  ) / 1 Ǐǡǒʉ ǖǔǡǒʉ  ǍǡǐǏ ǍǡǍǖ ǍǡǎǑ Ǎǡǒ /PX UP JOUFSQSFU UIF FTUJNBUF GPS  α XF IBWF UP SFNFNCFS UIBU UIF QBSBNFUFST JO B MPHJTUJD SFHSFTTJPO BSF PO UIF TDBMF PG MPHPEET 5P HFU UIFN CBDL POUP UIF QSPCBCJMJUZ TDBMF XF IBWF UP VTF UIF JOWFSTF MJOL GVODUJPO *O UIJT DBTF UIF JOWFSTF MJOL JT MPHJTUJD QSPWJEFE BT '*"$./$ JO UIF - /#$)&$)" QBDLBHF 4P UIF BCPWF TVNNBSZ JNQMJFT B ."1 QSPCBCJMJUZ PG QVMMJOH UIF MFę MFWFS PG '*"$./$ǿǍǡǐǏȀ≈ . XJUI B  JOUFSWBM PG . UP . 3 DPEF  '*"$./$ǿ ǿǍǡǎǑǢǍǡǒȀ Ȁ ȁǎȂ ǍǡǒǐǑǖǑǏǖ ǍǡǓǏǏǑǒǖǐ 4P XF IBWF UP OPUF CFGPSF DPOTJEFSJOH BOZ QSFEJDUPST UIBU UIF DIJNQBO[FFT FYIJCJUFE B QSFG FSFODF GPS UIF MFęIBOE MFWFS -BUFS XFMM TFF UIBU JOEJWJEVBM DIJNQBO[FFT WBSZ B MPU JO UIFJS IBOEFEOFTT TP UIJT PWFSBMM BWFSBHF JT NJTMFBEJOH  #*/0.*"- 3&(3&44*0/  /PX ĕU UIF OFYU UXP NPEFMT XJUI OP TVSQSJTFT JO UIF DPEF 3 DPEF  (ǎǍǡǏ ʚǶ (+ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ  ʔ +ȉ+-*.*Ǿ' !/ Ǣ  ʡ )*-(ǿǍǢǎǍȀ Ǣ + ʡ )*-(ǿǍǢǎǍȀ Ȁ Ǣ /ʙ Ȁ (ǎǍǡǐ ʚǶ (+ǿ '$./ǿ +0'' Ǿ' !/ ʡ $)*(ǿ ǎ Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ  ʔ ǿ+ ʔ +ȉ*)$/$*)Ȁȉ+-*.*Ǿ' !/ Ǣ  ʡ )*-(ǿǍǢǎǍȀ Ǣ + ʡ )*-(ǿǍǢǎǍȀ Ǣ + ʡ )*-(ǿǍǢǎǍȀ Ȁ Ǣ /ʙ Ȁ "OE UP DPNQBSF UIF UISFF NPEFMT VTF UIF *(+- GVODUJPO JOUSPEVDFE JO $IBQUFS  3 DPEF
41. Model comparison + ʡ )*-(ǿǍǢǎǍȀ Ǣ + ʡ )*-(ǿǍǢǎǍȀ Ȁ Ǣ /ʙ Ȁ "OE UP DPNQBSF UIF UISFF NPEFMT VTF UIF *(+- GVODUJPO JOUSPEVDFE JO $IBQUFS  3 DPEF  *(+- ǿ (ǎǍǡǎ Ǣ (ǎǍǡǏ Ǣ (ǎǍǡǐ Ȁ   +    2 $"#/   (ǎǍǡǏ ǓǕǍǡǓ Ǐ ǍǡǍ ǍǡǔǍ ǖǡǐǍ  (ǎǍǡǐ ǓǕǏǡǑ ǐ ǎǡǕ ǍǡǏǕ ǖǡǐǑ ǍǡǕǎ (ǎǍǡǎ ǓǕǕǡǍ ǎ ǔǡǑ ǍǡǍǏ ǔǡǎǐ ǓǡǎǑ ćF NPEFM UIBU JODMVEFT *)$/$*) EPFTOU EP CFTU CVU EPFT HFU NPSF UIBO  PG UIF 8"*$ XFJHIU :PV DBO BMTP QMPU UIF *(+- SFTVMUT XIJDI MPPL MJLF UIJT m10.1 m10.3 m10.2 675 680 685 690 695 deviance WAIC /PUJDF UIBU FWFO UIPVHI (ǎǍǡǏ JTOU IVHFMZ CFUUFS UIBO (ǎǍǡǐ UIF EJČFSFODF IBT B TNBMM TUBOEBSE FSSPS &WFO EPVCMJOH UIF TUBOEBSE FSSPS UP HFU B  JOUFSWBM UIF PSEFS PG UIF UXP NPEFMT XPVME OPU DIBOHF 4P PO UIF CBTJT PG JOGPSNBUJPO DSJUFSJB FWFO UIPVHI NPEFM (ǎǍǡǐ PG DPVSTF ĕUT UIF TBNQMF CFUUFS UIBO NPEFM (ǎǍǡǏ CFDBVTF JU IBT NPSF QBSBNFUFST JU EPFT OPU ĕU TVďDJFOUMZ CFUUFS UP PWFSDPNF UIF FYQFDUFE PWFSĕUUJOH /ʙ Ȁ "OE UP DPNQBSF UIF UISFF NPEFMT VTF UIF *(+- GVODUJPO JOUSPEVDFE JO $IBQUFS  *(+- ǿ (ǎǍǡǎ Ǣ (ǎǍǡǏ Ǣ (ǎǍǡǐ Ȁ   +    2 $"#/   (ǎǍǡǏ ǓǕǍǡǓ Ǐ ǍǡǍ ǍǡǔǍ ǖǡǐǍ  (ǎǍǡǐ ǓǕǏǡǑ ǐ ǎǡǕ ǍǡǏǕ ǖǡǐǑ ǍǡǕǎ (ǎǍǡǎ ǓǕǕǡǍ ǎ ǔǡǑ ǍǡǍǏ ǔǡǎǐ ǓǡǎǑ ćF NPEFM UIBU JODMVEFT *)$/$*) EPFTOU EP CFTU CVU EPFT HFU NPSF UIBO  PG UIF 8"*$ XFJHIU :PV DBO BMTP QMPU UIF *(+- SFTVMUT XIJDI MPPL MJLF UIJT m10.1 m10.3 m10.2 675 680 685 690 695 deviance WAIC /PUJDF UIBU FWFO UIPVHI (ǎǍǡǏ JTOU IVHFMZ CFUUFS UIBO (ǎǍǡǐ UIF EJČFSFODF IBT B TNBMM TUBOEBSE FSSPS &WFO EPVCMJOH UIF TUBOEBSE FSSPS UP HFU B  JOUFSWBM UIF PSEFS PG UIF UXP NPEFMT XPVME OPU DIBOHF 4P PO UIF CBTJT PG JOGPSNBUJPO DSJUFSJB FWFO UIPVHI NPEFM (ǎǍǡǐ PG DPVSTF ĕUT UIF TBNQMF CFUUFS UIBO NPEFM (ǎǍǡǏ CFDBVTF JU IBT NPSF QBSBNFUFST JU EPFT