Statistical Rethinking Fall 2017 Lecture 14

Week 8, Lecture 14, Statistical Rethinking: A Bayesian Course with Examples in R and Stan. This lecture covers Chapter 12 of the book.

Transcript

1. Week 8: Multilevel Models Richard McElreath Statistical Rethinking
2. Anterograde amnesia • Musicologist and conductor Clive Wearing • Lost parts of prefrontal and hippocampus • Can still play piano • Can’t remember what happened 1 min ago
3. Anterograde amnesia • “Fixed effects” models have anterograde amnesia • Every new cluster (individual, pond, road, classroom) is a new world • No information passed among clusters • Multilevel models remember and pool information • Properties of clusters come from a “population” • Inferred population defines pooling • If previous clusters improve your guess about a new cluster, you want to use pooling
4. Learning, forward and back
5. Depends upon variation
6. have shown that losses loom larger than the equivalent gains, a phenomenon known as loss aversion (19). Thus, changes in the de- fault may result in a change of choice. out default options for individuals’ deci- sions to become organ donors. Actual deci- sions about organ donation may be affected by governmental educational programs, the 4.25 27.5 17.17 12 99.98 98 99.91 99.97 99.5 99.64 85.9 0 10 20 30 40 50 60 70 80 90 100 Denmark Netherlands Effective consent percentage United Kingdom Germany Austria Belgium France Hungary Poland Portugal Sweden Effective consent rates, by country. Explicit consent (opt-in, gold) and presumed consent (opt- out, blue). EMBER 2003 VOL 302 SCIENCE www.sciencemag.org opt-in opt-out organ donation consent percentage
7. Multilevel should be default • Defaults are powerful things • Single-level regression is default • People justify multilevel models • This is backwards • Multilevel estimates usually better • Should have to justify not using multilevel model
8. Goals • Introduce multilevel models • How shrinkage and pooling work • Why they produce better estimates • How to fit with map2stan • Methods of plotting and comparing • Advanced: Continuous categories and Gaussian process regression
9. Multilevel models • Usual use is to model clustering • Classrooms within schools • Students within classrooms • Grades within students • Questions within exams • Repeat measures of units • Imbalance in sampling • “pseudoreplication”
10. • Examples from earlier: • !Kung individuals in families • Species in clades • Nations in continents • Applicants in departments Multilevel models
11. Example: Tadpole predation • Numbers of surviving tadpoles • Different densities/sizes • With and without predators • We’ll focus on variation across tanks β BOE τ BT QBSBNFUFST 5P HFU UIF QPTUFSJPS GPS θ ZPVE KVTU FYQPOFOUJBUF U GPS τ *MM VTF UIJT LJOE PG MJOL JO POF PG UIF NPEFM ĕUT UP DPNF TP ZPVMM HFU U MPPLT MJLF JO DPEF GPSN  &YBNQMF #PMLFST 3FFEGSPHT 8FMM VTF BO FYBNQMF BMTP GSPN #FO # UBMJUZ EBUB PO SFFE GSPH UBEQPMFT WBSJBCMZ FYQPTFE UP BRVBUJD QSFEBUPST BU FYQ EFUFSNJOFE EFOTJUJFT :PV DBO MPBE UIF QBDLBHF BOE EBUB XJUI )&//6ǯ/"1%&+(&+$ǰ !1ǯ/""!#/,$0ǰ ! ʆǦ /""!#/,$0 ćF EBUB GSBNF IBT  SPXT BOE  DPMVNOT 8FSF HPJOH UP CF JOUFSFTUFE JO QSFE UIF OVNCFS PG UBEQPMFT UIBU TVSWJWFE UIF EVSBUJPO PG UIF FYQFSJNFOU PVS PG !" BU UIF TUBSU ćF QSFEJDUPS WBSJBCMFT PG JOUFSFTU XJMM CF !"+0&16 -/"! UIF BCTFODF PG QSFEBUPST BOE 0&7" UIF TJ[F PG UBEQPMFT *OUFSDFQUPOMZ NPEFMT *UMM CF VTFGVM UP CFHJO XJUI TJNQMF NPEFMT UIBU E QSFEJDUPST TP ZPV DBO HFU B TFOTF GPS IPX B CFUBCJOPNJBM NPEFM DPNQBSF UJPO UP B QMBJO CJOPNJBM NPEFM 4P MFUT ĕU CPUI UP UIFTF EBUB BO PMEGBTIJPO SFHSFTTJPO BOE B CFUBCJOPNJBM SFHSFTTJPO 'JSTU UIF SFHVMBS CJOPNJBM NPEFM ćJT DPEF JT KVTU MJLF ZPVE FYQFDU BęFS *ƾƿǑǁ ʆǦ *-ǯ )&01ǯ
12. Tadpole models • Structure: • Tadpoles in tanks, different densities • Outcome: number surviving • Can fit two basic models: 1. Dummy variable for each tank 2. Multilevel model with varying intercepts by tank
13. Regularized intercepts number surviving, tank i   .6-5*-&7&- .0%&-4 )FSF JT B NPEFM GPS QSFEJDUJOH UBEQPMF NPSUBMJUZ JO FBDI UBOL VTJOH QSJPST PG FBSMJFS DIBQUFST TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >XQLTXHORJR αŁĮĻĸ ∼ /PSNBM(, ) >ZHDN "OE ZPV DBO ĕU UIJT UP UIF EBUB JO UIF TUBOEBSE XBZ VTJOH (+ PS (+Ǐ (+Ǐ./) GSPN IFSF POXBSET CFDBVTF UIF OFYU NPEFM XJMM OPU XPSL JO ( EBMMJFE JO JOTUBMMJOH 4UBO NDTUBOPSH UBLF B CSFBL OPX BOE HP EP UIBU (P PMMPX UIF JOTUSVDUJPOT GPS ZPVS QMBUGPSN ćFO FYFDVUF UIJT DPEF '$--4ǿ- /#$)&$)"Ȁ /ǿ- !-*".Ȁ regularizing prior
14. Regularized intercepts )FSF JT B NPEFM GPS QSFEJDUJOH UBEQPMF NPSUBMJUZ JO FBDI UBOL VTJOH QSJPST PG FBSMJFS DIBQUFST TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >XQLTXHORJR αŁĮĻĸ ∼ /PSNBM(, ) >ZHDN "OE ZPV DBO ĕU UIJT UP UIF EBUB JO UIF TUBOEBSE XBZ VTJOH (+ PS (+Ǐ (+Ǐ./) GSPN IFSF POXBSET CFDBVTF UIF OFYU NPEFM XJMM OPU XPSL JO ( EBMMJFE JO JOTUBMMJOH 4UBO NDTUBOPSH UBLF B CSFBL OPX BOE HP EP UIBU (P PMMPX UIF JOTUSVDUJPOT GPS ZPVS QMBUGPSN ćFO FYFDVUF UIJT DPEF '$--4ǿ- /#$)&$)"Ȁ /ǿ- !-*".Ȁ  ʚǶ - !-*". ȕ (& /# /)& '0./ - 1-$' ɶ/)& ʚǶ ǎǣ)-*2ǿȀ )FSF JT B NPEFM GPS QSFEJDUJOH UBEQPMF NPSUBMJUZ JO FBDI UBOL VTJOH UIF SFHVMBSJ[JOH QSJPST PG FBSMJFS DIBQUFST TJ ∼ #JOPNJBM(OJ, QJ) >OLNHOLKRRG@ MPHJU(QJ) = αŁĮĻĸ[J] >XQLTXHORJRGGVIRUHDFKWDQN J@ αŁĮĻĸ ∼ /PSNBM(, ) >ZHDNO\UHJXODUL]LQJSULRU@ "OE ZPV DBO ĕU UIJT UP UIF EBUB JO UIF TUBOEBSE XBZ VTJOH (+ PS (+Ǐ./) 8FMM VTF (+Ǐ./) GSPN IFSF POXBSET CFDBVTF UIF OFYU NPEFM XJMM OPU XPSL JO (+ 4P JG ZPVWF EBMMJFE JO JOTUBMMJOH 4UBO NDTUBOPSH UBLF B CSFBL OPX BOE HP EP UIBU (P UP UIF 63- BOE GPMMPX UIF JOTUSVDUJPOT GPS ZPVS QMBUGPSN ćFO FYFDVUF UIJT DPEF 3 DPEF  '$--4ǿ- /#$)&$)"Ȁ /ǿ- !-*".Ȁ  ʚǶ - !-*". ȕ (& /# /)& '0./ - 1-$' ɶ/)& ʚǶ ǎǣ)-*2ǿȀ ȕ !$/ (ǎǏǡǎ ʚǶ (+Ǐ./)ǿ '$./ǿ .0-1 ʡ $)*(ǿ  ).$/4 Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ Ǿ/)&ȁ/)&Ȃ Ǣ Ǿ/)&ȁ/)&Ȃ ʡ )*-(ǿ Ǎ Ǣ ǒ Ȁ ȀǢ /ʙ Ȁ
15. Adaptive regularization *G ZPV JOTQFDU UIF FTUJNBUFT +- $.ǿ(ǎǏǡǎǢ +/#ʙǏȀ ZPVMM TFF  EJČFS TFUT POF GPS FBDI UBOL 5P HFU FBDI UBOLT FYQFDUFE TVSWJWBM QSPCBCJMJUZ KVT Ǿ/)& WBMVFT BOE UIFO VTF UIF MPHJTUJD USBOTGPSN 4P GBS OPUIJOH OFX /PX MFUT ĕU UIF NVMUJMFWFM NPEFM XIJDI BEBQUJWFMZ QPPMT JOGPSNBUJPO UIBU JT SFRVJSFE UP FOBCMF BEBQUJWF QPPMJOH JT UP NBLF UIF QSJPS GPS UIF Ǿ/ GVODUJPO PG JUT PXO QBSBNFUFST )FSF JT UIF NVMUJMFWFM NPEFM JO NBUIFNBU TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >ORJRG αŁĮĻĸ ∼ /PSNBM(α, σ) >YDU\ α ∼ /PSNBM(, ) >SU σ ∼ )BMG$BVDIZ(, ) >SULRUIRUVWDQGDU /PUJDF UIBU UIF QSJPS GPS UIF αŁĮĻĸ JOUFSDFQUT JT OPX B GVODUJPO PG UXP QBSB ćJT JT XIFSF UIF iNVMUJw JO NVMUJMFWFM BSJTFT ćF (BVTTJBO EJTUSJCVUJPO X TUBOEBSE EFWJBUJPO σ JT UIF QSJPS GPS FBDI UBOLT JOUFSDFQU #VU UIBU QSJPS JUT α BOE σ 4P UIFSF BSF UXP MFWFMT JO UIF NPEFM FBDI SFTFNCMJOH B TJNQMFS N MFWFM UIF PVUDPNF JT T UIF QBSBNFUFST BSF αŁĮĻĸ BOE UIF QSJPS JT αŁĮĻĸ ∼ *O UIF TFDPOE MFWFM UIF iPVUDPNFw WBSJBCMF JT UIF WFDUPS PG JOUFSDFQU QBSBN
16. Adaptive regularization *G ZPV JOTQFDU UIF FTUJNBUFT +- $.ǿ(ǎǏǡǎǢ +/#ʙǏȀ ZPVMM TFF  EJČFS TFUT POF GPS FBDI UBOL 5P HFU FBDI UBOLT FYQFDUFE TVSWJWBM QSPCBCJMJUZ KVT Ǿ/)& WBMVFT BOE UIFO VTF UIF MPHJTUJD USBOTGPSN 4P GBS OPUIJOH OFX /PX MFUT ĕU UIF NVMUJMFWFM NPEFM XIJDI BEBQUJWFMZ QPPMT JOGPSNBUJPO UIBU JT SFRVJSFE UP FOBCMF BEBQUJWF QPPMJOH JT UP NBLF UIF QSJPS GPS UIF Ǿ/ GVODUJPO PG JUT PXO QBSBNFUFST )FSF JT UIF NVMUJMFWFM NPEFM JO NBUIFNBU TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >ORJRG αŁĮĻĸ ∼ /PSNBM(α, σ) >YDU\ α ∼ /PSNBM(, ) >SU σ ∼ )BMG$BVDIZ(, ) >SULRUIRUVWDQGDU /PUJDF UIBU UIF QSJPS GPS UIF αŁĮĻĸ JOUFSDFQUT JT OPX B GVODUJPO PG UXP QBSB ćJT JT XIFSF UIF iNVMUJw JO NVMUJMFWFM BSJTFT ćF (BVTTJBO EJTUSJCVUJPO X TUBOEBSE EFWJBUJPO σ JT UIF QSJPS GPS FBDI UBOLT JOUFSDFQU #VU UIBU QSJPS JUT α BOE σ 4P UIFSF BSF UXP MFWFMT JO UIF NPEFM FBDI SFTFNCMJOH B TJNQMFS N MFWFM UIF PVUDPNF JT T UIF QBSBNFUFST BSF αŁĮĻĸ BOE UIF QSJPS JT αŁĮĻĸ ∼ *O UIF TFDPOE MFWFM UIF iPVUDPNFw WBSJBCMF JT UIF WFDUPS PG JOUFSDFQU QBSBN varying intercepts
17. Terminology • Varying intercepts also called random intercepts • Neither of these terms makes much sense • “random”? Sometimes associated with research design, but design irrelevant • Ordinary dummy variables also “vary” across clusters • Distinctive because individual intercepts learn from one another • mnestic: opposite of amnestic
18. Adaptive regularization *G ZPV JOTQFDU UIF FTUJNBUFT +- $.ǿ(ǎǏǡǎǢ +/#ʙǏȀ ZPVMM TFF  EJČFS TFUT POF GPS FBDI UBOL 5P HFU FBDI UBOLT FYQFDUFE TVSWJWBM QSPCBCJMJUZ KVT Ǿ/)& WBMVFT BOE UIFO VTF UIF MPHJTUJD USBOTGPSN 4P GBS OPUIJOH OFX /PX MFUT ĕU UIF NVMUJMFWFM NPEFM XIJDI BEBQUJWFMZ QPPMT JOGPSNBUJPO UIBU JT SFRVJSFE UP FOBCMF BEBQUJWF QPPMJOH JT UP NBLF UIF QSJPS GPS UIF Ǿ/ GVODUJPO PG JUT PXO QBSBNFUFST )FSF JT UIF NVMUJMFWFM NPEFM JO NBUIFNBU TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >ORJRG αŁĮĻĸ ∼ /PSNBM(α, σ) >YDU\ α ∼ /PSNBM(, ) >SU σ ∼ )BMG$BVDIZ(, ) >SULRUIRUVWDQGDU /PUJDF UIBU UIF QSJPS GPS UIF αŁĮĻĸ JOUFSDFQUT JT OPX B GVODUJPO PG UXP QBSB ćJT JT XIFSF UIF iNVMUJw JO NVMUJMFWFM BSJTFT ćF (BVTTJBO EJTUSJCVUJPO X TUBOEBSE EFWJBUJPO σ JT UIF QSJPS GPS FBDI UBOLT JOUFSDFQU #VU UIBU QSJPS JUT α BOE σ 4P UIFSF BSF UXP MFWFMT JO UIF NPEFM FBDI SFTFNCMJOH B TJNQMFS N MFWFM UIF PVUDPNF JT T UIF QBSBNFUFST BSF αŁĮĻĸ BOE UIF QSJPS JT αŁĮĻĸ ∼ *O UIF TFDPOE MFWFM UIF iPVUDPNFw WBSJBCMF JT UIF WFDUPS PG JOUFSDFQU QBSBN mean standard deviation varying intercepts
19. Adaptive regularization *G ZPV JOTQFDU UIF FTUJNBUFT +- $.ǿ(ǎǏǡǎǢ +/#ʙǏȀ ZPVMM TFF  EJČFS TFUT POF GPS FBDI UBOL 5P HFU FBDI UBOLT FYQFDUFE TVSWJWBM QSPCBCJMJUZ KVT Ǿ/)& WBMVFT BOE UIFO VTF UIF MPHJTUJD USBOTGPSN 4P GBS OPUIJOH OFX /PX MFUT ĕU UIF NVMUJMFWFM NPEFM XIJDI BEBQUJWFMZ QPPMT JOGPSNBUJPO UIBU JT SFRVJSFE UP FOBCMF BEBQUJWF QPPMJOH JT UP NBLF UIF QSJPS GPS UIF Ǿ/ GVODUJPO PG JUT PXO QBSBNFUFST )FSF JT UIF NVMUJMFWFM NPEFM JO NBUIFNBU TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >ORJRG αŁĮĻĸ ∼ /PSNBM(α, σ) >YDU\ α ∼ /PSNBM(, ) >SU σ ∼ )BMG$BVDIZ(, ) >SULRUIRUVWDQGDU /PUJDF UIBU UIF QSJPS GPS UIF αŁĮĻĸ JOUFSDFQUT JT OPX B GVODUJPO PG UXP QBSB ćJT JT XIFSF UIF iNVMUJw JO NVMUJMFWFM BSJTFT ćF (BVTTJBO EJTUSJCVUJPO X TUBOEBSE EFWJBUJPO σ JT UIF QSJPS GPS FBDI UBOLT JOUFSDFQU #VU UIBU QSJPS JUT α BOE σ 4P UIFSF BSF UXP MFWFMT JO UIF NPEFM FBDI SFTFNCMJOH B TJNQMFS N MFWFM UIF PVUDPNF JT T UIF QBSBNFUFST BSF αŁĮĻĸ BOE UIF QSJPS JT αŁĮĻĸ ∼ *O UIF TFDPOE MFWFM UIF iPVUDPNFw WBSJBCMF JT UIF WFDUPS PG JOUFSDFQU QBSBN varying intercepts
20. Adaptive regularization *G ZPV JOTQFDU UIF FTUJNBUFT +- $.ǿ(ǎǏǡǎǢ +/#ʙǏȀ ZPVMM TFF  EJČFS TFUT POF GPS FBDI UBOL 5P HFU FBDI UBOLT FYQFDUFE TVSWJWBM QSPCBCJMJUZ KVT Ǿ/)& WBMVFT BOE UIFO VTF UIF MPHJTUJD USBOTGPSN 4P GBS OPUIJOH OFX /PX MFUT ĕU UIF NVMUJMFWFM NPEFM XIJDI BEBQUJWFMZ QPPMT JOGPSNBUJPO UIBU JT SFRVJSFE UP FOBCMF BEBQUJWF QPPMJOH JT UP NBLF UIF QSJPS GPS UIF Ǿ/ GVODUJPO PG JUT PXO QBSBNFUFST )FSF JT UIF NVMUJMFWFM NPEFM JO NBUIFNBU TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >ORJRG αŁĮĻĸ ∼ /PSNBM(α, σ) >YDU\ α ∼ /PSNBM(, ) >SU σ ∼ )BMG$BVDIZ(, ) >SULRUIRUVWDQGDU /PUJDF UIBU UIF QSJPS GPS UIF αŁĮĻĸ JOUFSDFQUT JT OPX B GVODUJPO PG UXP QBSB ćJT JT XIFSF UIF iNVMUJw JO NVMUJMFWFM BSJTFT ćF (BVTTJBO EJTUSJCVUJPO X TUBOEBSE EFWJBUJPO σ JT UIF QSJPS GPS FBDI UBOLT JOUFSDFQU #VU UIBU QSJPS JUT α BOE σ 4P UIFSF BSF UXP MFWFMT JO UIF NPEFM FBDI SFTFNCMJOH B TJNQMFS N MFWFM UIF PVUDPNF JT T UIF QBSBNFUFST BSF αŁĮĻĸ BOE UIF QSJPS JT αŁĮĻĸ ∼ *O UIF TFDPOE MFWFM UIF iPVUDPNFw WBSJBCMF JT UIF WFDUPS PG JOUFSDFQU QBSBN varying intercepts Survival across tanks has a distribution. This distribution is the prior for each tank. Distribution needs its own prior.
21. GVODUJPO PG JUT PXO QBSBNFUFST )FSF JT UIF NVMUJMFWFM NPEFM JO NBUIFNBU TJ ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αŁĮĻĸ[J] >ORJRG αŁĮĻĸ ∼ /PSNBM(α, σ) >YDU\ α ∼ /PSNBM(, ) >SU σ ∼ )BMG$BVDIZ(, ) >SULRUIRUVWDQGDU /PUJDF UIBU UIF QSJPS GPS UIF αŁĮĻĸ JOUFSDFQUT JT OPX B GVODUJPO PG UXP QBSB ćJT JT XIFSF UIF iNVMUJw JO NVMUJMFWFM BSJTFT ćF (BVTTJBO EJTUSJCVUJPO X TUBOEBSE EFWJBUJPO σ JT UIF QSJPS GPS FBDI UBOLT JOUFSDFQU #VU UIBU QSJPS JUT α BOE σ 4P UIFSF BSF UXP MFWFMT JO UIF NPEFM FBDI SFTFNCMJOH B TJNQMFS N MFWFM UIF PVUDPNF JT T UIF QBSBNFUFST BSF αŁĮĻĸ BOE UIF QSJPS JT αŁĮĻĸ ∼ *O UIF TFDPOE MFWFM UIF iPVUDPNFw WBSJBCMF JT UIF WFDUPS PG JOUFSDFQU QBSBN QBSBNFUFST BSF α BOE σ BOE UIFJS QSJPST BSF α ∼ /PSNBM(, ) BOE σ ∼ ) 4P JG ZPV IBWF B HPPE SFBTPO UP VTF BOPUIFS EJTUSJCVUJPO UIFO EP TP ćF QSBDUJDF QSPCMFNT BU UIF FOE PG UIF DIBQUFS QSPWJEF BO FYBNQMF 'JUUJOH UIF NPEFM UP EBUB FTUJNBUFT CPUI MFWFMT TJNVMUBOFPVTMZ JO UIF TBNF XBZ UIBU PVS SPCPU BU UIF TUBSU PG UIF DIBQUFS MFBSOFE CPUI BCPVU FBDI DBGÏ BOE UIF WBSJBUJPO BNPOH DBGÏT #VU ZPV DBOOPU ĕU UIJT NPEFM XJUI (+ 8IZ #FDBVTF UIF MJLFMJIPPE NVTU OPX BWFSBHF PWFS UIF MFWFM  QBSBNFUFST α BOE σ BOE (+ KVTU IJMM DMJNCT VTJOH TUBUJD WBMVFT GPS BMM PG UIF QBSBNFUFST *U DBOU TFF UIF MFWFMT 'PS NPSF FYQMBOBUJPO TFF UIF 0WFSUIJOLJOH CPY GVSUIFS EPXO :PV DBO IPXFWFS ĕU UIJT NPEFM XJUI (+Ǐ./) 3 DPEF  (ǎǏǡǏ ʚǶ (+Ǐ./)ǿ '$./ǿ .0-1 ʡ $)*(ǿ  ).$/4 Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ Ǿ/)&ȁ/)&Ȃ Ǣ Ǿ/)&ȁ/)&Ȃ ʡ )*-(ǿ  Ǣ .$"( Ȁ Ǣ  ʡ )*-(ǿǍǢǎȀ Ǣ .$"( ʡ 0#4ǿǍǢǎȀ ȀǢ /ʙ Ǣ $/ -ʙǑǍǍǍ Ǣ #$).ʙǑ Ȁ ćJT NPEFM ĕU QSPWJEFT FTUJNBUFT GPS  QBSBNFUFST POF PWFSBMM TBNQMF JOUFSDFQU α UIF WBSJ BODF BNPOH UBOLT σ BOE UIFO  QFSUBOL JOUFSDFQUT -FUT DIFDL 8"*$ UIPVHI UP TFF UIF FČFDUJWF OVNCFS PG QBSBNFUFST 3 DPEF   ǿ(ǎǏǡǏȀ
22.   .6-5*-&7&- .0%&-4 ćJT NPEFM ĕU QSPWJEFT FTUJNBUFT GPS  QBSBNFUFST POF PWFSBMM TBNQMF J WBSJBODF BNPOH UBOLT σ BOE UIFO  QFSUBOL JOUFSDFQUT -FUT DIFDL 8"*$ UIF FČFDUJWF OVNCFS PG QBSBNFUFST 8FMM DPNQBSF UIF FBSMJFS NPEFM *ƾƿǑ NVMUJMFWFM NPEFM 3 DPEF  ,*-/"ǯ *ƾƿǑƾ ǒ *ƾƿǑƿ ǰ   -  !  4"&$%1  ! *ƾƿǑƿ ƾƽƾƽǑƿ ǀDžǑƽ ƽǑƽ ƾ ǀDŽǑdžǁ  *ƾƿǑƾ ƾƽƿǀǑǀ ǁdžǑǁ ƾǀǑƾ ƽ ǁǀǑƽƾ ǃǑǂǁ ćFSF BSF UXP GBDUT UP OPUF IFSF 'JSTU UIF NVMUJMFWFM NPEFM IBT POMZ  FČFD ćFSF BSF  GFXFS FČFDUJWF QBSBNFUFST UIBO BDUVBM QBSBNFUFST CFDBVTF UIF UP FBDI JOUFSDFQU TISJOLT UIFN BMM UPXBSET UIF NFBO α *O UIJT DBTF UIF QSJ TUSPOH $IFDL UIF NFBO PG 0&$* XJUI -/" &0 PS ,"# BOE ZPVMM TFF JUT BSP B ĿIJĴłĹĮĿĶŇĶĻĴ ĽĿĶļĿ MJLF ZPVWF VTFE JO QSFWJPVT DIBQUFST CVU OPX UIF :PV DBO IPXFWFS ĕU UIJT NPEFM XJUI (+Ǐ./) 3 DPEF  (ǎǏǡǏ ʚǶ (+Ǐ./)ǿ '$./ǿ .0-1 ʡ $)*(ǿ  ).$/4 Ǣ + Ȁ Ǣ '*"$/ǿ+Ȁ ʚǶ Ǿ/)&ȁ/)&Ȃ Ǣ Ǿ/)&ȁ/)&Ȃ ʡ )*-(ǿ  Ǣ .$"( Ȁ Ǣ  ʡ )*-(ǿǍǢǎȀ Ǣ .$"( ʡ 0#4ǿǍǢǎȀ ȀǢ /ʙ Ǣ $/ -ʙǑǍǍǍ Ǣ #$).ʙǑ Ȁ ćJT NPEFM ĕU QSPWJEFT FTUJNBUFT GPS  QBSBNFUFST POF PWFSBMM TBNQMF JOUFSDFQU α UIF WBSJ BODF BNPOH UBOLT σ BOE UIFO  QFSUBOL JOUFSDFQUT -FUT DIFDL 8"*$ UIPVHI UP TFF UIF FČFDUJWF OVNCFS PG QBSBNFUFST 3 DPEF   ǿ(ǎǏǡǏȀ ȁǎȂ ǎǍǎǍǡǏǐǐ //-ǿǢǫ'++ǫȀ ȁǎȂ ǶǑǓǔǡǍǓǐǔ //-ǿǢǫ+ ǫȀ ȁǎȂ ǐǕǡǍǒǏǔ //-ǿǢǫ. ǫȀ ȁǎȂ ǐǕǡǍǐǍǎǎ *U IBT MFTT UIBO  FČFDUJWF QBSBNFUFST ćFSF BSF  GFXFS FČFDUJWF QBSBNFUFST UIBO BDUVBM 48 tanks + a + sigma => 50 parameters
23. 0.0 0.2 0.4 0.6 0.8 1.0 tank proportion survival 1 16 32 48 small tanks medium tanks large tanks 'ĶĴłĿIJ ƉƊƉ &NQJSJDBM QSPQPSUJPOT PG TVSWJWPST JO FBDI UBEQPMF UBOL TIPXO CZ UIF ĕMMFE CMVF QPJOUT QMPUUFE XJUI UIF  QFSUBOL FTUJNBUFT GSPN UIF NVMUJMFWFM NPEFM TIPXO CZ UIF CMBDL DJSDMFT ćF EBTIFE MJOF MPDBUFT UIF PWFSBMM BWFSBHF QSPQPSUJPO PG TVSWJWPST BDSPTT BMM UBOLT ćF WFSUJDBM Don’t expect predictions to match observations exactly. Instead expect shrinkage. Fixed estimate Multilevel estimate
24. 0.0 0.2 0.4 0.6 0.8 1.0 tank proportion survival 1 16 32 48 small tanks medium tanks large tanks 'ĶĴłĿIJ ƉƊƉ &NQJSJDBM QSPQPSUJPOT PG TVSWJWPST JO FBDI UBEQPMF UBOL TIPXO CZ UIF ĕMMFE CMVF QPJOUT QMPUUFE XJUI UIF  QFSUBOL FTUJNBUFT GSPN UIF NVMUJMFWFM NPEFM TIPXO CZ UIF CMBDL DJSDMFT ćF EBTIFE MJOF MPDBUFT UIF PWFSBMM BWFSBHF QSPQPSUJPO PG TVSWJWPST BDSPTT BMM UBOLT ćF WFSUJDBM Population mean not equal to raw empirical mean. Why? Imbalance in amount of evidence across tanks. Fixed estimate Multilevel estimate raw mean pop mean
25. 0.0 0.2 0.4 0.6 0.8 1.0 tank proportion survival 1 16 32 48 small tanks medium tanks large tanks 'ĶĴłĿIJ ƉƊƉ &NQJSJDBM QSPQPSUJPOT PG TVSWJWPST JO FBDI UBEQPMF UBOL TIPXO CZ UIF ĕMMFE CMVF QPJOUT QMPUUFE XJUI UIF  QFSUBOL FTUJNBUFT GSPN UIF NVMUJMFWFM NPEFM TIPXO CZ UIF CMBDL DJSDMFT ćF EBTIFE MJOF MPDBUFT UIF PWFSBMM BWFSBHF QSPQPSUJPO PG TVSWJWPST BDSPTT BMM UBOLT ćF WFSUJDBM Small tanks => high sampling variation. More shrinkage towards mean. Further from mean => more shrinkage. Fixed estimate Multilevel estimate
26. 0.0 0.2 0.4 0.6 0.8 1.0 tank proportion survival 1 16 32 48 small tanks medium tanks large tanks 'ĶĴłĿIJ ƉƊƉ &NQJSJDBM QSPQPSUJPOT PG TVSWJWPST JO FBDI UBEQPMF UBOL TIPXO CZ UIF ĕMMFE CMVF QPJOUT QMPUUFE XJUI UIF  QFSUBOL FTUJNBUFT GSPN UIF NVMUJMFWFM NPEFM TIPXO CZ UIF CMBDL DJSDMFT ćF EBTIFE MJOF MPDBUFT UIF PWFSBMM BWFSBHF QSPQPSUJPO PG TVSWJWPST BDSPTT BMM UBOLT ćF WFSUJDBM Large tanks => low sampling variation. Less shrinkage towards mean at all distances from mean. Fixed estimate Multilevel estimate
27. Careful comparing estimates • Typical for intercept (“fixed effect”) to change and become more uncertain • Meaning of parameter changes: no longer mean of data, but rather mean of distribution of intercepts • Uncertainty larger, because many combinations of alpha, sigma, a[tank]’s can produce same empirical mean of data 0.5 1.0 1.5 2.0 2.5 0 1 2 3 4 5 6 estimate Density alpha in fixed model alpha in vary intercept model
28. Shrinkage • Varying effect estimates shrink towards mean (alpha) • Further from mean, more shrinkage • Fewer data in cluster, more shrinkage • Same as regression to the mean, really 0.2 0.4 0.6 0.8 1.0 tank probability of survival in tank 1 16 32 10 25 25
29. Pooling • Shrinkage arises from pooling • Each tank informs estimates of other tanks • The model doesn’t have amnesia! • Effect of pooling influenced by • amount of data in cluster • amount of variation among clusters (sigma) Pool, or the bad guys win
30. Ulysses’ Compass again • Why are varying effects (partial pooling) more accurate than fixed effects (no pooling)? • Grand mean: maximum underfitting • Fixed effects: maximum overfitting • Varying effects: adaptive regularization