Statistical Rethinking Fall 2017 Lecture 17

Week 9, Lecture 17, Statistical Rethinking: A Bayesian Course with Examples in R and Stan. This lecture covers Chapter 13 of the book.

Transcript

1. Week 9: Multilevel Models II Adventures in Covariance Richard McElreath Statistical Rethinking
2. Varying slopes by dept QBSUNFOU EJČFSFOUMZ USFBUT NBMFT BOE GFNBMFT XJMM TISJOL UPXBSET UIF QPQVMBU DPOUSBTU EFQBSUNFOU ' SFDFJWFE IVOESFET PG BQQMJDBUJPOT GSPN CPUI NBMFT B QPPMJOH XJMM EP WFSZ MJUUMF UP UIF FTUJNBUFT GPS UIBU EFQBSUNFOU ćJT JT XIBU UIF WBSZJOH TMPQFT NPEFM MPPLT MJLF XJUI UIF WBSZJOH FČFDUT CMVF "J ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αıIJĽŁ[J] + βıIJĽŁ[J] NJ αıIJĽŁ βıIJĽŁ ∼ .7/PSNBM α β , 4 >MRLQWSULRUI 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) (σα, σβ) ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() >SULRUIRU ćF TZNCPM NJ JOEJDBUFT UIF WBMVF PG *)" GPS UIF JUI SPX *U JT NVMUJQMJFE C βıIJĽŁ[J] XIJDI JT B UPUBM TMPQF EFĕOFE CZ CPUI B WBMVF DPNNPO UP BMM EFQBSUN
3. Varying slopes by dept 3 ∼ -,+DPSS() >SULRUIRUFRUUHODWLRQPDWUL[@ ćF TZNCPM NJ JOEJDBUFT UIF WBMVF PG *)" GPS UIF JUI SPX *U JT NVMUJQMJFE CZ UIF TVN β + βıIJĽŁ[J] XIJDI JT B UPUBM TMPQF EFĕOFE CZ CPUI B WBMVF DPNNPO UP BMM EFQBSUNFOUT β BOE B WBMVF VOJRVF UP UIF EFQBSUNFOU GPS SPX J βıIJĽŁ[J]  5P ĕU UIJT NPEFM 3 DPEF  *ƾǀǑǀ ʆǦ *-ƿ01+ǯ )&01ǯ !*&1 ʍ !&+,*ǯ --)& 1&,+0 ǒ - ǰǒ ),$&1ǯ-ǰ ʆǦ Ǯ!"-1DZ!"-1Ǯ&!Dz ʀ *Ǯ!"-1DZ!"-1Ǯ&!Dzǹ*)"ǒ ǯǮ!"-1ǒ*Ǯ!"-1ǰDZ!"-1Ǯ&!Dz ʍ !*3+,/*ƿǯ ǯǒ*ǰ ǒ 0&$*Ǯ!"-1 ǒ %, ǰǒ  ʍ !+,/*ǯƽǒƾƽǰǒ * ʍ !+,/*ǯƽǒƾǰǒ 0&$*Ǯ!"-1 ʍ ! 2 %6ǯƽǒƿǰǒ %, ʍ !)(' ,//ǯƿǰ ǰ ǒ !1ʅ! ǒ 4/*2-ʅƾƽƽƽ ǒ &1"/ʅǂƽƽƽ ǒ %&+0ʅǁ ǒ ,/"0ʅǀ ǰ $IFDL GPS ZPVSTFMG UIBU UIF DIBJOT NJYFE BOE DPOWFSHFE FYDFMMFOUMZ :PV NJHIU HFU B XBSOJOH BCPVU B GFX iEJWFSHFOU JUFSBUJPOTw 8FMM GPDVT PO UIPTF JO UIF OFYU TFDUJPO 8FSF JOUFSFTUFE JO XIBU BEEJOH WBSZJOH TMPQFT IBT SFWFBMFE 4P MFUT MPPL BU UIF NBSHJOBM QPTUFSJPS EJTUSJCVUJPOT GPS UIF WBSZJOH FČFDUT POMZ
4. Correlated effects   "%7&/563&4 */ $07"3*"/$& -1.0 -0.5 0.0 0.5 1.0 0.0 0.2 0.4 0.6 0.8 1.0 correlation Density -2 -1 0 1 -1.0 -0.6 -0.2 0.2 intercept (a_dept) slope (bm_dept) A B C D E F 'ĶĴłĿIJ ƉƋƎ -Fę 1PTUFSJPS EJTUSJCVUJPO PG UIF DPSSFMBUJPO CFUXFFO JO UFSDFQUT BOE TMPQFT GPS UIF 6$# BENJTTJPOT NPEFM *ƾǀǑǀ 3JHIU 5XP EJNFOTJPOBM TISJOLBHF PG VOQPPMFE CMVF BOE BEBQUJWFMZ QPPMFE PQFO FT
5. Average effects can be a decoy • Average slope not necessarily of interest • Average can be zero, even when predictor very important for prediction  &9".1-& "%.*44*0/ %&$*4*0/4 "/% (&/%&3  3 DPE  -),1ǯ -/" &0ǯ*ƾǀǑǀǒ-/0ʅ ǯǛǮ!"-1ǛǒǛ*Ǯ!"-1Ǜǰǒ!"-1%ʅƿǰ ǰ a_dept[6] a_dept[5] a_dept[4] a_dept[3] a_dept[2] a_dept[1] bm_dept[6] bm_dept[5] bm_dept[4] bm_dept[3] bm_dept[2] bm_dept[1] -3 -2 -1 0 1 Value
6. Cross-classified varying slopes • More slopes: Higher dimension covariance matrix • More clusters: More than one multivariate prior • Reconsider chimpanzees data QSJPS 4P UIJT NFBOT POF NPSF TUBOEBSE EFWJBUJPO QBSBNFUFS BOE POF NPSF DPSSFMBUJPO NBUSJY 'PS FYBNQMF TVQQPTF UIF 6$# BENJTTJPOT EBUB BMTP SFDPSEFE UIF UF BQQMJDBOU ćFO XF DPVME BMTP JODMVEF UFTU TDPSF BT B QSFEJDUPS #VU XF E QSFEJDUPS GPS UIFTF EBUB 4P JOTUFBE XFMM UVSO UP BOPUIFS EBUB TFU JO UIF O EFFQFS JOUP WBSZJOH TMPQFT  &YBNQMF $SPTTDMBTTJĕFE DIJNQBO[FFT XJUI WBSZJOH ćF GVMM NPEFM JODMVEFT CPUI DMVTUFS UZQFT WBSZJOH JOUFSDFQUT WBSZJOH T BOE WBSZJOH TMPQFT PO UIF JOUFSBDUJPO CFUXFFO +-*.*Ǿ' !/ BOE *)$/$ -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = AJ + (B1,J + B1$,J $J)1J >OLQ AJ = α + αĮİŁļĿ[J] + αįĹļİĸ[J] B1,J = β1 + β1,ĮİŁļĿ[J] + β1,įĹļİĸ[J] B1$,J = β1 + β1$,ĮİŁļĿ[J] + β1$,įĹļİĸ[J] >1 × 4JODF UIFSF BSF UXP DMVTUFS UZQFT BDUPST BOE CMPDLT UIFSF BSF UXP NVMUJWBS PST ćF NVMUJWBSJBUF (BVTTJBO QSJPST BSF CPUI EJNFOTJPOBM JO UIJT FYBN
7. QSFEJDUPS GPS UIFTF EBUB 4P JOTUFBE XFMM UVSO UP BOPUIFS EBUB TFU JO UIF O EFFQFS JOUP WBSZJOH TMPQFT  &YBNQMF $SPTTDMBTTJĕFE DIJNQBO[FFT XJUI WBSZJOH ćF GVMM NPEFM JODMVEFT CPUI DMVTUFS UZQFT WBSZJOH JOUFSDFQUT WBSZJOH T BOE WBSZJOH TMPQFT PO UIF JOUFSBDUJPO CFUXFFO +-*.*Ǿ' !/ BOE *)$/$ -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = AJ + (B1,J + B1$,J $J)1J >OLQ AJ = α + αĮİŁļĿ[J] + αįĹļİĸ[J] B1,J = β1 + β1,ĮİŁļĿ[J] + β1,įĹļİĸ[J] B1$,J = β1 + β1$,ĮİŁļĿ[J] + β1$,įĹļİĸ[J] >1 × 4JODF UIFSF BSF UXP DMVTUFS UZQFT BDUPST BOE CMPDLT UIFSF BSF UXP NVMUJWBS PST ćF NVMUJWBSJBUF (BVTTJBO QSJPST BSF CPUI EJNFOTJPOBM JO UIJT FYBN FSBM ZPV DBO DIPPTF UP IBWF EJČFSFOU WBSZJOH FČFDUT JO EJČFSFOU DMVTUFS UZQ ⎡ ⎣ αĮİŁļĿ β1,ĮİŁļĿ β1$,ĮİŁļĿ ⎤ ⎦ ∼ .7/PSNBM ⎛ ⎝ ⎡ ⎣    ⎤ ⎦ , 4ĮİŁļĿ ⎞ ⎠
8. QSFEJDUPS GPS UIFTF EBUB 4P JOTUFBE XFMM UVSO UP BOPUIFS EBUB TFU JO UIF O EFFQFS JOUP WBSZJOH TMPQFT  &YBNQMF $SPTTDMBTTJĕFE DIJNQBO[FFT XJUI WBSZJOH ćF GVMM NPEFM JODMVEFT CPUI DMVTUFS UZQFT WBSZJOH JOUFSDFQUT WBSZJOH T BOE WBSZJOH TMPQFT PO UIF JOUFSBDUJPO CFUXFFO +-*.*Ǿ' !/ BOE *)$/$ -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = AJ + (B1,J + B1$,J $J)1J >OLQ AJ = α + αĮİŁļĿ[J] + αįĹļİĸ[J] B1,J = β1 + β1,ĮİŁļĿ[J] + β1,įĹļİĸ[J] B1$,J = β1 + β1$,ĮİŁļĿ[J] + β1$,įĹļİĸ[J] >1 × 4JODF UIFSF BSF UXP DMVTUFS UZQFT BDUPST BOE CMPDLT UIFSF BSF UXP NVMUJWBS PST ćF NVMUJWBSJBUF (BVTTJBO QSJPST BSF CPUI EJNFOTJPOBM JO UIJT FYBN FSBM ZPV DBO DIPPTF UP IBWF EJČFSFOU WBSZJOH FČFDUT JO EJČFSFOU DMVTUFS UZQ ⎡ ⎣ αĮİŁļĿ β1,ĮİŁļĿ β1$,ĮİŁļĿ ⎤ ⎦ ∼ .7/PSNBM ⎛ ⎝ ⎡ ⎣    ⎤ ⎦ , 4ĮİŁļĿ ⎞ ⎠
9. QSFEJDUPS GPS UIFTF EBUB 4P JOTUFBE XFMM UVSO UP BOPUIFS EBUB TFU JO UIF O EFFQFS JOUP WBSZJOH TMPQFT  &YBNQMF $SPTTDMBTTJĕFE DIJNQBO[FFT XJUI WBSZJOH ćF GVMM NPEFM JODMVEFT CPUI DMVTUFS UZQFT WBSZJOH JOUFSDFQUT WBSZJOH T BOE WBSZJOH TMPQFT PO UIF JOUFSBDUJPO CFUXFFO +-*.*Ǿ' !/ BOE *)$/$ -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = AJ + (B1,J + B1$,J $J)1J >OLQ AJ = α + αĮİŁļĿ[J] + αįĹļİĸ[J] B1,J = β1 + β1,ĮİŁļĿ[J] + β1,įĹļİĸ[J] B1$,J = β1 + β1$,ĮİŁļĿ[J] + β1$,įĹļİĸ[J] >1 × 4JODF UIFSF BSF UXP DMVTUFS UZQFT BDUPST BOE CMPDLT UIFSF BSF UXP NVMUJWBS PST ćF NVMUJWBSJBUF (BVTTJBO QSJPST BSF CPUI EJNFOTJPOBM JO UIJT FYBN FSBM ZPV DBO DIPPTF UP IBWF EJČFSFOU WBSZJOH FČFDUT JO EJČFSFOU DMVTUFS UZQ ⎡ ⎣ αĮİŁļĿ β1,ĮİŁļĿ β1$,ĮİŁļĿ ⎤ ⎦ ∼ .7/PSNBM ⎛ ⎝ ⎡ ⎣    ⎤ ⎦ , 4ĮİŁļĿ ⎞ ⎠
10. QSFEJDUPS GPS UIFTF EBUB 4P JOTUFBE XFMM UVSO UP BOPUIFS EBUB TFU JO UIF O EFFQFS JOUP WBSZJOH TMPQFT  &YBNQMF $SPTTDMBTTJĕFE DIJNQBO[FFT XJUI WBSZJOH ćF GVMM NPEFM JODMVEFT CPUI DMVTUFS UZQFT WBSZJOH JOUFSDFQUT WBSZJOH T BOE WBSZJOH TMPQFT PO UIF JOUFSBDUJPO CFUXFFO +-*.*Ǿ' !/ BOE *)$/$ -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = AJ + (B1,J + B1$,J $J)1J >OLQ AJ = α + αĮİŁļĿ[J] + αįĹļİĸ[J] B1,J = β1 + β1,ĮİŁļĿ[J] + β1,įĹļİĸ[J] B1$,J = β1 + β1$,ĮİŁļĿ[J] + β1$,įĹļİĸ[J] >1 × 4JODF UIFSF BSF UXP DMVTUFS UZQFT BDUPST BOE CMPDLT UIFSF BSF UXP NVMUJWBS PST ćF NVMUJWBSJBUF (BVTTJBO QSJPST BSF CPUI EJNFOTJPOBM JO UIJT FYBN FSBM ZPV DBO DIPPTF UP IBWF EJČFSFOU WBSZJOH FČFDUT JO EJČFSFOU DMVTUFS UZQ ⎡ ⎣ αĮİŁļĿ β1,ĮİŁļĿ β1$,ĮİŁļĿ ⎤ ⎦ ∼ .7/PSNBM ⎛ ⎝ ⎡ ⎣    ⎤ ⎦ , 4ĮİŁļĿ ⎞ ⎠ average effects
11. QSFEJDUPS GPS UIFTF EBUB 4P JOTUFBE XFMM UVSO UP BOPUIFS EBUB TFU JO UIF O EFFQFS JOUP WBSZJOH TMPQFT  &YBNQMF $SPTTDMBTTJĕFE DIJNQBO[FFT XJUI WBSZJOH ćF GVMM NPEFM JODMVEFT CPUI DMVTUFS UZQFT WBSZJOH JOUFSDFQUT WBSZJOH T BOE WBSZJOH TMPQFT PO UIF JOUFSBDUJPO CFUXFFO +-*.*Ǿ' !/ BOE *)$/$ -J ∼ #JOPNJBM(, QJ) MPHJU(QJ) = AJ + (B1,J + B1$,J $J)1J >OLQ AJ = α + αĮİŁļĿ[J] + αįĹļİĸ[J] B1,J = β1 + β1,ĮİŁļĿ[J] + β1,įĹļİĸ[J] B1$,J = β1 + β1$,ĮİŁļĿ[J] + β1$,įĹļİĸ[J] >1 × 4JODF UIFSF BSF UXP DMVTUFS UZQFT BDUPST BOE CMPDLT UIFSF BSF UXP NVMUJWBS PST ćF NVMUJWBSJBUF (BVTTJBO QSJPST BSF CPUI EJNFOTJPOBM JO UIJT FYBN FSBM ZPV DBO DIPPTF UP IBWF EJČFSFOU WBSZJOH FČFDUT JO EJČFSFOU DMVTUFS UZQ ⎡ ⎣ αĮİŁļĿ β1,ĮİŁļĿ β1$,ĮİŁļĿ ⎤ ⎦ ∼ .7/PSNBM ⎛ ⎝ ⎡ ⎣    ⎤ ⎦ , 4ĮİŁļĿ ⎞ ⎠ average effects actor offsets block offsets
12. Cross-classified varying slopes • Need two multivariate priors: actors and blocks • Each 3-dimensional with own covariance matrix  &9".1-& $3044$-"44*'*&% $)*.1"/;&&4 8*5) 7"3:*/( 4-01&4  WBSZJOH FČFDUT JO EJČFSFOU DMVTUFS UZQFT )FSF BSF UIF UXP QSJPST JO UIJT DBTF ⎡ ⎣ αĮİŁļĿ β1,ĮİŁļĿ β1$,ĮİŁļĿ ⎤ ⎦ ∼ .7/PSNBM ⎛ ⎝ ⎡ ⎣    ⎤ ⎦ , 4ĮİŁļĿ ⎞ ⎠ ⎡ ⎣ αįĹļİĸ β1,įĹļİĸ β1$,įĹļİĸ ⎤ ⎦ ∼ .7/PSNBM ⎛ ⎝ ⎡ ⎣    ⎤ ⎦ , 4įĹļİĸ ⎞ ⎠ 8IBU UIFTF QSJPST TUBUF JT UIBU BDUPST BOE CMPDLT DPNF GSPN UXP EJČFSFOU TUBUJTUJDBM QPQVMB UJPOT 8JUIJO FBDI UIF UISFF GFBUVSFT PG FBDI BDUPS PS CMPDL BSF SFMBUFE UISPVHI B DPWBSJBODF NBUSJY TQFDJĕD UP UIBU QPQVMBUJPO ćFSF BSF OP NFBOT JO UIFTF QSJPST KVTU CFDBVTF XF BM SFBEZ QMBDFE UIF BWFSBHF FČFDUT‰α β1 BOE β1$ ‰JO UIF MJOFBS NPEFMT "OE UIF (+Ǐ./) DPEF GPS UIJT NPEFM MPPLT BT ZPVE FYQFDU HJWFO QSFWJPVT FYBNQMFT
13. m13.6 <- map2stan alist( #likelihood pulled_left ~ dbinom(1,p), #linear models logit(p) <- A + (BP + BPC*condition)*prosoc_left, A <- a + a_actor[actor] + a_block[block_id], BP <- bp + bp_actor[actor] + bp_block[block_id], BPC <- bpc + bpc_actor[actor] + bpc_block[block_id], # adaptive priors c(a,bp,bpc) ~ dnorm(0,1), c(a_actor,bp_actor,bpc_actor)[actor] ~ dmvnorm2(0,sigma_actor,Rho_actor), c(a_block,bp_block,bpc_block)[block_id] ~ dmvnorm2(0,sigma_block,Rho_block), # fixed priors sigma_actor ~ dcauchy(0,2), sigma_block ~ dcauchy(0,2), Rho_actor ~ dlkjcorr(4), Rho_block ~ dlkjcorr(4) ) , data=d )
14. Cross-classified varying slopes • 54 parameters • 3 average effects • 3x7 = 21 varying effects on actor • 3x6 = 18 varying effects on block • 6 standard deviations • 6 free correlation parameters • WAIC says pWAIC ≈ 18 (sigmas are small) 1 1 1  &9".1-& $3044$-"44*'*&% $)*.1"/;&&4 8*5) 7"3:*/( 4-01&4  3 DPEF  -/" &0ǯ *ƾǀǑǃ ǒ !"-1%ʅƿ ǒ -/0ʅ ǯǛ0&$*Ǯ 1,/ǛǒǛ0&$*Ǯ), (Ǜǰ ǰ "+ 1!"3 ),4"/ ƽǑDždž 2--"/ ƽǑDždž +Ǯ"## %1 0&$*Ǯ 1,/DZƾDz ƿǑǀǀ ƽǑdžƽ ƾǑƾƿ ǀǑǁǃ ǀƿdžǃ ƾ 0&$*Ǯ 1,/DZƿDz ƽǑǁǃ ƽǑǀǃ ƽǑƽƽ ƽǑDžDž ǂǃDŽDŽ ƾ 0&$*Ǯ 1,/DZǀDz ƽǑǂƿ ƽǑǁdž ƽǑƽƽ ƾǑƽDž ǂDžǃDž ƾ 0&$*Ǯ), (DZƾDz ƽǑƿƿ ƽǑƿƽ ƽǑƽƽ ƽǑǁǃ ǂDžƽdž ƾ 0&$*Ǯ), (DZƿDz ƽǑǂDŽ ƽǑǁƽ ƽǑƽƽ ƾǑƽǀ ǀdžǀƾ ƾ 0&$*Ǯ), (DZǀDz ƽǑǂƾ ƽǑǁƿ ƽǑƽƽ ƾǑƽƾ ǂDžǀǁ ƾ
15. Non-centered form JUFSBUJPOT BSF B IJHIMZ UFDIOJDBM JOEJDBUPS PG UIBU GBDU ćJT JT XIFSF VTJOH UIF ĻļĻİIJĻŁIJĿIJı ĽĮĿĮĺIJŁIJĿĶŇĮŁĶļĻ XJMM IFMQ ćF 0WFSUIJOL JOH CPY BU UIF FOE PG UIF TFDUJPO FYQMBJOT XIBU UIJT NFBOT JO NPSF EFUBJM 'PS UIF NPNFOU MFUT KVTU SFFYQSFTT UIF NPEFM XJUI UIF BMUFSOBUJWF QBSBNFUFSJ[BUJPO ćF !*3+,/* EFOTJUZ JO *-ƿ01+ EPFT BMM UIF XPSL GPS ZPV IJEJOH UIF DIBOHF JO QBSBNFUFSJ[BUJPO BOE NBLJOH JU FBTZ UP TXJUDI CBDL BOE GPSUI )FSFT UIF TBNF NPEFM CVU OPX VTJOH !3+,/* GPS UIF WBSZJOH FČFDU QSJPST JOTUFBE PG UIF VTVBM !*3+,/*ƿ 3 DPEF  *ƾǀǑǃ ʆǦ *-ƿ01+ǯ )&01ǯ -2))"!Ǯ)"#1 ʍ !&+,*ǯƾǒ-ǰǒ ),$&1ǯ-ǰ ʆǦ  ʀ ǯ ʀ ǹ ,+!&1&,+ǰǹ-/,0, Ǯ)"#1ǒ  ʆǦ  ʀ Ǯ 1,/DZ 1,/Dz ʀ Ǯ), (DZ), (Ǯ&!Dzǒ  ʆǦ - ʀ -Ǯ 1,/DZ 1,/Dz ʀ -Ǯ), (DZ), (Ǯ&!Dzǒ  ʆǦ - ʀ - Ǯ 1,/DZ 1,/Dz ʀ - Ǯ), (DZ), (Ǯ&!Dzǒ ȅ !-1&3" Ǧ -/&,/0 ǯǮ 1,/ǒ-Ǯ 1,/ǒ- Ǯ 1,/ǰDZ 1,/Dz ʍ !*3+,/*ǯ0&$*Ǯ 1,/ǒ%,Ǯ 1,/ǰǒ ǯǮ), (ǒ-Ǯ), (ǒ- Ǯ), (ǰDZ), (Ǯ&!Dz ʍ !*3+,/*ǯ0&$*Ǯ), (ǒ%,Ǯ), (ǰǒ ǯǒ-ǒ- ǰ ʍ !+,/*ǯƽǒƾǰǒ 0&$*Ǯ 1,/ ʍ ! 2 %6ǯƽǒƿǰǒ 0&$*Ǯ), ( ʍ ! 2 %6ǯƽǒƿǰǒ %,Ǯ 1,/ ʍ !)(' ,//ǯǁǰǒ %,Ǯ), ( ʍ !)(' ,//ǯǁǰ ǰ ǒ !1ʅ! ǒ &1"/ʅǂƽƽƽ ǒ 4/*2-ʅƾƽƽƽ ǒ %&+0ʅǀ ǒ ,/"0ʅǀ ǰ dmvnormNC usually samples more efficiently
16. Non-centered form   "%7&/563&4 */ $07"3*"/$& m13.6 m13.6NC 0 2000 6000 10000 model effective samples 'ĶĴłĿIJ ƉƋƏ %JTUSJCVUJPOT PG FČFDUJWF TBNQMFT +Ǯ"## GPS UIF PSEJOBSZ BOE OPODFOUFSFE QBSBNFUFSJ[BUJPOT PG UIF DSPTT DMBTTJĕFE WBSZJOH TMPQFT NPEFM *ƾǀǑǃ BOE *ƾǀǑǃ SFTQFDUJWFMZ #PUI NPEFMT BSSJWF BU FRVJWBMFOU JOGFSFODFT CVU UIF OPODFOUFSFE WFSTJPO TBNQMFT NVDI NPSF FďDJFOUMZ /PUF UIBU UIFSF JT OP [FSP ƽ JO UIF BEBQUJWF QSJPST 8JUI UIF OPODFOUFSFE QBSBNFUFSJ[BUJPO UIF NFBOT  - BOE - JO UIJT DBTF BSF BMXBZT JO UIF MJOFBS NPEFM OFWFS JOTJEF UIF QSJPS
17. Non-centered form • Goal: Every dimension (parameter) in posterior shall be Normal(0,1) • Once you start embedding parameters inside priors, this is hard — Normal(a , sigma) e.g. • Solution: Factor things out of the prior NT IBWF BOBMPHPVT QSPCMFNT 4JHOT PG UIF QSPCMFN JODMVEF MPX +Ǯ"## EJWFSHFOU JUFSBUJPO XBSOJOHT 0ęFO SVOOJOH UIF DIBJOT MPOH FOPVHI GSPN UIF QPTUFSJPS CVU UIJT DBO CF WFSZ JOFďDJFOU ćJT XBT UIF DBTF X B JT UP SFQBSBNFUFSJ[F UIF NPEFM UP VTF B OPODFOUFSFE QBSBNFUFSJ[BUJP O WBSZJOH FČFDUT QSJPS ćF XPSE iOPODFOUFSFEw IFSF JT FOUJSFMZ VOIF NJOPMPHZ VOGPSUVOBUFMZ #FUUFS UP TBZ UIBU XFSF HPJOH UP VTF B TUBOE WBSZJOH FČFDUT ćJT NFBOT UIF NFBOT XJMM CF [FSP ZPVWF EPOF UIJT JBUJPOT XJMM BMM CF POF +VTU MJLF ZPV DBO NPWF UIF NFBOT UP UIF MJOFBS N BOEBSE EFWJBUJPOT UP UIF MJOFBS NPEFM )PX 'PS BOZ HJWFO (BVTTJBO CUSBDU PVU UIF NFBO BOE GBDUPS PVU UIF TUBOEBSE EFWJBUJPO 4P GPS FYBN Z ∼ /PSNBM(µ, σ) IJT Z = µ + [σ [ ∼ /PSNBM(, ) T UP UBLF UIF NFBOT BOE TUBOEBSE EFWJBUJPOT PVU PG B (BVTTJBO EJTUSJCVU
18. Non-centered form • Simple case: Varying intercepts • Factor the mean and sigma out of the prior • Centered form: mu <- a_actor[actor] + {stuff} a_actor[actor] ~ normal( a , sigma ) • Non-centered form: mu <- a + z_actor[actor]*sigma + {stuff} z_actor[actor] ~ normal( 0 , 1 ) • See page 408. NB lower=0 constraint on sigma
19. Non-centered form • What about varying slopes? • Now need to factor correlation matrix out of the prior and smuggle into linear model • Can be done: Cholesky! • See page 409 André-Louis Cholesky (1875–1918)
20. Cholesky magic N <- 1e4 sigma1 <- 2 sigma2 <- 0.5 rho <- 0.6 z1 <- rnorm( N ) z2 <- rnorm( N ) a1 <- z1 * sigma1 a2 <- ( rho*z1 + sqrt( 1-rho^2 )*z2 )*sigma2 > cor(z1,z2) [1] -0.0005542644 > cor(a1,a2) [1] 0.5999334 > sd(a1) [1] 1.997036 > sd(a2) [1] 0.4989456
21. http://elevanth.org/blog/2017/09/07/metamorphosis-multilevel-model/
22. Multilevel horoscopes • Begin with “empty” model with no predictors, but with varying intercepts on clusters of interest • Standardize all predictors • Use regularizing priors • Add in predictors and vary their slopes • Can drop varying effects with tiny sigmas • Consider two sorts of posterior prediction • Same units: What happened in these data? • New units: What might we expect for new units? • Your knowledge of domain trumps all
23. 00 04 08 12 Birth Year Republican Vote 2000 2004 2008 2012 1990 1970 1950 1930 0.3 0.4 0.5 0.6 0.7 Lining up by Birth Year VIOMIVLXZM[QLMV\QIT^W\QVOXZMN  <PMZMTI\QWV[PQXQ[KTMIZTaVWV \QKZMTI\QWV[PQX \PMK]Z^MKPIVOM[ V[I[_MTT \PW]OPVWKTMIZXI\\MZV Age Age−Specific Weights (w) • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • •• •• • • • •• •• • • • •••• • •••• • • •• • • • • • • • • • Posterior Mean 50% C.I. 95% C.I. 18 10 20 30 40 50 60 70 −0.01 0.00 0.01 0.02 0.03 0.04 .QO]ZM " -[\QUI\M[NWZ\PMOMVMZI\QWVITI[XMK\[WN  JMWN XIZIUW]V\QUXWZ\IVKMQV\PMNWZUI\QWVWN TWVO ^MZaaW]VOIOMPI^M^MZaTQ\\TMQUXIK\ IVLIN\MZ\PMIO UIOVQ\]LMNZWUIJW]\\PMIOMWN WV_IZL : < QUXTQMLJa\PMU IZM[]J[\IV\QITTaUWZMQUXWZ\IV\NWZ Republican Vote Age 2008 McCain Vote 20 40 60 80 0.3 0.4 0.5 0.6 0. Age Republican Vote 20 40 6 0.3 0.4 0.5 0.6 0. .QO]ZM " :I_LI\IIVL47-;; K]Z^M[ QVLQKI\QVO\PMZMTI <PM/ZMI\;WKQM\a :MIOIV¼[:M^WT]\QWV IVL /MVMZI\QWV[WN8ZM[QLMV\QIT>W\QVO AIQZ/PQ\bI )VLZM_/MTUIV‡ 2]Ta 
24. Continuous categories • Traditional clusters discrete, unordered => every category equally different from all others (in prior) • Continuous dimensions of difference: • Age, income, location, phylogenetic distance, social network distance, many others • No obvious cut points in continuum, but close values share common exposures/covariates/interactions • Would like to exploit pooling in these cases as well • Common approach: Gaussian process regression
25. GP e.g.: Spatial autocorrelation • Relationship between tool complexity and population • Close societies may share tools because of contact or similar geology/ecology • Use space as proxy • Spatial autocorrelation UBODF NBUSJY BNPOH UIF TPDJFUJFT ćFO XF DBO FTUJNBUF IPX QFOET VQPO HFPHSBQIJD EJTUBODF :PVMM TFF IPX UP TJNVMUB QSFEJDUPST TP UIBU UIF DPWBSJBUJPO BNPOH TPDJFUJFT XJUI EJTUBO DPOUSPMMFE CZ PUIFS GBDUPST UIBU JOĘVFODF UFDIOPMPHZ -FUT CFHJO CZ MPBEJOH UIF EBUB BOE JOTQFDUJOH UIF HFPHSB SFBEZ HPOF BIFBE BOE MPPLFE VQ UIF BTUIFDSPXĘJFT OBWJHBUJ PG TPDJFUJFT ćFTF EJTUBODFT BSF NFBTVSFE JO UIPVTBOET PG UIFN JT JO UIF - /#$)&$)" QBDLBHF 3 DPEF  ȕ '* /# $./) (/-$3 '$--4ǿ- /#$)&$)"Ȁ /ǿ$.').$.//-$3Ȁ ȕ $.+'4 ȕ .#*-/ *'0() )( .Ǣ .* !$/. *) .- ) (/ ʚǶ $.').$.//-$3 *')( .ǿ(/Ȁ ʚǶ ǿǫ'ǫǢǫ$ǫǢǫǫǢǫǫǢǫ$ǫǢǫ-ǫǢǫ# -*0)ǿ(/ǢǎȀ ' $   $ - # ) *  ' &0' ǍǡǍ Ǎǡǒ ǍǡǓ ǑǡǑ ǎǡǏ ǏǡǍ ǐǡǏ ǏǡǕ ǎǡǖ ǒǡǔ $&*+$ Ǎǡǒ ǍǡǍ Ǎǡǐ ǑǡǏ ǎǡǏ ǏǡǍ Ǐǡǖ Ǐǡǔ ǏǡǍ ǒǡǐ )/ -05 ǍǡǓ Ǎǡǐ ǍǡǍ ǐǡǖ ǎǡǓ ǎǡǔ ǏǡǓ ǏǡǑ Ǐǡǐ ǒǡǑ + ǑǡǑ ǑǡǏ ǐǡǖ ǍǡǍ ǒǡǑ Ǐǡǒ ǎǡǓ ǎǡǓ Ǔǡǎ ǔǡǏ 0 $%$ ǎǡǏ ǎǡǏ ǎǡǓ ǒǡǑ ǍǡǍ ǐǡǏ ǑǡǍ ǐǡǖ ǍǡǕ Ǒǡǖ -*-$) ǏǡǍ ǏǡǍ ǎǡǔ Ǐǡǒ ǐǡǏ ǍǡǍ ǎǡǕ ǍǡǕ ǐǡǖ Ǔǡǔ #00& ǐǡǏ Ǐǡǖ ǏǡǓ ǎǡǓ ǑǡǍ ǎǡǕ ǍǡǍ ǎǡǏ ǑǡǕ ǒǡǕ )0. ǏǡǕ Ǐǡǔ ǏǡǑ ǎǡǓ ǐǡǖ ǍǡǕ ǎǡǏ ǍǡǍ ǑǡǓ Ǔǡǔ *)" ǎǡǖ ǏǡǍ Ǐǡǐ Ǔǡǎ ǍǡǕ ǐǡǖ ǑǡǕ ǑǡǓ ǍǡǍ ǒǡǍ 2$$ ǒǡǔ ǒǡǐ ǒǡǑ ǔǡǏ Ǒǡǖ Ǔǡǔ ǒǡǕ Ǔǡǔ ǒǡǍ ǍǡǍ /PUJDF UIBU UIF EJBHPOBM JT BMM [FSPT CFDBVTF FBDI TPDJFUZ JT [F OPUJDF UIBU UIF NBUSJY JT TZNNFUSJD BSPVOE UIF EJBHPOBM CFDB distances in thousand km
26. Familiar likelihood OFUJD EJTUBODF PS EJTUBODF JO BHF PS BOZ PUIFS DPOUJOVPVT EJNFOTJPO PG TJNJMB ĘVFODFT PCTFSWBUJPOT ĕSTU QBSU PG UIF NPEFM JT B GBNJMJBS 1PJTTPO MJLFMJIPPE BOE B WBSZJOH JOUFSDFQ XJUI B MPH MJOL 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J ĮĻı QBSBNFUFST XJMM CF UIF WBSZJOH JOUFSDFQUT JO UIJT DBTF #VU VOMJLF UZQJDBM QUT UIFZ XJMM CF FTUJNBUFE JO MJHIU PG HFPHSBQIJD EJTUBODF OPU EJTUJODU DBUFHPS *WF BMTP JODMVEFE BO PSEJOBSZ DPFďDJFOU GPS MPH QPQVMBUJPO 8FMM CF DPO common mean island offset fixed log pop
27. Unfamiliar prior • Gaussian process prior: • Multivariate Gaussian • Means all zero (usually) • Model the covariance matrix using pairwise distances  $0/5*/6064 $"5&(03*&4 "/% 5)& ("644*"/ 130$&44 XJUI XIFUIFS JODMVEJOH TQBUJBM TJNJMBSJUZ XBTIFT PVU UIF BTTPDJBUJPO CFUXFFO BOE UIF UPUBM UPPMT ćF IFBSU PG UIF (BVTTJBO QSPDFTT JT UIF NVMUJWBSJBUF QSJPS GPS UIFTF JOUF γ ∼ .7/PSNBM [, . . . , ], , >S ,JK = η FYQ(−ρ% JK) + δJKσ >GHÀQH ćF ĕSTU MJOF JT UIF EJNFOTJPOBM (BVTTJBO QSJPS GPS UIF JOUFSDFQUT *U IBT CFDBVTF UIFSF BSF  JTMBOE TPDJFUJFT JO UIF EJTUBODF NBUSJY ćF WFDUPS PG N SPT CFDBVTF XFWF QVU UIF HSBOE NFBO α JO UIF MJOFBS NPEFM XIJDI NBLFT EFWJBUJPOT GSPN UIF FYQFDUBUJPO vector of offsets covariance matrix
28. Modeling covariance  $0/5*/6064 $"5&(03*&4 "/% 5)& ("644*"/ 130$&44 XJUI XIFUIFS JODMVEJOH TQBUJBM TJNJMBSJUZ XBTIFT PVU UIF BTTPDJBUJPO CFUXFFO BOE UIF UPUBM UPPMT ćF IFBSU PG UIF (BVTTJBO QSPDFTT JT UIF NVMUJWBSJBUF QSJPS GPS UIFTF JOUF γ ∼ .7/PSNBM [, . . . , ], , >S ,JK = η FYQ(−ρ% JK) + δJKσ >GHÀQH ćF ĕSTU MJOF JT UIF EJNFOTJPOBM (BVTTJBO QSJPS GPS UIF JOUFSDFQUT *U IBT CFDBVTF UIFSF BSF  JTMBOE TPDJFUJFT JO UIF EJTUBODF NBUSJY ćF WFDUPS PG N SPT CFDBVTF XFWF QVU UIF HSBOE NFBO α JO UIF MJOFBS NPEFM XIJDI NBLFT EFWJBUJPOT GSPN UIF FYQFDUBUJPO ćF DPWBSJBODF NBUSJY GPS UIFTF JOUFSDFQUT JT OBNFE , BOE UIF DPWBSJBO QBJS PG JTMBOET J BOE K JT ,JK  ćJT DPWBSJBODF JT EFĕOFE CZ UIF GPSNVMB PO BCPWF ćJT GPSNVMB VTFT UISFF QBSBNFUFST‰η ρ BOE σ‰UP NPEFM IPX DPW TPDJFUJFT DIBOHFT XJUI EJTUBODFT BNPOH UIFN *U QSPCBCMZ MPPLT WFSZ VOGBN ZPV UISPVHI JU JO QJFDFT covariance btw islands i & j max cov rate of decline with distance squared distance “jigger”
29. Modeling covariance XJUI XIFUIFS JODMVEJOH TQBUJBM TJNJMBSJUZ XBTIFT PVU UIF BTTPDJBUJPO CFUXFFO BOE UIF UPUBM UPPMT ćF IFBSU PG UIF (BVTTJBO QSPDFTT JT UIF NVMUJWBSJBUF QSJPS GPS UIFTF JOUF γ ∼ .7/PSNBM [, . . . , ], , >S ,JK = η FYQ(−ρ% JK) + δJKσ >GHÀQH ćF ĕSTU MJOF JT UIF EJNFOTJPOBM (BVTTJBO QSJPS GPS UIF JOUFSDFQUT *U IBT CFDBVTF UIFSF BSF  JTMBOE TPDJFUJFT JO UIF EJTUBODF NBUSJY ćF WFDUPS PG N SPT CFDBVTF XFWF QVU UIF HSBOE NFBO α JO UIF MJOFBS NPEFM XIJDI NBLFT EFWJBUJPOT GSPN UIF FYQFDUBUJPO ćF DPWBSJBODF NBUSJY GPS UIFTF JOUFSDFQUT JT OBNFE , BOE UIF DPWBSJBO QBJS PG JTMBOET J BOE K JT ,JK  ćJT DPWBSJBODF JT EFĕOFE CZ UIF GPSNVMB PO BCPWF ćJT GPSNVMB VTFT UISFF QBSBNFUFST‰η ρ BOE σ‰UP NPEFM IPX DPW TPDJFUJFT DIBOHFT XJUI EJTUBODFT BNPOH UIFN *U QSPCBCMZ MPPLT WFSZ VOGBN ZPV UISPVHI JU JO QJFDFT ćF QBSU PG UIF GPSNVMB GPS , UIBU HJWFT UIF DPWBSJBODF NPEFM JUT TIBQF %JK JT UIF EJTUBODF CFUXFFO UIF JUI BOE KUI TPDJFUJFT 4P XIBU UIJT GVODUJPO DPWBSJBODF CFUXFFO BOZ UXP TPDJFUJFT J BOE K EFDMJOFT FYQPOFOUJBMMZ XJUI UIF TR CFUXFFO UIFN ćF QBSBNFUFS ρ EFUFSNJOFT UIF SBUF PG EFDMJOF *G JU JT MBSHF U   .6-5*-&7&- .0%&-4 ** 0 1 2 3 4 0.0 0.2 0.4 0.6 0.8 1.0 distance correlation 'ĶĴłĿIJ ƉƋƎ 4IBQF PG UIF GVODUJP EJTUBODF UP UIF DPWBSJBODF ,JK  ćF UBM BYJT JT EJTUBODF ćF WFSUJDBM JT MBUJPO SFMBUJWF UP NBYJNVN CFUXFF TPDJFUJFT J BOE K ćF EBTIFE DVSWF FBS EJTUBODF GVODUJPO ćF TPMJE DV TRVBSFE EJTUBODF GVODUJPO squared linear Linear: Cov declines fastest at near distances. Squared: Cov declines fastest at intermediate distances.
30. Putting it all together CFDBVTF UIBUT DPNQVUBUJPOBM FBTJFS 8F EPOU OFFE σ JO UIJT NPEFM TP XFMM JOTUFBE KVTU ĕY JU BU BO JSSFMFWBOU DPOTUBOU /PX IFSFT UIF GVMM NPEFM XJUI UIF ĕYFE QSJPST GPS FBDI QBSBNFUFS BEEFE BU UIF CPUUPN 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J γ ∼ .7/PSNBM (, . . . , ), , ,JK = η FYQ(−ρ% JK) + δJK(.) α ∼ /PSNBM(, ) β1 ∼ /PSNBM(, ) η ∼ )BMG$BVDIZ(, ) ρ ∼ )BMG$BVDIZ(, ) /PUF UIBU ρ BOE η NVTU CF QPTJUJWF TP XF QMBDF IBMG$BVDIZ QSJPST PO UIFN ćFSFT OPUI OH TQFDJBM BCPVU UIF $BVDIZ IFSF *UT KVTU B VTFGVM XFBLMZJOGPSNBUJWF QSJPS GPS TDBMF QB BNFUFST MJLF UIFTF *G ZPV BSF DPODFSOFE BCPVU UIF JNQBDU PG UIF QSJPST ZPV TIPVME SFQFBU IF TBNQMJOH XJUI EJČFSFOU QSJPST " MJUUMF LOPXMFEHF PG 1BDJĕD OBWJHBUJPO XPVME QSPCBCMZ BMMPX VT B TNBSU JOGPSNBUJWF QSJPS PO ρ BU MFBTU 8FSF ĕOBMMZ SFBEZ UP ĕU UIF NPEFM ćF EJTUSJCVUJPO UP VTF UP TJHOBM UP (+Ǐ./) UIBU
31. Fitting m13.7 <- map2stan( alist( total_tools ~ dpois(lambda), log(lambda) <- a + g[society] + bp*logpop, g[society] ~ GPL2( Dmat , etasq , rhosq , 0.01 ), a ~ dnorm(0,10), bp ~ dnorm(0,1), etasq ~ dcauchy(0,1), rhosq ~ dcauchy(0,1) ), data=list( total_tools=d$total_tools, logpop=d$logpop, society=d$society, Dmat=islandsDistMatrix), warmup=2000 , iter=1e4 , chains=4 ) 0 1 2 3 4 0. distance CFDBVTF UIBUT DPNQVUBUJPOBM FBTJFS 8F EPOU OFFE σ JO UIJT NPEFM TP BU BO JSSFMFWBOU DPOTUBOU /PX IFSFT UIF GVMM NPEFM XJUI UIF ĕYFE QSJPST GPS FBDI QBSBNFUFS 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J γ ∼ .7/PSNBM (, . . . , ), , ,JK = η FYQ(−ρ% JK) + δJK(.) α ∼ /PSNBM(, ) β1 ∼ /PSNBM(, ) η ∼ )BMG$BVDIZ(, ) ρ ∼ )BMG$BVDIZ(, ) /PUF UIBU ρ BOE η NVTU CF QPTJUJWF TP XF QMBDF IBMG$BVDIZ QSJPST P JOH TQFDJBM BCPVU UIF $BVDIZ IFSF *UT KVTU B VTFGVM XFBLMZJOGPSNBUJ SBNFUFST MJLF UIFTF *G ZPV BSF DPODFSOFE BCPVU UIF JNQBDU PG UIF QSJP
32. Marginal posterior • Coefficients on log scale, so a bit opaque /*/'Ǿ/**'.ʙɶ/*/'Ǿ/**'.Ǣ '*"+*+ʙɶ'*"+*+Ǣ .*$ /4ʙɶ.*$ /4Ǣ (/ʙ$.').$.//-$3ȀǢ 2-(0+ʙǏǍǍǍ Ǣ $/ -ʙǎ Ǒ Ǣ #$).ʙǑ Ȁ #F TVSF UP DIFDL UIF DIBJOT ćFZ TIPVME TBNQMF WFSZ XFMM -FUT DIFDL UIF FTUJNBUFT KVTU UP DIFDL UIF DPOWFSHFODF EJBHOPTUJDT BOE UP WFSJGZ UIBU BT VTVBM UIF QBSBNFUFST UIFNTFMWFT BSF IBSE UP JOUFSQSFU 3 DPEF  +- $.ǿ(ǎǐǡǔǢ +/#ʙǏȀ  ) / 1 '*2 - Ǎǡǖǒ 0++ - Ǎǡǖǒ )Ǿ !! #/ "ȁǎȂ ǶǍǡǏǔ ǍǡǑǓ ǶǎǡǐǍ ǍǡǓǏ Ǐǎǐǎ ǎ "ȁǏȂ ǶǍǡǎǏ ǍǡǑǒ ǶǎǡǍǖ Ǎǡǔǔ ǏǍǍǕ ǎ "ȁǐȂ ǶǍǡǎǔ ǍǡǑǑ ǶǎǡǍǕ ǍǡǔǍ ǎǖǒǑ ǎ "ȁǑȂ ǍǡǐǍ Ǎǡǐǖ ǶǍǡǒǐ ǎǡǍǕ ǎǖǖǎ ǎ "ȁǒȂ ǍǡǍǏ Ǎǡǐǖ ǶǍǡǔǖ ǍǡǕǍ ǏǍǍǒ ǎ "ȁǓȂ ǶǍǡǑǓ ǍǡǑǍ ǶǎǡǐǍ ǍǡǏǔ ǏǎǐǕ ǎ "ȁǔȂ ǍǡǍǖ ǍǡǐǕ ǶǍǡǔǍ ǍǡǕǕ ǎǖǔǕ ǎ "ȁǕȂ ǶǍǡǏǔ Ǎǡǐǖ ǶǎǡǍǒ Ǎǡǒǎ Ǐǎǎǐ ǎ "ȁǖȂ ǍǡǏǐ ǍǡǐǓ ǶǍǡǒǐ ǍǡǖǑ Ǐǎǎǒ ǎ "ȁǎǍȂ ǶǍǡǎǐ ǍǡǑǔ ǶǎǡǍǖ ǍǡǕǐ ǑǎǔǏ ǎ  ǎǡǏǖ ǎǡǎǖ ǶǎǡǍǓ ǐǡǔǖ ǐǐǐǎ ǎ + ǍǡǏǒ ǍǡǎǏ ǍǡǍǏ ǍǡǑǖ ǒǍǍǒ ǎ /., ǍǡǐǓ ǍǡǓǐ ǍǡǍǍ ǎǡǎǒ ǑǐǑǐ ǎ -#*., ǎǡǓǐ ǎǔǡǎǎ ǍǡǍǍ ǑǡǏǐ Ǖǎǎǎ ǎ 'JSTU OPUF UIBU UIF DPFďDJFOU GPS MPH QPQVMBUJPO + JT WFSZ NVDI BT JU XBT CFGPSF XF BEEFE
33. Covariance function • Combination of eta and rho implies a covariance function K • Draw samples from posterior and plot variation in these functions • Yes, a posterior distribution of covariance functions 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J γ ∼ .7/PSNBM (, . . . , ), , ,JK = η FYQ(−ρ% JK) + δJK(.) α ∼ /PSNBM(, ) β1 ∼ /PSNBM(, ) η ∼ )BMG$BVDIZ(, ) ρ ∼ )BMG$BVDIZ(, ) BU ρ BOE η NVTU CF QPTJUJWF TP XF QMBDF IBMG$BVDIZ QSJPST PO UIFN ćFSFT DJBM BCPVU UIF $BVDIZ IFSF *UT KVTU B VTFGVM XFBLMZJOGPSNBUJWF QSJPS GPS TDB T MJLF UIFTF *G ZPV BSF DPODFSOFE BCPVU UIF JNQBDU PG UIF QSJPST ZPV TIPVME S QMJOH XJUI EJČFSFOU QSJPST " MJUUMF LOPXMFEHF PG 1BDJĕD OBWJHBUJPO XPVME QSP B TNBSU JOGPSNBUJWF QSJPS PO ρ BU MFBTU SF ĕOBMMZ SFBEZ UP ĕU UIF NPEFM ćF EJTUSJCVUJPO UP VTF UP TJHOBM UP (+Ǐ./ OU UP UIF TRVBSFE EJTUBODF (BVTTJBO QSPDFTT QSJPS JT  Ǐ ćF SFTU PG UIF DPEF T JBS   .6-5* 0 2 4 6 8 10 0.0 0.2 0.4 0.6 0.8 1.0 distance (thousand km) covariance
34. Implied correlations • Covariance (and variance) on log scale, so hard to understand • Compute correlations at posterior median:  $0/5*/6064 $"5&(03*&4 "/% 5)& ("644*"/ 130$&44 ȕ *)1 -/ /* *-- '/$*) (/-$3 #* ʚǶ -*0)ǿ *1Ǐ*-ǿ Ȁ Ǣ Ǐ Ȁ ȕ  -*2ȅ*' )( . !*- *)1 )$ ) *')( .ǿ#*Ȁ ʚǶ ǿǫ'ǫǢǫ$ǫǢǫǫǢǫǫǢǫ$ǫǢǫ-ǫǢǫ#ǫǢǫ)ǫǢǫ*ǫǢǫ  -*2)( .ǿ#*Ȁ ʚǶ *')( .ǿ#*Ȁ #* ' $   $ - # ) *  ' ǎǡǍǍ ǍǡǕǔ ǍǡǕǏ ǍǡǍǍ ǍǡǒǏ Ǎǡǎǖ ǍǡǍǏ ǍǡǍǑ ǍǡǏǑ Ǎ $ ǍǡǕǔ ǎǡǍǍ ǍǡǖǏ ǍǡǍǍ ǍǡǒǏ Ǎǡǎǖ ǍǡǍǑ ǍǡǍǓ ǍǡǏǎ Ǎ  ǍǡǕǏ ǍǡǖǏ ǎǡǍǍ ǍǡǍǍ Ǎǡǐǔ ǍǡǐǍ ǍǡǍǔ Ǎǡǎǎ ǍǡǎǏ Ǎ  ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǎǡǍǍ ǍǡǍǍ ǍǡǍǖ Ǎǡǐǔ ǍǡǐǑ ǍǡǍǍ Ǎ $ ǍǡǒǏ ǍǡǒǏ Ǎǡǐǔ ǍǡǍǍ ǎǡǍǍ ǍǡǍǏ ǍǡǍǍ ǍǡǍǍ ǍǡǔǓ Ǎ - Ǎǡǎǖ Ǎǡǎǖ ǍǡǐǍ ǍǡǍǖ ǍǡǍǏ ǎǡǍǍ ǍǡǏǓ ǍǡǔǏ ǍǡǍǍ Ǎ # ǍǡǍǏ ǍǡǍǑ ǍǡǍǔ Ǎǡǐǔ ǍǡǍǍ ǍǡǏǓ ǎǡǍǍ Ǎǡǒǐ ǍǡǍǍ Ǎ ) ǍǡǍǑ ǍǡǍǓ Ǎǡǎǎ ǍǡǐǑ ǍǡǍǍ ǍǡǔǏ Ǎǡǒǐ ǎǡǍǍ ǍǡǍǍ Ǎ * ǍǡǏǑ ǍǡǏǎ ǍǡǎǏ ǍǡǍǍ ǍǡǔǓ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǎǡǍǍ Ǎ  ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǎ ćF DMVTUFS PG TNBMM TPDJFUJFT JO UIF VQQFSMFę PG UIF NBUSJY‰.BMFLVMB .M
35. -40 -20 0 20 -20 -10 0 10 20 longitude latitude Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 7 8 9 10 11 12 20 30 40 50 60 70 log population total tools Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 'ĶĴłĿIJ ƉƋƐ -Fę 1PTUFSJPS NFEJBO DPSSFMBUJPOT BNPOH TPDJFUJFT JO HF PHSBQIJD TQBDF 3JHIU 4BNF QPTUFSJPS NFEJBO DPSSFMBUJPOT OPX TIPXO BHBJOTU SFMBUJPOTIJQ CFUXFFO UPUBM UPPMT BOE MPH QPQVMBUJPO
36. -40 -20 0 20 -20 -10 0 10 20 longitude latitude Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 7 8 9 10 11 12 20 30 40 50 60 70 log population total tools Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 'ĶĴłĿIJ ƉƋƐ -Fę 1PTUFSJPS NFEJBO DPSSFMBUJPOT BNPOH TPDJFUJFT JO HF PHSBQIJD TQBDF 3JHIU 4BNF QPTUFSJPS NFEJBO DPSSFMBUJPOT OPX TIPXO BHBJOTU SFMBUJPOTIJQ CFUXFFO UPUBM UPPMT BOE MPH QPQVMBUJPO
37. Gaussian process regression • Many applications, many covariance functions • Periodic functions of time (seasonality) • Phylogenetic (patristic) distance => phylogenetic regression • Social networks • Non-parametric splines on any predictor • Can use multiple dimensions in covariance, “automatic relevance determination” OUJBM OPOJOEFQFOEFODF PG TQFDJFT 'PS UIPTF JOUFSFTUFE JO TPDJBM OFUXPSLT OF F JT BOPUIFS UZQF PG BCTUSBDU EJTUBODF UIBU DBO CF QMVHHFE JOUP UIFTF NPEFMT PUIFS DPNNPO VTF GPS (BVTTJBO QSPDFTT SFHSFTTJPO JT UP NPEFM DZDMJDBM DPWBS NF *O UIPTF DBTFT UIF DPWBSJBODF NBUSJY , JT NPEFMFE VTJOH QFSJPEJD GVODUJPOT JOF BSF UIF FBTJFTU UP VTF‰PG EJTUBODF JO UJNF ćJT IFMQT NPEFM TFBTPOBM JOĘV JNQPTJOH BOZ IBSE DVUPČT GPS TFBTPOT IF EFĕOJUJPO PG , JTOU UIF TBNF JO BMM (BVTTJBO QSPDFTT NPEFMT CVU UIF CBTJD NPEFMJOH DPWBSJBODF BT B GVODUJPO PG EJTUBODF JT QSFTFOU JO BMM TVDI NPEFMT *U UP VTF NPSF UIBO POF EJNFOTJPO PG EJTUBODF BU UIF TBNF UJNF ćJT DPSSFTQP JOH TMPQFT TUSBUFHZ JO XIJDI WBSJBUJPO XJUIJO BOE CFUXFFO DBUFHPSJFT EFQFOET GFBUVSFT #VU UIF (BVTTJBO QSPDFTT NFSHFT BMM PG UIFTF JOĘVFODFT JOUP B DPNNP F NBUSJY BOE TP B DPNNPO JOUFSDFQU *U XPVME CF QPTTJCMF GPS FYBNQMF UP SFNP DF PG QPQVMBUJPO TJ[F JO UIF 0DFBOJD EBUB GSPN UIF MJOFBS NPEFM BOE NFSHF JU JO (BVTTJBO QSPDFTT *O UIBU DBTF B DPNNPO BQQSPBDI JT UP EFĕOF UIF DPWBSJBOD ,JK = η FYQ − ρ % % JK + ρ 1(MPH 1J − MPH 1K) + δJKσ
38. Next week, dénouement • Homework: 13M3, 13M4, 13H1 • Next week: • Missing data • Measurement error • Enlightenment