Statistical Rethinking - Lecture 19

Lecture 19 - Gaussian processes, measurement error - Statistical Rethinking: A Bayesian Course with R Examples

Transcript

1. Week 10: Gaussian Processes Richard McElreath Statistical Rethinking
2. Continuous categories • Traditional clusters discrete, unordered => every category equally different from all others (in prior) • Continuous dimensions of difference: • Age, income, location, phylogenetic distance, social network distance, many others • No obvious cut points in continuum, but close values share common exposures/covariates/interactions • Would like to exploit pooling in these cases as well • Common approach: Gaussian process regression
3. GP e.g.: Spatial autocorrelation • Relationship between tool complexity and population • Close societies may share tools because of contact or similar geology/ecology • Use space as proxy • Spatial autocorrelation UBODF NBUSJY BNPOH UIF TPDJFUJFT ćFO XF DBO FTUJNBUF IPX QFOET VQPO HFPHSBQIJD EJTUBODF :PVMM TFF IPX UP TJNVMUB QSFEJDUPST TP UIBU UIF DPWBSJBUJPO BNPOH TPDJFUJFT XJUI EJTUBO DPOUSPMMFE CZ PUIFS GBDUPST UIBU JOĘVFODF UFDIOPMPHZ -FUT CFHJO CZ MPBEJOH UIF EBUB BOE JOTQFDUJOH UIF HFPHSB SFBEZ HPOF BIFBE BOE MPPLFE VQ UIF BTUIFDSPXĘJFT OBWJHBUJ PG TPDJFUJFT ćFTF EJTUBODFT BSF NFBTVSFE JO UIPVTBOET PG UIFN JT JO UIF - /#$)&$)" QBDLBHF 3 DPEF  ȕ '* /# $./) (/-$3 '$--4ǿ- /#$)&$)"Ȁ /ǿ$.').$.//-$3Ȁ ȕ $.+'4 ȕ .#*-/ *'0() )( .Ǣ .* !$/. *) .- ) (/ ʚǶ $.').$.//-$3 *')( .ǿ(/Ȁ ʚǶ ǿǫ'ǫǢǫ$ǫǢǫǫǢǫǫǢǫ$ǫǢǫ-ǫǢǫ# -*0)ǿ(/ǢǎȀ ' $   $ - # ) *  ' &0' ǍǡǍ Ǎǡǒ ǍǡǓ ǑǡǑ ǎǡǏ ǏǡǍ ǐǡǏ ǏǡǕ ǎǡǖ ǒǡǔ $&*+$ Ǎǡǒ ǍǡǍ Ǎǡǐ ǑǡǏ ǎǡǏ ǏǡǍ Ǐǡǖ Ǐǡǔ ǏǡǍ ǒǡǐ )/ -05 ǍǡǓ Ǎǡǐ ǍǡǍ ǐǡǖ ǎǡǓ ǎǡǔ ǏǡǓ ǏǡǑ Ǐǡǐ ǒǡǑ + ǑǡǑ ǑǡǏ ǐǡǖ ǍǡǍ ǒǡǑ Ǐǡǒ ǎǡǓ ǎǡǓ Ǔǡǎ ǔǡǏ 0 $%$ ǎǡǏ ǎǡǏ ǎǡǓ ǒǡǑ ǍǡǍ ǐǡǏ ǑǡǍ ǐǡǖ ǍǡǕ Ǒǡǖ -*-$) ǏǡǍ ǏǡǍ ǎǡǔ Ǐǡǒ ǐǡǏ ǍǡǍ ǎǡǕ ǍǡǕ ǐǡǖ Ǔǡǔ #00& ǐǡǏ Ǐǡǖ ǏǡǓ ǎǡǓ ǑǡǍ ǎǡǕ ǍǡǍ ǎǡǏ ǑǡǕ ǒǡǕ )0. ǏǡǕ Ǐǡǔ ǏǡǑ ǎǡǓ ǐǡǖ ǍǡǕ ǎǡǏ ǍǡǍ ǑǡǓ Ǔǡǔ *)" ǎǡǖ ǏǡǍ Ǐǡǐ Ǔǡǎ ǍǡǕ ǐǡǖ ǑǡǕ ǑǡǓ ǍǡǍ ǒǡǍ 2$$ ǒǡǔ ǒǡǐ ǒǡǑ ǔǡǏ Ǒǡǖ Ǔǡǔ ǒǡǕ Ǔǡǔ ǒǡǍ ǍǡǍ /PUJDF UIBU UIF EJBHPOBM JT BMM [FSPT CFDBVTF FBDI TPDJFUZ JT [F OPUJDF UIBU UIF NBUSJY JT TZNNFUSJD BSPVOE UIF EJBHPOBM CFDB distances in thousand km
4. Familiar likelihood OFUJD EJTUBODF PS EJTUBODF JO BHF PS BOZ PUIFS DPOUJOVPVT EJNFOTJPO PG TJNJMB ĘVFODFT PCTFSWBUJPOT ĕSTU QBSU PG UIF NPEFM JT B GBNJMJBS 1PJTTPO MJLFMJIPPE BOE B WBSZJOH JOUFSDFQ XJUI B MPH MJOL 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J ĮĻı QBSBNFUFST XJMM CF UIF WBSZJOH JOUFSDFQUT JO UIJT DBTF #VU VOMJLF UZQJDBM QUT UIFZ XJMM CF FTUJNBUFE JO MJHIU PG HFPHSBQIJD EJTUBODF OPU EJTUJODU DBUFHPS *WF BMTP JODMVEFE BO PSEJOBSZ DPFďDJFOU GPS MPH QPQVMBUJPO 8FMM CF DPO common mean island offset fixed log pop
5. Unfamiliar prior • Gaussian process prior: • Multivariate Gaussian • Means all zero (usually) • Model the covariance matrix using pairwise distances  $0/5*/6064 $"5&(03*&4 "/% 5)& ("644*"/ 130$&44 XJUI XIFUIFS JODMVEJOH TQBUJBM TJNJMBSJUZ XBTIFT PVU UIF BTTPDJBUJPO CFUXFFO BOE UIF UPUBM UPPMT ćF IFBSU PG UIF (BVTTJBO QSPDFTT JT UIF NVMUJWBSJBUF QSJPS GPS UIFTF JOUF γ ∼ .7/PSNBM [, . . . , ], , >S ,JK = η FYQ(−ρ% JK) + δJKσ >GHÀQH ćF ĕSTU MJOF JT UIF EJNFOTJPOBM (BVTTJBO QSJPS GPS UIF JOUFSDFQUT *U IBT CFDBVTF UIFSF BSF  JTMBOE TPDJFUJFT JO UIF EJTUBODF NBUSJY ćF WFDUPS PG N SPT CFDBVTF XFWF QVU UIF HSBOE NFBO α JO UIF MJOFBS NPEFM XIJDI NBLFT EFWJBUJPOT GSPN UIF FYQFDUBUJPO vector of offsets covariance matrix
6. Modeling covariance  $0/5*/6064 $"5&(03*&4 "/% 5)& ("644*"/ 130$&44 XJUI XIFUIFS JODMVEJOH TQBUJBM TJNJMBSJUZ XBTIFT PVU UIF BTTPDJBUJPO CFUXF BOE UIF UPUBM UPPMT ćF IFBSU PG UIF (BVTTJBO QSPDFTT JT UIF NVMUJWBSJBUF QSJPS GPS UIFTF JO γ ∼ .7/PSNBM [, . . . , ], , ,JK = η FYQ(−ρ% JK) + δJKσ >GHÀQ ćF ĕSTU MJOF JT UIF EJNFOTJPOBM (BVTTJBO QSJPS GPS UIF JOUFSDFQUT *U IB CFDBVTF UIFSF BSF  JTMBOE TPDJFUJFT JO UIF EJTUBODF NBUSJY ćF WFDUPS P SPT CFDBVTF XFWF QVU UIF HSBOE NFBO α JO UIF MJOFBS NPEFM XIJDI NBLF EFWJBUJPOT GSPN UIF FYQFDUBUJPO ćF DPWBSJBODF NBUSJY GPS UIFTF JOUFSDFQUT JT OBNFE , BOE UIF DPWBSJB QBJS PG JTMBOET J BOE K JT ,JK  ćJT DPWBSJBODF JT EFĕOFE CZ UIF GPSNVMB P BCPWF ćJT GPSNVMB VTFT UISFF QBSBNFUFST‰η ρ BOE σ‰UP NPEFM IPX D TPDJFUJFT DIBOHFT XJUI EJTUBODFT BNPOH UIFN *U QSPCBCMZ MPPLT WFSZ VOG ZPV UISPVHI JU JO QJFDFT ćF QBSU PG UIF GPSNVMB GPS , UIBU HJWFT UIF DPWBSJBODF NPEFM JUT TIBQ covariance btw islands i & j max cov rate of decline with distance squared distance “jigger”
7. Modeling covariance XJUI XIFUIFS JODMVEJOH TQBUJBM TJNJMBSJUZ XBTIFT PVU UIF BTTPDJBUJPO CFUXFFO BOE UIF UPUBM UPPMT ćF IFBSU PG UIF (BVTTJBO QSPDFTT JT UIF NVMUJWBSJBUF QSJPS GPS UIFTF JOUF γ ∼ .7/PSNBM [, . . . , ], , >S ,JK = η FYQ(−ρ% JK) + δJKσ >GHÀQH ćF ĕSTU MJOF JT UIF EJNFOTJPOBM (BVTTJBO QSJPS GPS UIF JOUFSDFQUT *U IBT CFDBVTF UIFSF BSF  JTMBOE TPDJFUJFT JO UIF EJTUBODF NBUSJY ćF WFDUPS PG N SPT CFDBVTF XFWF QVU UIF HSBOE NFBO α JO UIF MJOFBS NPEFM XIJDI NBLFT EFWJBUJPOT GSPN UIF FYQFDUBUJPO ćF DPWBSJBODF NBUSJY GPS UIFTF JOUFSDFQUT JT OBNFE , BOE UIF DPWBSJBO QBJS PG JTMBOET J BOE K JT ,JK  ćJT DPWBSJBODF JT EFĕOFE CZ UIF GPSNVMB PO BCPWF ćJT GPSNVMB VTFT UISFF QBSBNFUFST‰η ρ BOE σ‰UP NPEFM IPX DPW TPDJFUJFT DIBOHFT XJUI EJTUBODFT BNPOH UIFN *U QSPCBCMZ MPPLT WFSZ VOGBN ZPV UISPVHI JU JO QJFDFT ćF QBSU PG UIF GPSNVMB GPS , UIBU HJWFT UIF DPWBSJBODF NPEFM JUT TIBQF %JK JT UIF EJTUBODF CFUXFFO UIF JUI BOE KUI TPDJFUJFT 4P XIBU UIJT GVODUJPO DPWBSJBODF CFUXFFO BOZ UXP TPDJFUJFT J BOE K EFDMJOFT FYQPOFOUJBMMZ XJUI UIF TR CFUXFFO UIFN ćF QBSBNFUFS ρ EFUFSNJOFT UIF SBUF PG EFDMJOF *G JU JT MBSHF U   .6-5*-&7&- .0%&-4 ** 0 1 2 3 4 0.0 0.2 0.4 0.6 0.8 1.0 distance correlation 'ĶĴłĿIJ ƉƋƎ 4IBQF PG UIF GVODUJP EJTUBODF UP UIF DPWBSJBODF ,JK  ćF UBM BYJT JT EJTUBODF ćF WFSUJDBM JT MBUJPO SFMBUJWF UP NBYJNVN CFUXFF TPDJFUJFT J BOE K ćF EBTIFE DVSWF FBS EJTUBODF GVODUJPO ćF TPMJE DV TRVBSFE EJTUBODF GVODUJPO squared linear Linear: Cov declines fastest at near distances. Squared: Cov declines fastest at intermediate distances.
8. Putting it all together IBUT DPNQVUBUJPOBM FBTJFS 8F EPOU OFFE σ JO UIJT NPEFM TP XFMM JOTUFBE KVTU ĕY JU FWBOU DPOTUBOU IFSFT UIF GVMM NPEFM XJUI UIF ĕYFE QSJPST GPS FBDI QBSBNFUFS BEEFE BU UIF CPUUPN 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J γ ∼ .7/PSNBM (, . . . , ), , ,JK = η FYQ(−ρ% JK) + δJK(.) α ∼ /PSNBM(, ) β1 ∼ /PSNBM(, ) η ∼ )BMG$BVDIZ(, ) ρ ∼ )BMG$BVDIZ(, ) ρ BOE η NVTU CF QPTJUJWF TP XF QMBDF IBMG$BVDIZ QSJPST PO UIFN ćFSFT OPUI M BCPVU UIF $BVDIZ IFSF *UT KVTU B VTFGVM XFBLMZJOGPSNBUJWF QSJPS GPS TDBMF QB JLF UIFTF *G ZPV BSF DPODFSOFE BCPVU UIF JNQBDU PG UIF QSJPST ZPV TIPVME SFQFBU JOH XJUI EJČFSFOU QSJPST " MJUUMF LOPXMFEHF PG 1BDJĕD OBWJHBUJPO XPVME QSPCBCMZ TNBSU JOGPSNBUJWF QSJPS PO ρ BU MFBTU ĕOBMMZ SFBEZ UP ĕU UIF NPEFM ćF EJTUSJCVUJPO UP VTF UP TJHOBM UP (+Ǐ./) UIBU
9. Fitting m13.7 <- map2stan( alist( total_tools ~ dpois(lambda), log(lambda) <- a + g[society] + bp*logpop, g[society] ~ GPL2( Dmat , etasq , rhosq , 0.01 ), a ~ dnorm(0,10), bp ~ dnorm(0,1), etasq ~ dcauchy(0,1), rhosq ~ dcauchy(0,1) ), data=list( total_tools=d$total_tools, logpop=d$logpop, society=d$society, Dmat=islandsDistMatrix), warmup=2000 , iter=1e4 , chains=4 ) 0 1 2 3 4 0. distance CFDBVTF UIBUT DPNQVUBUJPOBM FBTJFS 8F EPOU OFFE σ JO UIJT NPEFM TP BU BO JSSFMFWBOU DPOTUBOU /PX IFSFT UIF GVMM NPEFM XJUI UIF ĕYFE QSJPST GPS FBDI QBSBNFUFS 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J γ ∼ .7/PSNBM (, . . . , ), , ,JK = η FYQ(−ρ% JK) + δJK(.) α ∼ /PSNBM(, ) β1 ∼ /PSNBM(, ) η ∼ )BMG$BVDIZ(, ) ρ ∼ )BMG$BVDIZ(, ) /PUF UIBU ρ BOE η NVTU CF QPTJUJWF TP XF QMBDF IBMG$BVDIZ QSJPST P JOH TQFDJBM BCPVU UIF $BVDIZ IFSF *UT KVTU B VTFGVM XFBLMZJOGPSNBUJ SBNFUFST MJLF UIFTF *G ZPV BSF DPODFSOFE BCPVU UIF JNQBDU PG UIF QSJP
10. Marginal posterior • Coefficients on log scale, so a bit opaque /*/'Ǿ/**'.ʙɶ/*/'Ǿ/**'.Ǣ '*"+*+ʙɶ'*"+*+Ǣ .*$ /4ʙɶ.*$ /4Ǣ (/ʙ$.').$.//-$3ȀǢ 2-(0+ʙǏǍǍǍ Ǣ $/ -ʙǎ Ǒ Ǣ #$).ʙǑ Ȁ #F TVSF UP DIFDL UIF DIBJOT ćFZ TIPVME TBNQMF WFSZ XFMM -FUT DIFDL UIF FTUJNBUFT KVTU UP DIFDL UIF DPOWFSHFODF EJBHOPTUJDT BOE UP WFSJGZ UIBU BT VTVBM UIF QBSBNFUFST UIFNTFMWFT BSF IBSE UP JOUFSQSFU 3 DPEF  +- $.ǿ(ǎǐǡǔǢ +/#ʙǏȀ  ) / 1 '*2 - Ǎǡǖǒ 0++ - Ǎǡǖǒ )Ǿ !! #/ "ȁǎȂ ǶǍǡǏǔ ǍǡǑǓ ǶǎǡǐǍ ǍǡǓǏ Ǐǎǐǎ ǎ "ȁǏȂ ǶǍǡǎǏ ǍǡǑǒ ǶǎǡǍǖ Ǎǡǔǔ ǏǍǍǕ ǎ "ȁǐȂ ǶǍǡǎǔ ǍǡǑǑ ǶǎǡǍǕ ǍǡǔǍ ǎǖǒǑ ǎ "ȁǑȂ ǍǡǐǍ Ǎǡǐǖ ǶǍǡǒǐ ǎǡǍǕ ǎǖǖǎ ǎ "ȁǒȂ ǍǡǍǏ Ǎǡǐǖ ǶǍǡǔǖ ǍǡǕǍ ǏǍǍǒ ǎ "ȁǓȂ ǶǍǡǑǓ ǍǡǑǍ ǶǎǡǐǍ ǍǡǏǔ ǏǎǐǕ ǎ "ȁǔȂ ǍǡǍǖ ǍǡǐǕ ǶǍǡǔǍ ǍǡǕǕ ǎǖǔǕ ǎ "ȁǕȂ ǶǍǡǏǔ Ǎǡǐǖ ǶǎǡǍǒ Ǎǡǒǎ Ǐǎǎǐ ǎ "ȁǖȂ ǍǡǏǐ ǍǡǐǓ ǶǍǡǒǐ ǍǡǖǑ Ǐǎǎǒ ǎ "ȁǎǍȂ ǶǍǡǎǐ ǍǡǑǔ ǶǎǡǍǖ ǍǡǕǐ ǑǎǔǏ ǎ  ǎǡǏǖ ǎǡǎǖ ǶǎǡǍǓ ǐǡǔǖ ǐǐǐǎ ǎ + ǍǡǏǒ ǍǡǎǏ ǍǡǍǏ ǍǡǑǖ ǒǍǍǒ ǎ /., ǍǡǐǓ ǍǡǓǐ ǍǡǍǍ ǎǡǎǒ ǑǐǑǐ ǎ -#*., ǎǡǓǐ ǎǔǡǎǎ ǍǡǍǍ ǑǡǏǐ Ǖǎǎǎ ǎ 'JSTU OPUF UIBU UIF DPFďDJFOU GPS MPH QPQVMBUJPO + JT WFSZ NVDI BT JU XBT CFGPSF XF BEEFE
11. Covariance function • Combination of eta and rho implies a covariance function K • Draw samples from posterior and plot variation in these functions • Yes, a posterior distribution of covariance functions 5J ∼ 1PJTTPO(λJ) MPH λJ = α + γĶŀĹĮĻı[J] + β1 MPH 1J γ ∼ .7/PSNBM (, . . . , ), , ,JK = η FYQ(−ρ% JK) + δJK(.) α ∼ /PSNBM(, ) β1 ∼ /PSNBM(, ) η ∼ )BMG$BVDIZ(, ) ρ ∼ )BMG$BVDIZ(, ) BU ρ BOE η NVTU CF QPTJUJWF TP XF QMBDF IBMG$BVDIZ QSJPST PO UIFN ćFSFT DJBM BCPVU UIF $BVDIZ IFSF *UT KVTU B VTFGVM XFBLMZJOGPSNBUJWF QSJPS GPS TDB T MJLF UIFTF *G ZPV BSF DPODFSOFE BCPVU UIF JNQBDU PG UIF QSJPST ZPV TIPVME S QMJOH XJUI EJČFSFOU QSJPST " MJUUMF LOPXMFEHF PG 1BDJĕD OBWJHBUJPO XPVME QSP B TNBSU JOGPSNBUJWF QSJPS PO ρ BU MFBTU SF ĕOBMMZ SFBEZ UP ĕU UIF NPEFM ćF EJTUSJCVUJPO UP VTF UP TJHOBM UP (+Ǐ./ OU UP UIF TRVBSFE EJTUBODF (BVTTJBO QSPDFTT QSJPS JT  Ǐ ćF SFTU PG UIF DPEF T JBS   .6-5* 0 2 4 6 8 10 0.0 0.2 0.4 0.6 0.8 1.0 distance (thousand km) covariance
12. Implied correlations • Covariance (and variance) on log scale, so hard to understand • Compute correlations at posterior median:  $0/5*/6064 $"5&(03*&4 "/% 5)& ("644*"/ 130$&44 ȕ *)1 -/ /* *-- '/$*) (/-$3 #* ʚǶ -*0)ǿ *1Ǐ*-ǿ Ȁ Ǣ Ǐ Ȁ ȕ  -*2ȅ*' )( . !*- *)1 )$ ) *')( .ǿ#*Ȁ ʚǶ ǿǫ'ǫǢǫ$ǫǢǫǫǢǫǫǢǫ$ǫǢǫ-ǫǢǫ#ǫǢǫ)ǫǢǫ*ǫǢǫ  -*2)( .ǿ#*Ȁ ʚǶ *')( .ǿ#*Ȁ #* ' $   $ - # ) *  ' ǎǡǍǍ ǍǡǕǔ ǍǡǕǏ ǍǡǍǍ ǍǡǒǏ Ǎǡǎǖ ǍǡǍǏ ǍǡǍǑ ǍǡǏǑ Ǎ $ ǍǡǕǔ ǎǡǍǍ ǍǡǖǏ ǍǡǍǍ ǍǡǒǏ Ǎǡǎǖ ǍǡǍǑ ǍǡǍǓ ǍǡǏǎ Ǎ  ǍǡǕǏ ǍǡǖǏ ǎǡǍǍ ǍǡǍǍ Ǎǡǐǔ ǍǡǐǍ ǍǡǍǔ Ǎǡǎǎ ǍǡǎǏ Ǎ  ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǎǡǍǍ ǍǡǍǍ ǍǡǍǖ Ǎǡǐǔ ǍǡǐǑ ǍǡǍǍ Ǎ $ ǍǡǒǏ ǍǡǒǏ Ǎǡǐǔ ǍǡǍǍ ǎǡǍǍ ǍǡǍǏ ǍǡǍǍ ǍǡǍǍ ǍǡǔǓ Ǎ - Ǎǡǎǖ Ǎǡǎǖ ǍǡǐǍ ǍǡǍǖ ǍǡǍǏ ǎǡǍǍ ǍǡǏǓ ǍǡǔǏ ǍǡǍǍ Ǎ # ǍǡǍǏ ǍǡǍǑ ǍǡǍǔ Ǎǡǐǔ ǍǡǍǍ ǍǡǏǓ ǎǡǍǍ Ǎǡǒǐ ǍǡǍǍ Ǎ ) ǍǡǍǑ ǍǡǍǓ Ǎǡǎǎ ǍǡǐǑ ǍǡǍǍ ǍǡǔǏ Ǎǡǒǐ ǎǡǍǍ ǍǡǍǍ Ǎ * ǍǡǏǑ ǍǡǏǎ ǍǡǎǏ ǍǡǍǍ ǍǡǔǓ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǎǡǍǍ Ǎ  ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǍǡǍǍ ǎ ćF DMVTUFS PG TNBMM TPDJFUJFT JO UIF VQQFSMFę PG UIF NBUSJY‰.BMFLVMB .M BOE 4BOUB $SV[ 4$ ‰BSF IJHIMZ DPSSFMBUFE BMM BCPWF  XJUI POF BOPUIFS "T
13. -40 -20 0 20 -20 -10 0 10 20 longitude latitude Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 7 8 9 10 11 12 20 30 40 50 60 70 log population total tools Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 'ĶĴłĿIJ ƉƋƐ -Fę 1PTUFSJPS NFEJBO DPSSFMBUJPOT BNPOH TPDJFUJFT JO HF PHSBQIJD TQBDF 3JHIU 4BNF QPTUFSJPS NFEJBO DPSSFMBUJPOT OPX TIPXO BHBJOTU SFMBUJPOTIJQ CFUXFFO UPUBM UPPMT BOE MPH QPQVMBUJPO
14. -40 -20 0 20 -20 -10 0 10 20 longitude latitude Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 7 8 9 10 11 12 20 30 40 50 60 70 log population total tools Malekula Tikopia Santa Cruz Yap Lau Fiji Trobriand Chuuk Manus Tonga Hawaii 'ĶĴłĿIJ ƉƋƐ -Fę 1PTUFSJPS NFEJBO DPSSFMBUJPOT BNPOH TPDJFUJFT JO HF PHSBQIJD TQBDF 3JHIU 4BNF QPTUFSJPS NFEJBO DPSSFMBUJPOT OPX TIPXO BHBJOTU SFMBUJPOTIJQ CFUXFFO UPUBM UPPMT BOE MPH QPQVMBUJPO
15. Gaussian process regression • Many applications, many covariance functions • Periodic functions of time (seasonality) • Phylogenetic (patristic) distance => phylogenetic regression • Social networks • Non-parametric splines on any predictor • Can use multiple dimensions in covariance, “automatic relevance determination” OUJBM OPOJOEFQFOEFODF PG TQFDJFT 'PS UIPTF JOUFSFTUFE JO TPDJBM OFUXPSLT OF F JT BOPUIFS UZQF PG BCTUSBDU EJTUBODF UIBU DBO CF QMVHHFE JOUP UIFTF NPEFMT PUIFS DPNNPO VTF GPS (BVTTJBO QSPDFTT SFHSFTTJPO JT UP NPEFM DZDMJDBM DPWBS NF *O UIPTF DBTFT UIF DPWBSJBODF NBUSJY , JT NPEFMFE VTJOH QFSJPEJD GVODUJPOT JOF BSF UIF FBTJFTU UP VTF‰PG EJTUBODF JO UJNF ćJT IFMQT NPEFM TFBTPOBM JOĘV JNQPTJOH BOZ IBSE DVUPČT GPS TFBTPOT IF EFĕOJUJPO PG , JTOU UIF TBNF JO BMM (BVTTJBO QSPDFTT NPEFMT CVU UIF CBTJD NPEFMJOH DPWBSJBODF BT B GVODUJPO PG EJTUBODF JT QSFTFOU JO BMM TVDI NPEFMT *U UP VTF NPSF UIBO POF EJNFOTJPO PG EJTUBODF BU UIF TBNF UJNF ćJT DPSSFTQP JOH TMPQFT TUSBUFHZ JO XIJDI WBSJBUJPO XJUIJO BOE CFUXFFO DBUFHPSJFT EFQFOET GFBUVSFT #VU UIF (BVTTJBO QSPDFTT NFSHFT BMM PG UIFTF JOĘVFODFT JOUP B DPNNP F NBUSJY BOE TP B DPNNPO JOUFSDFQU *U XPVME CF QPTTJCMF GPS FYBNQMF UP SFNP DF PG QPQVMBUJPO TJ[F JO UIF 0DFBOJD EBUB GSPN UIF MJOFBS NPEFM BOE NFSHF JU JO (BVTTJBO QSPDFTT *O UIBU DBTF B DPNNPO BQQSPBDI JT UP EFĕOF UIF DPWBSJBOD ,JK = η FYQ − ρ % % JK + ρ 1(MPH 1J − MPH 1K) + δJKσ
16. Week 10: Missing Data & Other Opportunities Richard McElreath Statistical Rethinking
17. 1 2 3
18. 1 2 3 You are served: Probability other side is burnt?
19. Avoid being clever • Intuition terrible guide to probability • No need to be clever; just ruthlessly apply conditional probability • Pr(want to know|already know)
20. 1S(XBOU UP LOPX|BMSFBEZ LOPX) DBTF XF LOPX UIF VQ TJEF JT CVSOU 8F XBOU UP LOPX XIFUIFS PS OPU UIF EPX U ćF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ UFMMT VT 1S(CVSOU EPXO|CVSOU VQ) = 1S(CVSOU VQ, CVSOU EPXO) 1S(CVSOU VQ) KVTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI PVS QBODBLF QS OU UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO XF IBWF JT UIBU CVSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TUBUF PG JOGPSNB JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU UIF QSPCBCJMJUZ CVSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJEF VQ ćF QSPC VSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SBOEPN ćF QSP VQ TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B CVSOU UPQ TJEF F ćJT JT VSOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + (/)(/) = SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU PO CPUI TJEFT B SPN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS
21. 1S(XBOU UP LOPX|BMSFBEZ LOPX) DBTF XF LOPX UIF VQ TJEF JT CVSOU 8F XBOU UP LOPX XIFUIFS PS OPU UIF EPX U ćF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ UFMMT VT 1S(CVSOU EPXO|CVSOU VQ) = 1S(CVSOU VQ, CVSOU EPXO) 1S(CVSOU VQ) KVTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI PVS QBODBLF QS OU UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO XF IBWF JT UIBU CVSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TUBUF PG JOGPSNB JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU UIF QSPCBCJMJUZ CVSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJEF VQ ćF QSPC VSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SBOEPN ćF QSP VQ TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B CVSOU UPQ TJEF F ćJT JT VSOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + (/)(/) = SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU PO CPUI TJEFT B SPN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS ćJT JT KVTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI P 8F XBOU UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO X TJEF JT CVSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TU MJHIU PG JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU CVSOUCVSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJE PG UIF CVSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SB JUZ UIF VQ TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B QBODBLF ćJT JT 1S(CVSOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + "MM UIBU SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU P JT  GSPN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS 1S(CVSOU EPXO|CVSOU VQ) = / / =   *G ZPV EPOU RVJUF CFMJFWF UIJT BOTXFS ZPV DBO EP B RVJDL TJNVMBUJPO UP 
22. 1S(XBOU UP LOPX|BMSFBEZ LOPX) DBTF XF LOPX UIF VQ TJEF JT CVSOU 8F XBOU UP LOPX XIFUIFS PS OPU UIF EPX U ćF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ UFMMT VT 1S(CVSOU EPXO|CVSOU VQ) = 1S(CVSOU VQ, CVSOU EPXO) 1S(CVSOU VQ) KVTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI PVS QBODBLF QS OU UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO XF IBWF JT UIBU CVSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TUBUF PG JOGPSNB JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU UIF QSPCBCJMJUZ CVSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJEF VQ ćF QSPC VSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SBOEPN ćF QSP VQ TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B CVSOU UPQ TJEF F ćJT JT VSOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + (/)(/) = SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU PO CPUI TJEFT B SPN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS ćJT JT KVTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI P 8F XBOU UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO X TJEF JT CVSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TU MJHIU PG JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU CVSOUCVSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJE PG UIF CVSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SB JUZ UIF VQ TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B QBODBLF ćJT JT 1S(CVSOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + "MM UIBU SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU P JT  GSPN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS 1S(CVSOU EPXO|CVSOU VQ) = / / =   *G ZPV EPOU RVJUF CFMJFWF UIJT BOTXFS ZPV DBO EP B RVJDL TJNVMBUJPO UP  1S(CVSOU VQ) JUJPOBM QSPCBCJMJUZ MBCFMFE XJUI PVS QBODBLF QSPCMFN F JT CVSOU BOE UIF JOGPSNBUJPO XF IBWF JT UIBU UIF VQ JOGPSNBUJPO TP XF VQEBUF PVS TUBUF PG JOGPSNBUJPO JO IBU UIF QSPCBCJMJUZ XF XBOU JT KVTU UIF QSPCBCJMJUZ PG UIF F QSPCBCJMJUZ PG TFFJOH B CVSOU TJEF VQ ćF QSPCBCJMJUZ FDBVTF B QBODBLF XBT TFMFDUFE BU SBOEPN ćF QSPCBCJM F PWFS FBDI XBZ XF DBO HFU EFBMU B CVSOU UPQ TJEF PG UIF #6)(.) + 1S(66)() = (/) + (/)(/) = . G HFUUJOH UIF QBODBLF UIBU JT CVSOU PO CPUI TJEFT BOE UIJT 4P BMM UPHFUIFS EPXO|CVSOU VQ) = / / =   S ZPV DBO EP B RVJDL TJNVMBUJPO UP DPOĕSN JU
23. 1S(XBOU UP LOPX|BMSFBEZ LOPX) DBTF XF LOPX UIF VQ TJEF JT CVSOU 8F XBOU UP LOPX XIFUIFS PS OPU UIF EPX U ćF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ UFMMT VT 1S(CVSOU EPXO|CVSOU VQ) = 1S(CVSOU VQ, CVSOU EPXO) 1S(CVSOU VQ) KVTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI PVS QBODBLF QS OU UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO XF IBWF JT UIBU CVSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TUBUF PG JOGPSNB JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU UIF QSPCBCJMJUZ CVSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJEF VQ ćF QSPC VSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SBOEPN ćF QSP VQ TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B CVSOU UPQ TJEF F ćJT JT VSOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + (/)(/) = SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU PO CPUI TJEFT B SPN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS ćJT JT KVTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI P 8F XBOU UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO X TJEF JT CVSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TU MJHIU PG JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU CVSOUCVSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJE PG UIF CVSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SB JUZ UIF VQ TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B QBODBLF ćJT JT 1S(CVSOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + "MM UIBU SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU P JT  GSPN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS 1S(CVSOU EPXO|CVSOU VQ) = / / =   *G ZPV EPOU RVJUF CFMJFWF UIJT BOTXFS ZPV DBO EP B RVJDL TJNVMBUJPO UP  1S(CVSOU VQ) JUJPOBM QSPCBCJMJUZ MBCFMFE XJUI PVS QBODBLF QSPCMFN F JT CVSOU BOE UIF JOGPSNBUJPO XF IBWF JT UIBU UIF VQ JOGPSNBUJPO TP XF VQEBUF PVS TUBUF PG JOGPSNBUJPO JO IBU UIF QSPCBCJMJUZ XF XBOU JT KVTU UIF QSPCBCJMJUZ PG UIF F QSPCBCJMJUZ PG TFFJOH B CVSOU TJEF VQ ćF QSPCBCJMJUZ FDBVTF B QBODBLF XBT TFMFDUFE BU SBOEPN ćF QSPCBCJM F PWFS FBDI XBZ XF DBO HFU EFBMU B CVSOU UPQ TJEF PG UIF #6)(.) + 1S(66)() = (/) + (/)(/) = . G HFUUJOH UIF QBODBLF UIBU JT CVSOU PO CPUI TJEFT BOE UIJT 4P BMM UPHFUIFS EPXO|CVSOU VQ) = / / =   S ZPV DBO EP B RVJDL TJNVMBUJPO UP DPOĕSN JU VTU UIF EFĕOJUJPO PG DPOEJUJPOBM QSPCBCJMJUZ MBCFMFE XJUI PVS QBODBLF Q U UP LOPX JG UIF EPXO TJEF JT CVSOU BOE UIF JOGPSNBUJPO XF IBWF JT UIB VSOU 8F DPOEJUJPO PO UIF JOGPSNBUJPO TP XF VQEBUF PVS TUBUF PG JOGPSN JU ćF EFĕOJUJPO UFMMT VT UIBU UIF QSPCBCJMJUZ XF XBOU JT KVTU UIF QSPCBCJMJ VSOU QBODBLF EJWJEFE CZ UIF QSPCBCJMJUZ PG TFFJOH B CVSOU TJEF VQ ćF QSP VSOUCVSOU QBODBLF JT  CFDBVTF B QBODBLF XBT TFMFDUFE BU SBOEPN ćF Q Q TJEF JT CVSOU NVTU BWFSBHF PWFS FBDI XBZ XF DBO HFU EFBMU B CVSOU UPQ TJE  ćJT JT SOU VQ) = 1S(##)() + 1S(#6)(.) + 1S(66)() = (/) + (/)(/) SFNBJOT JT UIF QSPCBCJMJUZ PG HFUUJOH UIF QBODBLF UIBU JT CVSOU PO CPUI TJEFT PN UIF QSPCMFN EFĕOJUJPO 4P BMM UPHFUIFS 1S(CVSOU EPXO|CVSOU VQ) = / / =   POU RVJUF CFMJFWF UIJT BOTXFS ZPV DBO EP B RVJDL TJNVMBUJPO UP DPOĕSN JU 
24. Getting Ruthless • Express information as constraints and distributions => let logic discover implications • No need to be clever • Examples: • Measurement error • Missing data
25. Measurement error • Measurement always entails error • Typical linear regression: interpret sigma as “error” on outcome • What if error isn’t constant? • What if error is on predictors?
26. Error on outcome • data(WaffleDivorce) • Consider error on outcome, divorce rate • Heterogeneity in error • Small State => large error 23 24 25 26 27 28 29 4 6 8 10 12 14 Median age marriage Divorce rate 'ĶĴłĿIJ ƉƌƉ -Fę %JWPSDF SBUF CZ NF 6OJUFE 4UBUFT 7FSUJDBM CBST TIPX QMVT PG UIF (BVTTJBO VODFSUBJOUZ JO NFBTVSF BHBJO XJUI TUBOEBSE EFWJBUJPOT BHBJOTU M 4UBUFT QSPEVDF NPSF VODFSUBJO FTUJNBUF BEEJUJPOBM JOGPSNBUJPO #VU XF EPOU IBWF BOZ JU JT "T BMXBZT UIF (BVTTJBO DIPJDF JT OPU FRV FSSPS JT BDUVBMMZ (BVTTJBO *UT KVTU UIF NPTU DPOT WBSJBODF )FSFT IPX UP EFĕOF UIF EJTUSJCVUJPO GPS F %ļįŀ,J UIFSF XJMM CF POF QBSBNFUFS %IJŀŁ,J EFĕOF   .*44*/( %"5" "/% 05)&3 0110356/*5*&4 23 24 25 26 27 28 29 4 6 8 10 12 14 Median age marriage Divorce rate 0 1 2 3 4 6 8 10 12 14 log population Divorce rate
27. Error on outcome • Approach: • Treat true divorce rate as unknown parameter • Observed rate is sample from Gaussian distribution: BJO XJUI TUBOEBSE EFWJBUJPOT BHBJOTU MPH QPQVMBUJPO PG FBDI 4UBUF 4NB BUFT QSPEVDF NPSF VODFSUBJO FTUJNBUFT JOGPSNBUJPO #VU XF EPOU IBWF BOZ BEEJUJPOBM JOGPSNBUJPO IFSF T MXBZT UIF (BVTTJBO DIPJDF JT OPU FRVJWBMFOU UP BTTVNJOH UIBU UIF N UVBMMZ (BVTTJBO *UT KVTU UIF NPTU DPOTFSWBUJWF BTTVNQUJPO HJWFO POMZ IPX UP EFĕOF UIF EJTUSJCVUJPO GPS FBDI EJWPSDF SBUF 'PS FBDI PCT SF XJMM CF POF QBSBNFUFS %IJŀŁ,J EFĕOFE CZ %ļįŀ,J ∼ /PSNBM(%IJŀŁ,J, %ŀIJ,J) PFT JT EFĕOF UIF QBSBNFUFST %ļįŀ,J BT IBWJOH UIF TQFDJĕFE (BVTTJBO E PO %IJŀŁ,J  4P UIJT NFBOT UIF BCPWF EFĕOFT B QSPCBCJMJUZ GPS FBDI 4UBU UF HJWFO B LOPXO NFBTVSFNFOU FSSPS BOE USJFT UP FTUJNBUF UIF QMBVTJCMF XJUI UIF PCTFSWBUJPO IFO XF BMTP VTF UIFTF %IJŀŁ WBMVFT BT EBUB JO UIF SFHSFTTJPO FRVBUJPO ć observed (data) true (parameter) std error (data)
28. Error on outcome: model EJWPSDF SBUF HJWFO B LOPXO NFBTVSFNFOU FSSPS BOE USJFT UP FTUJNBUF UIF QMBVTJCM DPOTJTUFOU XJUI UIF PCTFSWBUJPO "OE UIFO XF BMTP VTF UIFTF %IJŀŁ WBMVFT BT EBUB JO UIF SFHSFTTJPO FRVBUJPO POMZ BMMPX VT UP FTUJNBUF DPFďDJFOUT GPS QSFEJDUJPOT UIBU UBLF JOUP BDDPVOU UIF JO UIF PVUDPNF CVU JU XJMM BMTP VQEBUF UIF QSJPS GPS EJWPSDF SBUF JO FBDI 4UBUF )FSFT XIBU UIF NPEFM MPPLT MJLF %IJŀŁ,J ∼ /PSNBM(µJ, σ) >´OLNHOLKRRG µJ = α + β" "J + β3 3J %ļįŀ,J ∼ /PSNBM(%IJŀŁ,J, %ŀIJ,J) >SULR α ∼ /PSNBM(, ) β" ∼ /PSNBM(, ) β3 ∼ /PSNBM(, ) σ ∼ $BVDIZ(, .)
29. Error on outcome: model divorce rate estimates EJWPSDF SBUF HJWFO B LOPXO NFBTVSFNFOU FSSPS BOE USJFT UP FTUJNBUF UIF QMBVTJCM DPOTJTUFOU XJUI UIF PCTFSWBUJPO "OE UIFO XF BMTP VTF UIFTF %IJŀŁ WBMVFT BT EBUB JO UIF SFHSFTTJPO FRVBUJPO POMZ BMMPX VT UP FTUJNBUF DPFďDJFOUT GPS QSFEJDUJPOT UIBU UBLF JOUP BDDPVOU UIF JO UIF PVUDPNF CVU JU XJMM BMTP VQEBUF UIF QSJPS GPS EJWPSDF SBUF JO FBDI 4UBUF )FSFT XIBU UIF NPEFM MPPLT MJLF %IJŀŁ,J ∼ /PSNBM(µJ, σ) >´OLNHOLKRRG µJ = α + β" "J + β3 3J %ļįŀ,J ∼ /PSNBM(%IJŀŁ,J, %ŀIJ,J) >SULR α ∼ /PSNBM(, ) β" ∼ /PSNBM(, ) β3 ∼ /PSNBM(, ) σ ∼ $BVDIZ(, .)
30. EJWPSDF SBUF HJWFO B LOPXO NFBTVSFNFOU FSSPS BOE USJFT UP FTUJNBUF UIF QMBVTJCM DPOTJTUFOU XJUI UIF PCTFSWBUJPO "OE UIFO XF BMTP VTF UIFTF %IJŀŁ WBMVFT BT EBUB JO UIF SFHSFTTJPO FRVBUJPO POMZ BMMPX VT UP FTUJNBUF DPFďDJFOUT GPS QSFEJDUJPOT UIBU UBLF JOUP BDDPVOU UIF JO UIF PVUDPNF CVU JU XJMM BMTP VQEBUF UIF QSJPS GPS EJWPSDF SBUF JO FBDI 4UBUF )FSFT XIBU UIF NPEFM MPPLT MJLF %IJŀŁ,J ∼ /PSNBM(µJ, σ) >´OLNHOLKRRG µJ = α + β" "J + β3 3J %ļįŀ,J ∼ /PSNBM(%IJŀŁ,J, %ŀIJ,J) >SULR α ∼ /PSNBM(, ) β" ∼ /PSNBM(, ) β3 ∼ /PSNBM(, ) σ ∼ $BVDIZ(, .) Error on outcome: model likelihood for each observation likelihood for each estimate estimate standard error of observation
31. " DPPM JNQMJDBUJPO UIBU XJMM BSJTF IFSF JT UIBU JOGPSNBUJPO ĘPXT JO CPUI EJSFDUJPOT‰UIF VO DFSUBJOUZ JO NFBTVSFNFOU JOĘVFODFT UIF SFHSFTTJPO QBSBNFUFST JO UIF MJOFBS NPEFM BOE UIF SFHSFTTJPO QBSBNFUFST JO UIF MJOFBS NPEFM BMTP JOĘVFODF UIF VODFSUBJOUZ JO UIF NFBTVSF NFOUT )FSF JT UIF (+Ǐ./) WFSTJPO PG UIF NPEFM 3 DPEF  '$./ ʚǶ '$./ǿ $1Ǿ*.ʙɶ$1*- Ǣ $1Ǿ.ʙɶ$1*- ǡǢ ʙɶ--$" Ǣ ʙɶ $)" --$" Ȁ (ǎǑǡǎ ʚǶ (+Ǐ./)ǿ '$./ǿ $1Ǿ ./ ʡ )*-(ǿ(0Ǣ.$"(ȀǢ (0 ʚǶ  ʔ ȉ ʔ ȉǢ $1Ǿ*. ʡ )*-(ǿ$1Ǿ ./Ǣ$1Ǿ.ȀǢ  ʡ )*-(ǿǍǢǎǍȀǢ  ʡ )*-(ǿǍǢǎǍȀǢ  ʡ )*-(ǿǍǢǎǍȀǢ .$"( ʡ 0#4ǿǍǢǏǡǒȀ Ȁ Ǣ /ʙ'$./ Ǣ ./-/ʙ'$./ǿ$1Ǿ ./ʙ'$./ɶ$1Ǿ*.Ȁ Ǣ  ʙ  Ǣ $/ -ʙǒǍǍǍ Ǣ #$).ʙǏ Ȁ ćFSF BSF UXP UIJOHT UP OPUF JO UIJT DPEF 'JSTU *WF UVSOFE PČ 8"*$ DBMDVMBUJPO CFDBVTF UIF EFGBVMU DPEF JO   XJMM OPU DPNQVUF UIF MJLFMJIPPE DPSSFDUMZ CZ JOUFHSBUJOH PWFS UIF Error on outcome: fitting %ļįŀ,J UIFSF XJMM CF POF QBSBNFUFS %IJŀŁ,J EFĕOFE CZ %ļįŀ,J ∼ /PSNBM(%IJŀŁ,J, %ŀIJ,J) "MM UIJT EPFT JT EFĕOF UIF QBSBNFUFST %ļįŀ,J BT IBWJOH UIF TQFD DFOUFSFE PO %IJŀŁ,J  4P UIJT NFBOT UIF BCPWF EFĕOFT B QSPCBCJM EJWPSDF SBUF HJWFO B LOPXO NFBTVSFNFOU FSSPS BOE USJFT UP FTUJN DPOTJTUFOU XJUI UIF PCTFSWBUJPO "OE UIFO XF BMTP VTF UIFTF %IJŀŁ WBMVFT BT EBUB JO UIF SFHSFT POMZ BMMPX VT UP FTUJNBUF DPFďDJFOUT GPS QSFEJDUJPOT UIBU UBLF J JO UIF PVUDPNF CVU JU XJMM BMTP VQEBUF UIF QSJPS GPS EJWPSDF SBUF )FSFT XIBU UIF NPEFM MPPLT MJLF %IJŀŁ,J ∼ /PSNBM(µJ, σ) µJ = α + β" "J + β3 3J %ļįŀ,J ∼ /PSNBM(%IJŀŁ,J, %ŀIJ,J) α ∼ /PSNBM(, ) β" ∼ /PSNBM(, ) β3 ∼ /PSNBM(, ) σ ∼ $BVDIZ(, .)
32.   .*44*/( %"5" "/% 05)&3 0110356/*5*&4 23 24 25 26 27 28 29 4 6 8 10 12 14 Median age marriage Divorce rate 0 1 2 3 4 6 8 10 12 14 log population Divorce rate 'ĶĴłĿIJ ƉƌƉ -Fę %JWPSDF SBUF CZ NFEJBO BHF PG NBSSJBHF 4UBUFT PG UIF 6OJUFE 4UBUFT 7FSUJDBM CBST TIPX QMVT BOE NJOVT POF TUBOEBSE EFWJBUJPO PG UIF (BVTTJBO VODFSUBJOUZ JO NFBTVSFE EJWPSDF SBUF 3JHIU %JWPSDF SBUF BHBJO XJUI TUBOEBSE EFWJBUJPOT BHBJOTU MPH QPQVMBUJPO PG FBDI 4UBUF 4NBMMFS 4UBUFT QSPEVDF NPSF VODFSUBJO FTUJNBUFT   .*44*/( %"5" "/% 05)&3 0110356/*5*&4 0.5 1.0 1.5 2.0 2.5 -2 -1 0 1 2 Divorce observed standard error Divorce estimated - divorce observed 23 24 25 26 27 28 29 4 6 8 10 12 14 Median age marriage Divorce rate (posterior) 'ĶĴłĿIJ ƉƌƊ -Fę 4ISJOLBHF SFTVMUJOH GSPN NPEFMJOH UIF NFBTVSFNFOU FSSPS ćF MFTT FSSPS JO UIF PSJHJOBM NFBTVSFNFOU UIF MFTT TISJOLBHF JO UIF QPTUFSJPS FTUJNBUF 3JHIU $PNQBSJTPO PG SFHSFTTJPO UIBU JHOPSFT NFBTVSF NFOU FSSPS EBTIFE MJOF BOE HSBZ TIBEJOH XJUI SFHSFTTJPO UIBU JODPSQPSBUFT NFBTVSFNFOU FSSPS CMVF MJOF BOE TIBEJOH  ćF QPJOUT BOE MJOF TFHNFOUT
33. Error on outcome: results • Association with age of marriage reduced • Could have increased • All about where the small States are • Divorce rate estimates do move from observed values. Why?   .*44*/( %"5" "/% 05)&3 0110356/*5*&4 0.5 1.0 1.5 2.0 2.5 -2 -1 0 1 2 Divorce observed standard error Divorce estimated - divorce observed 23 24 25 26 27 28 29 4 6 8 10 12 14 Median age marriage Divorce rate (posterior) 'ĶĴłĿIJ ƉƌƊ -Fę 4ISJOLBHF SFTVMUJOH GSPN NPEFMJOH UIF NFBTVSFNFOU FSSPS ćF MFTT FSSPS JO UIF PSJHJOBM NFBTVSFNFOU UIF MFTT TISJOLBHF JO UIF QPTUFSJPS FTUJNBUF 3JHIU $PNQBSJTPO PG SFHSFTTJPO UIBU JHOPSFT NFBTVSF NFOU FSSPS EBTIFE MJOF BOE HSBZ TIBEJOH XJUI SFHSFTTJPO UIBU JODPSQPSBUFT
34. Error on outcome: results • Q: Why do divorce rate estimates move? • A: Pooling! • Small States have highly uncertain rates => low influence on regression • Large States have more certain rates => high influence on regression • Divorce estimates should be consistent with regression => update estimates of each State’s divorce rate • Noisier estimates shrink more   .*44*/( %"5" "/% 0 0.5 1.0 1.5 2.0 2.5 -2 -1 0 1 2 Divorce observed standard error Divorce estimated - divorce observed 'ĶĴłĿIJ ƉƌƊ -Fę 4ISJOLBHF SFTVMUJO FSSPS ćF MFTT FSSPS JO UIF PSJHJOBM N QPTUFSJPS FTUJNBUF 3JHIU $PNQBSJTPO NFOU FSSPS EBTIFE MJOF BOE HSBZ TIBEJ