Statistical Rethinking Fall 2017 Lecture 16

Week 9, Lecture 16, Statistical Rethinking: A Bayesian Course with Examples in R and Stan. This lecture covers Chapter 13 of the book.

Transcript

1. Week 9: Multilevel Models II Adventures in Covariance Richard McElreath Statistical Rethinking
2. Kinds of varying effects • Varying intercepts: means differ by cluster • Varying slopes: effects of predictors vary by cluster • Any parameter can be made into a varying effect • (1) split into vector of parameters by cluster • (2) define population distribution -6 -4 -2 0 2 4 6 -6 -4 -2 0 2 4 6 -6 -4 -2 0 2 4 6 -6 -4 -2 0 2 4 6
3. Varying slopes • Why varying slopes? • drugs affect people differently • after school programs don’t work for everyone • not every unit has same relationship to predictor • variation is important, whether for intervention or inference • Average effect misleading? • Pooling, shrinkage, mnesia
4. Café Robot • Robot programmed to visit cafés, order coffee, record wait time • Visits in morning and afternoon • Intercepts: avg morning wait • Slopes: avg difference btw afternoon and morning • Are intercepts and slopes related? • Yes => pooling across parameter types!   .6-5* 2 4 6 8 wait time (minutes) M A M A M A M A M A 2 4 6 8 wait time (minutes) M A M A M A M A M A Café A Café B
5. Population of Cafés -3 -2 -1 0 1 2 3 0.0 0.2 0.4 intercept Density -3 -2 -1 0 1 2 3 0.0 0.4 0.8 slope Density -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 intercept slope intercepts slopes population
6. Population of Cafés • 2-dimensional Gaussian distribution • vector of means • variance-covariance matrix -3 -2 -1 0 1 2 3 -3 -2 -1 0 1 2 3 intercept slope  .6-5*-&7&- .0%&-4 **  JT UIF NFBO JOUFSDFQU UIF XBJU JO UIF NPSOJOH "OE UIF WBMVF JO  JT ČFSFODF JO XBJU CFUXFFO BęFSOPPO BOE NPSOJOH BODFT BOE DPWBSJBODFT JT BSSBOHFE MJLF UIJT σ α σασβρ σασβρ σ β QUT JT σ α BOE UIF WBSJBODF JO TMPQFT JT σ β  ćFTF BSF GPVOE BMPOH UIF ćF PUIFS UXP FMFNFOUT PG UIF NBUSJY BSF UIF TBNF σασβρ ćJT JT UIF FSDFQUT BOE TMPQFT *UT KVTU UIF QSPEVDU PG UIF UXP TUBOEBSE EFWJBUJPOT intercepts variance slopes variance covariance correlation
7. Simulated Cafés   .6-5*-&7&- .0%&-4 ** 2 3 4 5 6 -2.0 -1.5 -1.0 -0.5 intercepts (a_cafe) slopes (b_cafe) 'ĶĴłĿIJ ƉƋƊ  DBGÏT TBN UJTUJDBM QPQVMBUJPO ćF UIF JOUFSDFQU BWFSBHF NPSO DBGF ćF WFSUJDBM BYJT JT EJČFSFODF CFUXFFO BęFSOP XBJU GPS FBDI DBGÏ ćF HSB UIF NVMUJWBSJBUF (BVTTJBO Q DFQUT BOE TMPQFT 20 cafés 5 days morning & afternoon 200 observations
8. Varying slopes model  7"3:*/( 4-01&4 #: $0/4536$5*0/ WBSZJOH JOUFSDFQUT ćJT JT UIF WBSZJOH TMPQFT NPEFM XJUI FYQMBOBUJPO UP GPMMPX 8J ∼ /PSNBM(µJ, σ) µJ = αİĮij˦[J] + βİĮij˦[J] .J >OLQ αİĮij˦ βİĮij˦ ∼ .7/PSNBM α β , 4 >SRSXODWLRQRIYDU\ 4 = σα   σβ 3 σα   σβ >FRQVWUXFWFRYDULD α ∼ /PSNBM(, ) >SULRUIRUDYHUDJH β ∼ /PSNBM(, ) >SULRUIRUDYHU σ ∼ )BMG$BVDIZ(, ) >SULRUVWGGHYZ σα ∼ )BMG$BVDIZ(, ) >SULRUVWGGHYDPRQJ σβ ∼ )BMG$BVDIZ(, ) >SULRUVWGGHYDPR 3 ∼ -,+DPSS() >SULRUIRUFRUUHOD ćF MJLFMJIPPE BOE MJOFBS NPEFM OFFE OP FYQMBOBUJPO BU UIJT QPJOU JO UIF CPPL #VU MJOF XIJDI EFĕOFT UIF QPQVMBUJPO PG WBSZJOH JOUFSDFQUT BOE TMPQFT EFTFSWFT BUUFOU
9. WBSZJOH JOUFSDFQUT ćJT JT UIF WBSZJOH TMPQFT NPEFM XJUI FYQMB 8J ∼ /PSNBM(µJ, σ) µJ = αİĮij˦[J] + βİĮij˦[J] .J αİĮij˦ βİĮij˦ ∼ .7/PSNBM α β , 4 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ )BMG$BVDIZ(, ) σα ∼ )BMG$BVDIZ(, ) σβ ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() ćF MJLFMJIPPE BOE MJOFBS NPEFM OFFE OP FYQMBOBUJPO BU UIJT QP varying intercepts varying slopes
10. WBSZJOH JOUFSDFQUT ćJT JT UIF WBSZJOH TMPQFT NPEFM XJUI FYQMB 8J ∼ /PSNBM(µJ, σ) µJ = αİĮij˦[J] + βİĮij˦[J] .J αİĮij˦ βİĮij˦ ∼ .7/PSNBM α β , 4 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ )BMG$BVDIZ(, ) σα ∼ )BMG$BVDIZ(, ) σβ ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() ćF MJLFMJIPPE BOE MJOFBS NPEFM OFFE OP FYQMBOBUJPO BU UIJT QP multivariate prior
11. WBSZJOH JOUFSDFQUT ćJT JT UIF WBSZJOH TMPQFT NPEFM XJUI FYQMB 8J ∼ /PSNBM(µJ, σ) µJ = αİĮij˦[J] + βİĮij˦[J] .J αİĮij˦ βİĮij˦ ∼ .7/PSNBM α β , 4 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ )BMG$BVDIZ(, ) σα ∼ )BMG$BVDIZ(, ) σβ ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() ćF MJLFMJIPPE BOE MJOFBS NPEFM OFFE OP FYQMBOBUJPO BU UIJT QP pop avg intercept pop avg slope covariance matrix
12. Covariance matrix shuffle • m-by-m covariance matrix requires estimating • m standard deviations (or variances) • (m2 – m)/2 correlations (or covariances) • total of m(m + 1)/2 parameters • Several ways specify priors • Conjugate: inverse-Wishart (inv_wishart) • inverse-Wishart cannot pull apart stddev and correlations • Better to decompose: α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK ∼ /PSNBM(, σ) K = ... σ ∼ $BVDIZ(, ) "JK ∼ #JOPNJBM(OJ, QJK) MPHJU QJK = α + αK + (βN + βNK)NJK α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 S R {
13. Matrixes are nice • Matrix algebra just shortcuts for working with lists of numbers • A few simple rules • Can you make an omelet? You can multiply matrixes.
14. a b c d A B C D = αıIJĽŁ[J] + βıIJĽŁ[J] NJ ∼ .7/PSNBM α β , 4 = σα   σβ 3 σα   σβ ∼ /PSNBM(, ) βıIJĽŁ β 4 = σα   σβ 3 σ  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) (σα, σβ) ∼ )BMG$BVDIZ(, ) MPHJU(QJ) = αıIJĽŁ[J] + β αıIJĽŁ βıIJĽŁ ∼ .7/PSNBM 4 = σα   σβ α ∼ /PSNBM(, Ł β 4 = σα   σβ 3 σα  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) β) ∼ )BMG$BVDIZ(, )
15. a b c d A B C D = αıIJĽŁ[J] + βıIJĽŁ[J] NJ ∼ .7/PSNBM α β , 4 = σα   σβ 3 σα   σβ ∼ /PSNBM(, ) βıIJĽŁ β 4 = σα   σβ 3 σ  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) (σα, σβ) ∼ )BMG$BVDIZ(, ) MPHJU(QJ) = αıIJĽŁ[J] + β αıIJĽŁ βıIJĽŁ ∼ .7/PSNBM 4 = σα   σβ α ∼ /PSNBM(, Ł β 4 = σα   σβ 3 σα  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) β) ∼ )BMG$BVDIZ(, )
16. a b c d A B Aa + Bc C D = αıIJĽŁ[J] + βıIJĽŁ[J] NJ ∼ .7/PSNBM α β , 4 = σα   σβ 3 σα   σβ ∼ /PSNBM(, ) βıIJĽŁ β 4 = σα   σβ 3 σ  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) (σα, σβ) ∼ )BMG$BVDIZ(, ) MPHJU(QJ) = αıIJĽŁ[J] + β αıIJĽŁ βıIJĽŁ ∼ .7/PSNBM 4 = σα   σβ α ∼ /PSNBM(, Ł β 4 = σα   σβ 3 σα  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) β) ∼ )BMG$BVDIZ(, )
17. a b c d A B Aa + Bc Ab + Bd C D Ca + Dc Cb + Dd = αıIJĽŁ[J] + βıIJĽŁ[J] NJ ∼ .7/PSNBM α β , 4 = σα   σβ 3 σα   σβ ∼ /PSNBM(, ) βıIJĽŁ β 4 = σα   σβ 3 σ  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) (σα, σβ) ∼ )BMG$BVDIZ(, ) MPHJU(QJ) = αıIJĽŁ[J] + β αıIJĽŁ βıIJĽŁ ∼ .7/PSNBM 4 = σα   σβ α ∼ /PSNBM(, Ł β 4 = σα   σβ 3 σα  α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) β) ∼ )BMG$BVDIZ(, )
18. Matrixes are nice N αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 S
19. Matrixes are nice N αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 S N αK ∼ /PSNBM(, σ) K = ... σ ∼ $BVDIZ(, ) "JK ∼ #JOPNJBM(OJ, QJK) JU QJK = α + αK + (βN + βNK)NJK α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK NK ∼ .7/PSNBM   , Σ K = ... ρσα σβ σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 "JK ∼ #JOPNJBM(OJ, QJK) MPHJU QJK = α + αK + (βN + βNK)NJK α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK βNK ∼ .7/PSNBM   , Σ K = ... σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 ?
20. Matrixes are nice N αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 S N αK ∼ /PSNBM(, σ) K = ... σ ∼ $BVDIZ(, ) "JK ∼ #JOPNJBM(OJ, QJK) JU QJK = α + αK + (βN + βNK)NJK α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK NK ∼ .7/PSNBM   , Σ K = ... ρσα σβ σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 "JK ∼ #JOPNJBM(OJ, QJK) MPHJU QJK = α + αK + (βN + βNK)NJK α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK βNK ∼ .7/PSNBM   , Σ K = ... σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 σα ρσα ρσβ σβ
21. Matrixes are nice N αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 S "JK ∼ #JOPNJBM(OJ, QJK) MPHJU QJK = α + αK + (βN + βNK)NJK α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 σα ρσα ρσβ σβ ?
22. Matrixes are nice N αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 S "JK ∼ #JOPNJBM(OJ, QJK) MPHJU QJK = α + αK + (βN + βNK)NJK α ∼ /PSNBM(, ) βN ∼ /PSNBM(, ) αK βNK ∼ .7/PSNBM   , Σ K = ... Σ = σ α ρσα σβ ρσα σβ σ β = σα   σβ  ρ ρ  σα   σβ = 434 σα ρσα ρσβ σβ σα ρσα ρσβ σβ σ α ρσα σβ ρσα σβ σ β
23. WBSZJOH JOUFSDFQUT ćJT JT UIF WBSZJOH TMPQFT NPEFM XJUI FYQMB 8J ∼ /PSNBM(µJ, σ) µJ = αİĮij˦[J] + βİĮij˦[J] .J αİĮij˦ βİĮij˦ ∼ .7/PSNBM α β , 4 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ )BMG$BVDIZ(, ) σα ∼ )BMG$BVDIZ(, ) σβ ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() ćF MJLFMJIPPE BOE MJOFBS NPEFM OFFE OP FYQMBOBUJPO BU UIJT QP build cov matrix
24. WBSZJOH JOUFSDFQUT ćJT JT UIF WBSZJOH TMPQFT NPEFM XJUI FYQMB 8J ∼ /PSNBM(µJ, σ) µJ = αİĮij˦[J] + βİĮij˦[J] .J αİĮij˦ βİĮij˦ ∼ .7/PSNBM α β , 4 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ )BMG$BVDIZ(, ) σα ∼ )BMG$BVDIZ(, ) σβ ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() ćF MJLFMJIPPE BOE MJOFBS NPEFM OFFE OP FYQMBOBUJPO BU UIJT QP fixed (non-adaptive) priors
25. WBSZJOH JOUFSDFQUT ćJT JT UIF WBSZJOH TMPQFT NPEFM XJUI FYQMB 8J ∼ /PSNBM(µJ, σ) µJ = αİĮij˦[J] + βİĮij˦[J] .J αİĮij˦ βİĮij˦ ∼ .7/PSNBM α β , 4 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ )BMG$BVDIZ(, ) σα ∼ )BMG$BVDIZ(, ) σβ ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() ćF MJLFMJIPPE BOE MJOFBS NPEFM OFFE OP FYQMBOBUJPO BU UIJT QP correlation matrix prior
26. LKJ Correlation prior • After Lewandowski, Kurowicka, and Joe (LKJ) 2009 • One parameter, eta, specifies concentration or dispersion from identity matrix (zero correlations) • eta = 1, uniform correlation matrices • eta > 1, stomps on extreme correlations • eta < 1, elevates extreme correlations -1.0 -0.5 0.0 0.5 1.0 0.0 0.2 0.4 0.6 correlation Density -1.0 -0.5 0.0 0.5 1.0 0.0 0.4 0.8 correlation Density eta = 1 -1.0 -0.5 0.0 0.5 1.0 0.0 1.0 2.0 correlation Density eta = 2 eta = 0.5
27. Varying slopes estimation m13.1 <- map2stan( alist( wait ~ dnorm( mu , sigma ), mu <- a_cafe[cafe] + b_cafe[cafe]*afternoon, c(a_cafe,b_cafe)[cafe] ~ dmvnorm2(c(a,b),sigma_cafe,Rho), a ~ dnorm(0,10), b ~ dnorm(0,10), sigma_cafe ~ dcauchy(0,2), sigma ~ dcauchy(0,2), Rho ~ dlkjcorr(2) ) , data=d , iter=5000 , warmup=2000 , chains=2 )
28. Varying slopes estimation m13.1 <- map2stan( alist( wait ~ dnorm( mu , sigma ), mu <- a_cafe[cafe] + b_cafe[cafe]*afternoon, c(a_cafe,b_cafe)[cafe] ~ dmvnorm2(c(a,b),sigma_cafe,Rho), a ~ dnorm(0,10), b ~ dnorm(0,10), sigma_cafe ~ dcauchy(0,2), sigma ~ dcauchy(0,2), Rho ~ dlkjcorr(2) ) , data=d , iter=5000 , warmup=2000 , chains=2 )
29. Varying slopes estimation m13.1 <- map2stan( alist( wait ~ dnorm( mu , sigma ), mu <- a_cafe[cafe] + b_cafe[cafe]*afternoon, c(a_cafe,b_cafe)[cafe] ~ dmvnorm2(c(a,b),sigma_cafe,Rho), a ~ dnorm(0,10), b ~ dnorm(0,10), sigma_cafe ~ dcauchy(0,2), sigma ~ dcauchy(0,2), Rho ~ dlkjcorr(2) ) , data=d , iter=5000 , warmup=2000 , chains=2 )
30. Varying slopes estimation m13.1 <- map2stan( alist( wait ~ dnorm( mu , sigma ), mu <- a_cafe[cafe] + b_cafe[cafe]*afternoon, c(a_cafe,b_cafe)[cafe] ~ dmvnorm2(c(a,b),sigma_cafe,Rho), a ~ dnorm(0,10), b ~ dnorm(0,10), sigma_cafe ~ dcauchy(0,2), sigma ~ dcauchy(0,2), Rho ~ dlkjcorr(2) ) , data=d , iter=5000 , warmup=2000 , chains=2 )
31. Posterior correlation ǿǢȀ B WFDUPS PG TUBOEBSE EFWJBUJPOT .$"(Ǿ! BOE B DPSSFMBUJPO NBUSJY #* *U DPO TUSVDUT UIF DPWBSJBODF NBUSJY JOUFSOBMMZ *G ZPV BSF JOUFSFTUFE JO UIF EFUBJMT ZPV DBO QFFL BU UIF SBX 4UBO DPEF XJUI ./)* ǿ(ǎǐǡǎȀ /PX JOTUFBE PG MPPLJOH BU UIF NBSHJOBM FTUJNBUFT JO UIF +- $. PVUQVU MFUT HP TUSBJHIU UP JOTQFDUJOH UIF QPTUFSJPS EJTUSJCVUJPO PG WBSZJOH FČFDUT 'JSTU MFUT FYBNJOF UIF QPTUFSJPS DPSSFMBUJPO CFUXFFO JOUFSDFQUT BOE TMPQFT 3 DPEF  +*./ ʚǶ 3/-/ǡ.(+' .ǿ(ǎǐǡǎȀ  ).ǿ +*./ɶ#*ȁǢǎǢǏȂ Ȁ   .6-5*-&7&- .0%&-4 ** -1.0 -0.5 0.0 0.5 1.0 0.0 0.5 1.0 1.5 2.0 2.5 correlation Density prior posterior 'ĶĴłĿIJ ƉƋƋ 1PTUFS DPSSFMBUJPO CFUXFFO #MVF 1PTUFSJPS EJTUSJ SFMJBCMZ CFMPX [FSP UJPO UIF -,+DPSS 
32. Posterior shrinkage   .6-5 -1.0 -0.5 0.0 0.5 1.0 0.0 0.5 1.0 1.5 2.0 2.5 correlation Density prior posterior QSJPS JT ĘBU PWFS BMM WBMJE DPSSFMBUJPO NBUS   .6-5*-&7&- .0%&-4 ** 2.5 3.0 3.5 4.0 4.5 5.0 5.5 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 intercept slope 3.0 3.5 4.0 4.5 5.0 5.5 2.0 2.5 3.0 3.5 4.0 morning wait (mins) afternoon wait (mins) 'ĶĴłĿIJ ƉƋƌ 4ISJOLBHF JO UXP EJNFOTJPOT -Fę SBX VOQPPMFE JOUFSDFQUT BOE TMPQFT ĕMMFE CMVF DPNQBSFE UP QBSUJBMMZ QPPMFE QPTUFSJPS NFBOT PQFO DJSDMFT  ćF HSBZ DPOUPVST TIPX UIF JOGFSSFE QPQVMBUJPO PG WBSZJOH FČFDUT
33. 2.5 3.0 3.5 4.0 4.5 5.0 5.5 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 intercept slope 3 2.0 2.5 3.0 3.5 4.0 afternoon wait (mins) 10 30 50 80 99 unpooled pooled
34. 2.5 3.0 3.5 4.0 4.5 5.0 5.5 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 intercept slope 3 2.0 2.5 3.0 3.5 4.0 afternoon wait (mins)
35.  .6-5*-&7&- .0%&-4 ** 2.5 3.0 3.5 4.0 4.5 5.0 5.5 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 intercept slope 3.0 3.5 4.0 4.5 5.0 5.5 2.0 2.5 3.0 3.5 4.0 morning wait (mins) afternoon wait (mins) 'ĶĴłĿIJ ƉƋƌ 4ISJOLBHF JO UXP EJNFOTJPOT -Fę SBX VOQPPMFE JOUFSDFQUT BOE TMPQFT ĕMMFE CMVF DPNQBSFE UP QBSUJBMMZ QPPMFE QPTUFSJPS NFBOT PQFO DJSDMFT  ćF HSBZ DPOUPVST TIPX UIF JOGFSSFE QPQVMBUJPO PG WBSZJOH FČFDUT  .6-5*-&7&- .0%&-4 ** 2.5 3.0 3.5 4.0 4.5 5.0 5.5 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 intercept slope 3.0 3.5 4.0 4.5 5.0 5.5 2.0 2.5 3.0 3.5 4.0 morning wait (mins) afternoon wait (mins) 'ĶĴłĿIJ ƉƋƌ 4ISJOLBHF JO UXP EJNFOTJPOT -Fę SBX VOQPPMFE JOUFSDFQUT BOE TMPQFT ĕMMFE CMVF DPNQBSFE UP QBSUJBMMZ QPPMFE QPTUFSJPS NFBOT PQFO DJSDMFT  ćF HSBZ DPOUPVST TIPX UIF JOGFSSFE QPQVMBUJPO PG WBSZJOH FČFDUT parameter scale outcome scale
36. Multi-dimensional shrinkage • Joint distribution of varying effects pools information across intercepts & slopes • Correlation btw effects => shrinkage in one dimension induces shrinkage in others • Improved accuracy, just like varying intercepts   .6-5*-& 2.5 3.0 3.5 4.0 4.5 5.0 5.5 -2.5 -2.0 -1.5 -1.0 -0.5 0.0 intercept slope 'ĶĴłĿIJ ƉƋƌ 4ISJOLBHF JO UXP EJN BOE TMPQFT ĕMMFE CMVF DPNQBSFE UP DJSDMFT  ćF HSBZ DPOUPVST TIPX UIF 3JHIU ćF TBNF FTUJNBUFT PO UIF PV
37. Example: UCB admit data again dept applicant.gender admit reject applications male i j 1 A male 512 313 825 1 1 1 2 A female 89 19 108 0 2 1 3 B male 353 207 560 1 3 2 4 B female 17 8 25 0 4 2 5 C male 120 205 325 1 5 3 6 C female 202 391 593 0 6 3 7 D male 138 279 417 1 7 4 8 D female 131 244 375 0 8 4 9 E male 53 138 191 1 9 5 10 E female 94 299 393 0 10 5 11 F male 22 351 373 1 11 6 12 F female 24 317 341 0 12 6
38. Varying intercepts by dept  &9".1-& "%.*44*0/ %&$*4*0/4 "/% (&/%&3  7BSZJOHJOUFSDFQUT 8FMM CFHJO TMPXMZ CZ QSFTFOUJOHKVTUUIF WBS IFTF EBUB )FSFT UIF NPEFM XJUI UIF WBSZJOH JOUFSDFQU DPNQPOFOUT "J ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αıIJĽŁ[J] + βNJ αıIJĽŁ ∼ /PSNBM(α, σ) >SUL α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ )BMG$BVDIZ(, ) VUDPNF WBSJBCMF "J JT UIF OVNCFS PG BENJU EFDJTJPOT !*&1 BOE UI JT OJ --)& 1&,+0 /PUJDF UIBU * IBWF QMBDFE UIF BWFSBHF JOUFSD FM SBUIFS UIBO JOTJEF UIF WBSZJOH JOUFSDFQUT QSJPS ćJT GPSN JT QFSGFD
39. Varying slopes by dept QBSUNFOU EJČFSFOUMZ USFBUT NBMFT BOE GFNBMFT XJMM TISJOL UPXBSET UIF QPQVMBU DPOUSBTU EFQBSUNFOU ' SFDFJWFE IVOESFET PG BQQMJDBUJPOT GSPN CPUI NBMFT B QPPMJOH XJMM EP WFSZ MJUUMF UP UIF FTUJNBUFT GPS UIBU EFQBSUNFOU ćJT JT XIBU UIF WBSZJOH TMPQFT NPEFM MPPLT MJLF XJUI UIF WBSZJOH FČFDUT CMVF "J ∼ #JOPNJBM(OJ, QJ) MPHJU(QJ) = αıIJĽŁ[J] + βıIJĽŁ[J] NJ αıIJĽŁ βıIJĽŁ ∼ .7/PSNBM α β , 4 >MRLQWSULRUI 4 = σα   σβ 3 σα   σβ α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) (σα, σβ) ∼ )BMG$BVDIZ(, ) 3 ∼ -,+DPSS() >SULRUIRU ćF TZNCPM NJ JOEJDBUFT UIF WBMVF PG *)" GPS UIF JUI SPX *U JT NVMUJQMJFE C βıIJĽŁ[J] XIJDI JT B UPUBM TMPQF EFĕOFE CZ CPUI B WBMVF DPNNPO UP BMM EFQBSUN