L03 Statistical Rethinking Winter 2019

Lecture 03 of the Dec 2018 through March 2019 edition of Statistical Rethinking.

Transcript

1. Geocentric Models Statistical Rethinking Winter 2019 Lecture 03 / Week 2
3. Triumph of Geocentrism • Claudius Ptolemy (90–168) • Egyptian mathematician • Accurate model of planetary motion • Epicycles: orbits on orbits • Fourier series   -*/&" Earth equant planet epicycle deferent
4. Geocentrism • Descriptively accurate • Mechanistically wrong • General method of approximation • Known to be wrong Regression • Descriptively accurate • Mechanistically wrong • General method of approximation • Taken too seriously
5. Linear regression • Simple statistical golems • Model of mean and variance of normally (Gaussian) distributed measure • Mean as additive combination of weighted variables • Constant variance
6. 1809 Bayesian argument for normal error and least-squares estimation
7. Why normal? • Why are normal (Gaussian) distributions so common in statistics? 1. Easy to calculate with 2. Common in nature 3. Very conservative assumption 0.0 0.1 0.2 0.3 0.4 x density −4σ −2σ 0 2σ 4σ 95%
13. -6 -3 0 3 6 0.00 0.10 0.20 position Density 16 steps -6 -3 0 3 6 0.0 0.1 0.2 0.3 position Density 4 steps -6 -3 0 3 6 0.00 0.10 0.20 position Density 8 steps 0 4 8 12 16 -6 -3 0 3 6 step number position 'ĶĴłĿIJ ƌƊ 3BOEPN XBMLT PO UIF TPDDFS ĕFME DPOWFSHF UP B OPSNBM EJT USJCVUJPO ćF NPSF TUFQT BSF UBLFO UIF DMPTFS UIF NBUDI CFUXFFO UIF SFBM Figure 4.2
14. -6 -3 0 3 6 0.00 0.10 0.20 position Density 16 steps -6 -3 0 3 6 0.0 0.1 0.2 0.3 position Density 4 steps -6 -3 0 3 6 0.00 0.10 0.20 position Density 8 steps 0 4 8 12 16 -6 -3 0 3 6 step number position 'ĶĴłĿIJ ƌƊ 3BOEPN XBMLT PO UIF TPDDFS ĕFME DPOWFSHF UP B OPSNBM EJT USJCVUJPO ćF NPSF TUFQT BSF UBLFO UIF DMPTFS UIF NBUDI CFUXFFO UIF SFBM Figure 4.2
15. -6 -3 0 3 6 0.00 0.10 0.20 position Density 16 steps -6 -3 0 3 6 0.0 0.1 0.2 0.3 position Density 4 steps -6 -3 0 3 6 0.00 0.10 0.20 position Density 8 steps 0 4 8 12 16 -6 -3 0 3 6 step number position 'ĶĴłĿIJ ƌƊ 3BOEPN XBMLT PO UIF TPDDFS ĕFME DPOWFSHF UP B OPSNBM EJT USJCVUJPO ćF NPSF TUFQT BSF UBLFO UIF DMPTFS UIF NBUDI CFUXFFO UIF SFBM Figure 4.2
16. Why normal? • Processes that produce normal distributions • Addition • Products of small deviations • Logarithms of products Francis Galton’s 1894 “bean machine” for simulating normal distributions
17. Why normal? • Ontological perspective • Processes which add fluctuations result in dampening • Damped fluctuations end up Gaussian • No information left, except mean and variance • Can’t infer process from distribution! • Epistemological perspective • Know only mean and variance • Then least surprising and most conservative (maximum entropy) distribution is Gaussian • Nature likes maximum entropy distributions
18. Why normal? • Ontological perspective • Processes which add fluctuations result in dampening • Damped fluctuations end up Gaussian • No information left, except mean and variance • Can’t infer process from distribution! • Epistemological perspective • Know only mean and variance • Then least surprising and most conservative (maximum entropy) distribution is Gaussian • Nature likes maximum entropy distributions
19. Linear models • Models of normally distributed data common • “General Linear Model”: t-test, single regression, multiple regression, ANOVA, ANCOVA, MANOVA, MANCOVA, yadda yadda yadda • All the same thing • Learn strategy, not procedure Willard Boepple
20. Language for modeling • Revisit globe tossing model: outcome parameter to estimate data distribution (likelihood) prior distribution “is distributed” σα ρσα ρσβ σβ σ α ρσα σβ ρσα σβ σ β 8 ∼ #JOPNJBM(/, Q) Q ∼ 6OJGPSN(, )
21. Language for modeling QQMJFT UP NPEFMT JO FWFSZ ĕFME GSPN BTUSPOPNZ UP BSU IJTUPSZ MZ MJFT JO UIF TVCKFDU NBUUFS‰XIJDI WBSJBCMFT NBUUFS BOE IPX EPF IFN ‰OPU JO UIF NBUIFNBUJDT IFTF EFDJTJPOT BSF NBEF‰BOE NPTU PG UIFN XJMM DPNF UP TFFN B H‰XF TVNNBSJ[F UIF NPEFM XJUI TPNFUIJOH NBUIZ MJLF ZJ ∼ /PSNBM(µJ, σ) µJ = βYJ β ∼ /PSNBM(, ) σ ∼ &YQPOFOUJBM() YJ ∼ /PSNBM(, ) NBLF NVDI TFOTF HPPE ćBU JOEJDBUFT UIBU ZPV BSF IPMEJOH UIF SJH L UFBDIFT ZPV IPX UP SFBE BOE XSJUF UIFTF NBUIFNBUJDBM NPEFM E OZ NBUIFNBUJDBM NBOJQVMBUJPO PG UIFN *OTUFBE UIFZ QSPWJEF BO OF BOE DPNNVOJDBUF PVS NPEFMT 0ODF ZPV HFU DPNGPSUBCMF XJUI Define the variables:
22. Language for modeling QQMJFT UP NPEFMT JO FWFSZ ĕFME GSPN BTUSPOPNZ UP BSU IJTUPSZ MZ MJFT JO UIF TVCKFDU NBUUFS‰XIJDI WBSJBCMFT NBUUFS BOE IPX EPF IFN ‰OPU JO UIF NBUIFNBUJDT IFTF EFDJTJPOT BSF NBEF‰BOE NPTU PG UIFN XJMM DPNF UP TFFN B H‰XF TVNNBSJ[F UIF NPEFM XJUI TPNFUIJOH NBUIZ MJLF ZJ ∼ /PSNBM(µJ, σ) µJ = βYJ β ∼ /PSNBM(, ) σ ∼ &YQPOFOUJBM() YJ ∼ /PSNBM(, ) NBLF NVDI TFOTF HPPE ćBU JOEJDBUFT UIBU ZPV BSF IPMEJOH UIF SJH L UFBDIFT ZPV IPX UP SFBE BOE XSJUF UIFTF NBUIFNBUJDBM NPEFM E OZ NBUIFNBUJDBM NBOJQVMBUJPO PG UIFN *OTUFBE UIFZ QSPWJEF BO OF BOE DPNNVOJDBUF PVS NPEFMT 0ODF ZPV HFU DPNGPSUBCMF XJUI Give them definitions:
23. Some data: Kalahari foragers  ćF EBUB ćF EBUB DPOUBJOFE JO /ǿ *2 ''ǎȀ BSF QBSUJBM DFOTVT EBUB GPS UIF %PCF BSFB ,VOH 4BO DPNQJMFE GSPN JOUFSWJFXT DPOEVDUFE CZ /BODZ )PXFMM JO UIF MBUF T 'PS UIF OPOBOUISPQPMPHJTUT SFBEJOH BMPOH UIF ,VOH 4BO BSF UIF NPTU GBNPVT GPSBHJOH QPQVMBUJPO PG UIF UI DFOUVSZ MBSHFMZ CFDBVTF PG EFUBJMFE RVBOUJUBUJWF TUVEJFT CZ QFPQMF MJLF )PXFMM -PBE UIF EBUB BOE QMBDF UIFN JOUP B DPOWFOJFOU PCKFDU XJUI '$--4ǿ- /#$)&$)"Ȁ /ǿ *2 ''ǎȀ  ʚǶ *2 ''ǎ 8IBU ZPV IBWF OPX JT B EBUB GSBNF OBNFE TJNQMZ  * VTF UIF OBNF  PWFS BOE PWFS BHBJO JO UIJT CPPL UP SFGFS UP UIF EBUB GSBNF XF BSF XPSLJOH XJUI BU UIF NPNFOU * LFFQ JUT OBNF TIPSU UP TBWF ZPV UZQJOH " EBUB GSBNF JT B TQFDJBM LJOE PG PCKFDU JO 3 *U JT B UBCMF XJUI OBNFE DPMVNOT DPSSFTQPOEJOH UP WBSJBCMFT BOE OVNCFSFE SPXT DPSSFTQPOEJOH UP JOEJWJEVBM DBTFT *O UIJT FYBNQMF UIF DBTFT BSF JOEJWJEVBMT *OTQFDU UIF TUSVDUVSF PG UIF EBUB GSBNF UIF TBNF XBZ ZPV DBO JOTQFDU UIF TUSVDUVSF PG BOZ TZNCPM JO 3 ./-ǿ  Ȁ Ǫ/ǡ!-( Ǫǣ ǒǑǑ *.ǡ *! Ǒ 1-$' .ǣ ɶ # $"#/ǣ )0( ǎǒǏ ǎǑǍ ǎǐǔ ǎǒǔ ǎǑǒ ǡǡǡ ɶ 2 $"#/ǣ )0( ǑǔǡǕ ǐǓǡǒ ǐǎǡǖ ǒǐ Ǒǎǡǐ ǡǡǡ ɶ " ǣ )0( Ǔǐ Ǔǐ Ǔǒ Ǒǎ ǒǎ ǐǒ ǐǏ Ǐǔ ǎǖ ǒǑ ǡǡǡ ɶ (' ǣ $)/ ǎ Ǎ Ǎ ǎ Ǎ ǎ Ǎ ǎ Ǎ ǎ ǡǡǡ   (&0$&/53*$ .0%&-4 Ǫ/ǡ!-( Ǫǣ ǒǑǑ *.ǡ *! Ǒ 1-$' .ǣ ( ) . ǒǡǒʉ ǖǑǡǒʉ #$./*"-( # $"#/ ǎǐǕǡǏǓ ǏǔǡǓǍ Ǖǎǡǎǎ ǎǓǒǡǔǑ ΤΤΤΤΤΤΤΥΤΪΪΨΤ 2 $"#/ ǐǒǡǓǎ ǎǑǡǔǏ ǖǡǐǓ ǒǑǡǒǍ ΤΥΦΥΥΥΥΨΪΪΦΥΤ " ǏǖǡǐǑ ǏǍǡǔǒ ǎǡǍǍ ǓǓǡǎǐ ΪΨΨΦΨΥΥΤΤ (' ǍǡǑǔ ǍǡǒǍ ǍǡǍǍ ǎǡǍǍ ΪΤΤΤΤΤΤΤΤΪ ćJT EBUB GSBNF DPOUBJOT GPVS DPMVNOT &BDI DPMVNO IBT  FOUSJFT TP UIFSF BSF  JOEJWJE VBMT JO UIFTF EBUB &BDI JOEJWJEVBM IBT B SFDPSEFE IFJHIU DFOUJNFUFST XFJHIU LJMPHSBNT BHF ZFBST BOE iNBMFOFTTw  JOEJDBUJOH GFNBMF BOE  JOEJDBUJOH NBMF  8FSF HPJOH UP XPSL XJUI KVTU UIF # $"#/ DPMVNO GPS UIF NPNFOU ćF DPMVNO DPO UBJOJOH UIF IFJHIUT JT SFBMMZ KVTU B SFHVMBS PME 3 WFDUPS UIF LJOE PG MJTU XF IBWF CFFO XPSLJOH DPMVNOT DPSSFTQPOEJOH UP WBSJBCMFT BOE OVNCFSFE SPXT DPSSFTQPOEJOH UP JOEJWJEVBM DBTFT *O UIJT FYBNQMF UIF DBTFT BSF JOEJWJEVBMT *OTQFDU UIF TUSVDUVSF PG UIF EBUB GSBNF UIF TBNF XBZ ZPV DBO JOTQFDU UIF TUSVDUVSF PG BOZ TZNCPM JO 3 3 DPEF  ./-ǿ  Ȁ Ǫ/ǡ!-( Ǫǣ ǒǑǑ *.ǡ *! Ǒ 1-$' .ǣ ɶ # $"#/ǣ )0( ǎǒǏ ǎǑǍ ǎǐǔ ǎǒǔ ǎǑǒ ǡǡǡ ɶ 2 $"#/ǣ )0( ǑǔǡǕ ǐǓǡǒ ǐǎǡǖ ǒǐ Ǒǎǡǐ ǡǡǡ ɶ " ǣ )0( Ǔǐ Ǔǐ Ǔǒ Ǒǎ ǒǎ ǐǒ ǐǏ Ǐǔ ǎǖ ǒǑ ǡǡǡ ɶ (' ǣ $)/ ǎ Ǎ Ǎ ǎ Ǎ ǎ Ǎ ǎ Ǎ ǎ ǡǡǡ 8F DBO BMTP VTF - /#$)&$)"T +- $. TVNNBSZ GVODUJPO XIJDI XFMM BMTP VTF UP TVNNBSJ[F QPTUFSJPS EJTUSJCVUJPOT MBUFS PO 3 DPEF  +- $.ǿ  Ȁ
24. Gaussian model • A first model: 140 150 160 170 180 0.00 0.02 0.04 0.06 height (cm) Density UIF QMPUUFE PVUDPNF WBSJBCMF MPPLT (BVTTJBO UP ZPV (BXLJOH BU USZ UP EFDJEF IPX UP NPEFM UIFN JT VTVBMMZ OPU B HPPE JEFB ćF NJYUVSF PG EJČFSFOU OPSNBM EJTUSJCVUJPOT GPS FYBNQMF BOE JO UIBU CF BCMF UP EFUFDU UIF VOEFSMZJOH OPSNBMJUZ KVTU CZ FZFCBMMJOH UIF IUT BSF BQQSPYJNBUFMZ OPSNBMMZ EJTUSJCVUFE CVU XIJDI OPSNBM EJT F BSF BO JOĕOJUF OVNCFS PG UIFN XJUI BO JOĕOJUF OVNCFS PG EJG OE WBSJBODFT 8FSF SFBEZ UP XSJUF EPXO UIF HFOFSBM NPEFM BOE QBSBNFUFST UIBU NBYJNJ[F UIF MJLFMJIPPE 5P EFĕOF UIF IFJHIUT SJCVUFE XJUI B NFBO µ BOE TUBOEBSE EFWJBUJPO σ XF XSJUF IJ ∼ /PSNBM(µ, σ). ZPVMM TFF UIF TBNF NPEFM XSJUUFO BT IJ ∼ N(µ, σ) XIJDI NFBOT ćF TZNCPM I SFGFST UP UIF MJTU PG IFJHIUT BOE UIF TVCTDSJQU J JWJEVBM FMFNFOU PG UIJT MJTU *U JT DPOWFOUJPOBM UP VTF J CFDBVTF JU Y ćF JOEFY J UBLFT PO SPX OVNCFST BOE TP JO UIJT FYBNQMF DBO SPN  UP  UIF OVNCFS PG IFJHIUT JO ƽɠ# $"#/  "T TVDI UIF TBZJOH UIBU BMM UIF HPMFN LOPXT BCPVU FBDI IFJHIU NFBTVSFNFOU
25. Gaussian model DPVME CF B NJYUVSF PG EJČFSFOU OPSNBM EJTUSJCVUJPOT GPS FYBNQMF BOE J ZPV XPOU CF BCMF UP EFUFDU UIF VOEFSMZJOH OPSNBMJUZ KVTU CZ FZFCBMMJO TJUZ QMPU 4P UIF IFJHIUT BSF BQQSPYJNBUFMZ OPSNBMMZ EJTUSJCVUFE CVU XIJDI OPSNB VUJPO ćFSF BSF BO JOĕOJUF OVNCFS PG UIFN XJUI BO JOĕOJUF OVNCFS P OU NFBOT BOE WBSJBODFT 8FSF SFBEZ UP XSJUF EPXO UIF HFOFSBM NPEF UIF VOJRVF QBSBNFUFST UIBU NBYJNJ[F UIF MJLFMJIPPE 5P EFĕOF UIF I PSNBMMZ EJTUSJCVUFE XJUI B NFBO µ BOE TUBOEBSE EFWJBUJPO σ XF XSJUF IJ ∼ /PSNBM(µ, σ). NBOZ CPPLT ZPVMM TFF UIF TBNF NPEFM XSJUUFO BT IJ ∼ N(µ, σ) XIJDI N TBNF UIJOH ćF TZNCPM I SFGFST UP UIF MJTU PG IFJHIUT BOE UIF TVCTD OT FBDI JOEJWJEVBM FMFNFOU PG UIJT MJTU *U JT DPOWFOUJPOBM UP VTF J CFDB ET GPS JOEFY ćF JOEFY J UBLFT PO SPX OVNCFST BOE TP JO UIJT FYBNQ BOZ WBMVF GSPN  UP  UIF OVNCFS PG IFJHIUT JO ƽɠ# $"#/  "T TVD EFM BCPWF JT TBZJOH UIBU BMM UIF HPMFN LOPXT BCPVU FBDI IFJHIU NFBTVSF FĕOFE CZ UIF TBNF OPSNBM EJTUSJCVUJPO XJUI B DPOTUBOU NFBO µ BOE TUBO BUJPO σ #FGPSF MPOH UIPTF MJUUMF JT BSF HPJOH UP TIPX VQ PO UIF SJHIUIBO outcome “is distributed” Height hi of an individual i is distributed normally, with mean mu and standard deviation sigma. mean standard deviation
26. Gaussian model • Add priors: IJHIMZ DPSSFMBUFE IFJHIUT #VU UIF PWFSBMM EJTUSJCVUJPO PG GFNBMF IFJHIU SFNBJOT NBM *O TVDI DBTFT JJE SFNBJOT QFSGFDUMZ VTFGVM EFTQJUF JHOPSJOH UIF DPSSFMB YBNQMF UIBU .BSLPW DIBJO .POUF $BSMP $IBQUFS  DBO VTF IJHIMZ DPSSFMBUFE FTUJNBUF NPTU BOZ JJE EJTUSJCVUJPO XF MJLF F NPEFM XFSF HPJOH UP OFFE TPNF QSJPST ćF QBSBNFUFST UP CF FTUJNBUFE P XF OFFE B QSJPS 1S(µ, σ) UIF KPJOU QSJPS QSPCBCJMJUZ GPS BMM QBSBNFUFST ST BSF TQFDJĕFE JOEFQFOEFOUMZ GPS FBDI QBSBNFUFS XIJDI BNPVOUT UP BT 1S(µ) 1S(σ) ćFO XF DBO XSJUF IJ ∼ /PSNBM(µ, σ) >OLNHOLKRRG@ µ ∼ /PSNBM(, ) >µ SULRU@ σ ∼ 6OJGPSN(, ) >σ SULRU@ HIU BSF OPU QBSU PG UIF NPEFM CVU JOTUFBE KVTU OPUFT UP IFMQ ZPV LFFQ USBDL BDI MJOF ćF QSJPS GPS µ JT B CSPBE (BVTTJBO QSJPS DFOUFSFE PO DN JMJUZ CFUXFFO  ±  :PVS BVUIPS JT  DN UBMM "OE UIF SBOHF GSPN  DN UP  DN FODPN F PG QMBVTJCMF NFBO IFJHIUT GPS IVNBO QPQVMBUJPOT 4P EPNBJOTQFDJĕD Figure 4.3 0.000 0.010 0.020 mu Density mu ~ dnorm( 178 , 20 ) 100 178 250 0.000 0.004 0.008 0.012 height Density h ~ dnorm(mu,sigma) 0 73 178 283  " ("644*"/ .0%&- 0' )&*()5 0.000 0.010 0.020 mu Density mu ~ dnorm( 178 , 20 ) 100 178 250 0.000 0.010 0.020 sigma Density sigma ~ dunif( 0 , 50 ) 0 50
27. Prior predictive distribution • What do these priors imply about height, before we see data? Simulate! => prior predictive distribution 100 cm = 3.3 feet 200 cm = 6.5 feet 0.000 0.010 0.020 mu Density mu ~ dnorm( 178 , 20 ) 100 178 250 0.000 0.010 0.020 sigma Density sigma ~ dunif( 0 , 50 ) 0 50 0.000 0.004 0.008 0.012 height Density h ~ dnorm(mu,sigma) 0 73 178 283 0.000 0.002 0.004 height Density h ~ dnorm(mu,sigma) mu ~ dnorm(178,100) -128 0 178 484 Figure 4.3 UJPO PG JOEJWJEVBM IFJHIUT ćJT JT BO FTTFOUJBM QBSU PG ZPVS NPEFMJOH UIF ĽĿĶļĿ ĽĿIJıĶİŁĶŃIJ TJNVMBUJPO 0ODF ZPVWF DIPTFO QSJPST GPS I µ BOE σ UIFTF JNQMZ B KPJOU QSJPS EJTUSJCVUJPO PG JOEJWJEVBM IFJHIUT #Z TJNVMBUJOH GSPN UIJT EJTUSJCVUJPO ZPV DBO TFF XIBU ZPVS DIPJDFT JNQMZ BCPVU PCTFSWBCMF IFJHIU ćJT IFMQT ZPV EJBHOPTF CBE DIPJDFT -PUT PG DPOWFOUJPOBM DIPJDFT BSF JOEFFE CBE POFT BOE XFMM CF BCMF UP TFF UIJT CZ DPOEVDUJOH QSJPS QSFEJDUJWF TJN VMBUJPOT 0LBZ TP IPX UP EP UIJT :PV DBO RVJDLMZ TJNVMBUF IFJHIUT CZ TBNQMJOH GSPN UIF QSJPS MJLF ZPV TBNQMFE GSPN UIF QPTUFSJPS CBDL JO $IBQUFS  3FNFNCFS FWFSZ QPTUFSJPS JT BMTP QPUFOUJBMMZ B QSJPS GPS B TVCTFRVFOU BOBMZTJT TP ZPV DBO QSPDFTT QSJPST KVTU MJLF QPTUFSJPST 3 DPEF  .(+' Ǿ(0 ʚǶ -)*-(ǿ ǎ Ǒ Ǣ ǎǔǕ Ǣ ǏǍ Ȁ .(+' Ǿ.$"( ʚǶ -0)$!ǿ ǎ Ǒ Ǣ Ǎ Ǣ ǒǍ Ȁ +-$*-Ǿ# ʚǶ -)*-(ǿ ǎ Ǒ Ǣ .(+' Ǿ(0 Ǣ .(+' Ǿ.$"( Ȁ  ).ǿ +-$*-Ǿ# Ȁ
28. Prior predictive distribution Figure 4.3 mu 178 250 0.000 0.010 sigma Densit 0 50 height dnorm(mu,sigma) 178 283 0.000 0.002 0.004 height Density h ~ dnorm(mu,sigma) mu ~ dnorm(178,100) -128 0 178 484 JPS QSFEJDUJWF TJNVMBUJPO GPS UIF IFJHIU NPEFM 5PQ SPX POT GPS µ BOE σ #PUUPN MFę ćF QSJPS QSFEJDUJWF TJNVMBUJPO H UIF QSJPST JO UIF UPQ SPX 7BMVFT BU  TUBOEBSE EFWJBUJPOT World’s tallest person (272cm) Fertilized egg (0cm)
29. Computing the posterior • Aim for the posterior distribution, which is now 2-dimensional • Grid approximation: Compute posterior for many combinations of mu and sigma 153.0 154.0 155.0 156.0 7.0 7.5 8.0 8.5 9.0 mu sigma
30. 153.0 154.0 155.0 156.0 7.0 7.5 8.0 8.5 9.0 mu sigma 50x50 153.0 154.0 155.0 156.0 7.0 7.5 8.0 8.5 9.0 mu sigma 100x100 153.0 154.0 155.0 156.0 7.0 7.5 8.0 8.5 9.0 mu sigma 200x200
31. 153.0 154.0 155.0 156.0 0.0 0.2 0.4 0.6 0.8 1.0 mu Density 7.0 7.5 8.0 8.5 9.0 0.0 0.4 0.8 1.2 sigma Density mu sigma Figure 4.4 Drawing samples to work with
32. Quadratic approximation • Approximate posterior as Gaussian • Can estimate with two things: • Peak of posterior, maximum a posteriori (MAP) • Standard deviations & correlations of parameters (covariance matrix) • With flat priors, same as conventional maximum likelihood estimation
33. Using quap 5P DPNQMFUF UIF NPEFM XFSF HPJOH UP OFFE TPNF QSJPST ć CF FTUJNBUFE BSF CPUI µ BOE σ TP XF OFFE B QSJPS 1S(µ, σ) UIF BCJMJUZ GPS BMM QBSBNFUFST *O NPTU DBTFT QSJPST BSF TQFDJĕFE J FBDI QBSBNFUFS XIJDI BNPVOUT UP BTTVNJOH 1S(µ, σ) = 1S(µ DBO XSJUF IJ ∼ /PSNBM(µ, σ) µ ∼ /PSNBM(, ) σ ∼ 6OJGPSN(, ) ćF MBCFMT PO UIF SJHIU BSF OPU QBSU PG UIF NPEFM CVU JOTUFBE KVTU LFFQ USBDL PG UIF QVSQPTF PG FBDI MJOF ćF QSJPS GPS µ JT B CSPB DFOUFSFE PO DN XJUI  PG QSPCBCJMJUZ CFUXFFO  ±  *UT B WFSZ HPPE JEFB UP QMPU ZPVS QSJPST TP ZPV IBWF B TFOTF UIFZ CVJME JOUP UIF NPEFM *O UIJT DBTF 0-1 ǭ )*-(ǭ 3 ǐ Ƽǀǁ ǐ Ƽƻ Ǯ ǐ !-*(ʃƼƻƻ ǐ /*ʃƽƻƻ Ǯ &YFDVUF UIBU DPEF ZPVSTFMG UP TFF UIBU UIF HPMFN JT BTTVNJOH IFJHIU OPU FBDI JOEJWJEVBM IFJHIU JT BMNPTU DFSUBJOMZ CFUXFFO  4P UIJT QSJPS DBSSJFT B MJUUMF JOGPSNBUJPO CVU OPU B MPU ćF σ Q 178, 20 IJ ∼ /PSNBM(µ, σ) # $"#/ ʡ )*-(ǿ µ ∼ /PSNBM(, ) (0 ʡ )*-(ǿ σ ∼ 6OJGPSN(, ) .$"( ʡ 0)$!ǿ /PX QMBDF UIF 3 DPEF FRVJWBMFOUT JOUP BO '$./ )FSFT BO '$./ PG UIF GPSNVMB 3 DPEF  !'$./ ʚǶ '$./ǿ # $"#/ ʡ )*-(ǿ (0 Ǣ .$"( Ȁ Ǣ (0 ʡ )*-(ǿ ǎǔǕ Ǣ ǏǍ Ȁ Ǣ .$"( ʡ 0)$!ǿ Ǎ Ǣ ǒǍ Ȁ Ȁ /PUF UIF DPNNBT BU UIF FOE PG FBDI MJOF FYDFQU UIF MBTU ćFTF DPNNBT TFQBSBU PG UIF NPEFM EFĕOJUJPO 'JU UIF NPEFM UP UIF EBUB JO UIF EBUB GSBNF Ǐ XJUI 3 DPEF  (Ǒǡǎ ʚǶ ,0+ǿ !'$./ Ǣ /ʙǏ Ȁ
34. Using quap 5P DPNQMFUF UIF NPEFM XFSF HPJOH UP OFFE TPNF QSJPST ć CF FTUJNBUFE BSF CPUI µ BOE σ TP XF OFFE B QSJPS 1S(µ, σ) UIF BCJMJUZ GPS BMM QBSBNFUFST *O NPTU DBTFT QSJPST BSF TQFDJĕFE J FBDI QBSBNFUFS XIJDI BNPVOUT UP BTTVNJOH 1S(µ, σ) = 1S(µ DBO XSJUF IJ ∼ /PSNBM(µ, σ) µ ∼ /PSNBM(, ) σ ∼ 6OJGPSN(, ) ćF MBCFMT PO UIF SJHIU BSF OPU QBSU PG UIF NPEFM CVU JOTUFBE KVTU LFFQ USBDL PG UIF QVSQPTF PG FBDI MJOF ćF QSJPS GPS µ JT B CSPB DFOUFSFE PO DN XJUI  PG QSPCBCJMJUZ CFUXFFO  ±  *UT B WFSZ HPPE JEFB UP QMPU ZPVS QSJPST TP ZPV IBWF B TFOTF UIFZ CVJME JOUP UIF NPEFM *O UIJT DBTF 0-1 ǭ )*-(ǭ 3 ǐ Ƽǀǁ ǐ Ƽƻ Ǯ ǐ !-*(ʃƼƻƻ ǐ /*ʃƽƻƻ Ǯ &YFDVUF UIBU DPEF ZPVSTFMG UP TFF UIBU UIF HPMFN JT BTTVNJOH IFJHIU OPU FBDI JOEJWJEVBM IFJHIU JT BMNPTU DFSUBJOMZ CFUXFFO  4P UIJT QSJPS DBSSJFT B MJUUMF JOGPSNBUJPO CVU OPU B MPU ćF σ Q 178, 20 IJ ∼ /PSNBM(µ, σ) # $"#/ ʡ )*-(ǿ µ ∼ /PSNBM(, ) (0 ʡ )*-(ǿ σ ∼ 6OJGPSN(, ) .$"( ʡ 0)$!ǿ /PX QMBDF UIF 3 DPEF FRVJWBMFOUT JOUP BO '$./ )FSFT BO '$./ PG UIF GPSNVMB 3 DPEF  !'$./ ʚǶ '$./ǿ # $"#/ ʡ )*-(ǿ (0 Ǣ .$"( Ȁ Ǣ (0 ʡ )*-(ǿ ǎǔǕ Ǣ ǏǍ Ȁ Ǣ .$"( ʡ 0)$!ǿ Ǎ Ǣ ǒǍ Ȁ Ȁ /PUF UIF DPNNBT BU UIF FOE PG FBDI MJOF FYDFQU UIF MBTU ćFTF DPNNBT TFQBSBU PG UIF NPEFM EFĕOJUJPO 'JU UIF NPEFM UP UIF EBUB JO UIF EBUB GSBNF Ǐ XJUI 3 DPEF  (Ǒǡǎ ʚǶ ,0+ǿ !'$./ Ǣ /ʙǏ Ȁ /PX QMBDF UIF 3 DPEF FRVJWBMFOUT JOUP BO '$./ )FSFT BO '$./ PG UIF GPSNVMBT 3 DPEF  !'$./ ʚǶ '$./ǿ # $"#/ ʡ )*-(ǿ (0 Ǣ .$"( Ȁ Ǣ (0 ʡ )*-(ǿ ǎǔǕ Ǣ ǏǍ Ȁ Ǣ .$"( ʡ 0)$!ǿ Ǎ Ǣ ǒǍ Ȁ Ȁ /PUF UIF DPNNBT BU UIF FOE PG FBDI MJOF FYDFQU UIF MBTU ćFTF DPNNBT TFQBSBUF PG UIF NPEFM EFĕOJUJPO 'JU UIF NPEFM UP UIF EBUB JO UIF EBUB GSBNF Ǐ XJUI 3 DPEF  (Ǒǡǎ ʚǶ ,0+ǿ !'$./ Ǣ /ʙǏ Ȁ
35. Using quap 5P DPNQMFUF UIF NPEFM XFSF HPJOH UP OFFE TPNF QSJPST ć CF FTUJNBUFE BSF CPUI µ BOE σ TP XF OFFE B QSJPS 1S(µ, σ) UIF BCJMJUZ GPS BMM QBSBNFUFST *O NPTU DBTFT QSJPST BSF TQFDJĕFE J FBDI QBSBNFUFS XIJDI BNPVOUT UP BTTVNJOH 1S(µ, σ) = 1S(µ DBO XSJUF IJ ∼ /PSNBM(µ, σ) µ ∼ /PSNBM(, ) σ ∼ 6OJGPSN(, ) ćF MBCFMT PO UIF SJHIU BSF OPU QBSU PG UIF NPEFM CVU JOTUFBE KVTU LFFQ USBDL PG UIF QVSQPTF PG FBDI MJOF ćF QSJPS GPS µ JT B CSPB DFOUFSFE PO DN XJUI  PG QSPCBCJMJUZ CFUXFFO  ±  *UT B WFSZ HPPE JEFB UP QMPU ZPVS QSJPST TP ZPV IBWF B TFOTF UIFZ CVJME JOUP UIF NPEFM *O UIJT DBTF 0-1 ǭ )*-(ǭ 3 ǐ Ƽǀǁ ǐ Ƽƻ Ǯ ǐ !-*(ʃƼƻƻ ǐ /*ʃƽƻƻ Ǯ &YFDVUF UIBU DPEF ZPVSTFMG UP TFF UIBU UIF HPMFN JT BTTVNJOH IFJHIU OPU FBDI JOEJWJEVBM IFJHIU JT BMNPTU DFSUBJOMZ CFUXFFO  4P UIJT QSJPS DBSSJFT B MJUUMF JOGPSNBUJPO CVU OPU B MPU ćF σ Q 178, 20 IJ ∼ /PSNBM(µ, σ) # $"#/ ʡ )*-(ǿ µ ∼ /PSNBM(, ) (0 ʡ )*-(ǿ σ ∼ 6OJGPSN(, ) .$"( ʡ 0)$!ǿ /PX QMBDF UIF 3 DPEF FRVJWBMFOUT JOUP BO '$./ )FSFT BO '$./ PG UIF GPSNVMB 3 DPEF  !'$./ ʚǶ '$./ǿ # $"#/ ʡ )*-(ǿ (0 Ǣ .$"( Ȁ Ǣ (0 ʡ )*-(ǿ ǎǔǕ Ǣ ǏǍ Ȁ Ǣ .$"( ʡ 0)$!ǿ Ǎ Ǣ ǒǍ Ȁ Ȁ /PUF UIF DPNNBT BU UIF FOE PG FBDI MJOF FYDFQU UIF MBTU ćFTF DPNNBT TFQBSBU PG UIF NPEFM EFĕOJUJPO 'JU UIF NPEFM UP UIF EBUB JO UIF EBUB GSBNF Ǐ XJUI 3 DPEF  (Ǒǡǎ ʚǶ ,0+ǿ !'$./ Ǣ /ʙǏ Ȁ /PX QMBDF UIF 3 DPEF FRVJWBMFOUT JOUP BO '$./ )FSFT BO '$./ PG UIF GPSNVMBT 3 DPEF  !'$./ ʚǶ '$./ǿ # $"#/ ʡ )*-(ǿ (0 Ǣ .$"( Ȁ Ǣ (0 ʡ )*-(ǿ ǎǔǕ Ǣ ǏǍ Ȁ Ǣ .$"( ʡ 0)$!ǿ Ǎ Ǣ ǒǍ Ȁ Ȁ /PUF UIF DPNNBT BU UIF FOE PG FBDI MJOF FYDFQU UIF MBTU ćFTF DPNNBT TFQBSBUF PG UIF NPEFM EFĕOJUJPO 'JU UIF NPEFM UP UIF EBUB JO UIF EBUB GSBNF Ǐ XJUI 3 DPEF  (Ǒǡǎ ʚǶ ,0+ǿ !'$./ Ǣ /ʙǏ Ȁ  " ("644*"/ .0%&- 0' )&*()5 "ęFS FYFDVUJOH UIJT DPEF ZPVMM IBWF B ĕU NPEFM TUPSFE JO UIF TZNCPM (Ǒǡǎ /PX BU UIF QPTUFSJPS EJTUSJCVUJPO +- $.ǿ (Ǒǡǎ Ȁ  ) / 1 ǒǡǒʉ ǖǑǡǒʉ (0 ǎǒǑǡǓǎ ǍǡǑǎ ǎǒǐǡǖǒ ǎǒǒǡǏǔ .$"( ǔǡǔǐ ǍǡǏǖ ǔǡǏǔ ǕǡǏǍ ćFTF OVNCFST QSPWJEF (BVTTJBO BQQSPYJNBUJPOT GPS FBDI QBSBNFUFST NBSHJOBM E ćJT NFBOT UIF QMBVTJCJMJUZ PG FBDI WBMVF PG µ BęFS BWFSBHJOH PWFS UIF QMBVTJCJMJU
36. 153.0 154.0 155.0 156.0 0.0 0.2 0.4 0.6 0.8 1.0 mu Density 7.0 7.5 8.0 8.5 9.0 0.0 0.4 0.8 1.2 sigma Density Samples Approximation WFOJFODF GVODUJPO UP EP FYBDUMZ UIBU 3 DPEF  '$--4ǿ- /#$)&$)"Ȁ +*./ ʚǶ 3/-/ǡ.(+' .ǿ (Ǒǡǎ Ǣ )ʙǎ Ǒ Ȁ # ǿ+*./Ȁ (0 .$"( ǎ ǎǒǒǡǍǍǐǎ ǔǡǑǑǐǕǖǐ Ǐ ǎǒǑǡǍǐǑǔ ǔǡǔǔǎǏǒǒ ǐ ǎǒǑǡǖǎǒǔ ǔǡǕǏǏǎǔǕ Ǒ ǎǒǑǡǑǏǒǏ ǔǡǒǐǍǐǐǎ ǒ ǎǒǑǡǒǐǍǔ ǔǡǓǒǒǑǖǍ Ǔ ǎǒǒǡǎǔǔǏ ǔǡǖǔǑǓǍǐ :PV FOE VQ XJUI B EBUB GSBNF +*./ XJUI   ǎ Ǒ SPXT BOE UXP DPMVNOT POF DPMVNO GPS µ BOE POF GPS σ &BDI WBMVF JT B TBNQMF GSPN UIF QPTUFSJPS TP UIF NFBO BOE TUBOEBSE EFWJBUJPO PG FBDI DPMVNO XJMM CF WFSZ DMPTF UP UIF ."1 WBMVFT GSPN CFGPSF :PV DBO DPOĕSN UIJT CZ TVNNBSJ[JOH UIF TBNQMFT 3 DPEF  +- $.ǿ+*./Ȁ ,0+ +*./ -$*-ǣ ǎǍǍǍǍ .(+' . !-*( (Ǒǡǎ ( ) . ǒǡǒʉ ǖǑǡǒʉ #$./*"-( (0 ǎǒǑǡǓǎ ǍǡǑǎ ǎǒǐǡǖǒ ǎǒǒǡǏǔ ΤΤΤΨΪΥΤΤ .$"( ǔǡǔǏ ǍǡǏǖ ǔǡǏǓ ǕǡǎǕ ΤΤΤΥΨΪΪΦΤΤΤΤ $PNQBSF UIFTF WBMVFT UP UIF PVUQVU GSPN +- $.ǿ(ǑǡǎȀ "OE ZPV DBO VTF +'*/ǿ+*./Ȁ UP TFF IPX NVDI UIFZ SFTFNCMF UIF TBNQMFT GSPN UIF HSJE BQQSPYJNBUJPO JO 'ĶĴłĿIJ ƌƌ QBHF   ćFTF TBNQMFT BMTP QSFTFSWF UIF DPWBSJBODF CFUXFFO µ BOE σ ćJT IBSEMZ NBUUFST SJHIU OPX CFDBVTF µ BOE σ EPOU DPWBSZ BU BMM JO UIJT NPEFM #VU PODF ZPV BEE B QSFEJDUPS WBSJBCMF UP ZPVS NPEFM DPWBSJBODF XJMM NBUUFS B MPU
37. Scaffolds • quap is a scaffold • Forces full specification of model, so you learn it • Works with a very wide class of models • Same as penalized maximum likelihood • Not always a good way to approximate posterior
38. Adding a predictor variable • How does weight describe height? 30 35 40 45 50 55 60 140 150 160 170 180 d2$weight d2$height
39. Adding a predictor variable • Use a linear model of the mean, mu: IJ ∼ /PSNBM(µ, σ) [likelihood] µ ∼ /PSNBM(, ) [µ prior] σ ∼ 6OJGPSN(, ) [σ prior] XFJHIU JOUP B (BVTTJBO NPEFM PG IFJHIU -FU Y CF UIF OBNF GPS UIF TVSFNFOUT Ǐɶ2 $"#/ -FU UIF BWFSBHF PG UIF Y WBMVFT CF ¯ Y iFY CBSw PS WBSJBCMF Y XIJDI JT B MJTU PG NFBTVSFT PG UIF TBNF MFOHUI BT I 5P HFU XF EFĕOF UIF NFBO µ BT B GVODUJPO PG UIF WBMVFT JO Y ćJT JT XIBU JU BUJPO UP GPMMPX IJ ∼ /PSNBM(µJ, σ) [likelihood] µJ = α + β(YJ − ¯ Y) [linear model] α ∼ /PSNBM(, ) [α prior] β ∼ /PSNBM(, ) [β prior] σ ∼ 6OJGPSN(, ) [σ prior] I MJOF PO UIF SJHIUIBOE TJEF CZ UIF UZQF PG EFĕOJUJPO JU FODPEFT 8FMM Z PG UIF EBUB -FUT CFHJO XJUI KVTU UIF QSPCBCJMJUZ PG UIF PCTFSWFE
40. Adding a predictor variable mean when xi = 0 “intercept” change in mean, per unit change xi “slope” weight on row i mean on row i IJ ∼ /PSNBM(µ, σ) [likeliho µ ∼ /PSNBM(, ) [µ pr σ ∼ 6OJGPSN(, ) [σ pr /PX IPX EP XF HFU XFJHIU JOUP B (BVTTJBO NPEFM PG IFJHIU -FU Y CF UIF OBNF GPS U DPMVNO PG XFJHIU NFBTVSFNFOUT Ǐɶ2 $"#/ -FU UIF BWFSBHF PG UIF Y WBMVFT CF ¯ Y iFY CB /PX XF IBWF B QSFEJDUPS WBSJBCMF Y XIJDI JT B MJTU PG NFBTVSFT PG UIF TBNF MFOHUI BT I 5P H 2 $"#/ JOUP UIF NPEFM XF EFĕOF UIF NFBO µ BT B GVODUJPO PG UIF WBMVFT JO Y ćJT JT XIBU MPPLT MJLF XJUI FYQMBOBUJPO UP GPMMPX IJ ∼ /PSNBM(µJ, σ) [likeliho µJ = α + β(YJ − ¯ Y) [linear mod α ∼ /PSNBM(, ) [α pr β ∼ /PSNBM(, ) [β pr σ ∼ 6OJGPSN(, ) [σ pr "HBJO *WF MBCFMFE FBDI MJOF PO UIF SJHIUIBOE TJEF CZ UIF UZQF PG EFĕOJUJPO JU FODPEFT 8F EJTDVTT FBDI JO UVSO  1SPCBCJMJUZ PG UIF EBUB -FUT CFHJO XJUI KVTU UIF QSPCBCJMJUZ PG UIF PCTFSW IFJHIU UIF ĕSTU MJOF PG UIF NPEFM ćJT JT OFBSMZ JEFOUJDBM UP CFGPSF FYDFQU OPX UIFSF JT MJUUMF JOEFY J PO UIF µ BT XFMM BT UIF I :PV DBO SFBE IJ BT iFBDI Iw BOE µJ BT iFBDI µw ć NFBO µ OPX EFQFOET VQPO VOJRVF WBMVFT PO FBDI SPX J 4P UIF MJUUMF J PO µ JOEJDBUFT UI mean weight
41. Prior predictive distribution • What do these priors mean? • Let’s simulate to find out! IJ ∼ /PSNBM(µ, σ) [likelihood] µ ∼ /PSNBM(, ) [µ prior] σ ∼ 6OJGPSN(, ) [σ prior] XFJHIU JOUP B (BVTTJBO NPEFM PG IFJHIU -FU Y CF UIF OBNF GPS UIF TVSFNFOUT Ǐɶ2 $"#/ -FU UIF BWFSBHF PG UIF Y WBMVFT CF ¯ Y iFY CBSw PS WBSJBCMF Y XIJDI JT B MJTU PG NFBTVSFT PG UIF TBNF MFOHUI BT I 5P HFU XF EFĕOF UIF NFBO µ BT B GVODUJPO PG UIF WBMVFT JO Y ćJT JT XIBU JU BUJPO UP GPMMPX IJ ∼ /PSNBM(µJ, σ) [likelihood] µJ = α + β(YJ − ¯ Y) [linear model] α ∼ /PSNBM(, ) [α prior] β ∼ /PSNBM(, ) [β prior] σ ∼ 6OJGPSN(, ) [σ prior] I MJOF PO UIF SJHIUIBOE TJEF CZ UIF UZQF PG EFĕOJUJPO JU FODPEFT 8FMM Z PG UIF EBUB -FUT CFHJO XJUI KVTU UIF QSPCBCJMJUZ PG UIF PCTFSWFE
42. Prior predictive distribution XFJHIU IBT OP SFMBUJPOTIJQ UP IFJHIU 5P ĕHVSF PVU XIBU UIJT QSJPS JNQMJFT MFUT TJNVMBUF‰ UIF QSJPS QSFEJDUJWF TJNVMBUJPO ćFSF JT OP PUIFS XBZ UP VOEFSTUBOE ćF HPBM JT UP TJNVMBUF PCTFSWFE IFJHIUT GSPN UIF NPEFM 'JSTU MFUT DPOTJEFS B SBOHF PG XFJHIU WBMVFT UP TJNVMBUF PWFS ćF SBOHF PG PCTFSWFE XFJHIUT XJMM EP ĕOF ćFO XF OFFE UP TJNVMBUF B CVODI PG MJOFT UIF MJOFT JNQMJFE CZ UIF QSJPST GPS α BOE β )FSFT IPX UP EP JU TFUUJOH B TFFE TP ZPV DBO SFQSPEVDF JU FYBDUMZ 3 DPEF  . /ǡ. ǿǏǖǔǎȀ  ʚǶ ǎǍǍ ȕ ǎǍǍ '$) .  ʚǶ -)*-(ǿ  Ǣ ǎǔǕ Ǣ ǏǍ Ȁ  ʚǶ -)*-(ǿ  Ǣ Ǎ Ǣ ǎǍ Ȁ /PX XF IBWF  QBJST PG α BOE β WBMVFT /PX UP QMPU UIF MJOFT 3 DPEF  +'*/ǿ  Ǣ 3'$(ʙ-)" ǿǏɶ2 $"#/Ȁ Ǣ 4'$(ʙǿǶǎǍǍǢǑǍǍȀ Ǣ 3'ʙǫ2 $"#/ǫ Ǣ 4'ʙǫ# $"#/ǫ Ȁ '$) ǿ #ʙǍ Ǣ '/4ʙǏ Ȁ '$) ǿ #ʙǏǔǏ Ǣ '/4ʙǎ Ǣ '2ʙǍǡǒ Ȁ (/ 3/ǿ ǫ ʡ )*-(ǿǍǢǎǍȀǫ Ȁ 3- ʚǶ ( )ǿǏɶ2 $"#/Ȁ !*- ǿ $ $) ǎǣ Ȁ 0-1 ǿ ȁ$Ȃ ʔ ȁ$Ȃȉǿ3 Ƕ 3-Ȁ Ǣ !-*(ʙ($)ǿǏɶ2 $"#/Ȁ Ǣ /*ʙ(3ǿǏɶ2 $"#/Ȁ Ǣ ʙ Ǣ *'ʙ*'ǡ'+#ǿǫ'&ǫǢǍǡǏȀ Ȁ ćF SFTVMU JT EJTQMBZFE JO 'ĶĴłĿIJ ƌƍ 'PS SFGFSFODF *WF BEEFE B EBTIFE MJOF BU [FSP‰OP POF JT TIPSUFS UIBO [FSP‰BOE UIF i8BEMPXw MJOF BU DN GPS UIF XPSMET UBMMFTU QFSTPO ćF QBUUFSO EPFTOU MPPL MJLF BOZ IVNBO QPQVMBUJPO BU BMM *U FTTFOUJBMMZ TBZT UIBU UIF SFMBUJPOTIJQ CFUXFFO XFJHIU BOE IFJHIU DPVME CF BCTVSEMZ QPTJUJWF PS OFHBUJWF #FGPSF XFWF FWFO TFFO UIF EBUB UIJT JT B CBE NPEFM $BO XF EP CFUUFS 8F DBO EP CFUUFS JNNFEJBUFMZ 8F LOPX UIBU BWFSBHF IFJHIU JODSFBTFT XJUI BWFSBHF 2 $"#/ JOUP UIF NPEFM XF EFĕOF UIF NFBO µ BT B GVODUJPO PG UIF WBMV MPPLT MJLF XJUI FYQMBOBUJPO UP GPMMPX IJ ∼ /PSNBM(µJ, σ) µJ = α + β(YJ − ¯ Y) α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ 6OJGPSN(, ) "HBJO *WF MBCFMFE FBDI MJOF PO UIF SJHIUIBOE TJEF CZ UIF UZQF PG EFĕO EJTDVTT FBDI JO UVSO  1SPCBCJMJUZ PG UIF EBUB -FUT CFHJO XJUI KVTU UIF QSPCB IFJHIU UIF ĕSTU MJOF PG UIF NPEFM ćJT JT OFBSMZ JEFOUJDBM UP CFGPSF MJUUMF JOEFY J PO UIF µ BT XFMM BT UIF I :PV DBO SFBE IJ BT iFBDI Iw BO NFBO µ OPX EFQFOET VQPO VOJRVF WBMVFT PO FBDI SPX J 4P UIF MJUUMF UIF NFBO EFQFOET VQPO UIF SPX
43. Prior predictive distribution XFJHIU IBT OP SFMBUJPOTIJQ UP IFJHIU 5P ĕHVSF PVU XIBU UIJT QSJPS JNQMJFT MFUT TJNVMBUF‰ UIF QSJPS QSFEJDUJWF TJNVMBUJPO ćFSF JT OP PUIFS XBZ UP VOEFSTUBOE ćF HPBM JT UP TJNVMBUF PCTFSWFE IFJHIUT GSPN UIF NPEFM 'JSTU MFUT DPOTJEFS B SBOHF PG XFJHIU WBMVFT UP TJNVMBUF PWFS ćF SBOHF PG PCTFSWFE XFJHIUT XJMM EP ĕOF ćFO XF OFFE UP TJNVMBUF B CVODI PG MJOFT UIF MJOFT JNQMJFE CZ UIF QSJPST GPS α BOE β )FSFT IPX UP EP JU TFUUJOH B TFFE TP ZPV DBO SFQSPEVDF JU FYBDUMZ 3 DPEF  . /ǡ. ǿǏǖǔǎȀ  ʚǶ ǎǍǍ ȕ ǎǍǍ '$) .  ʚǶ -)*-(ǿ  Ǣ ǎǔǕ Ǣ ǏǍ Ȁ  ʚǶ -)*-(ǿ  Ǣ Ǎ Ǣ ǎǍ Ȁ /PX XF IBWF  QBJST PG α BOE β WBMVFT /PX UP QMPU UIF MJOFT 3 DPEF  +'*/ǿ  Ǣ 3'$(ʙ-)" ǿǏɶ2 $"#/Ȁ Ǣ 4'$(ʙǿǶǎǍǍǢǑǍǍȀ Ǣ 3'ʙǫ2 $"#/ǫ Ǣ 4'ʙǫ# $"#/ǫ Ȁ '$) ǿ #ʙǍ Ǣ '/4ʙǏ Ȁ '$) ǿ #ʙǏǔǏ Ǣ '/4ʙǎ Ǣ '2ʙǍǡǒ Ȁ (/ 3/ǿ ǫ ʡ )*-(ǿǍǢǎǍȀǫ Ȁ 3- ʚǶ ( )ǿǏɶ2 $"#/Ȁ !*- ǿ $ $) ǎǣ Ȁ 0-1 ǿ ȁ$Ȃ ʔ ȁ$Ȃȉǿ3 Ƕ 3-Ȁ Ǣ !-*(ʙ($)ǿǏɶ2 $"#/Ȁ Ǣ /*ʙ(3ǿǏɶ2 $"#/Ȁ Ǣ ʙ Ǣ *'ʙ*'ǡ'+#ǿǫ'&ǫǢǍǡǏȀ Ȁ ćF SFTVMU JT EJTQMBZFE JO 'ĶĴłĿIJ ƌƍ 'PS SFGFSFODF *WF BEEFE B EBTIFE MJOF BU [FSP‰OP POF JT TIPSUFS UIBO [FSP‰BOE UIF i8BEMPXw MJOF BU DN GPS UIF XPSMET UBMMFTU QFSTPO ćF QBUUFSO EPFTOU MPPL MJLF BOZ IVNBO QPQVMBUJPO BU BMM *U FTTFOUJBMMZ TBZT UIBU UIF SFMBUJPOTIJQ CFUXFFO XFJHIU BOE IFJHIU DPVME CF BCTVSEMZ QPTJUJWF PS OFHBUJWF #FGPSF XFWF FWFO TFFO UIF EBUB UIJT JT B CBE NPEFM $BO XF EP CFUUFS 8F DBO EP CFUUFS JNNFEJBUFMZ 8F LOPX UIBU BWFSBHF IFJHIU JODSFBTFT XJUI BWFSBHF 2 $"#/ JOUP UIF NPEFM XF EFĕOF UIF NFBO µ BT B GVODUJPO PG UIF WBMV MPPLT MJLF XJUI FYQMBOBUJPO UP GPMMPX IJ ∼ /PSNBM(µJ, σ) µJ = α + β(YJ − ¯ Y) α ∼ /PSNBM(, ) β ∼ /PSNBM(, ) σ ∼ 6OJGPSN(, ) "HBJO *WF MBCFMFE FBDI MJOF PO UIF SJHIUIBOE TJEF CZ UIF UZQF PG EFĕO EJTDVTT FBDI JO UVSO  1SPCBCJMJUZ PG UIF EBUB -FUT CFHJO XJUI KVTU UIF QSPCB IFJHIU UIF ĕSTU MJOF PG UIF NPEFM ćJT JT OFBSMZ JEFOUJDBM UP CFGPSF MJUUMF JOEFY J PO UIF µ BT XFMM BT UIF I :PV DBO SFBE IJ BT iFBDI Iw BO NFBO µ OPX EFQFOET VQPO VOJRVF WBMVFT PO FBDI SPX J 4P UIF MJUUMF UIF NFBO EFQFOET VQPO UIF SPX   (&0$&/53*$ .0%&-4 30 35 40 45 50 55 60 -100 0 100 200 300 400 weight height b ~ dnorm(0,10) 30 35 40 45 50 55 60 -100 0 100 200 300 400 weight height log(b) ~ dnorm(0,1) World's tallest person (272cm) Embryo 'ĶĴłĿIJ ƌƍ 1SJPS QSFEJDUJWF TJNVMBUJPO GPS UIF IFJHIU BOE XFJHIU NPEFM Figure 4.5 World’s tallest person (272cm) Fertilized egg (0cm)
44. Prior predictive distribution %FĕOJOH β BT -PH/PSNBM NFBOT UP DMBJN UIBU UIF MPHBSJUIN PG β IBT B OPSNBM EJT USJCVUJPO 1MBJOMZ β ∼ -PH/PSNBM(, ) 3 QSPWJEFT UIF ')*-( BOE -')*-( EFOTJUJFT GPS XPSLJOH XJUI MPHOPSNBM EJTUSJCVUJPOT :PV DBO TJNVMBUF UIJT SFMBUJPOTIJQ UP TFF XIBU UIJT NFBOT GPS β 3 DPEF   ʚǶ -')*-(ǿ ǎ Ǒ Ǣ Ǎ Ǣ ǎ Ȁ  ).ǿ  Ǣ 3'$(ʙǿǍǢǒȀ Ǣ %ʙǍǡǎ Ȁ *G UIF MPHBSJUIN PG β JT OPSNBM UIFO β JUTFMG JT TUSJDUMZ QPTJUJWF ćF SFBTPO JT UIBU FYQ(Y) JT HSFBUFS UIBO [FSP GPS BOZ SFBM OVNCFS Y ćJT JT UIF SFBTPO UIBU -PH/PSNBM QSJPST BSF DPNNPOQMBDF ćFZ BSF BO FBTZ XBZ UP FOGPSDF QPTJUJWF SFMBUJPOTIJQT 4P XIBU EPFT UIJT FBSO VT %P UIF QSJPS QSFEJDUJWF TJNVMBUJPO BHBJO OPX XJUI UIF -PH/PSNBM QSJPS 3 DPEF  . /ǡ. ǿǏǖǔǎȀ  ʚǶ ǎǍǍ ȕ ǎǍǍ '$) .  ʚǶ -)*-(ǿ  Ǣ ǎǔǕ Ǣ ǏǍ Ȁ  ʚǶ -')*-(ǿ  Ǣ Ǎ Ǣ ǎ Ȁ 1MPUUJOH BT CFGPSF QSPEVDFT UIF SJHIUIBOE QMPU JO 'ĶĴłĿIJ ƌƍ ćJT JT NVDI NPSF TFOTJCMF ćFSF JT TUJMM B SBSF JNQPTTJCMF SFMBUJPOTIJQ #VU OFBSMZ BMM MJOFT JO UIF KPJOU QSJPS GPS α BOE β BSF OPX XJUIJO IVNBO SFBTPO 8FSF GVTTJOH BCPVU UIJT QSJPS FWFO UIPVHI BT ZPVMM TFF JO UIF OFYU TFDUJPO UIFSF JT TP NVDI EBUB JO UIJT FYBNQMF UIBU UIF QSJPST FOE VQ OPU NBUUFSJOH 8F GVTT GPS UXP SFBTPOT 'JSTU UIFSF BSF NBOZ BOBMZTFT JO XIJDI OP BNPVOU PG EBUB NBLFT UIF QSJPS JSSFMFWBOU *O TVDI MPHBSJUINT UIBUT PLBZ *MM TIPX FBDI TUFQ BOE UIFSFT TPNF NPSF EPXO %FĕOJOH β BT -PH/PSNBM NFBOT UP DMBJN UIBU UIF MPHBSJUIN USJCVUJPO 1MBJOMZ β ∼ -PH/PSNBM(, ) 3 QSPWJEFT UIF ')*-( BOE -')*-( EFOTJUJFT GPS XPSLJOH XJUI MPHOP DBO TJNVMBUF UIJT SFMBUJPOTIJQ UP TFF XIBU UIJT NFBOT GPS β 3 DPEF   ʚǶ -')*-(ǿ ǎ Ǒ Ǣ Ǎ Ǣ ǎ Ȁ  ).ǿ  Ǣ 3'$(ʙǿǍǢǒȀ Ǣ %ʙǍǡǎ Ȁ *G UIF MPHBSJUIN PG β JT OPSNBM UIFO β JUTFMG JT TUSJDUMZ QPTJUJWF ć JT HSFBUFS UIBO [FSP GPS BOZ SFBM OVNCFS Y ćJT JT UIF SFBTPO UIBU DPNNPOQMBDF ćFZ BSF BO FBTZ XBZ UP FOGPSDF QPTJUJWF SFMBUJPOT FBSO VT %P UIF QSJPS QSFEJDUJWF TJNVMBUJPO BHBJO OPX XJUI UIF -P 3 DPEF  . /ǡ. ǿǏǖǔǎȀ  ʚǶ ǎǍǍ ȕ ǎǍǍ '$) . 0 1 2 3 4 5 0.0 0.5 1.0 1.5 simulated b values Density
45. Prior predictive distribution 0 1 2 3 4 5 0.0 0.5 1.0 1.5 simulated b values Density *G UIF MPHBSJUIN PG β JT OPSNBM UIFO β JUTFMG JT TUSJDUMZ QPTJUJWF ćF SFBTPO JT UIBU FYQ(Y) JT HSFBUFS UIBO [FSP GPS BOZ SFBM OVNCFS Y ćJT JT UIF SFBTPO UIBU -PH/PSNBM QSJPST BSF DPNNPOQMBDF ćFZ BSF BO FBTZ XBZ UP FOGPSDF QPTJUJWF SFMBUJPOTIJQT 4P XIBU EPFT UIJT FBSO VT %P UIF QSJPS QSFEJDUJWF TJNVMBUJPO BHBJO OPX XJUI UIF -PH/PSNBM QSJPS 3 DPEF  . /ǡ. ǿǏǖǔǎȀ  ʚǶ ǎǍǍ ȕ ǎǍǍ '$) .  ʚǶ -)*-(ǿ  Ǣ ǎǔǕ Ǣ ǏǍ Ȁ  ʚǶ -')*-(ǿ  Ǣ Ǎ Ǣ ǎ Ȁ 1MPUUJOH BT CFGPSF QSPEVDFT UIF SJHIUIBOE QMPU JO 'ĶĴłĿIJ ƌƍ ćJT JT NVDI NPSF TFOTJCMF ćFSF JT TUJMM B SBSF JNQPTTJCMF SFMBUJPOTIJQ #VU OFBSMZ BMM MJOFT JO UIF KPJOU QSJPS GPS α BOE β BSF OPX XJUIJO IVNBO SFBTPO 8FSF GVTTJOH BCPVU UIJT QSJPS FWFO UIPVHI BT ZPVMM TFF JO UIF OFYU TFDUJPO UIFSF JT TP NVDI EBUB JO UIJT FYBNQMF UIBU UIF QSJPST FOE VQ OPU NBUUFSJOH 8F GVTT GPS UXP SFBTPOT 'JSTU UIFSF BSF NBOZ BOBMZTFT JO XIJDI OP BNPVOU PG EBUB NBLFT UIF QSJPS JSSFMFWBOU *O TVDI   (&0$&/53*$ .0%&-4 30 35 40 45 50 55 60 -100 0 100 200 300 400 weight height b ~ dnorm(0,10) 30 35 40 45 50 55 60 -100 0 100 200 300 400 weight height log(b) ~ dnorm(0,1) World's tallest person (272cm) Embryo 'ĶĴłĿIJ ƌƍ 1SJPS QSFEJDUJWF TJNVMBUJPO GPS UIF IFJHIU BOE XFJHIU NPEFM Figure 4.5
46. /PUJDF UIBU UIF MJOFBS NPEFM JO UIF 3 DPEF PO UIF SJHIUIBOE TJEF VTFT UIF 3 BTTJHONFOU PQFSBUPS ʚǶ FWFO UIPVHI UIF NBUIFNBUJDBM EFĕOJUJPO VTFT UIF TZNCPM  ćJT JT B DPEF DPOWFOUJPO TIBSFE CZ TFWFSBM #BZFTJBO NPEFM ĕUUJOH FOHJOFT TP JUT XPSUI HFUUJOH VTFE UP UIF TXJUDI :PV KVTU IBWF UP SFNFNCFS UP VTF ʚǶ JOTUFBE PG ʙ XIFO EFĕOJOH B MJOFBS NPEFM ćBUT JU "OE UIF BCPWF BMMPXT VT UP CVJME UIF QPTUFSJPS BQQSPYJNBUJPO 3 DPEF  ȕ '* / "$)Ǣ .$) $/Ǫ.  '*)" 24 & '$--4ǿ- /#$)&$)"Ȁ /ǿ *2 ''ǎȀ  ʚǶ *2 ''ǎ Ǐ ʚǶ ȁ ɶ" ʛʙ ǎǕ Ǣ Ȃ ȕ  !$) /# 1 -" 2 $"#/Ǣ 3Ƕ- 3- ʚǶ ( )ǿǏɶ2 $"#/Ȁ ȕ !$/ (* ' (Ǒǡǐ ʚǶ ,0+ǿ '$./ǿ # $"#/ ʡ )*-(ǿ (0 Ǣ .$"( Ȁ Ǣ (0 ʚǶ  ʔ ȉǿ 2 $"#/ Ƕ 3- Ȁ Ǣ  ʡ )*-(ǿ ǎǔǕ Ǣ ǏǍ Ȁ Ǣ  ʡ ')*-(ǿ Ǎ Ǣ ǎ Ȁ Ǣ .$"( ʡ 0)$!ǿ Ǎ Ǣ ǒǍ Ȁ Ȁ Ǣ /ʙǏ Ȁ 3FUIJOLJOH &WFSZUIJOH UIBU EFQFOET VQPO QBSBNFUFST IBT B QPTUFSJPS EJTUSJCVUJPO *O UIF NPEFM TUBUJTUJDT JT iQIBDLJOH w UIF QSBDUJDF PG BEKVTUJOH UIF NPEFM ćF EFTJSFE SFTVMU JT VTVBMMZ B QWBMVF MFTT UIFO  ćF QSP NBEF EFQFOEFOU PO UIF PCTFSWFE EBUB UIFO QWBMVFT OP MPO SFTVMUT BSF UP CF FYQFDUFE 8F EPOU QBZ BOZ BUUFOUJPO UP QWBM JG XF DIPPTF PVS QSJPST DPOEJUJPOBM PO UIF PCTFSWFE TBNQMF DFEVSF XFWF QFSGPSNFE JO UIJT DIBQUFS JT UP DIPPTF QSJPST D EBUB‰JUT DPOTUSBJOUT SBOHFT BOE UIFPSFUJDBM SFMBUJPOTIJQT ć JO UIF FBSMJFS TFDUJPO 8F BSF KVEHJOH PVS QSJPST BHBJOTU HFO QFSGPSNT BHBJOTU UIF SFBM EBUB OFYU  'JOEJOH UIF QPTUFSJPS EJTUSJCVUJPO ćF DPEF O B TUSBJHIUGPSXBSE NPEJĕDBUJPO PG UIF LJOE PG DPEF ZP JT JODPSQPSBUF PVS OFX NPEFM GPS UIF NFBO JOUP UIF NP TVSF UP BEE B QSJPS GPS UIF OFX QBSBNFUFS β -FUT SFQF DPSSFTQPOEJOH 3 DPEF PO UIF SJHIUIBOE TJEF IJ ∼ /PSNBM(µJ, σ) µJ = α + β(YJ − ¯ Y) α ∼ /PSNBM(, ) β ∼ -PH/PSNBM(, ) σ ∼ 6OJGPSN(, ) Approximate the posterior
47.  5BCMFT PG NBSHJOBM EJTUSJCVUJPOT 8JUI UIF OFX MJOFBS SFHSFTTJPO USBJOFE PO UIF ,BMBIBSJ EBUB XF JOTQFDU UIF NBSHJOBM QPTUFSJPS EJTUSJCVUJPOT PG UIF QBSBNFUFST 3 DPEF  +- $.ǿ (Ǒǡǐ Ȁ ( ) . ǒǡǒʉ ǖǑǡǒʉ  ǎǒǑǡǓǍ ǍǡǏǔ ǎǒǑǡǎǔ ǎǒǒǡǍǐ  ǍǡǖǍ ǍǡǍǑ ǍǡǕǑ Ǎǡǖǔ .$"( ǒǡǍǔ Ǎǡǎǖ Ǒǡǔǔ ǒǡǐǕ ćF ĕSTU SPX HJWFT UIF RVBESBUJD BQQSPYJNBUJPO GPS α UIF TFDPOE UIF BQQSPYJNBUJPO GPS β BOE UIF UIJSE BQQSPYJNBUJPO GPS σ -FUT USZ UP NBLF TPNF TFOTF PG UIFN -FUT GPDVT PO  β CFDBVTF JUT UIF OFX QBSBNFUFS 4JODF β JT B TMPQF UIF WBMVF  DBO CF SFBE BT B QFSTPO  LH IFBWJFS JT FYQFDUFE UP CF  DN UBMMFS  PG UIF QPTUFSJPS QSPCBCJMJUZ MJFT CFUXFFO  BOE  ćBU TVHHFTUT UIBU β WBMVFT DMPTF UP [FSP PS HSFBUMZ BCPWF POF BSF IJHIMZ JODPNQBUJCMF XJUI UIFTF EBUB BOE UIJT NPEFM *U JT NPTU DFSUBJOMZ OPU FWJEFODF UIBU UIF SFMBUJPOTIJQ CFUXFFO XFJHIU BOE IFJHIU JT MJOFBS CFDBVTF UIF NPEFM POMZ DPOTJEFSFE MJOFT *U KVTU TBZT UIBU JG ZPV BSF DPNNJUUFE UP B MJOF UIFO MJOFT XJUI B TMPQF BSPVOE  BSF QMBVTJCMF POFT 3FNFNCFS UIF OVNCFST JO UIF EFGBVMU +- $. PVUQVU BSFOU TVďDJFOU UP EFTDSJCF UIF RVBESBUJD QPTUFSJPS DPNQMFUFMZ 'PS UIBU XF BMTP SFRVJSF UIF WBSJBODFDPWBSJBODF NBUSJY :PV DBO TFF UIF DPWBSJBODFT BNPOH UIF QBSBNFUFST XJUI 1*1 3 DPEF  -*0)ǿ 1*1ǿ (Ǒǡǐ Ȁ Ǣ ǐ Ȁ 30 35 40 45 50 55 60 140 150 160 170 180 weight height ' Q U J UIF NPEFM TBZT #VU GPS FWFO TMJHIUMZ NPSF DP JOUFSBDUJPO FČFDUT $IBQUFS  JOUFSQSFUJOH Q UIJT UIF QSPCMFN PG JODPSQPSBUJOH UIF JOGPSNBU QMPUT BSF JSSFQMBDFBCMF 8FSF HPJOH UP TUBSU XJUI B TJNQMF WFSTJPO NFBO WBMVFT PWFS UIF IFJHIU BOE XFJHIU EBUB NBUJPO UP UIF QSFEJDUJPO QMPUT VOUJM XFWF VTFE 8FMM TUBSU XJUI KVTU UIF SBX EBUB BOE B TJ Figure 4.6 30 35 40 45 50 55 60 140 150 1 weight heig 'ĶĴłĿIJ ƌƎ )FJHIU JO DFOUJNFUFST WFSUJDBM QMPUUFE BHBJOTU XFJHIU JO LJMPHSBNT IPSJ[PO UBM XJUI UIF MJOF BU UIF QPTUFSJPS NFBO QMPUUFE JO CMBDL UIF NPEFM TBZT #VU GPS FWFO TMJHIUMZ NPSF DPNQMFY NPEFMT FTQFDJBMMZ UIPTF UIBU JODMVEF JOUFSBDUJPO FČFDUT $IBQUFS  JOUFSQSFUJOH QPTUFSJPS EJTUSJCVUJPOT JT IBSE $PNCJOF XJUI UIJT UIF QSPCMFN PG JODPSQPSBUJOH UIF JOGPSNBUJPO JO 1*1 JOUP ZPVS JOUFSQSFUBUJPOT BOE UIF QMPUT BSF JSSFQMBDFBCMF 8FSF HPJOH UP TUBSU XJUI B TJNQMF WFSTJPO PG UIBU UBTL TVQFSJNQPTJOH KVTU UIF QPTUFSJPS NFBO WBMVFT PWFS UIF IFJHIU BOE XFJHIU EBUB ćFO XFMM TMPXMZ BEE NPSF BOE NPSF JOGPS NBUJPO UP UIF QSFEJDUJPO QMPUT VOUJM XFWF VTFE UIF FOUJSF QPTUFSJPS EJTUSJCVUJPO 8FMM TUBSU XJUI KVTU UIF SBX EBUB BOE B TJOHMF MJOF ćF DPEF CFMPX QMPUT UIF SBX EBUB DPNQVUFT UIF QPTUFSJPS NFBO WBMVFT GPS  BOE  UIFO ESBXT UIF JNQMJFE MJOF 3 DPEF  +'*/ǿ # $"#/ ʡ 2 $"#/ Ǣ /ʙǏ Ǣ *'ʙ-)"$Ǐ Ȁ +*./ ʚǶ 3/-/ǡ.(+' .ǿ (Ǒǡǐ Ȁ Ǿ(+ ʚǶ ( )ǿ+*./ɶȀ Ǿ(+ ʚǶ ( )ǿ+*./ɶȀ 0-1 ǿ Ǿ(+ ʔ Ǿ(+ȉǿ3 Ƕ 3-Ȁ Ǣ ʙ Ȁ :PV DBO TFF UIF SFTVMUJOH QMPU JO 'ĶĴłĿIJ ƌƎ &BDI QPJOU JO UIJT QMPU JT B TJOHMF JOEJWJEVBM ćF CMBDL MJOF JT EFĕOFE CZ UIF NFBO TMPQF β BOE NFBO JOUFSDFQU α ćJT JT OPU B CBE MJOF *U DFSUBJOMZ MPPLT IJHIMZ QMBVTJCMF #VU UIFSF BO JOĕOJUF OVNCFS PG PUIFS IJHIMZ QMBVTJCMF MJOFT
48. Sampling from the posterior • Want to get uncertainty onto that graph • Again, sample from posterior 1. Use mean and standard deviation to approximate posterior 2. Sample from multivariate normal distribution of parameters 3. Use samples to generate predictions that “integrate over” the uncertainty
49. Sampling from the posterior   (&0$&/53*$ .0%&-4 ćFO XF DPVME EJTQMBZ UIPTF MJOFT PO UIF QMPU UP WJTVBMJ[F UIF VODFSUBJOUZ JO UIF SFHSFTTJPO SFMBUJPOTIJQ 5P CFUUFS BQQSFDJBUF IPX UIF QPTUFSJPS EJTUSJCVUJPO DPOUBJOT MJOFT XF XPSL XJUI BMM PG UIF TBNQMFT GSPN UIF NPEFM -FUT UBLF B DMPTFS MPPL BU UIF TBNQMFT OPX 3 DPEF  +*./ ʚǶ 3/-/ǡ.(+' .ǿ (Ǒǡǐ Ȁ +*./ȁǎǣǒǢȂ   .$"( ǎ ǎǒǑǡǒǒǍǒ ǍǡǖǏǏǏǐǔǏ ǒǡǎǕǕǓǐǎ Ǐ ǎǒǑǡǑǖǓǒ ǍǡǖǏǕǓǏǏǔ ǒǡǏǔǕǐǔǍ ǐ ǎǒǑǡǑǔǖǑ ǍǡǖǑǖǍǐǏǖ Ǒǡǖǐǔǒǎǐ Ǒ ǎǒǒǡǏǏǕǖ ǍǡǖǏǒǏǍǑǕ ǑǡǕǓǖǕǍǔ ǒ ǎǒǑǡǖǒǑǒ ǍǡǕǎǖǏǒǐǒ ǒǡǍǓǐǓǔǏ &BDI SPX JT B DPSSFMBUFE SBOEPN TBNQMF GSPN UIF KPJOU QPTUFSJPS PG BMM UISFF QBSBNFUFST VTJOH UIF DPWBSJBODFT QSPWJEFE CZ 1*1ǿ(ǑǡǐȀ ćF QBJSFE WBMVFT PG  BOE  PO FBDI SPX EFĕOF B MJOF ćF BWFSBHF PG WFSZ NBOZ PG UIFTF MJOFT JT UIF QPTUFSJPS NFBO MJOF #VU UIF TDBUUFS BSPVOE UIBU BWFSBHF JT NFBOJOHGVM CFDBVTF JU BMUFST PVS DPOĕEFODF JO UIF SFMBUJPOTIJQ CFUXFFO UIF QSFEJDUPS BOE UIF PVUDPNF 4P OPX MFUT EJTQMBZ B CVODI PG UIFTF MJOFT TP ZPV DBO TFF UIF TDBUUFS ćJT MFTTPO XJMM CF FBTJFS UP BQQSFDJBUF JG XF VTF POMZ TPNF PG UIF EBUB UP CFHJO ćFO ZPV DBO TFF IPX BEEJOH JO NPSF EBUB DIBOHFT UIF TDBUUFS PG UIF MJOFT 4P XFMM CFHJO XJUI KVTU UIF ĕSTU  DBTFT JO Ǐ ćF GPMMPXJOH DPEF FYUSBDUT UIF ĕSTU  DBTFT BOE SFFTUJNBUFT UIF NPEFM 3 DPEF   ʚǶ ǎǍ
50. Posterior is full of lines  "%%*/( " 13&%*$503  30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 150 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 352 Figure 4.7 ćFO XF DPVME EJTQMBZ UIPTF MJOFT PO UIF QM SFMBUJPOTIJQ 5P CFUUFS BQQSFDJBUF IPX UIF QPTUFSJPS UIF TBNQMFT GSPN UIF NPEFM -FUT UBLF B DMP 3 DPEF  +*./ ʚǶ 3/-/ǡ.(+' .ǿ (Ǒǡǐ Ȁ +*./ȁǎǣǒǢȂ   .$"( ǎ ǎǒǑǡǒǒǍǒ ǍǡǖǏǏǏǐǔǏ ǒǡǎǕǕǓǐǎ Ǐ ǎǒǑǡǑǖǓǒ ǍǡǖǏǕǓǏǏǔ ǒǡǏǔǕǐǔǍ ǐ ǎǒǑǡǑǔǖǑ ǍǡǖǑǖǍǐǏǖ Ǒǡǖǐǔǒǎǐ Ǒ ǎǒǒǡǏǏǕǖ ǍǡǖǏǒǏǍǑǕ ǑǡǕǓǖǕǍǔ ǒ ǎǒǑǡǖǒǑǒ ǍǡǕǎǖǏǒǐǒ ǒǡǍǓǐǓǔǏ &BDI SPX JT B DPSSFMBUFE SBOEPN TBNQMF GSPN UIF DPWBSJBODFT QSPWJEFE CZ 1*1ǿ(ǑǡǐȀ ć MJOF ćF BWFSBHF PG WFSZ NBOZ PG UIFTF MJOFT UIBU BWFSBHF JT NFBOJOHGVM CFDBVTF JU BMUFST QSFEJDUPS BOE UIF PVUDPNF 4P OPX MFUT EJTQMBZ B CVODI PG UIFTF MJO FBTJFS UP BQQSFDJBUF JG XF VTF POMZ TPNF PG JO NPSF EBUB DIBOHFT UIF TDBUUFS PG UIF MJOF ćF GPMMPXJOH DPEF FYUSBDUT UIF ĕSTU  DBTF 3 DPEF   ʚǶ ǎǍ  ʚǶ Ǐȁ ǎǣ Ǣ Ȃ ( ʚǶ ,0+ǿ '$./ǿ # $"#/ ʡ )*-(ǿ (0 Ǣ .$"( Ȁ
51. Posterior is full of lines  "%%*/( " 13&%*$503  30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 150 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 352 Figure 4.7
52. Posterior is full of lines  "%%*/( " 13&%*$503  30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 150 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 352 Figure 4.7
53. Posterior is full of lines  "%%*/( " 13&%*$503  30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 150 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 352 Figure 4.7