Statistical Rethinking - Lecture 04

Statistical Rethinking: A Bayesian Course with R Examples, Lecture 04, Linear Models

Transcript

1. Week 2: Linear Models Richard McElreath Statistical Rethinking
2. TQPOEJOH 3 DPEF PO UIF SJHIUIBOE TJEF IJ ∼ /PSNBM(µJ, σ) # $"#/ ʋ )*-(ǭ(0ǐ.$"(Ǯ µJ = α + βYJ (0 ʄǤ  ɾ Ƿ2 $"#/ α ∼ /PSNBM(, )  ʋ )*-(ǭƼǀǁǐƼƻƻǮ β ∼ /PSNBM(, )  ʋ )*-(ǭƻǐƼƻǮ σ ∼ 6OJGPSN(, ) .$"( ʋ 0)$!ǭƻǐǀƻǮ F UIBU UIF MJOFBS NPEFM JO UIF 3 DPEF PO UIF SJHIUIBOE TJEF VTFT UIF 3 BTTJHONFOU UPS ʄǤ FWFO UIPVHI UIF NBUIFNBUJDBM EFĕOJUJPO VTFT UIF TZNCPM  ćJT JT B DPEF FOUJPO TIBSFE CZ TFWFSBM #BZFTJBO NPEFM ĕUUJOH FOHJOFT TP JUT XPSUI HFUUJOH VTFE UP XJUDI :PV KVTU IBWF UP SFNFNCFS UP VTF ʄǤ JOTUFBE PG ʃ XIFO EFĕOJOH B MJOFBS NPEFM JU "OE UIF BCPWF BMMPXT VT UP CVJME UIF ."1 NPEFM ĕU  / "$)ǐ .$) $/ǘ.  '*)" 24 & -4ǭ- /#$)&$)"Ǯ ǭ *2 ''ƼǮ *2 ''Ƽ Ǥ ǯ ɠ" ʅʃ Ƽǃ ǐ ǰ / (* ' ʄǤ (+ǭ '$./ǭ # $"#/ ʋ )*-(ǭ (0 ǐ .$"( Ǯ ǐ /PUJDF UIBU UIF MJOFBS NPEFM JO UIF 3 DPEF PO UIF SJHIUIBOE TJEF VTFT UIF 3 BTTJHONFOU PQFSBUPS ʄǤ FWFO UIPVHI UIF NBUIFNBUJDBM EFĕOJUJPO VTFT UIF TZNCPM  ćJT JT B DPEF DPOWFOUJPO TIBSFE CZ TFWFSBM #BZFTJBO NPEFM ĕUUJOH FOHJOFT TP JUT XPSUI HFUUJOH VTFE UP UIF TXJUDI :PV KVTU IBWF UP SFNFNCFS UP VTF ʄǤ JOTUFBE PG ʃ XIFO EFĕOJOH B MJOFBS NPEFM ćBUT JU "OE UIF BCPWF BMMPXT VT UP CVJME UIF ."1 NPEFM ĕU 3 DPEF  ȃ '* / "$)ǐ .$) $/ǘ.  '*)" 24 & '$--4ǭ- /#$)&$)"Ǯ /ǭ *2 ''ƼǮ  ʄǤ *2 ''Ƽ ƽ ʄǤ ǯ ɠ" ʅʃ Ƽǃ ǐ ǰ ȃ !$/ (* ' (ƿǏƾ ʄǤ (+ǭ '$./ǭ # $"#/ ʋ )*-(ǭ (0 ǐ .$"( Ǯ ǐ (0 ʄǤ  ɾ Ƿ2 $"#/ ǐ  ʋ )*-(ǭ Ƽǀǁ ǐ Ƽƻƻ Ǯ ǐ  ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ .$"( ʋ 0)$!ǭ ƻ ǐ ǀƻ Ǯ Ǯ ǐ /ʃƽ Ǯ ćF QBSBNFUFS (0 JT OP MPOHFS SFBMMZ B QBSBNFUFS IFSF CFDBVTF JU IBT CFFO SFQMBDFE CZ UIF MJOFBS NPEFM ɾǷ2 $"#/ XIFSF  JT α BOE  JT β BOE 2 $"#/ JT PG DPVSTF PVS Y JO UIJT JOTUBODF 4P UIFSF JT B QSJPS GPS UIF QBSBNFUFS  OPX CVU OPU POF GPS (0 TJODF (0 JT EFĕOFE CZ UIF MJOFBS NPEFM JOTUFBE *O UIF ./-/ MJTU (0 JT SFQMBDFE CZ  XIJDI TUBSUT BU UIF PWFSBMM NFBO KVTU MJLF (0 VTFE
3. Figure 4.5 PG QBSBNFUFS WBMVFT EFTDSJCF UIF SFMBUJWF DPNQBUJCJMJUZ PG EJČFSFOU TUBUFT PG UIF XPSME XJUI UIF EBUB BD DPSEJOH UP UIF NPEFM ćFTF BSF TNBMM XPSME $IBQUFS  OVNCFST 4P SFBTPOBCMF QFPQMF NBZ EJTBHSFF BCPVU UIF MBSHF XPSME NFBOJOH BOE UIF EFUBJMT PG UIPTF EJTBHSFFNFOUT EFQFOE TUSPOHMZ VQPO DPOUFYU 4VDI EJTBHSFFNFOUT BSF QSPEVDUJWF CFDBVTF UIFZ MFBE UP NPEFM DSJUJDJTN BOE SFWJTJPO TPNFUIJOH UIBU HPMFNT DBOOPU EP GPS UIFNTFMWFT  5BCMFT PG FTUJNBUFT 8JUI UIF OFX MJOFBS SFHSFTTJPO ĕU UP UIF ,BMBIBSJ EBUB XF JOTQFDU UIF FTUJNBUFT 3 DPEF  +- $.ǭ (ƿǏƾ Ǯ  ) / 1 ƽǏǀɳ DŽǂǏǀɳ  ƼƼƾǏǃDŽ ƼǏDŽƼ ƼƼƻǏƼǁ ƼƼǂǏǁƾ .$"( ǀǏƻǂ ƻǏƼDŽ ƿǏǂƻ ǀǏƿǀ  ƻǏDŽƻ ƻǏƻƿ ƻǏǃƽ ƻǏDŽDŽ ćF ĕSTU SPX HJWFT UIF RVBESBUJD BQQSPYJNBUJPO GPS α UIF TFDPOE UIF BQQSPYJNBUJPO GPS σ BOE UIF UIJSE BQQSPYJNBUJPO GPS β -FUT USZ UP NBLF TPNF TFOTF PG UIFN JO UIJT WFSZ TJNQMF NPEFM #FTU UP CFHJO GSPN UIF CPUUPN XJUI  β CFDBVTF JUT UIF OFX QBSBNFUFS 4JODF β JT B TMPQF UIF WBMVF  DBO CF SFBE BT B QFSTPO  LH IFBWJFS JT FYQFDUFE UP CF  DN UBMMFS  PG UIF QPTUFSJPS QSPCBCJMJUZ MJFT CFUXFFO  BOE  ćBU TVHHFTUT UIBU β WBMVFT DMPTF UP [FSP PS HSFBUMZ BCPWF POF BSF IJHIMZ JODPNQBUJCMF XJUI UIFTF EBUB BOE UIJT NPEFM *G ZPV XFSF UIJOLJOH UIBU QFSIBQT UIFSF XBT OP SFMBUJPOTIJQ BU BMM CFUXFFO IFJHIU BOE XFJHIU UIFO UIJT FTUJNBUF CFDPNFT TUSPOH FWJEFODF PG B QPTJUJWF SFMBUJPOTIJQ JOTUFBE #VU NBZCF ZPV KVTU XBOUFE BT QSFDJTF B NFBTVSFNFOU BT QPTTJCMF PG UIF SFMBUJPOTIJQ CFUXFFO IFJHIU BOE XFJHIU ćJT FTUJNBUF FNCPEJFT UIBU NFBTVSFNFOU DPOEJUJPOBM PO UIF NPEFM BT BMXBZT 'PS B EJČFSFOU NPEFM UIF NFBTVSF PG UIF SFMBUJPOTIJQ NJHIU CF EJČFSFOU 30 35 40 45 50 55 60 140 150 160 170 180 weight height SFMBUJWF QMBVTJCJMJUZ UP FBDI ćJT NFBOT QSPCBCJMJUZ *U DPVME CF UIBU UIFSF BSF NB BT UIF ."1 MJOF 0S JU DPVME CF JOTUFBE U UIF ."1 MJOF
4. Sampling from the posterior • Want to get uncertainty onto that graph • Again, sample from posterior 1. Use MAP and standard deviation to approximate posterior 2. Sample from multivariate normal distribution of parameters 3. Use samples to generate predictions that “integrate over” the uncertainty
5. Sampling from the posterior correlation matrix • MAP estimates, standard deviations, and correlation matrix define multivariate normal posterior distribution • Can easily sample from this distribution  QFSDFOUJMF JOUFSWBM TBZT UIBU  PG IFJHIUT MJF XJUIJO DN UP DN PG UIF NFBO *O PUIFS XPSET  PG UIF QMBVTJCMF WBMVFT PG σ MJF CFUXFFO  BOE  ćF OVNCFST JO UIF EFGBVMU +- $. PVUQVU BSFOU TVďDJFOU UP EFTDSJCF UIF RVBESBUJD QPT UFSJPS DPNQMFUFMZ 'PS UIBU XF BMTP SFRVJSF UIF WBSJBODFDPWBSJBODF NBUSJY 8FSF JOUFSFTUFE JO DPSSFMBUJPOT BNPOH QBSBNFUFST‰XF BMSFBEZ IBWF UIFJS WBSJBODF JO UIF UBCMF BCPWF‰TP MFUT HP TUSBJHIU UP UIF DPSSFMBUJPO NBUSJY 3 DPEF  +- $.ǭ (ƿǏƾ ǐ *--ʃ Ǯ  ) / 1 ƽǏǀɳ DŽǂǏǀɳ  .$"(   ƼƼƾǏǃDŽ ƼǏDŽƼ ƼƼƻǏƼǁ ƼƼǂǏǁƾ ƼǏƻƻ ƻ ǤƻǏDŽDŽ .$"( ǀǏƻǂ ƻǏƼDŽ ƿǏǂƻ ǀǏƿǀ ƻǏƻƻ Ƽ ƻǏƻƻ  ƻǏDŽƻ ƻǏƻƿ ƻǏǃƽ ƻǏDŽDŽ ǤƻǏDŽDŽ ƻ ƼǏƻƻ ćF OFX DPMVNOT PO UIF GBS SJHIU TIPX UIF DPSSFMBUJPOT BNPOH UIF QBSBNFUFST ćJT JT UIF TBNF JOGPSNBUJPO ZPVE HFU CZ VTJOH *1ƽ*-ǭ1*1ǭ(ƿǏƾǮǮ /PUJDF UIBU α BOE β BSF BM NPTU QFSGFDUMZ OFHBUJWFMZ DPSSFMBUFE 3JHIU OPX UIJT JT IBSNMFTT *U KVTU NFBOT UIBU UIFTF UXP QBSBNFUFST DBSSZ UIF TBNF JOGPSNBUJPO‰BT ZPV DIBOHF UIF TMPQF PG UIF MJOF UIF CFTU JOUFSDFQU DIBOHFT UP NBUDI JU #VU JO NPSF DPNQMFY NPEFMT TUSPOH DPSSFMBUJPOT MJLF UIJT DBO NBLF JU EJďDVMU UP ĕU UIF NPEFM UP UIF EBUB 4P XFMM XBOU UP VTF TPNF HPMFN FOHJOFFSJOH USJDLT UP BWPJE JU XIFO QPTTJCMF ćF ĕSTU USJDL JT İIJĻŁIJĿĶĻĴ $FOUFSJOH JT UIF QSPDFEVSF PG TVCUSBDUJOH UIF NFBO PG B WBSJBCMF GSPN FBDI WBMVF 5P DSFBUF B DFOUFSFE WFSTJPO PG UIF XFJHIU WBSJBCMF 3 DPEF  ƽɠ2 $"#/Ǐ ʄǤ ƽɠ2 $"#/ Ǥ ( )ǭƽɠ2 $"#/Ǯ :PV DBO DPOĕSN UIBU UIF BWFSBHF WBMVF PG 2 $"#/Ǐ JT [FSP ( )ǭƽɠ2 $"#/ǏǮ /PX MFUT SFĕU UIF NPEFM BOE TFF XIBU UIJT HBJOT VT 3 DPEF
6. ćF FTUJNBUF PG α  JO UIF +- $. UBCMF JOEJDBUFT UIBU B QFSTPO PG XFJHIU  TIPVME CF DN UBMM ćJT JT OPOTFOTF TJODF SFBM QFPQMF BMXBZT IBWF QPTJUJWF XFJHIU ZFU JU JT BMTP USVF 1BSBNFUFST MJLF α BSF iJOUFSDFQUTw UIBU UFMM VT UIF WBMVF PG µ XIFO BMM PG UIF QSFEJDUPS WBSJBCMFT IBWF WBMVF [FSP "T B DPOTFRVFODF UIF WBMVF PG UIF JOUFSDFQU JT GSFRVFOUMZ VOJOUFSQSFUBCMF XJUIPVU BMTP TUVEZJOH BOZ β QBSBNFUFST 'JOBMMZ UIF FTUJNBUF GPS σ .$"( JOGPSNT VT PG UIF XJEUI PG UIF EJTUSJCVUJPO PG IFJHIUT BSPVOE UIF NFBO " RVJDL XBZ UP JOUFSQSFU JU JT UP SFDBMM UIBU BCPVU  PG UIF QSPCBCJMJUZ JO B (BVTTJBO EJTUSJCVUJPO MJFT CFUXFFO UXP TUBOEBSE EFWJBUJPOT 4P JO UIJT DBTF UIF FTUJNBUF UFMMT VT UIBU JU JT NPTU MJLFMZ UIBU  PG IFJHIUT MJF XJUIJO DN σ PG UIF NFBO IFJHIU #VU UIFSF JT BMTP VODFSUBJOUZ BCPVU UIJT BT BMXBZT 4P XF DBO BMTP UBLF OPUF PG UIF GBDU UIBU UIF  QFSDFOUJMF JOUFSWBM TBZT UIBU  PG IFJHIUT MJF XJUIJO DN UP DN PG UIF NFBO *O PUIFS XPSET  PG UIF QMBVTJCMF WBMVFT PG σ MJF CFUXFFO  BOE  ćF OVNCFST JO UIF EFGBVMU +- $. PVUQVU BSFOU TVďDJFOU UP EFTDSJCF UIF RVBESBUJD QPT UFSJPS DPNQMFUFMZ 'PS UIBU XF BMTP SFRVJSF UIF WBSJBODFDPWBSJBODF NBUSJY 8FSF JOUFSFTUFE JO DPSSFMBUJPOT BNPOH QBSBNFUFST‰XF BMSFBEZ IBWF UIFJS WBSJBODF JO UIF UBCMF BCPWF‰TP MFUT HP TUSBJHIU UP UIF DPSSFMBUJPO NBUSJY 3 DPEF  +- $.ǭ (ƿǏƾ ǐ *--ʃ Ǯ  ) / 1 ƽǏǀɳ DŽǂǏǀɳ  .$"(   ƼƼƾǏǃDŽ ƼǏDŽƼ ƼƼƻǏƼǁ ƼƼǂǏǁƾ ƼǏƻƻ ƻ ǤƻǏDŽDŽ .$"( ǀǏƻǂ ƻǏƼDŽ ƿǏǂƻ ǀǏƿǀ ƻǏƻƻ Ƽ ƻǏƻƻ  ƻǏDŽƻ ƻǏƻƿ ƻǏǃƽ ƻǏDŽDŽ ǤƻǏDŽDŽ ƻ ƼǏƻƻ ćF OFX DPMVNOT PO UIF GBS SJHIU TIPX UIF DPSSFMBUJPOT BNPOH UIF QBSBNFUFST ćJT JT UIF TBNF JOGPSNBUJPO ZPVE HFU CZ VTJOH *1ƽ*-ǭ1*1ǭ(ƿǏƾǮǮ /PUJDF UIBU α BOE β BSF BM NPTU QFSGFDUMZ OFHBUJWFMZ DPSSFMBUFE 3JHIU OPX UIJT JT IBSNMFTT *U KVTU NFBOT UIBU UIFTF UXP QBSBNFUFST DBSSZ UIF TBNF JOGPSNBUJPO‰BT ZPV DIBOHF UIF TMPQF PG UIF MJOF UIF CFTU
7. Sampling from the posterior UIF ."1 MJOF 4P IPX DBO XF HFU UIBU VODFSUBJOUZ POUP UIF QMPU 5PHFUIFS B DPNCJOBUJPO PG α BOE β EFĕOF B MJOF "OE TP XF DPVME TBNQMF B CVODI PG MJOFT GSPN UIF QPTUFSJPS EJTUSJCVUJPO ćFO XF DPVME EJTQMBZ UIPTF MJOFT PO UIF QMPU UP WJTVBMJ[F UIF VODFSUBJOUZ JO UIF SFHSFTTJPO SFMBUJPOTIJQ 5P CFUUFS BQQSFDJBUF IPX UIF QPTUFSJPS EJTUSJCVUJPO DPOUBJOT MJOFT FYUSBDU TPNF TBNQMFT GSPN UIF NPEFM 3 DPEF  +*./ ʄǤ 3/-/Ǐ.(+' .ǭ (ƿǏƾ Ǯ ćFO JOTQFDU UIF ĕSTU  SPXT PG UIF TBNQMFT 3 DPEF  +*./ǯƼǑǀǐǰ  .$"(  Ƽ ƼƼǀǏƼDŽǁƿ ƿǏDŽDŽƽƽǁǂ ƻǏǃǂǂǁƾDŽƾ ƽ ƼƼƼǏƻƾǃDŽ ǀǏƼǁDŽǀƼǀ ƻǏDŽǂǀǃǀǀƿ ƾ ƼƼǀǏƿǃƾƾ ǀǏƼƾƾƿǁƾ ƻǏǃǂƽǁǂǀǂ ƿ ƼƻDŽǏǁƿǃǃ ǀǏƻƻǀǃƾǂ ƻǏDŽǃƼƽǁDŽƽ ǀ ƼƼƽǏƿǁƾǂ ƿǏǁǂǃƾƼƿ ƻǏDŽƾǃƿǃƼƿ &BDI SPX JT B DPSSFMBUFE SBOEPN TBNQMF GSPN UIF KPJOU QPTUFSJPS PG BMM UISFF QBSBNFUFST VTJOH UIF DPWBSJBODFT QSPWJEFE CZ 1*1ǭ(ƿǏƾǮ ćF QBJSFE WBMVFT PG  BOE  PO FBDI SPX EFĕOF B MJOF ćF BWFSBHF PG WFSZ NBOZ PG UIFTF MJOFT JT UIF ."1 MJOF #VU UIF TDBUUFS BSPVOE UIBU BWFSBHF JT NFBOJOHGVM CFDBVTF JU BMUFST PVS DPOĕEFODF JO UIF SFMBUJPOTIJQ CFUXFFO UIF QSFEJDUPS BOE UIF PVUDPNF 4P OPX MFUT EJTQMBZ B CVODI PG UIFTF MJOFT TP ZPV DBO TFF UIF TDBUUFS ćJT MFTTPO XJMM CF FBTJFS UP BQQSFDJBUF JG XF VTF POMZ TPNF PG UIF EBUB UP CFHJO ćFO ZPV DBO TFF IPX BEEJOH JO NPSF EBUB DIBOHFT UIF TDBUUFS PG UIF MJOFT 4P XFMM CFHJO XJUI KVTU UIF ĕSTU  DBTFT JO ƽ UIF ."1 MJOF 4P IPX DBO XF HFU UIBU VODFSUBJOUZ POUP UIF QMPU 5PHFUIFS B DPNCJOBUJPO PG α BOE β EFĕOF B MJOF "OE TP XF DPVME TBNQMF B CVODI PG MJOFT GSPN UIF QPTUFSJPS EJTUSJCVUJPO ćFO XF DPVME EJTQMBZ UIPTF MJOFT PO UIF QMPU UP WJTVBMJ[F UIF VODFSUBJOUZ JO UIF SFHSFTTJPO SFMBUJPOTIJQ 5P CFUUFS BQQSFDJBUF IPX UIF QPTUFSJPS EJTUSJCVUJPO DPOUBJOT MJOFT FYUSBDU TPNF TBNQMFT GSPN UIF NPEFM 3 DPEF  +*./ ʄǤ 3/-/Ǐ.(+' .ǭ (ƿǏƾ Ǯ ćFO JOTQFDU UIF ĕSTU  SPXT PG UIF TBNQMFT 3 DPEF  +*./ǯƼǑǀǐǰ  .$"(  Ƽ ƼƼǀǏƼDŽǁƿ ƿǏDŽDŽƽƽǁǂ ƻǏǃǂǂǁƾDŽƾ ƽ ƼƼƼǏƻƾǃDŽ ǀǏƼǁDŽǀƼǀ ƻǏDŽǂǀǃǀǀƿ ƾ ƼƼǀǏƿǃƾƾ ǀǏƼƾƾƿǁƾ ƻǏǃǂƽǁǂǀǂ ƿ ƼƻDŽǏǁƿǃǃ ǀǏƻƻǀǃƾǂ ƻǏDŽǃƼƽǁDŽƽ ǀ ƼƼƽǏƿǁƾǂ ƿǏǁǂǃƾƼƿ ƻǏDŽƾǃƿǃƼƿ &BDI SPX JT B DPSSFMBUFE SBOEPN TBNQMF GSPN UIF KPJOU QPTUFSJPS PG BMM UISFF QBSBNFUFST VTJOH UIF DPWBSJBODFT QSPWJEFE CZ 1*1ǭ(ƿǏƾǮ ćF QBJSFE WBMVFT PG  BOE  PO FBDI SPX EFĕOF B MJOF ćF BWFSBHF PG WFSZ NBOZ PG UIFTF MJOFT JT UIF ."1 MJOF #VU UIF TDBUUFS BSPVOE UIBU BWFSBHF JT NFBOJOHGVM CFDBVTF JU BMUFST PVS DPOĕEFODF JO UIF SFMBUJPOTIJQ CFUXFFO UIF QSFEJDUPS BOE UIF PVUDPNF 4P OPX MFUT EJTQMBZ B CVODI PG UIFTF MJOFT TP ZPV DBO TFF UIF TDBUUFS ćJT MFTTPO XJMM CF FBTJFS UP BQQSFDJBUF JG XF VTF POMZ TPNF PG UIF EBUB UP CFHJO ćFO ZPV DBO TFF IPX BEEJOH JO NPSF EBUB DIBOHFT UIF TDBUUFS PG UIF MJOFT 4P XFMM CFHJO XJUI KVTU UIF ĕSTU  DBTFT JO ƽ ćF GPMMPXJOH DPEF FYUSBDUT UIF ĕSTU  DBTFT BOE SFFTUJNBUFT UIF NPEFM
8. Posterior is full of lines  "%%*/( " 13&%*$503  30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 150 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 352 GSPN UIF NPEFM +*./ ʄǤ 3/-/Ǐ.(+' .ǭ (ƿǏƾ Ǯ ćFO JOTQFDU UIF ĕSTU  SPXT PG UIF TBNQMFT +*./ǯƼǑǀǐǰ  .$"(  Ƽ ƼƼǀǏƼDŽǁƿ ƿǏDŽDŽƽƽǁǂ ƻǏǃǂǂǁƾDŽƾ ƽ ƼƼƼǏƻƾǃDŽ ǀǏƼǁDŽǀƼǀ ƻǏDŽǂǀǃǀǀƿ ƾ ƼƼǀǏƿǃƾƾ ǀǏƼƾƾƿǁƾ ƻǏǃǂƽǁǂǀǂ ƿ ƼƻDŽǏǁƿǃǃ ǀǏƻƻǀǃƾǂ ƻǏDŽǃƼƽǁDŽƽ ǀ ƼƼƽǏƿǁƾǂ ƿǏǁǂǃƾƼƿ ƻǏDŽƾǃƿǃƼƿ &BDI SPX JT B DPSSFMBUFE SBOEPN TBNQMF GSPN VTJOH UIF DPWBSJBODFT QSPWJEFE CZ 1*1ǭ(ƿǏƾǮ EFĕOF B MJOF ćF BWFSBHF PG WFSZ NBOZ PG UIFTF M UIBU BWFSBHF JT NFBOJOHGVM CFDBVTF JU BMUFST PVS QSFEJDUPS BOE UIF PVUDPNF 4P OPX MFUT EJTQMBZ B CVODI PG UIFTF MJOFT T FBTJFS UP BQQSFDJBUF JG XF VTF POMZ TPNF PG UIF E JO NPSF EBUB DIBOHFT UIF TDBUUFS PG UIF MJOFT 4P ćF GPMMPXJOH DPEF FYUSBDUT UIF ĕSTU  DBTFT BOE Figure 4.6
9. Posterior is full of lines  "%%*/( " 13&%*$503  30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 150 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 352 Figure 4.6
10. Predict mu 3 DPEF  +*./ ʄǤ .Ǐ/Ǐ!-( ǭ (1-)*-(ǭ Ƽ ƿ ǐ (0ʃ* !ǭ(ƿǏƾǮ ǐ $"(ʃ1*1ǭ(ƿǏƾǮ Ǯ Ǯ 8IBU ZPV FOE VQ XJUI JO UIF TZNCPM +*./ JT UIPVTBOE SPXT FBDI B TBNQMF BOE UISFF DPMVNOT FBDI B QBSBNFUFS 5BLF B MPPL BU UIF ĕSTU  SPXT PG UIF SFTVMUJOH QPTUFSJPS TBNQMFT 3 DPEF  +*./ǯƼǑƼƻǐǰ  .$"(  Ƽ ƼƼǀǏƼDŽǁƿ ƿǏDŽDŽƽƽǁǂ ƻǏǃǂǂǁƾDŽƾ ƽ ƼƼƼǏƻƾǃDŽ ǀǏƼǁDŽǀƼǀ ƻǏDŽǂǀǃǀǀƿ ƾ ƼƼǀǏƿǃƾƾ ǀǏƼƾƾƿǁƾ ƻǏǃǂƽǁǂǀǂ ƿ ƼƻDŽǏǁƿǃǃ ǀǏƻƻǀǃƾǂ ƻǏDŽǃƼƽǁDŽƽ ǀ ƼƼƽǏƿǁƾǂ ƿǏǁǂǃƾƼƿ ƻǏDŽƾǃƿǃƼƿ ǁ ƼƼƿǏƼƿƻƼ ƿǏǁǀDŽƾƻDŽ ƻǏǃDŽƿƻƾƼǂ ǂ ƼƼǀǏǃǃƻƿ ƿǏǂƽDŽǃƽDŽ ƻǏǃǀǂƾǁǃƻ ǃ ƼƼƿǏƻƿǁǃ ǀǏƻƽƼDŽƿƽ ƻǏǃDŽƼDŽǂƽƻ DŽ ƼƼƽǏǁƼǃƻ ƿǏǃǁƻDŽƾǃ ƻǏDŽƾǁǁǁƽƻ Ƽƻ ƼƼƽǏDŽƻǁƿ ƿǏǃǃƾǁǃǃ ƻǏDŽƾƿƻǀǃǃ &BDI SPX JT B DPSSFMBUFE SBOEPN TBNQMF GSPN UIF KPJOU QPTUFSJPS PG BMM UISFF QB SBNFUFST VTJOH UIF DPWBSJBODFT ZPV QSPWJEFE GSPN 1*1ǭ(ƿǏƾǮ :PV DBO QMPU UIF WBMVFT JO +*./ OPX UP WJTVBMJ[F UIF DPWBSJBODF BNPOH UIF QBSBNFUFS FTUJNBUFT * EP UIJT JO 'ĶĴłĿIJ ƌƎ #Z OPX ZPV TIPVME CF DPOĕEFOU JO IPX UP BDDPNQMJTI TVDI B QMPU PO ZPVS PXO TP * XPOU TIPX BOZ DPEF UP IFMQ ZPV "MM ZPV OFFE BSF UIF DPMVNOT JO +*./ BOE UIF +'*/ GVODUJPO JO 3 )FSFT IPX UP QMPU B  DPOĕEFODF JOUFSWBM BSPVOE UIF TMPQF UIBU JODPSQP SBUFT VODFSUBJOUZ JO CPUI UIF TMPQF β BOE JOUFSDFQU α BU UIF TBNF UJNF 'PDVT GPS UIF NPNFOU PO B TJOHMF 2 $"#/ WBMVF TBZ  LJMPHSBNT :PV DBO RVJDLMZ NBLF B  4BNQMFT GSPN UIF RVBESBUJD BQQSPYJNBUF QPTUFSJPS O GPS UIF IFJHIUXFJHIU NPEFM (ƿǏƾ &BDI QPJOU JT QMF GSPN UIF QPTUFSJPS BOE UIF DSPTT JO FBDI QMPU JT TUJNBUF &TUJNBUFT PG UIF TMPQF  BOE UIF JOUFSDFQU HMZ OFHBUJWFMZ DPSSFMBUFE MFę XIJMF FTUJNBUFT PG UIF IF TUBOEBSE EFWJBUJPO .$"( BSF FTTFOUJBMMZ VODPSSF  ćF JOUFSDFQU BOE .$"( BSF BMTP VODPSSFMBUFE OPU QSFEJDUFE NFBOT GPS  LJMPHSBNT CZ VTJOH ZPVS TBNQMFT GSPN 3 DPEF  ./ɠ Ƿ ǀƻ IU PG UIF ʄǤ BCPWF UBLFT JUT GPSN GSPN UIF FRVBUJPO GPS µJ µJ = α + βYJ. IJT DBTF JT  (P BIFBE BOE UBLF B MPPL JOTJEF UIF SFTVMU (0 JDUFE NFBOT POF GPS FBDI SBOEPN TBNQMF GSPN UIF QPTUFSJPS XFOU JOUP DPNQVUJOH FBDI UIF WBSJBUJPO BDSPTT UIPTF NFBOT JO 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 150 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 352 'ĶĴłĿIJ ƌƎ 4BNQMFT GSPN UIF RVBESBUJD BQQSPYJNBUF QPTUFSJPS EJTUSJCV UJPO GPS UIF IFJHIUXFJHIU NPEFM (ƿǏƾ XJUI JODSFBTJOH BNPVOUT PG EBUB *O FBDI QMPU  MJOFT TBNQMFE GSPN UIF QPTUFSJPS EJTUSJCVUJPO TIPXJOH UIF VODFSUBJOUZ JO UIF SFHSFTTJPO SFMBUJPOTIJQ 3 DPEF  (0Ǭ/Ǭǀƻ ʄǤ +*./ɠ ɾ +*./ɠ Ƿ ǀƻ ćF DPEF UP UIF SJHIU PG UIF ʄǤ BCPWF UBLFT JUT GPSN GSPN UIF FRVBUJPO GPS µJ  µJ = α + βYJ. ćF WBMVF PG YJ JO UIJT DBTF JT  (P BIFBE BOE UBLF B MPPL JOTJEF UIF SFTVMU (0Ǭ/Ǭǀƻ *UT B
11. Predict mu Figure 4.7 'ĶĴłĿIJ ƌƎ 4BNQMFT GSPN UIF RVBESBUJD BQQSPYJNBUF QPTUFSJPS EJTUSJCV UJPO GPS UIF IFJHIUXFJHIU NPEFM (ƿǏƾ XJUI JODSFBTJOH BNPVOUT PG EBUB *O FBDI QMPU  MJOFT TBNQMFE GSPN UIF QPTUFSJPS EJTUSJCVUJPO TIPXJOH UIF VODFSUBJOUZ JO UIF SFHSFTTJPO SFMBUJPOTIJQ 3 DPEF  (0Ǭ/Ǭǀƻ ʄǤ +*./ɠ ɾ +*./ɠ Ƿ ǀƻ ćF DPEF UP UIF SJHIU PG UIF ʄǤ BCPWF UBLFT JUT GPSN GSPN UIF FRVBUJPO GPS µJ  µJ = α + βYJ. ćF WBMVF PG YJ JO UIJT DBTF JT  (P BIFBE BOE UBLF B MPPL JOTJEF UIF SFTVMU (0Ǭ/Ǭǀƻ *UT B WFDUPS PG QSFEJDUFE NFBOT POF GPS FBDI SBOEPN TBNQMF GSPN UIF QPTUFSJPS 4JODF KPJOU  BOE  XFOU JOUP DPNQVUJOH FBDI UIF WBSJBUJPO BDSPTT UIPTF NFBOT JODPSQPSBUFT UIF VODFSUBJOUZ JO BOE DPSSFMBUJPO CFUXFFO CPUI QBSBNFUFST *U NJHIU CF IFMQGVM BU UIJT QPJOU UP BDUVBMMZ QMPU UIF EFOTJUZ GPS UIJT WFDUPS PG NFBOT   -*/&"3 .0%&-4 158.0 159.0 160.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 mu|weight=50 Density 'ĶĴłĿIJ ƌƏ ć UFSJPS EJTUSJCV XFJHIU JT L UIF SFMBUJWF Q UIF NFBO 3 DPEF  ).ǭ (0Ǭ/Ǭǀƻ ǐ *'ʃ-)"$ƽ ǐ '2ʃƽ ǐ 3'ʃǙ(0Ǵ2 $"
12. Predict every mu /PX XIBU DBO XF EP XJUI UIJT CJH NBUSJY -PUT PG UIJOHT ćF GVODUJPO '$)& QSPWJEFT B QPTUFSJPS EJTUSJCVUJPO PG µ GPS FBDI DBTF XF GFFE JU 4P BCPWF XF IBWF B EJTUSJCVUJPO PG µ GPS FBDI JOEJWJEVBM JO UIF PSJHJOBM EBUB 8F BDUVBMMZ XBOU TPNFUIJOH TMJHIUMZ EJČFSFOU B EJTUSJCVUJPO PG µ GPS FBDI VOJRVF XFJHIU WBMVF PO UIF IPSJ[POUBM BYJT *UT POMZ TMJHIUMZ IBSEFS UP DPNQVUF UIBU CZ KVTU QBTTJOH '$)& TPNF OFX EBUB 3 DPEF  ȃ  !$) . ,0 ) *! 2 $"#/. /* *(+0/ +- $/$*). !*- ȃ /# . 1'0 . 2$''  *) /# #*-$5*)/' 3$. 2 $"#/Ǐ. , ʄǤ . ,ǭ !-*(ʃƽǀ ǐ /*ʃǂƻ ǐ 4ʃƼ Ǯ ȃ 0. '$)& /* *(+0/ (0 ȃ !*- # .(+' !-*( +*./ -$*- ȃ ) !*- # 2 $"#/ $) 2 $"#/Ǐ. , (0 ʄǤ '$)&ǭ (ƿǏƾ ǐ /ʃ'$./ǭ2 $"#/ʃ2 $"#/Ǐ. ,Ǯ Ǯ ./-ǭ(0Ǯ )0( ǯƼǑƼƻƻƻǐ ƼǑƿǁǰ Ƽƾǂ Ƽƾǁ Ƽƾǂ Ƽƾǂ Ƽƾǁ ǏǏǏ "OE OPX UIFSF BSF POMZ  DPMVNOT JO (0 CFDBVTF XF GFE JU  EJČFSFOU WBMVFT GPS 2 $"#/ 5P WJTVBMJ[F XIBU ZPVWF HPU IFSF MFUT QMPU UIF EJTUSJCVUJPO PG µ WBMVFT BU FBDI IFJHIU PO UIF QMPU 3 DPEF  ȃ 0. /4+ ʃǙ)Ǚ /* #$ -2 / +'*/ǭ # $"#/ ʋ 2 $"#/ ǐ ƽ ǐ /4+ ʃǙ)Ǚ Ǯ ȃ '**+ *1 - .(+' . ) +'*/ # (0 1'0 !*- ǭ $ $) ƼǑƼƻƻ Ǯ +*$)/.ǭ 2 $"#/Ǐ. , ǐ (0ǯ$ǐǰ ǐ +#ʃƼǁ ǐ *'ʃ*'Ǐ'+#ǭ-)"$ƽǐƻǏƼǮ Ǯ
13. How link works ćJT SFDJQF XPSLT GPS FWFSZ NPEFM XF ĕU JO UIF CPPL "T MPOH BT ZPV LOPX IPX QBSBNFUFST SFMBUF UP UIF EBUB ZPV DBO VTF TBNQMFT GSPN UIF QPTUFSJPS UP EFTDSJCF BOZ BTQFDU PG UIF NPEFMT CFIBWJPS 3FUIJOLJOH 0WFSDPOĕEFOU DPOĕEFODF JOUFSWBMT ćF DPOĕEFODF JOUFSWBM GPS UIF SFHSFTTJPO MJOF JO 'ĶĴłĿIJ ƌƐ DMJOHT UJHIUMZ UP UIF ."1 MJOF ćVT UIFSF JT WFSZ MJUUMF VODFSUBJOUZ BCPVU UIF BWFSBHF IFJHIU BT B GVODUJPO PG BWFSBHF XFJHIU #VU ZPV IBWF UP LFFQ JO NJOE UIBU UIFTF JOGFSFODFT BSF BMXBZT DPOEJUJPOBM PO UIF NPEFM &WFO B WFSZ CBE NPEFM DBO IBWF WFSZ UJHIU DPOĕEFODF JOUFSWBMT *U NBZ IFMQ JG ZPV UIJOL PG UIF SFHSFTTJPO MJOF JO 'ĶĴłĿIJ ƌƐ BT TBZJOH DPOEJUJPOBM PO UIF BTTVNQUJPO UIBU IFJHIU BOE XFJHIU BSF SFMBUFE CZ B TUSBJHIU MJOF UIFO UIJT JT UIF NPTU QMBVTJCMF MJOF BOE UIFTF BSF JUT QMBVTJCMF CPVOET 0WFSUIJOLJOH )PX '$)& XPSLT ćF GVODUJPO '$)& JT OPU SFBMMZ WFSZ TPQIJTUJDBUFE "MM JU JT EPJOH JT VTJOH UIF GPSNVMB ZPV QSPWJEFE XIFO ZPV ĕU UIF NPEFM UP DPNQVUF UIF WBMVF PG UIF MJOFBS NPEFM *U EPFT UIJT GPS FBDI TBNQMF GSPN UIF QPTUFSJPS EJTUSJCVUJPO GPS FBDI DBTF JO UIF EBUB :PV DPVME BDDPNQMJTI UIF TBNF UIJOH GPS BOZ NPEFM ĕU CZ BOZ NFBOT CZ QFSGPSNJOH UIFTF TUFQT ZPVSTFMG ćJT JT IPX JUE MPPL GPS (ƿǏƾ 3 DPEF  +*./ ʄǤ 3/-/Ǐ.(+' .ǭ(ƿǏƾǮ (0Ǐ'$)& ʄǤ !0)/$*)ǭ2 $"#/Ǯ +*./ɠ ɾ +*./ɠǷ2 $"#/ 2 $"#/Ǐ. , ʄǤ . ,ǭ !-*(ʃƽǀ ǐ /*ʃǂƻ ǐ 4ʃƼ Ǯ (0 ʄǤ .++'4ǭ 2 $"#/Ǐ. , ǐ (0Ǐ'$)& Ǯ (0Ǐ( ) ʄǤ ++'4ǭ (0 ǐ ƽ ǐ ( ) Ǯ (0Ǐ  ʄǤ ++'4ǭ (0 ǐ ƽ ǐ  Ǯ "OE UIF WBMVFT JO (0Ǐ( ) BOE (0Ǐ  TIPVME CF WFSZ TJNJMBS BMMPXJOH GPS TJNVMBUJPO WBSJBODF UP XIBU ZPV HPU UIF BVUPNBUFE XBZ VTJOH '$)& ,OPXJOH UIJT NBOVBM NFUIPE JT VTFGVM CPUI GPS  VOEFSTUBOEJOH BOE  TIFFS QPXFS 8IBU FWFS UIF NPEFM ZPV ĕOE ZPVSTFMG XJUI UIJT BQQSPBDI DBO CF VTFE UP HFOFSBUF QPTUFSJPS QSFEJDUJPOT GPS • Sample from posterior • Define series of predictor (weight) values • For each predictor value • For each sample from posterior • Compute mu: a + b*weight • Summarize
14. 'data.frame': 10000 obs. of 3 variables: $ a : num 115 116 114 111 111 ... $ b : num 0.885 0.871 0.9 0.989 0.981 ... $ sigma: num 5.07 5.23 5.3 4.84 5.07 ... > post <- extract.samples(m4.3) > str(post) > mu.link <- function(weight) post$a + post$b*weight > str( mu.link(50) ) num [1:10000] 159 159 159 160 160 ... > weight.seq <- seq( from=25 , to=70 , by=1 ) > str( weight.seq ) num [1:46] 25 26 27 28 29 30 31 32 33 34 ... > mu <- sapply( weight.seq , mu.link ) > str( mu ) num [1:10000, 1:46] 137 138 136 135 135 ... > mu.mean <- apply( mu , 2 , mean ) > str( mu.mean ) num [1:46] 137 137 138 139 140 ... 1. sample from posterior 2. define link function 3. define weight values to compute predictions for 4. compute prediction for each sample in posterior, for each weight value 5. summarize
15. sapply (simplified apply) 1:10 [1] 1 2 3 4 5 6 7 8 9 10 sapply( 1:10 , function(z) z^2 ) [1] 1 4 9 16 25 36 49 64 81 100 sapply( 1:10 , function(z) prod(1:z)^(1/z) ) [1] 1.000000 1.414214 1.817121 2.213364 2.605171 2.993795 [7] 3.380015 3.764351 4.147166 4.528729
16. 'data.frame': 10000 obs. of 3 variables: $ a : num 115 116 114 111 111 ... $ b : num 0.885 0.871 0.9 0.989 0.981 ... $ sigma: num 5.07 5.23 5.3 4.84 5.07 ... > post <- extract.samples(m4.3) > str(post) > mu.link <- function(weight) post$a + post$b*weight > str( mu.link(50) ) num [1:10000] 159 159 159 160 160 ... > weight.seq <- seq( from=25 , to=70 , by=1 ) > str( weight.seq ) num [1:46] 25 26 27 28 29 30 31 32 33 34 ... > mu <- sapply( weight.seq , mu.link ) > str( mu ) num [1:10000, 1:46] 137 138 136 135 135 ... > mu.mean <- apply( mu , 2 , mean ) > str( mu.mean ) num [1:46] 137 137 138 139 140 ... 1. sample from posterior 2. define link function 3. define weight values to compute predictions for 4. compute prediction for each sample in posterior, for each weight value 5. summarize
17. Figure 4.8 30 35 40 45 50 55 60 140 150 160 170 180 weight height 30 35 40 45 50 55 60 140 150 160 170 180 weight height 'ĶĴłĿIJ ƌƐ -Fę ćF ĕSTU  WBMVFT JO UIF EJTUSJCVUJPO PG µ BU FBDI XFJHIU WBMVF 3JHIU ćF ,VOH IFJHIU EBUB BHBJO OPX XJUI  )1%* PG UIF NFBO JOEJDBUFE CZ UIF TIBEFE SFHJPO $PNQBSF UIJT SFHJPO UP UIF EJTUSJCVUJPOT PG CMVF QPJOUT PO UIF MFę +'*/ǭ # $"#/ ʋ 2 $"#/ ǐ /ʃƽ ǐ *'ʃ*'Ǐ'+#ǭ-)"$ƽǐƻǏǀǮ Ǯ ȃ +'*/ /#  '$) ǐ & /# ( ) (0 !*- # 2 $"#/ UP DPNQVUF UIBU CZ KVTU QBTTJOH '$)& TPNF OFX EBUB 3 DPEF  ȃ  !$) . ,0 ) *! 2 $"#/. /* *(+0/ +- $/$*). !*- ȃ /# . 1'0 . 2$''  *) /# #*-$5*)/' 3$. 2 $"#/Ǐ. , ʄǤ . ,ǭ !-*(ʃƽǀ ǐ /*ʃǂƻ ǐ 4ʃƼ Ǯ ȃ 0. '$)& /* *(+0/ (0 ȃ !*- # .(+' !-*( +*./ -$*- ȃ ) !*- # 2 $"#/ $) 2 $"#/Ǐ. , (0 ʄǤ '$)&ǭ (ƿǏƾ ǐ /ʃ'$./ǭ2 $"#/ʃ2 $"#/Ǐ. ,Ǯ Ǯ ./-ǭ(0Ǯ )0( ǯƼǑƼƻƻƻǐ ƼǑƿǁǰ Ƽƾǂ Ƽƾǁ Ƽƾǂ Ƽƾǂ Ƽƾǁ ǏǏǏ "OE OPX UIFSF BSF POMZ  DPMVNOT JO (0 CFDBVTF XF GFE JU  EJČFSFOU WBMVFT GPS 2 $"#/ 5P WJTVBMJ[F XIBU ZPVWF HPU IFSF MFUT QMPU UIF EJTUSJCVUJPO PG µ WBMVFT BU FBDI IFJHIU PO UIF QMPU 3 DPEF  ȃ 0. /4+ ʃǙ)Ǚ /* #$ -2 / +'*/ǭ # $"#/ ʋ 2 $"#/ ǐ ƽ ǐ /4+ ʃǙ)Ǚ Ǯ ȃ '**+ *1 - .(+' . ) +'*/ # (0 1'0 !*- ǭ $ $) ƼǑƼƻƻ Ǯ +*$)/.ǭ 2 $"#/Ǐ. , ǐ (0ǯ$ǐǰ ǐ +#ʃƼǁ ǐ *'ʃ*'Ǐ'+#ǭ-)"$ƽǐƻǏƼǮ Ǯ ćF SFTVMU JT TIPXO PO UIF MFęIBOE TJEF PG 'ĶĴłĿIJ ƌƐ "U FBDI XFJHIU WBMVF JO 2 $"#/Ǐ. , B QJMF PG DPNQVUFE µ WBMVFT BSF TIPXO &BDI PG UIFTF QJMFT JT B (BVTTJBO EJTUSJCVUJPO MJLF UIBU JO 'ĶĴłĿIJ ƌƏ :PV DBO TFF OPX UIBU UIF BNPVOU PG VODFSUBJOUZ JO µ EFQFOET VQPO UIF WBMVF PG 2 $"#/ "OE UIJT JT UIF TBNF GBDU ZPV TBX JO UIF SJHIUIBOE QMPU JO 'ĶĴłĿIJ ƌƎ ćF ĕOBM TUFQ JT UP TVNNBSJ[F UIF EJTUSJCVUJPO GPS FBDI XFJHIU WBMVF 8FMM VTF ++'4
18. Figure 4.8 30 35 40 45 50 55 60 140 150 160 170 180 weight height 30 35 40 45 50 55 60 140 150 160 170 180 weight height 'ĶĴłĿIJ ƌƐ -Fę ćF ĕSTU  WBMVFT JO UIF EJTUSJCVUJPO PG µ BU FBDI XFJHIU WBMVF 3JHIU ćF ,VOH IFJHIU EBUB BHBJO OPX XJUI  )1%* PG UIF NFBO JOEJDBUFE CZ UIF TIBEFE SFHJPO $PNQBSF UIJT SFHJPO UP UIF EJTUSJCVUJPOT PG CMVF QPJOUT PO UIF MFę +'*/ǭ # $"#/ ʋ 2 $"#/ ǐ /ʃƽ ǐ *'ʃ*'Ǐ'+#ǭ-)"$ƽǐƻǏǀǮ Ǯ ȃ +'*/ /#  '$) ǐ & /# ( ) (0 !*- # 2 $"#/ 5P WJTVBMJ[F XIBU ZPVWF HPU IFSF MFUT QMPU UIF EJTUSJCVU QMPU ȃ 0. /4+ ʃǙ)Ǚ /* #$ -2 / +'*/ǭ # $"#/ ʋ 2 $"#/ ǐ ƽ ǐ /4+ ʃǙ)Ǚ Ǯ ȃ '**+ *1 - .(+' . ) +'*/ # (0 1'0 !*- ǭ $ $) ƼǑƼƻƻ Ǯ +*$)/.ǭ 2 $"#/Ǐ. , ǐ (0ǯ$ǐǰ ǐ +#ʃƼǁ ǐ *'ʃ ćF SFTVMU JT TIPXO PO UIF MFęIBOE TJEF PG 'ĶĴłĿIJ ƌƐ B QJMF PG DPNQVUFE µ WBMVFT BSF TIPXO &BDI PG UIFTF UIBU JO 'ĶĴłĿIJ ƌƏ :PV DBO TFF OPX UIBU UIF BNPVOU P WBMVF PG 2 $"#/ "OE UIJT JT UIF TBNF GBDU ZPV TBX JO U ćF ĕOBM TUFQ JT UP TVNNBSJ[F UIF EJTUSJCVUJPO GPS XIJDI BQQMJFT B GVODUJPO PG ZPVS DIPJDF UP B NBUSJY ȃ .0((-$5 /# $./-$0/$*) *! (0 (0Ǐ( ) ʄǤ ++'4ǭ (0 ǐ ƽ ǐ ( ) Ǯ (0Ǐ  ʄǤ ++'4ǭ (0 ǐ ƽ ǐ  Ǯ 3FBE ++'4ǭ(0ǐƽǐ( )Ǯ BT DPNQVUF UIF NFBO PG FBDI (0 /PX (0Ǐ( ) DPOUBJOT UIF BWFSBHF µ BU FBDI XFJH MPXFS BOE VQQFS CPVOET GPS FBDI XFJHIU WBMVF #F TVS (0Ǐ  UP EFNZTUJGZ UIFN ćFZ BSF KVTU EJČFSFOU LJO JO (0 XJUI FBDI DPMVNO CFJOH GPS B EJČFSFOU XFJHIU WB :PV DBO QMPU UIFTF TVNNBSJFT PO UPQ PG UIF EBUB X ȃ +'*/ -2 / ȃ !$)" *0/ +*$)/. /* (& '$) ) $)/ -1' ( QMPU 3 DPEF  ȃ 0. /4+ ʃǙ)Ǚ /* #$ -2 / +'*/ǭ # $"#/ ʋ 2 $"#/ ǐ ƽ ǐ /4+ ʃǙ)Ǚ Ǯ ȃ '**+ *1 - .(+' . ) +'*/ # (0 1'0 !*- ǭ $ $) ƼǑƼƻƻ Ǯ +*$)/.ǭ 2 $"#/Ǐ. , ǐ (0ǯ$ǐǰ ǐ +#ʃƼǁ ǐ *'ʃ*'Ǐ'+#ǭ-)"$ƽǐƻǏƼǮ Ǯ ćF SFTVMU JT TIPXO PO UIF MFęIBOE TJEF PG 'ĶĴłĿIJ ƌƐ "U FBDI XFJHIU WBMVF JO 2 $"#/Ǐ. , B QJMF PG DPNQVUFE µ WBMVFT BSF TIPXO &BDI PG UIFTF QJMFT JT B (BVTTJBO EJTUSJCVUJPO MJLF UIBU JO 'ĶĴłĿIJ ƌƏ :PV DBO TFF OPX UIBU UIF BNPVOU PG VODFSUBJOUZ JO µ EFQFOET VQPO UIF WBMVF PG 2 $"#/ "OE UIJT JT UIF TBNF GBDU ZPV TBX JO UIF SJHIUIBOE QMPU JO 'ĶĴłĿIJ ƌƎ ćF ĕOBM TUFQ JT UP TVNNBSJ[F UIF EJTUSJCVUJPO GPS FBDI XFJHIU WBMVF 8FMM VTF ++'4 XIJDI BQQMJFT B GVODUJPO PG ZPVS DIPJDF UP B NBUSJY 3 DPEF  ȃ .0((-$5 /# $./-$0/$*) *! (0 (0Ǐ( ) ʄǤ ++'4ǭ (0 ǐ ƽ ǐ ( ) Ǯ (0Ǐ  ʄǤ ++'4ǭ (0 ǐ ƽ ǐ  Ǯ 3FBE ++'4ǭ(0ǐƽǐ( )Ǯ BT DPNQVUF UIF NFBO PG FBDI DPMVNO EJNFOTJPO iw PG UIF NBUSJY (0 /PX (0Ǐ( ) DPOUBJOT UIF BWFSBHF µ BU FBDI XFJHIU WBMVF BOE (0Ǐ  DPOUBJOT  MPXFS BOE VQQFS CPVOET GPS FBDI XFJHIU WBMVF #F TVSF UP UBLF B MPPL JOTJEF (0Ǐ( ) BOE (0Ǐ  UP EFNZTUJGZ UIFN ćFZ BSF KVTU EJČFSFOU LJOET PG TVNNBSJFT PG UIF EJTUSJCVUJPOT JO (0 XJUI FBDI DPMVNO CFJOH GPS B EJČFSFOU XFJHIU WBMVF :PV DBO QMPU UIFTF TVNNBSJFT PO UPQ PG UIF EBUB XJUI B GFX MJOFT PG 3 DPEF 3 DPEF  ȃ +'*/ -2 / ȃ !$)" *0/ +*$)/. /* (& '$) ) $)/ -1' (*- 1$.$' 30 35 40 45 50 55 60 140 150 160 170 weight height 30 35 40 45 50 55 60 140 150 160 170 weight height 'ĶĴłĿIJ ƌƐ -Fę ćF ĕSTU  WBMVFT JO UIF EJTUSJCVUJPO PG µ BU FBDI XFJHIU WBMVF 3JHIU ćF ,VOH IFJHIU EBUB BHBJO OPX XJUI  )1%* PG UIF NFBO JOEJDBUFE CZ UIF TIBEFE SFHJPO $PNQBSF UIJT SFHJPO UP UIF EJTUSJCVUJPOT PG CMVF QPJOUT PO UIF MFę +'*/ǭ # $"#/ ʋ 2 $"#/ ǐ /ʃƽ ǐ *'ʃ*'Ǐ'+#ǭ-)"$ƽǐƻǏǀǮ Ǯ ȃ +'*/ /#  '$) ǐ & /# ( ) (0 !*- # 2 $"#/ '$) .ǭ 2 $"#/Ǐ. , ǐ (0Ǐ( ) Ǯ ȃ +'*/  .#  - "$*) !*- DŽǀɳ  .# ǭ (0Ǐ  ǐ 2 $"#/Ǐ. , Ǯ
19. 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 20 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 100 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 200 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 350
20. 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 10 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 20 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 50 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 100 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 200 30 35 40 45 50 55 60 140 150 160 170 180 weight height N = 350
21. Prediction intervals, too • What about predicted heights, not just mean height? • Uncertainty from posterior and uncertainty from Gaussian process => distribution of predicted height • Could use rnorm to simulate sampling — like your Chapter 3 homework • Can automate with sim   -*/&"3 .0%&-4 8IBU ZPVWF EPOF TP GBS JT KVTU VTF TBNQMFT GSPN UIF QPTUFSJPS UP WJTVBMJ[F UIF VODFSUB JO µJ UIF MJOFBS NPEFM PG UIF NFBO #VU BDUVBM QSFEJDUJPOT PG IFJHIUT EFQFOE BMTP VQPO TUPDIBTUJD EFĕOJUJPO JO UIF ĕSTU MJOF ćF (BVTTJBO EJTUSJCVUJPO PO UIF ĕSTU MJOF UFMMT VT UIF NPEFM FYQFDUT PCTFSWFE IFJHIUT UP CF EJTUSJCVUFE BSPVOE µ OPU SJHIU PO UPQ PG JU UIF TQSFBE BSPVOE µ JT HPWFSOFE CZ σ "MM PG UIJT TVHHFTUT XF OFFE UP JODPSQPSBUF σ JO QSFEJDUJPOT TPNFIPX )FSFT IPX ZPV EP JU *NBHJOF TJNVMBUJOH IFJHIUT 'PS BOZ VOJRVF XFJHIU WBMVF TBNQMF GSPN B (BVTTJBO EJTUSJCVUJPO XJUI UIF DPSSFDU NFBO µ GPS UIBU XFJHIU VTJOH UIF SFDU WBMVF PG σ TBNQMFE GSPN UIF TBNF QPTUFSJPS EJTUSJCVUVPO *G ZPV EP UIJT GPS FWFSZ TBN GSPN UIF QPTUFSJPS GPS FWFSZ XFJHIU WBMVF PG JOUFSFTU ZPV FOE VQ XJUI B DPMMFDUJPO PG T MBUFE IFJHIUT UIBU FNCPEZ UIF VODFSUBJOUZ JO UIF QPTUFSJPS BT XFMM BT UIF VODFSUBJOUZ JO (BVTTJBO MJLFMJIPPE 3 DPEF  .$(Ǐ# $"#/ ʄǤ .$(ǭ (ƿǏƾ ǐ /ʃ'$./ǭ2 $"#/ʃ2 $"#/Ǐ. ,Ǯ Ǯ ./-ǭ.$(Ǐ# $"#/Ǯ )0( ǯƼǑƼƻƻƻǐ ƼǑƿǁǰ ƼƾDŽ Ƽƿƿ ƼƿƼ Ƽƿƻ Ƽƾƻ ǏǏǏ ćJT NBUSJY JT NVDI MJLF UIF FBSMJFS POF (0 CVU JU DPOUBJOT TJNVMBUFE IFJHIUT OPU EJTUS UJPOT PG QMBVTJCMF BWFSBHF IFJHIU µ 8F DBO TVNNBSJ[F UIFTF TJNVMBUFE IFJHIUT JO UIF TBNF XBZ XF TVNNBSJ[FE UIF E
22. Figure 4.9 95% prediction intervals Nothing special about 95% Try 50%, 80%, 99% Interested in shape, not boundaries  "%%*/( " 13&%*$503 30 35 40 45 50 55 60 140 150 160 170 180 weight height 'ĶĴłĿIJ ƌƑ  QSFEJDUJPO J IFJHIU BT B GVODUJPO PG XFJHIU MJOF JT UIF ."1 FTUJNBUF PG UIF N BU FBDI XFJHIU ćF UXP TIB TIPX EJČFSFOU  QMBVTJCMF SF OBSSPX TIBEFE JOUFSWBM BSPVOE UI EJTUSJCVUJPO PG µ ćF XJEFS TI SFQSFTFOUT UIF SFHJPO XJUIJO XIJD FYQFDUT UP ĕOE  PG BDUVBM IF QPQVMBUJPO BU FBDI XFJHIU UIF SFBEFS +VTU HP CBDL UP UIF DPEF BCPWF BOE BEE +-*ʃƻǏǃ GPS FYBNQMF UP UI ćBU XJMM HJWF ZPV  JOUFSWBMT JOTUFBE PG  POFT
23. How sim works   -*/&"3 .0%&-4 (BVTTJBO EJTUSJCVUJPO UIF DPNQBOJPO JT -)*-( BOE JU TJNVMBUFT TBNQMJOH GSPN B UJPO 8IBU XF XBOU 3 UP EP TJNVMBUF B IFJHIU GPS FBDI TFU PG TBNQMFT BOE UP EP UI XFJHIU ćF GPMMPXJOH XJMM EP JU 3 DPEF  +*./ ʄǤ 3/-/Ǐ.(+' .ǭ(ƿǏƾǮ 2 $"#/Ǐ. , ʄǤ ƽǀǑǂƻ .$(Ǐ# $"#/ ʄǤ .++'4ǭ 2 $"#/Ǐ. , ǐ !0)/$*)ǭ2 $"#/Ǯ -)*-(ǭ )ʃ)-*2ǭ+*./Ǯ ǐ ( )ʃ+*./ɠ ɾ +*./ɠǷ2 $"#/ ǐ .ʃ+*./ɠ.$"( Ǯ Ǯ # $"#/Ǐ ʄǤ ++'4ǭ .$(Ǐ# $"#/ ǐ ƽ ǐ  Ǯ ćF WBMVFT JO # $"#/Ǐ XJMM CF QSBDUJDBMMZ JEFOUJDBM UP UIF POFT DPNQVUFE JO EJTQMBZFE JO 'ĶĴłĿIJ ƌƑ • For each weight • For each sample from posterior • Simulate a height: rnorm(n,mu,sigma)
24. Polynomial regression • Linear trends can make absurd predictions • Some relationships obviously not linear 30 40 50 60 70 80 130 150 170 190 weight height 0 20 40 60 80 60 100 140 180 age height
25. Polynomial regression • Purely descriptive (geocentric) strategy: use polynomial of predictor variable ɠ (' Ǒ $)/ Ƽ ƻ ƻ Ƽ ƻ Ƽ ƻ Ƽ ƻ Ƽ ǏǏǏ (P BIFBE BOE QMPU # $"#/ BHBJOTU 2 $"#/ ćF SFMBUJPOTIJQ JT WJTJCMZ DVSWF UIBU XFWF JODMVEFE UIF OPOBEVMU JOEJWJEVBMT ćFSF BSF NBOZ XBZT UP NPEFM B DVSWFE SFMBUJPOTIJQ CFUXFFO UXP WBS )FSF *MM TIPX ZPV B WFSZ DPNNPO POF ĽļĹņĻļĺĶĮĹ ĿIJĴĿIJŀŀĶļĻ *O UI UFYU iQPMZOPNJBMw NFBOT FRVBUJPOT GPS µJ UIBU BEE BEEJUJPOBM UFSNT XJUI TR DVCFT BOE FWFO IJHIFS QPXFST PG UIF QSFEJDUPS WBSJBCMF ćFSFT TUJMM POMZ PO EJDUPS WBSJBCMF JO UIF NPEFM TP UIJT JT TUJMM B CJWBSJBUF SFHSFTTJPO #VU UIF EFĕ PG µJ IBT NPSF QBSBNFUFST OPX )FSFT UIF NPTU DPNNPO QPMZOPNJBM SFHSFTTJPO B QBSBCPMJD NPEFM NFBO µJ = α + β YJ + β Y J ćF BCPWF JT B QBSBCPMJD TFDPOE PSEFS QPMZOPNJBM ćF α+β YJ QBSU JT UI MJOFBS GVODUJPO PG Y JO B MJOFBS SFHSFTTJPO KVTU XJUI B MJUUMF iw TVCTDSJQU BE UIFQBSBNFUFSOBNF TP XFDBOUFMM JUBQBSUGSPNUIFOFX QBSBNFUFS ćFBEE UFSN VTFT UIF TRVBSF PG YJ UP DPOTUSVDU B QBSBCPMB SBUIFS UIBO B QFSGFDUMZ T MJOF ćF OFX QBSBNFUFS β NFBTVSFT UIF DVSWBUVSF PG UIF SFMBUJPOTIJQ 1st order (line): (P BIFBE BOE QMPU # $"#/ BHBJOTU 2 $"#/ ćF SFMBUJPOTIJQ JT WJTJCMZ DVSWF UIBU XFWF JODMVEFE UIF OPOBEVMU JOEJWJEVBMT ćFSF BSF NBOZ XBZT UP NPEFM B DVSWFE SFMBUJPOTIJQ CFUXFFO UXP WBS )FSF *MM TIPX ZPV B WFSZ DPNNPO POF ĽļĹņĻļĺĶĮĹ ĿIJĴĿIJŀŀĶļĻ *O UI UFYU iQPMZOPNJBMw NFBOT FRVBUJPOT GPS µJ UIBU BEE BEEJUJPOBM UFSNT XJUI TR DVCFT BOE FWFO IJHIFS QPXFST PG UIF QSFEJDUPS WBSJBCMF ćFSFT TUJMM POMZ PO EJDUPS WBSJBCMF JO UIF NPEFM TP UIJT JT TUJMM B CJWBSJBUF SFHSFTTJPO #VU UIF EFĕ PG µJ IBT NPSF QBSBNFUFST OPX )FSFT UIF NPTU DPNNPO QPMZOPNJBM SFHSFTTJPO B QBSBCPMJD NPEFM NFBO µJ = α + β YJ + β Y J ćF BCPWF JT B QBSBCPMJD TFDPOE PSEFS QPMZOPNJBM ćF α+β YJ QBSU JT UI MJOFBS GVODUJPO PG Y JO B MJOFBS SFHSFTTJPO KVTU XJUI B MJUUMF iw TVCTDSJQU BE UIFQBSBNFUFSOBNF TP XFDBOUFMM JUBQBSUGSPNUIFOFX QBSBNFUFS ćFBEE UFSN VTFT UIF TRVBSF PG YJ UP DPOTUSVDU B QBSBCPMB SBUIFS UIBO B QFSGFDUMZ T MJOF ćF OFX QBSBNFUFS β NFBTVSFT UIF DVSWBUVSF PG UIF SFMBUJPOTIJQ 3FUIJOLJOH -JOFBS BEEJUJWF GVOLZ ćF QBSBCPMJD NPEFM PG µJ BCPWF JT TUJMM D iMJOFBS NPEFMw PG UIF NFBO ćJT JT TP FWFO UIPVHI UIF FRVBUJPO JT DMFBSMZ OPU PG B 2nd order (parabola):
26. Polynomial regression • We’ll use full !Kung height/weight data  1PMZOPNJBM SFHSFTTJPO *O UIF OFYU DIBQUFS ZPVMM TFF IPX UP VTF MJOFBS NPEFMT UP CVJME SFHSFTTJPOT XJUI NPSF UIBO POF QSFEJDUPS WBSJBCMF #VU CFGPSF UIFO JU IFMQT UP TFF IPX UP NPEFM UIF PVUDPNF BT B DVSWFE GVODUJPO PG B TJOHMF QSFEJDUPS ćF NPEFMT TP GBS BMM BTTVNF UIBU B TUSBJHIU MJOF EFTDSJCFT UIF SFMBUJPOTIJQ #VU UIFSFT OPUIJOH TQFDJBM BCPVU TUSBJHIU MJOFT BTJEF GSPN UIFJS TJNQMJDJUZ -FUT XPSL UISPVHI BO FYBNQMF VTJOH UIF GVMM ,VOH EBUB 3 DPEF  /ǭ *2 ''ƼǮ  ʄǤ *2 ''Ƽ ./-ǭǮ ǘ/Ǐ!-( ǘǑ ǀƿƿ *.Ǐ *! ƿ 1-$' .Ǒ ɠ # $"#/Ǒ )0( Ƽǀƽ Ƽƿƻ Ƽƾǂ Ƽǀǂ Ƽƿǀ ǏǏǏ 'data.frame': 544 obs. of 4 variables: $ height: num 152 140 137 157 145 ... $ weight: num 47.8 36.5 31.9 53 41.3 ... $ age : num 63 63 65 41 51 35 32 27 19 54 ... $ male : int 1 0 0 1 0 1 0 1 0 1 ... 10 30 50 60 100 140 180 weight height adults (18+)
27. Standardized predictors • Very helpful to standardize predictor variables before fitting • Makes estimation easier • Makes interpretation kinda easier • To standardize: • subtract mean • divide by standard deviation • result: mean of zero and standard deviation of 1 ıĮĿıĶŇIJ UIF QSFEJDUPS WBSJBCMF ćJT NFBOT UP ĕSTU DFOUFS UIF WBSJBCMF BOE UIFO EJWJEF JU CZ JUT TUBOEBSE EFWJBUJPO 8IZ EP UIJT :PV BMSFBEZ IBWF TPNF TFOTF PG UIF WBMVF PG DFOUFSJOH (PJOH GVSUIFS UP TUBOEBSEJ[F MFBWFT UIF NFBO BU [FSP CVU BMTP SFTDBMFT UIF SBOHF PG UIF EBUB ćJT JT IFMQGVM GPS UXP SFBTPOT  *OUFSQSFUBUJPO NJHIU CF FBTJFS 'PS B TUBOEBSEJ[FE WBSJBCMF B DIBOHF PG POF VOJU JT FRVJWBMFOU UP B DIBOHF PG POF TUBOEBSE EFWJBUJPO *O NBOZ DPOUFYUT UIJT JT NPSF JOUFSFTUJOH BOE NPSF SFWFBMJOH UIBO B POF VOJU DIBOHF PO UIF OBUVSBM TDBMF "OE PODF ZPV TUBSU NBLJOH SFHSFTTJPOT XJUI NPSF UIBO POF LJOE PG QSFEJDUPS WBSJBCMF TUBOEBSEJ[JOH BMM PG UIFN NBLFT JU FBTJFS UP DPNQBSF UIFJS SFMBUJWF JOĘVFODF PO UIF PVUDPNF VTJOH POMZ FTUJNBUFT 0O UIF PUIFS IBOE ZPV NJHIU XBOU UP JOUFSQSFU UIF EBUB PO UIF OBUVSBM TDBMF 4P TUBOEBSEJ[BUJPO DBO NBLF JOUFSQSFUBUJPO IBSEFS OPU FBTJFS 8JUI B MJUUMF QSBDUJDF UIPVHI ZPV DBO MFBSO UP RVJDLMZ DPOWFSU CBDL BOE GPSUI CFUXFFO OBUVSBM BOE TUBOEBSE TDBMFT  .PSF JNQPSUBOU UIPVHI BSF UIF BEWBOUBHFT GPS ĕUUJOH UIF NPEFM UP UIF EBUB 8IFO QSFEJDUPS WBSJBCMFT IBWF WFSZ MBSHF WBMVFT JO UIFN UIFSF BSF TPNFUJNFT OVNFSJ DBM HMJUDIFT &WFO XFMMLOPXO TUBUJTUJDBM TPęXBSF DBO TVČFS GSPN UIFTF HMJUDIFT MFBEJOH UP NJTUBLFO FTUJNBUFT ćFTF QSPCMFNT BSF WFSZ DPNNPO GPS QPMZOPNJBM SFHSFTTJPO CFDBVTF UIF TRVBSF PS DVCF PG B MBSHF OVNCFS DBO CF USVMZ NBTTJWF 4UBO EBSEJ[JOH MBSHFMZ SFTPMWFT UIJT JTTVF 5P TUBOEBSEJ[F 2 $"#/ BMM ZPV EP JT TVCUSBDU UIF NFBO BOE UIFO EJWJEF CZ UIF TUBOEBSE EFWJBUJPO ćJT XJMM EP JU 3 DPEF  ɠ2 $"#/Ǐ. ʄǤ ǭ ɠ2 $"#/ Ǥ ( )ǭɠ2 $"#/Ǯ Ǯdz.ǭɠ2 $"#/Ǯ ćJT OFX WBSJBCMF 2 $"#/Ǐ. IBT NFBO [FSP BOE TUBOEBSE EFWJBUJPO  /P JOGPSNBUJPO IBT CFFO MPTU JO UIJT QSPDFEVSF (P BIFBE BOE QMPU # $"#/ PO 2 $"#/Ǐ. UP WFSJGZ UIBU :PVMM
28. Parabolic regression • Parabolic model of height as function of weight:   -*/&"3 .0%&-4 TFF UIF TBNF DVSWFE SFMBUJPOTIJQ BT CFGPSF CVU OPX XJUI B EJČFSFOU SBOHF PO UIF IPSJ[POUBM BYJT 5P ĕU UIF QBSBCPMJD NPEFM KVTU NPEJGZ UIF EFĕOJUJPO PG µJ  )FSFT UIF NPEFM XJUI WFSZ XFBL QSJPST  IJ ∼ /PSNBM(µJ, σ) # $"#/ ʋ )*-(ǭ(0ǐ.$"(Ǯ µJ = α + β YJ + β Y J (0 ʄǤ  ɾ ƼǷ2 $"#/Ǐ. ɾ ƽǷ2 $"#/Ǐ.ʉƽ α ∼ /PSNBM(, )  ʋ )*-(ǭƼƿƻǐƼƻƻǮ β ∼ /PSNBM(, ) Ƽ ʋ )*-(ǭƻǐƼƻǮ β ∼ /PSNBM(, ) ƽ ʋ )*-(ǭƻǐƼƻǮ σ ∼ 6OJGPSN(, ) .$"( ʋ 0)$!ǭƻǐǀƻǮ "OE ĕUUJOH JT TUSBJHIUGPSXBSE (ƿǏǀ ʄǤ (+ǭ '$./ǭ # $"#/ ʋ )*-(ǭ (0 ǐ .$"( Ǯ ǐ (0 ʄǤ  ɾ ƼǷ2 $"#/Ǐ. ɾ ƽǷ2 $"#/Ǐ.ʉƽ ǐ  ʋ )*-(ǭ Ƽƿƻ ǐ Ƽƻƻ Ǯ ǐ Ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ .$"( ʋ 0)$!ǭ ƻ ǐ ǀƻ Ǯ
29. ƿǂǏǃ ƾǁǏǀ ƾƼǏDŽ ǀƾ ƿƼǏƾ ǏǏǏ ǁƾ ǁƾ ǁǀ ƿƼ ǀƼ ƾǀ ƾƽ ƽǂ ƼDŽ ǀƿ ǏǏǏ Ƽ ƻ ƻ Ƽ ƻ Ƽ ƻ Ƽ ƻ Ƽ ǏǏǏ # $"#/ BHBJOTU 2 $"#/ ćF SFMBUJPOTIJQ JT WJTJCMZ DVSWFE OPX E UIF OPOBEVMU JOEJWJEVBMT OZ XBZT UP NPEFM B DVSWFE SFMBUJPOTIJQ CFUXFFO UXP WBSJBCMFT V B WFSZ DPNNPO POF ĽļĹņĻļĺĶĮĹ ĿIJĴĿIJŀŀĶļĻ *O UIJT DPO NFBOT FRVBUJPOT GPS µJ UIBU BEE BEEJUJPOBM UFSNT XJUI TRVBSFT HIFS QPXFST PG UIF QSFEJDUPS WBSJBCMF ćFSFT TUJMM POMZ POF QSF IF NPEFM TP UIJT JT TUJMM B CJWBSJBUF SFHSFTTJPO #VU UIF EFĕOJUJPO BNFUFST OPX TU DPNNPO QPMZOPNJBM SFHSFTTJPO B QBSBCPMJD NPEFM PG UIF µJ = α + β YJ + β Y J BCPMJD TFDPOE PSEFS QPMZOPNJBM ćF α+β YJ QBSU JT UIF TBNF Y JO B MJOFBS SFHSFTTJPO KVTU XJUI B MJUUMF iw TVCTDSJQU BEEFE UP NF TP XFDBOUFMM JUBQBSUGSPNUIFOFX QBSBNFUFS ćFBEEJUJPOBM "OE ĕUUJOH JT TUSBJHIUGPSXBSE 3 DPEF  (ƿǏǀ ʄǤ (+ǭ '$./ǭ # $"#/ ʋ )*-(ǭ (0 ǐ .$"( Ǯ ǐ (0 ʄǤ  ɾ ƼǷ2 $"#/Ǐ. ɾ ƽǷ2 $"#/Ǐ.ʉƽ ǐ  ʋ )*-(ǭ Ƽƿƻ ǐ Ƽƻƻ Ǯ ǐ Ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ .$"( ʋ 0)$!ǭ ƻ ǐ ǀƻ Ǯ Ǯ ǐ /ʃ Ǯ "MM UIBU DIBOHFT JT UIF FYQSFTTJPO GPS (0 *OUFSQSFUJOH UIF FTUJNBUFT GSPN UIF TVNNBSZ UBCMF DBO CF IBSE UIPVHI -FUT UBLF B MPPL 3 DPEF  +- $.ǭ (ƿǏǀ Ǯ  ) / 1 ƽǏǀɳ DŽǂǏǀɳ  ƼƿǁǏǁǁ ƻǏƾǂ ƼƿǀǏDŽƾ ƼƿǂǏƿƻ Ƽ ƽƼǏƿƻ ƻǏƽDŽ ƽƻǏǃƾ ƽƼǏDŽǂ ƽ ǤǃǏƿƼ ƻǏƽǃ ǤǃǏDŽǂ ǤǂǏǃǁ .$"( ǀǏǂǀ ƻǏƼǂ ǀǏƿƼ ǁǏƻDŽ ćF QBSBNFUFS α  JT TUJMM UIF JOUFSDFQU TP JU UFMMT VT UIF FYQFDUFE WBMVF PG # $"#/ XIFO 2 $"#/Ǐ. JT [FSP #VU JU JT OP MPOHFS FRVBM UP UIF NFBO IFJHIU JO UIF TBNQMF TJODF UIFSF JT OP HVBSBOUFF JU TIPVME JO B QPMZOPNJBM SFHSFTTJPO "OE UIPTF β BOE β QBSBNFUFST BSF UIF MJOFBS BOE TRVBSF DPNQPOFOUT PG UIF DVSWF SFTQFDUJWFMZ #VU UIBU EPFTOU NBLF UIFN USBOTQBSFOU :PV IBWF UP QMPU UIFTF NPEFM ĕUT UP VOEFSTUBOE XIBU UIFZ BSF TBZJOH 4P MFUT EP UIBU 8FMM DBMDVMBUF UIF NFBO SFMBUJPOTIJQ BOE UIF  JOUFSWBMT PG UIF NFBO BOE UIF QSFEJDUJPOT 3 DPEF  (ƿǏǀ ʄǤ (+ǭ '$./ǭ # $"#/ ʋ )*-(ǭ (0 ǐ .$"( Ǯ ǐ (0 ʄǤ  ɾ ƼǷ2 $"#/Ǐ. ɾ ƽǷ2 $"#/Ǐ.ʉƽ ǐ  ʋ )*-(ǭ Ƽƿƻ ǐ Ƽƻƻ Ǯ ǐ Ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ .$"( ʋ 0)$!ǭ ƻ ǐ ǀƻ Ǯ Ǯ ǐ /ʃ Ǯ "MM UIBU DIBOHFT JT UIF FYQSFTTJPO GPS (0 *OUFSQSFUJOH UIF FTUJNBUFT GSPN UIF TVNNBSZ UBCMF DBO CF IBSE UIPVHI -FUT UBLF B MPPL 3 DPEF  +- $.ǭ (ƿǏǀ Ǯ  ) / 1 ƽǏǀɳ DŽǂǏǀɳ  ƼƿǁǏǁǁ ƻǏƾǂ ƼƿǀǏDŽƾ ƼƿǂǏƿƻ Ƽ ƽƼǏƿƻ ƻǏƽDŽ ƽƻǏǃƾ ƽƼǏDŽǂ ƽ ǤǃǏƿƼ ƻǏƽǃ ǤǃǏDŽǂ ǤǂǏǃǁ .$"( ǀǏǂǀ ƻǏƼǂ ǀǏƿƼ ǁǏƻDŽ ćF QBSBNFUFS α  JT TUJMM UIF JOUFSDFQU TP JU UFMMT VT UIF FYQFDUFE WBMVF PG # $"#/ XIFO 2 $"#/Ǐ. JT [FSP #VU JU JT OP MPOHFS FRVBM UP UIF NFBO IFJHIU JO UIF TBNQMF TJODF UIFSF JT OP HVBSBOUFF JU TIPVME JO B QPMZOPNJBM SFHSFTTJPO "OE UIPTF β BOE β QBSBNFUFST BSF UIF MJOFBS BOE TRVBSF DPNQPOFOUT PG UIF DVSWF SFTQFDUJWFMZ #VU UIBU EPFTOU NBLF UIFN USBOTQBSFOU :PV IBWF UP QMPU UIFTF NPEFM ĕUT UP VOEFSTUBOE XIBU UIFZ BSF TBZJOH 4P MFUT EP UIBU 8FMM DBMDVMBUF UIF NFBO SFMBUJPOTIJQ BOE UIF  JOUFSWBMT PG UIF NFBO BOE UIF QSFEJDUJPOT MJLF JO UIF QSFWJPVT TFDUJPO )FSFT UIF XPSLJOH DPEF  10-:/0.*"- 3&(3&44*0/ (a) (b) ( -2 -1 0 1 2 60 80 100 140 180 weight.s height -2 60 80 100 140 180 height -2 -1 0 1 2 60 80 100 140 180 weight.s height
30. -2 -1 0 1 2 60 100 140 180 weight.s height N = 10 -2 -1 0 1 2 60 100 140 180 weight.s height N = 20 -2 -1 0 1 2 60 100 140 180 weight.s height N = 50 -2 -1 0 1 2 60 100 140 180 weight.s height N = 100 -2 -1 0 1 2 60 100 140 180 weight.s height N = 300 -2 -1 0 1 2 60 100 140 180 weight.s height N = 544
31. Cubic model • Can go further down the rabbit hole: TPNF TQFDUVMBSMZ QPPS QSFEJDUJPOT BU CPUI WFSZ MPX BOE NJEEMF XFJHIUT $PN QBSF UIJT UP QBOFM C PVS OFX QBSBCPMJD SFHSFTTJPO ćF DVSWF EPFT B NVDI CFUUFS KPC PG ĕOEJOH B DFOUSBM QBUI UISPVHI UIF EBUB 1BOFM D JO 'ĶĴłĿIJ ƌƉƈ TIPXT B IJHIFSPSEFS QPMZOPNJBM SFHSFTTJPO B DVCJD SFHSFTTJPO PO XFJHIU ćF NPEFM JT BHBJO XJUI MB[Z ĘBU QSJPST  IJ ∼ /PSNBM(µJ, σ) µJ = α + β YJ + β Y J + β Y J 'JU UIF NPEFM XJUI B TMJHIU NPEJĕDBUJPO PG UIF QBSBCPMJD NPEFMT DPEF (ƿǏǁ ʄǤ (+ǭ '$./ǭ # $"#/ ʋ )*-(ǭ ( )ʃ(0 ǐ .ʃ.$"( Ǯ ǐ (0 ʋ  ɾ ƼǷ2 $"#/Ǐ. ɾ ƽǷ2 $"#/Ǐ.ʉƽ ɾ ƾǷ2 $"#/Ǐ.ʉƾ Ǯ ǐ /ʃ ǐ ./-/ʃ'$./ǭ ʃ( )ǭɠ# $"#/Ǯ ǐ Ƽʃƻ ǐ ƽʃƻ ǐ ƾʃƻ ǐ .$"(ʃƼƻ Ǯ Ǯ $PNQVUJOH UIF DVSWF BOE DPOĕEFODF JOUFSWBMT JT TJNJMBSMZ B TNBMM NPEJĕDBUJPO   -*/&"3 .0%&-4 'JU UIF NPEFM XJUI B TMJHIU NPEJĕDBUJPO PG UIF QBSBCPMJD NPEFMT DPEF 3 DPEF  (ƿǏǁ ʄǤ (+ǭ '$./ǭ # $"#/ ʋ )*-(ǭ (0 ǐ .$"( Ǯ ǐ (0 ʄǤ  ɾ ƼǷ2 $"#/Ǐ. ɾ ƽǷ2 $"#/Ǐ.ʉƽ ɾ ƾǷ2 $"#/Ǐ.ʉƾ ǐ  ʋ )*-(ǭ Ƽƿƻ ǐ Ƽƻƻ Ǯ ǐ Ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ ƽ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ ƾ ʋ )*-(ǭ ƻ ǐ Ƽƻ Ǯ ǐ .$"( ʋ 0)$!ǭ ƻ ǐ ǀƻ Ǯ Ǯ ǐ /ʃ Ǯ $PNQVUJOH UIF DVSWF BOE JOUFSWBMT JT TJNJMBSMZ B TNBMM NPEJĕDBUJPO PG UIF QSFWJPVT DPEF
32. Polynomial regression 1st order 2nd order 3rd order  10-:/0.*"- 3&(3&44*0/  (a) (b) (c) -2 -1 0 1 2 60 80 100 140 180 weight.s height -2 -1 0 1 2 60 80 100 140 180 weight.s height -2 -1 0 1 2 60 80 100 140 180 weight.s height 'ĶĴłĿIJ ƌƉƈ 1PMZOPNJBM SFHSFTTJPOT PG IFJHIU PO XFJHIU TUBOEBSEJ[FE GPS UIF GVMM ,VOH EBUB *O FBDI QMPU UIF SBX EBUB BSF TIPXO CZ UIF DJSDMFT
33. Next week • Multiple regression • Categorical data • Steady practice with these same tools • quadratic approximation • plotting implied predictions • model criticism