Statistical Rethinking 2022 Lecture 02

Course materials: https://github.com/rmcelreath/stat_rethinking_2022

Transcript

1. Statistical Rethinking 02: Foundations of Bayesian Inference 2022
3. What proportion of the surface is covered with water?
4. How should we use the sample? How to produce a summary? How to represent uncertainty?
5. Bayesian data analysis For each possible explanation of the data, Count all the ways data can happen. Explanations with more ways to produce the data are more plausible.
7. Garden of Forking Data Contains 4 marbles ? Observe: (1) (2) (3) (4) (5) Possible contents:
8. Garden of Forking Data Contains 4 marbles ? Observe: (1) (2) (3) (4) (5) Possible contents:
9. How many ways to observe ? Garden of Forking Data Contains 4 marbles ? assume (1) (2) (3) (4) (5) Possible contents:
10. First Possibility Figure 2.2
11. Second Possibility Figure 2.2
12. Third Possibility Figure 2.2
13. Figure 2.2 First Observation
14. Figure 2.2 Second Observation
15. Figure 2.2 Third Observation
16. Figure 2.2 3 Ways to see if the bag contains
17. Garden of Forking Data (1) (2) (3) (4) (5) Possible contents: Ways to produce ? 3 ? ? ?
18. Garden of Forking Data (1) (2) (3) (4) (5) Possible contents: Ways to produce 0 3 ? ? ?
19. Garden of Forking Data (1) (2) (3) (4) (5) Possible contents: Ways to produce 0 3 ? ? 0
20. (3)
21. (4)
22. Garden of Forking Data (1) (2) (3) (4) (5) Possible contents: Ways to produce 0 3 8 9 0
23. Counts to plausibility Unglamorous basis of applied probability: Things that can happen more ways are more plausible. "OE UIFO EJWJEF FBDI QSPEVDU CZ UIF TVN PG QSPEVDUT QMBVTJCJMJUZ PG Q BęFS %OFX = XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q TVN PG QSPEVDUT ćFSFT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH UI VNCFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQMBV CJMJUZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    PV DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3
24. Counts to plausibility Unglamorous basis of applied probability: Things that can happen more ways are more plausible. "OE UIFO EJWJEF FBDI QSPEVDU CZ UIF TVN PG QSPEVDUT QMBVTJCJMJUZ PG Q BęFS %OFX = XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q TVN PG QSPEVDUT ćFSFT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH UI VNCFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQMBV CJMJUZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    PV DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3
25. Counts to plausibility Unglamorous basis of applied probability: Things that can happen more ways are more plausible. "OE UIFO EJWJEF FBDI QSPEVDU CZ UIF TVN PG QSPEVDUT QMBVTJCJMJUZ PG Q BęFS %OFX = XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q TVN PG QSPEVDUT ćFSFT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH UI VNCFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQMBV CJMJUZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    PV DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3
26. Counts to plausibility QMBVTJCJMJUZ PG Q BęFS %OFX = XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q TVN PG QSPEVDUT ćFSFT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH UIF OVNCFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQMBV TJCJMJUZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    :PV DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3 3 DPEF  24. ʄǤ ǭ ƾ ǐ ǃ ǐ DŽ Ǯ 24.dz.0(ǭ24.Ǯ ǯƼǰ ƻǏƼǀ ƻǏƿƻ ƻǏƿǀ ćFTF QMBVTJCJMJUJFT BSF BMTP QSPCBCJMJUJFT‰UIFZ BSF OPOOFHBUJWF [FSP PS QPTJUJWF SFBM OVNCFST UIBU TVN UP POF "OE BMM PG UIF NBUIFNBUJDBM UIJOHT ZPV DBO EP XJUI QSPCBCJMJUJFT ćFSFT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH UI VNCFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQMBV CJMJUZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    PV DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3 4. ʄǤ ǭ ƾ ǐ ǃ ǐ DŽ Ǯ 4.dz.0(ǭ24.Ǯ Ƽǰ ƻǏƼǀ ƻǏƿƻ ƻǏƿǀ
27. Updating Another draw from the bag: UT PWFS DPOKFDUVSFT      BOE KVTU VQEBUF UIFN JO MJHIU PG UIF OFX PCTFSWB OT PVU UIBU UIFTF UXP NFUIPET BSF NBUIFNBUJDBMMZ JEFOUJDBM "T MPOH BT UIF OFX P O JT MPHJDBMMZ JOEFQFOEFOU PG UIF QSFWJPVT PCTFSWBUJPOT )FSFT IPX UP EP JU 'JSTU XF DPVOU UIF OVNCFST PG XBZT FBDI DPOKFDUVSF DPVME QSP FX PCTFSWBUJPO  ćFO XF NVMUJQMZ FBDI PG UIFTF OFX DPVOUT CZ UIF QSFWJPVT OVN BZT GPS FBDI DPOKFDUVSF *O UBCMF GPSN $POKFDUVSF 8BZT UP QSPEVDF 1SFWJPVT DPVOUT /FX DPVOU < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  OFX DPVOUT JO UIF SJHIUIBOE DPMVNO BCPWF TVNNBSJ[F BMM UIF FWJEFODF GPS FBDI DP "T OFX EBUB BSSJWF BOE QSPWJEFE UIPTF EBUB BSF JOEFQFOEFOU PG QSFWJPVT PCTFSWBU UIF OVNCFS PG MPHJDBMMZ QPTTJCMF XBZT GPS B DPOKFDUVSF UP QSPEVDF BMM UIF EBUB VQ UP 4
28. Updating Another draw from the bag: UT PWFS DPOKFDUVSFT      BOE KVTU VQEBUF UIFN JO MJHIU PG UIF OFX PCTFSWB OT PVU UIBU UIFTF UXP NFUIPET BSF NBUIFNBUJDBMMZ JEFOUJDBM "T MPOH BT UIF OFX P O JT MPHJDBMMZ JOEFQFOEFOU PG UIF QSFWJPVT PCTFSWBUJPOT )FSFT IPX UP EP JU 'JSTU XF DPVOU UIF OVNCFST PG XBZT FBDI DPOKFDUVSF DPVME QSP FX PCTFSWBUJPO  ćFO XF NVMUJQMZ FBDI PG UIFTF OFX DPVOUT CZ UIF QSFWJPVT OVN BZT GPS FBDI DPOKFDUVSF *O UBCMF GPSN $POKFDUVSF 8BZT UP QSPEVDF 1SFWJPVT DPVOUT /FX DPVOU < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  OFX DPVOUT JO UIF SJHIUIBOE DPMVNO BCPWF TVNNBSJ[F BMM UIF FWJEFODF GPS FBDI DP "T OFX EBUB BSSJWF BOE QSPWJEFE UIPTF EBUB BSF JOEFQFOEFOU PG QSFWJPVT PCTFSWBU UIF OVNCFS PG MPHJDBMMZ QPTTJCMF XBZT GPS B DPOKFDUVSF UP QSPEVDF BMM UIF EBUB VQ UP 4
29. Updating Another draw from the bag: UT PWFS DPOKFDUVSFT      BOE KVTU VQEBUF UIFN JO MJHIU PG UIF OFX PCTFSWB OT PVU UIBU UIFTF UXP NFUIPET BSF NBUIFNBUJDBMMZ JEFOUJDBM "T MPOH BT UIF OFX P O JT MPHJDBMMZ JOEFQFOEFOU PG UIF QSFWJPVT PCTFSWBUJPOT )FSFT IPX UP EP JU 'JSTU XF DPVOU UIF OVNCFST PG XBZT FBDI DPOKFDUVSF DPVME QSP FX PCTFSWBUJPO  ćFO XF NVMUJQMZ FBDI PG UIFTF OFX DPVOUT CZ UIF QSFWJPVT OVN BZT GPS FBDI DPOKFDUVSF *O UBCMF GPSN $POKFDUVSF 8BZT UP QSPEVDF 1SFWJPVT DPVOUT /FX DPVOU < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  OFX DPVOUT JO UIF SJHIUIBOE DPMVNO BCPWF TVNNBSJ[F BMM UIF FWJEFODF GPS FBDI DP "T OFX EBUB BSSJWF BOE QSPWJEFE UIPTF EBUB BSF JOEFQFOEFOU PG QSFWJPVT PCTFSWBU UIF OVNCFS PG MPHJDBMMZ QPTTJCMF XBZT GPS B DPOKFDUVSF UP QSPEVDF BMM UIF EBUB VQ UP 4
30. Updating Another draw from the bag: UT PWFS DPOKFDUVSFT      BOE KVTU VQEBUF UIFN JO MJHIU PG UIF OFX PCTFSWB OT PVU UIBU UIFTF UXP NFUIPET BSF NBUIFNBUJDBMMZ JEFOUJDBM "T MPOH BT UIF OFX P O JT MPHJDBMMZ JOEFQFOEFOU PG UIF QSFWJPVT PCTFSWBUJPOT )FSFT IPX UP EP JU 'JSTU XF DPVOU UIF OVNCFST PG XBZT FBDI DPOKFDUVSF DPVME QSP FX PCTFSWBUJPO  ćFO XF NVMUJQMZ FBDI PG UIFTF OFX DPVOUT CZ UIF QSFWJPVT OVN BZT GPS FBDI DPOKFDUVSF *O UBCMF GPSN $POKFDUVSF 8BZT UP QSPEVDF 1SFWJPVT DPVOUT /FX DPVOU < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  OFX DPVOUT JO UIF SJHIUIBOE DPMVNO BCPWF TVNNBSJ[F BMM UIF FWJEFODF GPS FBDI DP "T OFX EBUB BSSJWF BOE QSPWJEFE UIPTF EBUB BSF JOEFQFOEFOU PG QSFWJPVT PCTFSWBU UIF OVNCFS PG MPHJDBMMZ QPTTJCMF XBZT GPS B DPOKFDUVSF UP QSPEVDF BMM UIF EBUB VQ UP 4
31. Bayesian updating The rules: 1. State a causal model for how the observations arise, given each possible explanation 2. Count ways data could arise for each explanation 3. Relative plausibility is relative value from (2)
32. Globe of Forking Water For each possible proportion of water, Count number of ways data could happen. Must state how observations are generated
33. Toss The First
34. posterior prior observation
35. “density”:
 relative plausibility of proportion of water
36. Toss The Second
37. × = Toss The Second relative plausibility of L relative plausibility of W relative plausibility of LW
38. plot( 0:10 * 10:0 ) 2 4 6 8 10 0 5 10 15 20 25 Index 0:10 * 10:0
39. × × plot( 10:0 * 0:10 * 10:0 ) Toss The Third
40. Toss The Ten
42. (1) No minimum sample size
43. (2) Shape embodies sample size
44. (3) No point estimate mean mode The distribution is the estimate Always use the entire distribution
45. (4) No one true interval Intervals communicate shape of posterior 0.0 1.0 2.0 proportion water density 0 0.5 1
46. 0.0 1.0 2.0 proportion water density 0 0.5 1 (4) No one true interval Intervals communicate shape of posterior 50%
47. 0.0 1.0 2.0 proportion water density 0 0.5 1 (4) No one true interval Intervals communicate shape of posterior 89%
48. 0.0 1.0 2.0 proportion water density 0 0.5 1 (4) No one true interval Intervals communicate shape of posterior 95% is obvious superstition. Nothing magical happens at the boundary. 99%
49. Letters From My Reviewers “The author uses these cute 89% intervals, but we need to see the 95% intervals so we can tell whether any of the effects are robust.” That an arbitrary interval contains an arbitrary value is not meaningful. Use the whole distribution.
50. PAUSE
52. The Formalities In practice, we write the model in a way that communicates all of the probability assumptions. The observations (data) and explanations (parameters) are variables For each variable, must say how it is generated
53. The Formalities Data: W and L, the number of water and land observations Parameters: p, the proportion of water on the globe HIU TPVOE DIBMMFOHJOH CVU JUT UIF LJOE PG UIJOH ZPV HFU HPPE BU WFSZ RVJDLMZ SBDUJDJOH PODF XF BEE PVS BTTVNQUJPOT UIBU  FWFSZ UPTT JT JOEFQFOEFOU PG UIF PUIFS IF QSPCBCJMJUZ PG 8 JT UIF TBNF PO FWFSZ UPTT QSPCBCJMJUZ UIFPSZ QSPWJEFT S LOPXO BT UIF CJOPNJBM EJTUSJCVUJPO ćJT JT UIF DPNNPO iDPJO UPTTJOHw OE TP UIF QSPCBCJMJUZ PG PCTFSWJOH 8 XBUFST BOE - MBOET XJUI B QSPCBCJMJUZ Q I UPTT JT 1S(8, -|Q) = (8 + -)! 8!-! Q8 ( − Q)- BT VOUT PG iXBUFSw 8 BOE iMBOE - BSF EJTUSJCVUFE CJOPNJBMMZ XJUI QSPC Q PG iXBUFSw PO FBDI UPTT BM EJTUSJCVUJPO GPSNVMB JT CVJMU JOUP 3 TP ZPV DBO FBTJMZ DPNQVUF UIF MJLFMJ B‰TJY 8T JO OJOF UPTTFT‰VOEFS BOZ WBMVF PG Q XJUI 3 DPEF  ize=9 , prob=0.5 ) OF XIP IBT WJTJUFE B TUBUJTUJDT IFMQ EFTL BU B VOJWFSTJUZ IBT QSPCBCMZ FYQFSJFODFE UI DUJWJUZ‰TUBUJTUJDJBOT EP OPU JO HFOFSBM FYBDUMZ BHSFF PO IPX UP BOBMZ[F BOZUIJOH CVU UI FTU PG QSPCMFNT ćF GBDU UIBU TUBUJTUJDBM JOGFSFODF VTFT NBUIFNBUJDT EPFT OPU JNQMZ UI JT POMZ POF SFBTPOBCMF PS VTFGVM XBZ UP DPOEVDU BO BOBMZTJT &OHJOFFSJOH VTFT NBUI B CVU UIFSF BSF NBOZ XBZT UP CVJME B CSJEHF FZPOE BMM PG UIF BCPWF UIFSFT OP MBX NBOEBUJOH XF VTF POMZ POF QSJPS *G ZPV EPO B TUSPOH BSHVNFOU GPS BOZ QBSUJDVMBS QSJPS UIFO USZ EJČFSFOU POFT #FDBVTF UIF QSJPS VNQUJPO JU TIPVME CF JOUFSSPHBUFE MJLF PUIFS BTTVNQUJPOT CZ BMUFSJOH JU BOE DIFDLJO FOTJUJWF JOGFSFODF JT UP UIF BTTVNQUJPO /P POF JT SFRVJSFE UP TXFBS BO PBUI UP UI NQUJPOT PG B NPEFM BOE OP TFU PG BTTVNQUJPOT EFTFSWFT PVS PCFEJFODF IJOLJOH 1SJPS BT QSPCBCJMJUZ EJTUSJCVUJPO :PV DPVME XSJUF UIF QSJPS JO UIF FYBNQMF IFSF BT 1S(Q) =   −  = . JPS JT B QSPCBCJMJUZ EJTUSJCVUJPO GPS UIF QBSBNFUFS *O HFOFSBM GPS B VOJGPSN QSJPS GSPN B UP C UI CJMJUZ PG BOZ QPJOU JO UIF JOUFSWBM JT /(C − B) *G ZPVSF CPUIFSFE CZ UIF GBDU UIBU UIF QSPCBCJMJ Binomial probability function dbinom( W , W+L , p ) The number of ways to realize W,L given p
54. The Formalities Data: W and L, the number of water and land observations Parameters: p, the proportion of water on the globe JTJUFE B TUBUJTUJDT IFMQ EFTL BU B VOJWFSTJUZ IBT QSPCBCMZ FYQFSJFODFE UIJT UJDJBOT EP OPU JO HFOFSBM FYBDUMZ BHSFF PO IPX UP BOBMZ[F BOZUIJOH CVU UIF NT ćF GBDU UIBU TUBUJTUJDBM JOGFSFODF VTFT NBUIFNBUJDT EPFT OPU JNQMZ UIBU BTPOBCMF PS VTFGVM XBZ UP DPOEVDU BO BOBMZTJT &OHJOFFSJOH VTFT NBUI BT NBOZ XBZT UP CVJME B CSJEHF IF BCPWF UIFSFT OP MBX NBOEBUJOH XF VTF POMZ POF QSJPS *G ZPV EPOU NFOU GPS BOZ QBSUJDVMBS QSJPS UIFO USZ EJČFSFOU POFT #FDBVTF UIF QSJPS JT IPVME CF JOUFSSPHBUFE MJLF PUIFS BTTVNQUJPOT CZ BMUFSJOH JU BOE DIFDLJOH SFODF JT UP UIF BTTVNQUJPO /P POF JT SFRVJSFE UP TXFBS BO PBUI UP UIF NPEFM BOE OP TFU PG BTTVNQUJPOT EFTFSWFT PVS PCFEJFODF BT QSPCBCJMJUZ EJTUSJCVUJPO :PV DPVME XSJUF UIF QSJPS JO UIF FYBNQMF IFSF BT 1S(Q) =   −  = . MJUZ EJTUSJCVUJPO GPS UIF QBSBNFUFS *O HFOFSBM GPS B VOJGPSN QSJPS GSPN B UP C UIF JOU JO UIF JOUFSWBM JT /(C − B) *G ZPVSF CPUIFSFE CZ UIF GBDU UIBU UIF QSPCBCJMJUZ HIU TPVOE DIBMMFOHJOH CVU JUT UIF LJOE PG UIJOH ZPV HFU HPPE BU WFSZ RVJDLMZ SBDUJDJOH PODF XF BEE PVS BTTVNQUJPOT UIBU  FWFSZ UPTT JT JOEFQFOEFOU PG UIF PUIFS IF QSPCBCJMJUZ PG 8 JT UIF TBNF PO FWFSZ UPTT QSPCBCJMJUZ UIFPSZ QSPWJEFT S LOPXO BT UIF CJOPNJBM EJTUSJCVUJPO ćJT JT UIF DPNNPO iDPJO UPTTJOHw OE TP UIF QSPCBCJMJUZ PG PCTFSWJOH 8 XBUFST BOE - MBOET XJUI B QSPCBCJMJUZ Q I UPTT JT 1S(8, -|Q) = (8 + -)! 8!-! Q8 ( − Q)- BT VOUT PG iXBUFSw 8 BOE iMBOE - BSF EJTUSJCVUFE CJOPNJBMMZ XJUI QSPC Q PG iXBUFSw PO FBDI UPTT BM EJTUSJCVUJPO GPSNVMB JT CVJMU JOUP 3 TP ZPV DBO FBTJMZ DPNQVUF UIF MJLFMJ B‰TJY 8T JO OJOF UPTTFT‰VOEFS BOZ WBMVF PG Q XJUI 3 DPEF  ize=9 , prob=0.5 ) The number of ways to realize W,L given p Relative plausibility of each possible p
55. The Formalities PODF XF BEE PVS BTTVNQUJPOT UIBU  FWFSZ UPTT JT JOEFQFOEFOU PG UIF PUIFS UIF QSPCBCJMJUZ PG 8 JT UIF TBNF PO FWFSZ UPTT QSPCBCJMJUZ UIFPSZ QSPWJEFT FS LOPXO BT UIF CJOPNJBM EJTUSJCVUJPO ćJT JT UIF DPNNPO iDPJO UPTTJOHw OE TP UIF QSPCBCJMJUZ PG PCTFSWJOH 8 XBUFST BOE - MBOET XJUI B QSPCBCJMJUZ Q I UPTT JT 1S(8, -|Q) = (8 + -)! 8!-! Q8 ( − Q)- BT PVOUT PG iXBUFSw 8 BOE iMBOE - BSF EJTUSJCVUFE CJOPNJBMMZ XJUI QSPC Z Q PG iXBUFSw PO FBDI UPTT NJBM EJTUSJCVUJPO GPSNVMB JT CVJMU JOUP 3 TP ZPV DBO FBTJMZ DPNQVUF UIF MJLFMJ UB‰TJY 8T JO OJOF UPTTFT‰VOEFS BOZ WBMVF PG Q XJUI 3 DPEF  ize=9 , prob=0.5 ) IPX TFOTJUJWF JOGFSFODF JT UP UIF BTTVNQUJPO /P POF JT SFRVJSFE BTTVNQUJPOT PG B NPEFM BOE OP TFU PG BTTVNQUJPOT EFTFSWFT PVS PC 0WFSUIJOLJOH 1SJPS BT QSPCBCJMJUZ EJTUSJCVUJPO :PV DPVME XSJUF UIF QSJ 1S(Q) =   −  = . ćF QSJPS JT B QSPCBCJMJUZ EJTUSJCVUJPO GPS UIF QBSBNFUFS *O HFOFSBM GPS B VO QSPCBCJMJUZ PG BOZ QPJOU JO UIF JOUFSWBM JT /(C − B) *G ZPVSF CPUIFSFE CZ PG FWFSZ WBMVF PG Q JT  SFNFNCFS UIBU FWFSZ QSPCBCJMJUZ EJTUSJCVUJPO NVT FYQSFTTJPO /(C − B) FOTVSFT UIBU UIF BSFB VOEFS UIF ĘBU MJOF GSPN B UP C NPSF UP TBZ BCPVU UIJT JO $IBQUFS  3FUIJOLJOH %BUVN PS QBSBNFUFS *U JT UZQJDBM UP DPODFJWF PG EBUB BOE EJČFSFOU LJOET PG FOUJUJFT %BUB BSF NFBTVSFE BOE LOPXO QBSBNFUFST B Posterior is (normalized) product: PO #VU JUT KVTU BT USVF UIBU 1S(8, -, Q) = 1S(Q|8, -) 1S(8, -) T SFWFSTF XIJDI QSPCBCJMJUZ JT DPOEJUJPOBM PO UIF SJHIUIBOE TJEF *U JT TUJMM B O *UT MJLF TBZJOH UIBU UIF QSPCBCJMJUZ PG SBJO BOE DPME PO UIF TBNF EBZ JT FRVBM MJUZ UIBU JUT DPME XIFO JUT SBJOJOH UJNFT UIF QSPCBCJMJUZ PG SBJO $PNQBSF UIJT IF POF JO UIF QSFWJPVT QBSBHSBQI F CPUI SJHIUIBOE TJEFT BCPWF BSF FRVBM UP UIF TBNF UIJOH 1S(8, -, Q) UIFZ BSF POF BOPUIFS 1S(8, -|Q) 1S(Q) = 1S(Q|8, -) 1S(8, -) X TPMWF GPS UIF UIJOH UIBU XF XBOU 1S(Q|8, -) 1S(Q|8, -) = 1S(8, -|Q) 1S(Q) 1S(8, -) BZFT UIFPSFN *U TBZT UIBU UIF QSPCBCJMJUZ PG BOZ QBSUJDVMBS WBMVF PG Q DPOTJE JT FRVBM UP UIF QSPEVDU PG UIF SFMBUJWF QMBVTJCJMJUZ PG UIF EBUB DPOEJUJPOBM PO PS QMBVTJCJMJUZ PG Q EJWJEFE CZ UIJT UIJOH 1S(8, -) XIJDI *MM DBMM UIF BWFSBHF Relative plausibility of each possible p, 
 after learning W,L We multiply because that’s how the garden counts!
56. With Numbers Ignore the mathematics for the moment and just draw the owl with numbers 1. For each possible value of p 2. Compute product Pr(W,L|p)Pr(p) 3. Relative sizes of products in (2) are posterior probabilities Bayesian owl
57. 0.0 0.2 0.4 0.6 0.8 1.0 proportion of water 0 0.5 1 3 values
58. 0.0 0.2 0.4 proportion of water posterior probability 0 0.25 0.5 0.75 1 5 values 0.0 0.2 0.4 0.6 0.8 1.0 proportion of water 0 0.5 1 3 values
59. 10 values 0.00 0.10 0.20 0.30 proportion of water posterior probability 0 0.22 0.44 0.67 0.89 0.0 0.2 0.4 proportion of water posterior probability 0 0.25 0.5 0.75 1 5 values 0.0 0.2 0.4 0.6 0.8 1.0 proportion of water 0 0.5 1 3 values
60. 0.00 0.04 0.08 0.12 proportion of water posterior probability 0 0.21 0.47 0.74 1 20 values 0.0000 0.0010 0.0020 proportion of water posterior probability 0 0.17 0.39 0.6 0.8 1 1000 values
61. Grid Approximation Bę  4BNQMJOH GSPN B HSJEBQQSPYJNBUF QPTUFSJPS #FGPSF CFHJOOJOH UP XPSL XJUI TBNQMFT XF OFFE UP HFOFSBUF UIFN )FSFT B SFNJO GPS IPX UP DPNQVUF UIF QPTUFSJPS GPS UIF HMPCF UPTTJOH NPEFM VTJOH HSJE BQQSPYJNBUJ 3FNFNCFS UIF QPTUFSJPS IFSF NFBOT UIF QSPCBCJMJUZ PG Q DPOEJUJPOBM PO UIF EBUB 3 DPEF  +Ǿ"-$ ʚǶ . ,ǿ !-*(ʙǍ Ǣ /*ʙǎ Ǣ ' )"/#ǡ*0/ʙǎǍǍǍ Ȁ +-*Ǿ+ ʚǶ - +ǿ ǎ Ǣ ǎǍǍǍ Ȁ +-*Ǿ/ ʚǶ $)*(ǿ Ǔ Ǣ .$5 ʙǖ Ǣ +-*ʙ+Ǿ"-$ Ȁ +*./ -$*- ʚǶ +-*Ǿ/ ȉ +-*Ǿ+ +*./ -$*- ʚǶ +*./ -$*- ȅ .0(ǿ+*./ -$*-Ȁ /PX XF XJTI UP ESBX   TBNQMFT GSPN UIJT QPTUFSJPS *NBHJOF UIF QPTUFSJPS JT B CVD GVMM PG QBSBNFUFS WBMVFT OVNCFST TVDI BT     FUD 8JUIJO UIF CVDLFU FBDI WB FYJTUT JO QSPQPSUJPO UP JUT QPTUFSJPS QSPCBCJMJUZ TVDI UIBU WBMVFT OFBS UIF QFBL BSF NVDI N DPNNPO UIBO UIPTF JO UIF UBJMT 8FSF HPJOH UP TDPPQ PVU   WBMVFT GSPN UIF CVD 1SPWJEFE UIF CVDLFU JT XFMM NJYFE UIF SFTVMUJOH TBNQMFT XJMM IBWF UIF TBNF QSPQPSUJPOT UIF FYBDU QPTUFSJPS EFOTJUZ ćFSFGPSF UIF JOEJWJEVBM WBMVFT PG Q XJMM BQQFBS JO PVS TBNQ JO QSPQPSUJPO UP UIF QPTUFSJPS QMBVTJCJMJUZ PG FBDI WBMVF Let’s draw the owl
62. 0 200 600 1000 0.0 0.4 0.8 Index p_grid 0 200 600 1000 0.6 0.8 1.0 1.2 1.4 Index prob_p 0 200 600 1000 0.00 0.10 0.20 Index prob_data 0 200 600 1000 0.0000 0.0015 Index posterior p_grid <- seq( from=0 , to=1 , len=1000 ) prob_p <- rep( 1 , 1000 ) prob_data <- dbinom( 6 , 9 , prob=p_grid ) posterior <- prob_data * prob_p posterior <- posterior / sum(posterior)
63. 0 200 600 1000 0.0 0.4 0.8 Index p_grid 0 200 600 1000 0.00 0.10 0.20 Index prob_data 0 200 600 1000 0.0 1.0 2.0 3.0 Index prob_p 0 200 600 1000 0.0000 0.0015 Index posterior 0 200 600 1000 0.0000 0.0015 0.0030 Index posterior p_grid <- seq( from=0 , to=1 , len=1000 ) prob_p <- dbeta( p_grid , 3 , 1 ) prob_data <- dbinom( 6 , 9 , prob=p_grid ) posterior <- prob_data * prob_p posterior <- posterior / sum(posterior)
64. Many Ways to Count Grid Approximation inefficient Other methods: Quadratic approximation Markov chain Monte Carlo (MCMC)
65. From Posterior to Prediction Implications of model depend upon entire posterior Must average any inference over entire posterior This usually requires integral calculus OR we can just take samples from the posterior
66. Uncertainty Causal model Implications ⇨ ⇨
67. Uncertainty Causal model Implications ⇨ ⇨
68. Sample from posterior  4BNQMJOH GSPN B HSJEBQQSPYJNBUF QPTUFSJPS #FGPSF CFHJOOJOH UP XPSL XJUI TBNQMFT XF OFFE UP HFOFSBUF UIFN )FSFT B SFNJOEFS GPS IPX UP DPNQVUF UIF QPTUFSJPS GPS UIF HMPCF UPTTJOH NPEFM VTJOH HSJE BQQSPYJNBUJPO 3FNFNCFS UIF QPTUFSJPS IFSF NFBOT UIF QSPCBCJMJUZ PG Q DPOEJUJPOBM PO UIF EBUB 3 DPEF  p_grid <- seq( from=0 , to=1 , length.out=1000 ) prob_p <- rep( 1 , 1000 ) prob_data <- dbinom( 6 , size=9 , prob=p_grid ) posterior <- prob_data * prob_p posterior <- posterior / sum(posterior) /PX XF XJTI UP ESBX   TBNQMFT GSPN UIJT QPTUFSJPS *NBHJOF UIF QPTUFSJPS JT B CVDLFU GVMM PG QBSBNFUFS WBMVFT OVNCFST TVDI BT     FUD 8JUIJO UIF CVDLFU FBDI WBMVF FYJTUT JO QSPQPSUJPO UP JUT QPTUFSJPS QSPCBCJMJUZ TVDI UIBU WBMVFT OFBS UIF QFBL BSF NVDI NPSF DPNNPO UIBO UIPTF JO UIF UBJMT 8FSF HPJOH UP TDPPQ PVU   WBMVFT GSPN UIF CVDLFU 1SPWJEFE UIF CVDLFU JT XFMM NJYFE UIF SFTVMUJOH TBNQMFT XJMM IBWF UIF TBNF QSPQPSUJPOT BT UIF FYBDU QPTUFSJPS EFOTJUZ ćFSFGPSF UIF JOEJWJEVBM WBMVFT PG Q XJMM BQQFBS JO PVS TBNQMFT JO QSPQPSUJPO UP UIF QPTUFSJPS QMBVTJCJMJUZ PG FBDI WBMVF )FSFT IPX ZPV DBO EP UIJT JO 3 XJUI POF MJOF PG DPEF 3 DPEF  samples <- sample( p_grid , prob=posterior , size=1e4 , replace=TRUE ) 3 DPEF  p_grid <- seq( from=0 , to=1 , length.out=1000 ) prob_p <- rep( 1 , 1000 ) prob_data <- dbinom( 6 , size=9 , prob=p_grid ) posterior <- prob_data * prob_p posterior <- posterior / sum(posterior) /PX XF XJTI UP ESBX   TBNQMFT GSPN UIJT QPTUFSJPS *NBHJOF UIF QPTUFSJPS JT B CVDLFU GVMM PG QBSBNFUFS WBMVFT OVNCFST TVDI BT     FUD 8JUIJO UIF CVDLFU FBDI WBMVF FYJTUT JO QSPQPSUJPO UP JUT QPTUFSJPS QSPCBCJMJUZ TVDI UIBU WBMVFT OFBS UIF QFBL BSF NVDI NPSF DPNNPO UIBO UIPTF JO UIF UBJMT 8FSF HPJOH UP TDPPQ PVU   WBMVFT GSPN UIF CVDLFU 1SPWJEFE UIF CVDLFU JT XFMM NJYFE UIF SFTVMUJOH TBNQMFT XJMM IBWF UIF TBNF QSPQPSUJPOT BT UIF FYBDU QPTUFSJPS EFOTJUZ ćFSFGPSF UIF JOEJWJEVBM WBMVFT PG Q XJMM BQQFBS JO PVS TBNQMFT JO QSPQPSUJPO UP UIF QPTUFSJPS QMBVTJCJMJUZ PG FBDI WBMVF )FSFT IPX ZPV DBO EP UIJT JO 3 XJUI POF MJOF PG DPEF 3 DPEF  samples <- sample( p_grid , prob=posterior , size=1e4 , replace=TRUE ) ćF XPSLIPSTF IFSF JT sample XIJDI SBOEPNMZ QVMMT WBMVFT GSPN B WFDUPS ćF WFDUPS JO UIJT DBTF JT p_grid UIF HSJE PG QBSBNFUFS WBMVFT ćF QSPCBCJMJUZ PG FBDI WBMVF JT HJWFO CZ posterior XIJDI ZPV DPNQVUFE KVTU BCPWF
69. Sample from posterior Figure 3.1  4".1-*/( 50 46.."3*;&  0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.5 1.0 1.5 2.0 2.5 proportion water (p) Density 'ĶĴłĿIJ ƋƉ 4BNQMJOH QBSBNFUFS WBMVFT GSPN UIF QPTUFSJPS EJTUSJCVUJPO -Fę UIPVTBOE TBNQMFT GSPN UIF QPTUFSJPS JNQMJFE CZ UIF HMPCF UPTTJOH EBUB BOE NPEFM 3JHIU ćF EFOTJUZ PG TBNQMFT WFSUJDBM BU FBDI QBSBNFUFS FYJTUT JO QSPQPSUJPO UP JUT QPTUFSJPS QSPCBCJMJUZ TVDI UIBU WBMVFT OFBS UIF QFBL BSF NVDI NPSF DPNNPO UIBO UIPTF JO UIF UBJMT 8FSF HPJOH UP TDPPQ PVU   WBMVFT GSPN UIF CVDLFU 1SPWJEFE UIF CVDLFU JT XFMM NJYFE UIF SFTVMUJOH TBNQMFT XJMM IBWF UIF TBNF QSPQPSUJPOT BT UIF FYBDU QPTUFSJPS EFOTJUZ ćFSFGPSF UIF JOEJWJEVBM WBMVFT PG Q XJMM BQQFBS JO PVS TBNQMFT JO QSPQPSUJPO UP UIF QPTUFSJPS QMBVTJCJMJUZ PG FBDI WBMVF )FSFT IPX ZPV DBO EP UIJT JO 3 XJUI POF MJOF PG DPEF 3 DPEF  samples <- sample( p_grid , prob=posterior , size=1e4 , replace=TRUE ) ćF XPSLIPSTF IFSF JT sample XIJDI SBOEPNMZ QVMMT WBMVFT GSPN B WFDUPS ćF WFDUPS JO UIJT DBTF JT p_grid UIF HSJE PG QBSBNFUFS WBMVFT ćF QSPCBCJMJUZ PG FBDI WBMVF JT HJWFO CZ posterior XIJDI ZPV DPNQVUFE KVTU BCPWF
70. Sample predictions FYJTUT JO QSPQPSUJPO UP JUT QPTUFSJPS QSPCBCJMJUZ TVDI UIBU WBMVFT OFBS UIF QFBL BSF NVDI NPSF DPNNPO UIBO UIPTF JO UIF UBJMT 8FSF HPJOH UP TDPPQ PVU   WBMVFT GSPN UIF CVDLFU 1SPWJEFE UIF CVDLFU JT XFMM NJYFE UIF SFTVMUJOH TBNQMFT XJMM IBWF UIF TBNF QSPQPSUJPOT BT UIF FYBDU QPTUFSJPS EFOTJUZ ćFSFGPSF UIF JOEJWJEVBM WBMVFT PG Q XJMM BQQFBS JO PVS TBNQMFT JO QSPQPSUJPO UP UIF QPTUFSJPS QMBVTJCJMJUZ PG FBDI WBMVF )FSFT IPX ZPV DBO EP UIJT JO 3 XJUI POF MJOF PG DPEF 3 DPEF  samples <- sample( p_grid , prob=posterior , size=1e4 , replace=TRUE ) ćF XPSLIPSTF IFSF JT sample XIJDI SBOEPNMZ QVMMT WBMVFT GSPN B WFDUPS ćF WFDUPS JO UIJT DBTF JT p_grid UIF HSJE PG QBSBNFUFS WBMVFT ćF QSPCBCJMJUZ PG FBDI WBMVF JT HJWFO CZ posterior XIJDI ZPV DPNQVUFE KVTU BCPWF 3 DPEF  w <- rbinom( 1e4 , size=9 , prob=0.6 ) ćJT HFOFSBUFT   1e4 TJNVMBUFE QSFEJDUJPOT PG  HMPCF UPTTFT size=9 BTTVNJOH Q = . ćF QSFEJDUJPOT BSF TUPSFE BT DPVOUT PG XBUFS TP UIF UIFPSFUJDBM NJOJNVN JT [FSP BOE UIF UIFPSFUJDBM NBYJNVN JT OJOF :PV DBO VTF simplehist(w) JO UIF rethinking QBDLBHF UP HFU B DMFBO IJTUPHSBN PG ZPVS TJNVMBUFE PVUDPNFT "MM ZPV OFFE UP QSPQBHBUF QBSBNFUFS VODFSUBJOUZ JOUP UIFTF QSFEJDUJPOT JT SFQMBDF UIF WBMVF 0.6 XJUI TBNQMFT GSPN UIF QPTUFSJPS 3 DPEF  w <- rbinom( 1e4 , size=9 , prob=samples ) ćF TZNCPM samples BCPWF JT UIF TBNF MJTU PG SBOEPN TBNQMFT GSPN UIF QPTUFSJPS EJTUSJCV UJPO UIBU ZPVWF VTFE JO QSFWJPVT TFDUJPOT 'PS FBDI TBNQMFE WBMVF B SBOEPN CJOPNJBM PCTFS WBUJPO JT HFOFSBUFE 4JODF UIF TBNQMFE WBMVFT BQQFBS JO QSPQPSUJPO UP UIFJS QPTUFSJPS QSPCB CJMJUJFT UIF SFTVMUJOH TJNVMBUFE PCTFSWBUJPOT BSF BWFSBHFE PWFS UIF QPTUFSJPS :PV DBO NBOJQ VMBUF UIFTF TJNVMBUFE PCTFSWBUJPOT KVTU MJLF ZPV NBOJQVMBUF TBNQMFT GSPN UIF QPTUFSJPS‰ZPV DBO DPNQVUF JOUFSWBMT BOE QPJOU TUBUJTUJDT VTJOH UIF TBNF QSPDFEVSFT *G ZPV QMPU UIFTF TBN QMFT ZPVMM TFF UIF EJTUSJCVUJPO TIPXO JO UIF SJHIUIBOE QMPU JO 'ĶĴłĿIJ ƋƎ ćF TJNVMBUFE NPEFM QSFEJDUJPOT BSF RVJUF DPOTJTUFOU XJUI UIF PCTFSWFE EBUB JO UIJT DBTF‰UIF BDUVBM DPVOU PG  MJFT SJHIU JO UIF NJEEMF PG UIF TJNVMBUFE EJTUSJCVUJPO ćFSF JT RVJUF B MPU PG TQSFBE UP UIF QSFEJDUJPOT CVU B MPU PG UIJT TQSFBE BSJTFT GSPN UIF CJOPNJBM QSP DFTT JUTFMG OPU VODFSUBJOUZ BCPVU Q 4UJMM JUE CF QSFNBUVSF UP DPODMVEF UIBU UIF NPEFM JT QFSGFDU 4P GBS XFWF POMZ WJFXFE UIF EBUB KVTU BT UIF NPEFM WJFXT JU &BDI UPTT PG UIF HMPCF
71. Sampling is Fun & Easy Sample from posterior, compute desired quantity for each sample, profit Much easier than doing integrals Turn a calculus problem into 
 a data summary problem MCMC produces only samples anyway
72. Sampling is Handsome & Handy Things we’ll compute with sampling: Model-based forecasts Causal effects Counterfactuals Prior predictions ???
73. Prior Prior
74. Bayesian data analysis For each possible explanation of the data, Count all the ways data can happen. Explanations with more ways to produce the data are more plausible.
75. Bayesian modesty No guarantees except logical Probability theory is a method of logically deducing implications of data under assumptions that you must choose Any framework selling you more is hiding assumptions
76. Course Schedule Week 1 Bayesian inference Chapters 1, 2, 3 Week 2 Linear models & Causal Inference Chapter 4 Week 3 Causes, Confounds & Colliders Chapters 5 & 6 Week 4 Overfitting / Interactions Chapters 7 & 8 Week 5 MCMC & Generalized Linear Models Chapters 9, 10, 11 Week 6 Integers & Other Monsters Chapters 11 & 12 Week 7 Multilevel models I Chapter 13 Week 8 Multilevel models II Chapter 14 Week 9 Measurement & Missingness Chapter 15 Week 10 Generalized Linear Madness Chapter 16 https://github.com/rmcelreath/statrethinking_2022