The Golem of Prague Richard McElreath Statistical Rethinking Week 1 

No content

Topics Week 1 Bayesian inference Chapters 1, 2, 3 Week 2 Linear models Chapters 3 & 4 Week 3 Multivariate models Chapter 5 Week 4 Model comparison Chapter 6 Week 5 Interactions Chapter 7 Week 6 MCMC Chapter 8 Week 7 GLMs I: Counts Chapters 9 & 10 Week 8 GLMs II: Mixtures Chapter 11 Week 9 Multilevel models Chapters 12 & 13 Week 10 Measurement error etc. Chapter 14 

Course procedures • Notes on website. Lectures recorded. • Homework: Assigned Thurs, due following Thurs • Should work together on homework • Upload to your smartsite dropbox • Format PDF (not MS Word), plain text scripts okay • Final exam • Take-home, work alone • Given out on the last day of class • Due one week later • Grade: 50% homework, 50% final exam • A lot of work, and you’ll learn a lot 

Learning curve week knowledge 

The Golem of Prague go•lem |gōlǝm| noun • (in Jewish legend) a clay figure brought to life by magic. • an automaton or robot. ORIGIN late 19th cent.: from Yiddish goylem, from Hebrew gōlem ‘shapeless mass.’ 

The Golem of Prague • How-To: (1) Get a ton of clay (2) Form into humanoid (3) Inscribe brow with emeth, “truth” (4) Give commands, very carefully 

The Golem of Prague “Even the most perfect of Golem, risen to life to protect us, can easily change into a destructive force. Therefore let us treat carefully that which is strong, just as we bow kindly and patiently to that which is weak.” Rabbi Judah Loew ben Bezalel (1512–1609) From Breath of Bones: A Tale of the Golem 

The Golems of Science Golem • Made of clay • Animated by “truth” • Powerful • Blind to creator’s intent • Easy to misuse • Fictional Model • Made of...silicon? • Animated by “truth” • Hopefully powerful • Blind to creator’s intent • Easy to misuse • Always false 

Against Tests • Specialized, pre-made golems, “procedures” • Most developed in early 20th century, fragile, eclipsed by more recent tools • Users don’t know they are using models • Symptom of naive falsificationism O, that way madness lies 

The Failure of Falsification • Popperism: Science progresses by logical falsification, ergo statistics should aim to falsify • Burden on individuals and individual procedures • But falsification impossible (1) Hypotheses not models (2) Measurement matters Sir Karl Popper (1902–1994) with a headache, because people keep misunderstanding him 

H0 H1 “Evolution is neutral” “Selection matters” P0A Neutral, non-equilibrium P0B Neutral, equilibrium P1B Fluctuating selection P1A Constant selection MI MII MIII Hypotheses Process models Statistical models Figure 1.2 

Measurement Matters • Modus tollens • Hypothesis H • Implication D • Observe not-D, deduce not-H • Observe D, infer nothing • e.g. H: all swans white • black swan falsifies H 

Measurement Matters • Problems with modus tollens • Observation error • Really a black swan? Ivory-billed Woodpecker? Maybe just a Sasquatch. 

Measurement Matters Observe D Observe not-D Believe H “confirmation” “measurement error” Disbelieve H “measurement error” “falsification” 

Measurement Matters • Problems with modus tollens • Observation error • Really a black swan? • Continuous hypotheses • H: most swans are white Photo: Marcin Ryczek 

The Failure of Falsification • Strict falsification impossible (1) Hypotheses not models (2) Measurement matters • Falsification is consensual, not logical • Falsifiability about demarcation, not method • Science is a social technology “There is even something like a methodological justification for individual scientists to be dogmatic and biased. Since the method of science is that of critical discussion, it is of great importance that the theories criticized should be tenaciously defended. For only in this way can we learn their real power.” —Karl Popper, The Myth of the Framework 

Golem Engineering • Need a framework for developing and vetting statistical golems • Several options • We’ll use this one • Bayesian data analysis • Multilevel modeling • Model comparison and information criteria From Breath of Bones: A Tale of the Golem 

Bayesian data analysis • Use probability to describe uncertainty • Extends discrete logic (true/false) to continuous plausibility • Computationally difficult • Markov chain Monte Carlo (MCMC) to the rescue • Used to be controversial • Ronald Fisher: Bayesian analysis “must be wholly rejected.” Pierre-Simon Laplace (1749–1827) Sir Harold Jeffreys (1891–1989) with Bertha Swirles, aka Lady Jeffreys (1903–1999) 

Bayesian data analysis • Contrast with frequentist view • Probability is just limiting frequency • Uncertainty arises from sampling variation • Bayesian probability much more general • Probability is in the golem, not in the world • Coins are not random, but our ignorance makes them so Saturn as Galileo saw it 

Multilevel models • Models with multiple levels of uncertainty • Replace parameters with models • Common uses • Repeat & imbalanced sampling • Study variation • Avoid averaging • Phylogenetics, factor and path analysis, networks, spatial models • Natural Bayesian strategy 

Model comparison • Instead of falsifying a null model, compare meaningful models • Basic problems • How to compare? • Overfitting • Ockham’s razor: Goofy • Information theory less goofy • Criteria like AIC, DIC, WAIC • Information theory inherently Bayesian 

R is the right tool (for now) • R Environment for Statistical Computing • Free, platform neutral • Not user friendly • Very powerful • Wide user base • Growing fast • Lots of help available • Interactive, flexible Stare into the abyss 

Small Worlds and Large Worlds Richard McElreath Statistical Rethinking (Chapter 2) Week 1 

Colombo’s Mistake Behaim’s globe, as detailed in 1492 

Colombo’s Mistake Behaim’s globe, as detailed in 1492 

Small and Large Worlds • Sensu L.J. Savage (1954) • Small world: The world of the golem’s assumptions. Bayesian golems are optimal, in the small world. • Large world: The real world. No guarantee of optimality for any kind of golem. • Have to worry about both 

Garden of Forking Data • The future: • Full of branching paths • Each choice closes some • The data: • Many possible events • Each observation eliminates some 

Garden of Forking Data (1) (2) (3) (4) (5) Contains 4 marbles ? Possible contents: Observe: 

Conjecture: Data: 

Conjecture: Data: 

Conjecture: Data: 

Conjecture: Data: 3 paths consistent with data 

Garden of Forking Data (1) (2) (3) (4) (5) Possible contents: Ways to produce ? 3 ? ? ? 

Garden of Forking Data (1) (2) (3) (4) (5) Possible contents: Ways to produce 0 3 ? ? 0 

3 ways 9 ways 8 ways 

3 ways 9 ways 8 ways 

3 ways 9 ways 8 ways 

Garden of Forking Data OE  XIJUF UIFSF BSF  QBUIT UIBU TVSWJWF WF DPOTJEFSFE ĕWF EJČFSFOU DPOKFDUVSFT BCPVU UIF DPOUFOUT PG UIF CBH F NBSCMFT UP GPVS CMVF NBSCMFT 'PS FBDI PG UIFTF DPOKFDUVSFT XFWF TFRVFODFT QBUIT UISPVHI UIF HBSEFO PG GPSLJOH EBUB DPVME QPUFOUJBMMZ EBUB  $POKFDUVSF 8BZT UP QSPEVDF < >  ×  ×  =  < >  ×  ×  =  < >  ×  ×  =  < >  ×  ×  =  < >  ×  ×  =  S PG XBZT UP QSPEVDF UIF EBUB GPS FBDI DPOKFDUVSF DBO CF DPNQVUFE VNCFS PG QBUIT JO FBDI iSJOHw PG UIF HBSEFO BOE UIFO CZ NVMUJQMZJOH ćJT JT KVTU B DPNQVUBUJPOBM EFWJDF *U UFMMT VT UIF TBNF UIJOH BT 'ĶĴ BWJOH UP ESBX UIF HBSEFO ćF GBDU UIBU OVNCFST BSF NVMUJQMJFE EVSJOH OHF UIF GBDU UIBU UIJT JT TUJMM KVTU DPVOUJOH PG MPHJDBMMZ QPTTJCMF QBUIT 

Updating Another draw from the bag: QBUIT DPNQBUJCMF XJUI UIF EBUB TFRVFODF  0S ZPV DPVME UBLF UIF Q PWFS DPOKFDUVSFT      BOE KVTU VQEBUF UIFN JO MJHIU PG UIF OFX PCTFS PVU UIBU UIFTF UXP NFUIPET BSF NBUIFNBUJDBMMZ JEFOUJDBM "T MPOH BT UIF OFX JT MPHJDBMMZ JOEFQFOEFOU PG UIF QSFWJPVT PCTFSWBUJPOT SFT IPX UP EP JU 'JSTU XF DPVOU UIF OVNCFST PG XBZT FBDI DPOKFDUVSF DPVME Q X PCTFSWBUJPO  ćFO XF NVMUJQMZ FBDI PG UIFTF OFX DPVOUT CZ UIF QSFWJPVT OV GPS FBDI DPOKFDUVSF *O UBCMF GPSN $POKFDUVSF 8BZT UP QSPEVDF 1SFWJPVT DPVOUT /FX DPVOU < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  X DPVOUT JO UIF SJHIUIBOE DPMVNO BCPWF TVNNBSJ[F BMM UIF FWJEFODF GPS FBDI T OFX EBUB BSSJWF BOE QSPWJEFE UIPTF EBUB BSF JOEFQFOEFOU PG QSFWJPVT PCTFSW F OVNCFS PG MPHJDBMMZ QPTTJCMF XBZT GPS B DPOKFDUVSF UP QSPEVDF BMM UIF EBUB VQ BO CF DPNQVUFE KVTU CZ NVMUJQMZJOH UIF OFX DPVOU CZ UIF PME DPVOU T VQEBUJOH BQQSPBDI BNPVOUT UP OPUIJOH NPSF UIBO BTTFSUJOH UIBU  XIFO X VT JOGPSNBUJPO TVHHFTUJOH UIFSF BSF 8QSJPS XBZT GPS B DPOKFDUVSF UP QSPEVDF B Q 4 

Using prior information marbles rare, but every bag contains at least one. Factory says: IJT FYBNQMF UIF QSJPS EBUB BOE OFX EBUB BSF PG UIF TBNF UZQF NBSCMFT ESBX #VU JO HFOFSBM UIF QSJPS EBUB BOE OFX EBUB DBO CF PG EJČFSFOU UZQFT 4VQ UIBU TPNFPOF GSPN UIF NBSCMF GBDUPSZ UFMMT ZPV UIBU CMVF NBSCMFT BSF SBSF H DPOUBJOJOH < > UIFZ NBEF  CBHT DPOUBJOJOH < > BOE  CBHT D > ćFZ BMTP FOTVSFE UIBU FWFSZ CBH DPOUBJOFE BU MFBTU POF CMVF BOE PO 8F DBO VQEBUF PVS DPVOUT BHBJO $POKFDUVSF 1SJPS XBZT 'BDUPSZ DPVOU /FX DPVOU < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  DPOKFDUVSF < > JT NPTU QMBVTJCMF CVU CBSFMZ CFUUFS UIBO < > * E EJČFSFODF JO UIFTF DPVOUT BU XIJDI XF DBO TBGFMZ EFDJEF UIBU POF PG UIF DPO SSFDU POF :PVMM TQFOE UIF OFYU DIBQUFS FYQMPSJOH UIBU RVFTUJPO OH 0SJHJOBM JHOPSBODF 8IJDI BTTVNQUJPO TIPVME XF VTF XIFO UIFSF JT OP QSF UIF QSJPS EBUB BOE OFX EBUB BSF PG UIF TBNF UZQF NBSCMFT ESBXO GSPN FSBM UIF QSJPS EBUB BOE OFX EBUB DBO CF PG EJČFSFOU UZQFT 4VQQPTF GPS POF GSPN UIF NBSCMF GBDUPSZ UFMMT ZPV UIBU CMVF NBSCMFT BSF SBSF 4P GPS H < > UIFZ NBEF  CBHT DPOUBJOJOH < > BOE  CBHT DPOUBJO Z BMTP FOTVSFE UIBU FWFSZ CBH DPOUBJOFE BU MFBTU POF CMVF BOE POF XIJUF EBUF PVS DPVOUT BHBJO $POKFDUVSF 1SJPS XBZT 'BDUPSZ DPVOU /FX DPVOU >    ×  =  >    ×  =  >    ×  =  >    ×  =  >    ×  =  < > JT NPTU QMBVTJCMF CVU CBSFMZ CFUUFS UIBO < > *T UIFSF B F JO UIFTF DPVOUT BU XIJDI XF DBO TBGFMZ EFDJEF UIBU POF PG UIF DPOKFDUVSFT :PVMM TQFOE UIF OFYU DIBQUFS FYQMPSJOH UIBU RVFTUJPO BM JHOPSBODF 8IJDI BTTVNQUJPO TIPVME XF VTF XIFO UIFSF JT OP QSFWJPVT JO 

Using prior information marbles rare. Factory says: IJT FYBNQMF UIF QSJPS EBUB BOE OFX EBUB BSF PG UIF TBNF UZQF NBSCMFT ESBX #VU JO HFOFSBM UIF QSJPS EBUB BOE OFX EBUB DBO CF PG EJČFSFOU UZQFT 4VQ UIBU TPNFPOF GSPN UIF NBSCMF GBDUPSZ UFMMT ZPV UIBU CMVF NBSCMFT BSF SBSF H DPOUBJOJOH < > UIFZ NBEF  CBHT DPOUBJOJOH < > BOE  CBHT D > ćFZ BMTP FOTVSFE UIBU FWFSZ CBH DPOUBJOFE BU MFBTU POF CMVF BOE PO 8F DBO VQEBUF PVS DPVOUT BHBJO $POKFDUVSF 1SJPS XBZT 'BDUPSZ DPVOU /FX DPVOU < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  < >    ×  =  DPOKFDUVSF < > JT NPTU QMBVTJCMF CVU CBSFMZ CFUUFS UIBO < > * E EJČFSFODF JO UIFTF DPVOUT BU XIJDI XF DBO TBGFMZ EFDJEF UIBU POF PG UIF DPO SSFDU POF :PVMM TQFOE UIF OFYU DIBQUFS FYQMPSJOH UIBU RVFTUJPO OH 0SJHJOBM JHOPSBODF 8IJDI BTTVNQUJPO TIPVME XF VTF XIFO UIFSF JT OP QSF 

Counts to plausibility Unglamorous basis of applied probability: Things that can happen more ways are more plausible. J[F JT UP BEE VQ BMM PG UIF QSPEVDUT POF GPS FBDI WBMVF Q DBO UBLF XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q UIFO EJWJEF FBDI QSPEVDU CZ UIF TVN PG QSPEVDUT QMBVTJCJMJUZ PG Q BęFS %OFX = XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q TVN PG QSPEVDUT FT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH CFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQ UZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3 ʄǤ ǭ ƾ ǐ ǃ ǐ DŽ Ǯ 

Counts to plausibility TVN PG QSPEVDUT FT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH CFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQ UZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3 ʄǤ ǭ ƾ ǐ ǃ ǐ DŽ Ǯ dz.0(ǭ24.Ǯ ƻǏƼǀ ƻǏƿƻ ƻǏƿǀ ćFTF QMBVTJCJMJUJFT BSF BMTP QSPCBCJMJUJFT‰UIFZ BSF OPOOFHBUJWF [FSP PS QPTJUJWF CFST UIBU TVN UP POF "OE BMM PG UIF NBUIFNBUJDBM UIJOHT ZPV DBO EP XJUI QSPCBCJ UIF QMBVTJCJMJUJFT GPS BMM QPTTJCMF DPOKFDUVSFT XJMM CF POF "MM ZPV OFFE UP EP JO PSEFS UP TUBO EBSEJ[F JT UP BEE VQ BMM PG UIF QSPEVDUT POF GPS FBDI WBMVF Q DBO UBLF XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q "OE UIFO EJWJEF FBDI QSPEVDU CZ UIF TVN PG QSPEVDUT QMBVTJCJMJUZ PG Q BęFS %OFX = XBZT Q DBO QSPEVDF %OFX × QSJPS QMBVTJCJMJUZ Q TVN PG QSPEVDUT ćFSFT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH UIF OVNCFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQMBV TJCJMJUZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    :PV DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3 3 DPEF  24. ʄǤ ǭ ƾ ǐ ǃ ǐ DŽ Ǯ 24.dz.0(ǭ24.Ǯ ǯƼǰ ƻǏƼǀ ƻǏƿƻ ƻǏƿǀ ćFTF QMBVTJCJMJUJFT BSF BMTP QSPCBCJMJUJFT‰UIFZ BSF OPOOFHBUJWF [FSP PS QPTJUJWF SFBM OVNCFST UIBU TVN UP POF "OE BMM PG UIF NBUIFNBUJDBM UIJOHT ZPV DBO EP XJUI QSPCBCJMJUJFT ZPV DBO BMTP EP XJUI UIFTF WBMVFT 4QFDJĕDBMMZ FBDI QJFDF PG UIF DBMDVMBUJPO IBT B EJSFDU QBSUOFS JO BQQMJFE QSPCBCJMJUZ UIFPSZ ćFTF QBSUOFST IBWF TUFSFPUZQFE OBNFT TP JUT XPSUI 

Counts to plausibility TVN PG QSPEVDUT FT OPUIJOH TQFDJBM SFBMMZ BCPVU TUBOEBSEJ[JOH UP POF "OZ WBMVF XJMM EP #VU VTJOH CFS  FOET VQ NBLJOH UIF NBUIFNBUJDT NPSF DPOWFOJFOU $POTJEFS BHBJO UIF UBCMF GSPN CFGPSF OPX VQEBUFE VTJOH PVS EFĕOJUJPOT PG Q BOE iQ UZw 1PTTJCMF DPNQPTJUJPO Q XBZT UP QSPEVDF EBUB QMBVTJCJMJUZ < >    < > .   < > .   < > .   < >    DBO RVJDLMZ DPNQVUF UIFTF QMBVTJCJMJUJFT JO 3 ʄǤ ǭ ƾ ǐ ǃ ǐ DŽ Ǯ dz.0(ǭ24.Ǯ ƻǏƼǀ ƻǏƿƻ ƻǏƿǀ ćFTF QMBVTJCJMJUJFT BSF BMTP QSPCBCJMJUJFT‰UIFZ BSF OPOOFHBUJWF [FSP PS QPTJUJWF CFST UIBU TVN UP POF "OE BMM PG UIF NBUIFNBUJDBM UIJOHT ZPV DBO EP XJUI QSPCBCJ Plausibility is probability: Set of non-negative real numbers that sum to one. Probability theory is just a set of shortcuts for counting possibilities.