Avoid being clever • Intuition terrible guide to probability • No need to be clever; just ruthlessly apply conditional probability • Pr(want to know|already know)
Getting Ruthless • Express information as constraints and distributions => let logic discover implications • No need to be clever • Examples: • Measurement error • Missing data
Measurement error • Measurement always entails error • Typical linear regression: interpret sigma as “error” on outcome • What if error isn’t constant? • What if error is on predictors?
• Divorce rate estimates move from observed values. Why? 1.0 1.2 1.4 AK DC E RI SD VT WY -2 -1 0 1 2 3 -2 -1 0 1 2 median age marriage (std) divorce rate (std) AR ID ME MN ND RI WY SJOLBHF SFTVMUJOH GSPN NPEFMJOH UIF NFBTVSFNFOU FS IF PSJHJOBM NFBTVSFNFOU
UIF MFTT TISJOLBHF JO UIF QPT
Error on predictor • What about error on predictor? • Many procedures invented • errors-in-variables • reduced major axis • total least squares • Our approach will be logical • State information • Deduce implications • Garbage in? You know what comes out. 0 1 2 3 15 20 25 30 log population Marriage rate
Error on predictor • Both divorce rate and marriage rate shrink • Divorce shrinks more. Why? • Marriage rate not strongly associated with outcome => not much pooling through regression => not much shrinkage . -1 0 1 2 -2 -1 0 1 2 marriage rate (std) divorce rate (std) +*$)/.ΰ ά ./ ά ./ α !*- ΰ $ $) ͠)-*2ΰα α
Measurement error • Common malady: “data” come from uncertain procedure, but uncertainty discarded at analysis • Examples: • Predicting with averages • Parentage analysis • Phylogenetics: distribution of trees • Archaeology/paleontology/forensics: identification, sexing, aging, dating • Propagate uncertainty
Missing data • Missing values commonplace • Usual approach: complete-case analysis • drop all cases with any missing values • Discards a lot of information • Alternatives • replace missing with mean of column: NEVER DO THIS • Multiple imputation • Bayesian imputation • others
Three Types of Missingness K JT OFPDPSUFY QFSDFOU
, JT NJML FOFSHZ
BOE 6 JT TPNF VOPCTFSWFE OE # QPTJUJWFMZ DPSSFMBUFE 8F XBOU UP BEE NJTTJOHOFTT UP UIJT T SFBMJ[JOH UIBU XF IBWFOU PCTFSWFE # OFPDPSUFY QFSDFOU 8FWF QBSUJBMMZ PCTFSWFE TFU PG WBMVFT HFOFSBUFE CZ # BOE TPNF QSPDFTT U HFOFSBUFT NJTTJOH WBMVFT 3# BOE OPX BEE JU UP PVS HSBQI B B_obs K M R_B U UIJOL PG UIF PCTFSWFEXJUINJTTJOHOFTT #ļįŀ BT CFJOH B GVODUJPO CTFSWFE # BOE UIF NJTTJOHOFTT QSPDFTT 3# 8F DBO USFBU 3# MJLF E ćFO XF DBO VTF PVS GSJFOE UIF CBDLEPPS DSJUFSJPO UP ĕHVSF PVU O BCPVU OFFE UP DPOEJUJPO PO 3# *O UIF HSBQI BCPWF
XF XBOU O , 5P ĕHVSF PVU XIFO UIF FTUJNBUF JT DPOGPVOEFE
XF ĕOE BMM UIF OZ PG UIFN BSF CBDLEPPST
XF OFFE UP DMPTF UIPTF UP HFU UIF DBVTBM BCPWF
XF OFFE UP DPOEJUJPO PO . UP DMPTF UIF JOEJSFDU QBUI
KVTU UIFSF JT OP QBUI UISPVHI 3# 4P JU EPFTOU DPOGPVOE JOGFSFODF .*44*/( %"5" "OPUIFS QPTTJCJMJUZ JT UIBU TPNF PUIFS WBSJBCMF JOĘVFODFT UIF NJTTJOHOFTT QSPDFTT B B_obs K M R_B U /PX . JOĘVFODFT 3#
XIJDI NFBOT GPS FYBNQMF UIBU TQFDJFT XJUI TNBMMFS CPEJFT BSF NPSF PS MFTT MJLFMZ UP IBWF NJTTJOH WBMVFT JO #ļįŀ ćJT DPVME IBQQFO JG SFTFBSDIFST BSF MFTT JOUFS FTUFE JO TNBMM TQFDJFT BOE TP EP OPU PęFO HP UISPVHI UIF USPVCMF PG NBLJOH EFUBJMFE CSBJO NFBTVSFNFOUT GPS UIFN 8IBU IBQQFOT JO UIJT DBTF ćFSF JT OPX B CBDLEPPS QBUI GSPN #ļįŀ UIPVHI 3# UP , 4P UIF NJTTJOHOFTT QSPDFTT DBO DPOGPVOE PVS JOGFSFODF
VOMFTT XF DBO DMPTF UIF CBDLEPPS *O UIJT DBTF
XF DBO TIVU UIF CBDLEPPS CZ DPOEJUJPOJOH PO . 8F NJHIU IBWF EPOF UIJT BOZXBZ
CFDBVTF XF XBOU UP UIF EJSFDU JOĘVFODF PG # PO , ćJT UZQF PG NJTT JOHOFTT JT LOPXO CZ BOPUIFS VOGPSUVOBUFMZ BXLXBSE OBNF
ĺĶŀŀĶĻĴ ĮŁ ĿĮĻıļĺ ."3
8F EPOU OFFE UP EJTDPWFS UIF NJTTJOHOFTT QSPDFTT BCPWF #VU UIFSF JT TPNFUIJOH FMTF XF JU EJEOU QSPEVDF CJBT /PX JU XJMM QSPEVDF CJBT
CFDBVTF JU SFNPW PUIFS WBSJBCMFT BOE DPOGPVOET PVS JOGFSFODF #VU XF DBO MVDLJMZ BOE QPTTJCMZ NBLF WBMJE JOGFSFODFT 8JUI .$"3 UIF ĕSTU UZQF
J VTFGVM 8JUI ."3 UIJT TFDPOE UZQF
JU JT NBOEBUPSZ ćF UIJSE UZQF PG NJTTJOHOFTT JT B USVF UFSSPS 4VQQPTF OPX UI PG # BSF UIF POFT UIBU UFOE UP CF NJTTJOH ćJT DPVME IBQQFO GPS F PG OFPDPSUFY BSF TUVEJFE FYBDUMZ GPS UIBU SFBTPO ćFSF XPVME CF N BCPVU TVDI CSBJOT
CVU GFX QSFDJTF NFBTVSFNFOUT BCPVU CSBJOT XJUI NJHIU MPPL MJLF UIJT B B_obs K M R_B U ćJT JT B SFBM QSPCMFN /PX UIFSF JT B CBDLEPPS GSPN #PCT UISPVHI XBZ UP , ćFSF JT OP XBZ UP DMPTF UIJT CBDLEPPS
CFDBVTF XF DBOU D WBSJBCMF # $POEJUJPOJOH PO . EPFTOU IFMQ *G XF DBO NPEFM UIF N UIFSF JT TUJMM IPQF #VU JO HFOFSBM UIFSF BSF OP HVBSBOUFFT IFSF ć MISSING COMPLETELY AT RANDOM MISSING AT RANDOM MISSING NOT AT RANDOM MCAR MAR MNAR Possibly most confusing statistical terms ever invented.
Missing Completely At Random • MCAR: K is unconditionally independent of R_B • Do not need to condition on anything for R_B not to be a confound • On right, no path through R_B, conditioning on B_obs • Do not NEED to impute • But imputation adds precision K XIFSF . JT CPEZ NBTT
# JT OFPDPSUFY QFSDFOU
, JT NJML FOFSHZ
BOE WBSJBCMF UIBU SFOEFST . BOE # QPTJUJWFMZ DPSSFMBUFE 8F XBOU UP B HSBQI 8IBU UIBU NFBOT JT SFBMJ[JOH UIBU XF IBWFOU PCTFSWFE # OF JOTUFBE PCTFSWFE #PCT
B QBSUJBMMZ PCTFSWFE TFU PG WBMVFT HFOFSBUFE -FUT OBNF UIF QSPDFTT UIBU HFOFSBUFT NJTTJOH WBMVFT 3# BOE OPX BE B B_obs K M R_B U ćF XBZ UP SFBE UIJT JT UP UIJOL PG UIF PCTFSWFEXJUINJTTJOHOFTT # PG UIF DPNQMFUFCVUVOPCTFSWFE # BOE UIF NJTTJOHOFTT QSPDFTT 3# BOPUIFS QPTTJCMF DPOGPVOE ćFO XF DBO VTF PVS GSJFOE UIF CBDLEPP XIFO XF OFFE JOGPSNBUJPO BCPVU OFFE UP DPOEJUJPO PO 3# *O UI JOGFS UIF JOĘVFODF PG # PO , 5P ĕHVSF PVU XIFO UIF FTUJNBUF JT DPO QBUIT GSPN #ļįŀ UP , *G BOZ PG UIFN BSF CBDLEPPST
XF OFFE UP DMPT
Missing (Simply) At Random • MAR: K is conditionally independent of R_B • Must to condition on M for R_B not to be a confound • Still must impute to de-bias estimates • Why? If you delete cases of M/K where B is missing, missingness obscures causation. .*44*/( %"5" "OPUIFS QPTTJCJMJUZ JT UIBU TPNF PUIFS WBSJBCMF JOĘVFODFT UI B B_obs K M R_B U /PX . JOĘVFODFT 3#
XIJDI NFBOT GPS FYBNQMF UIBU TQFDJFT XJ PS MFTT MJLFMZ UP IBWF NJTTJOH WBMVFT JO #ļįŀ ćJT DPVME IBQQFO FTUFE JO TNBMM TQFDJFT BOE TP EP OPU PęFO HP UISPVHI UIF USPVC NFBTVSFNFOUT GPS UIFN 8IBU IBQQFOT JO UIJT DBTF ćFSF JT O #ļįŀ UIPVHI 3# UP , 4P UIF NJTTJOHOFTT QSPDFTT DBO DPOGPVOE P DMPTF UIF CBDLEPPS *O UIJT DBTF
XF DBO TIVU UIF CBDLEPPS CZ DPO
Missing Not At Random Missingness more likely for specific values of B. How can this happen? FGVM 8JUI ."3 UIJT TFDPOE UZQF
JU JT NBOEBUPSZ ćF UIJSE UZQF PG NJTTJOHOFTT JT B USVF UFSSPS 4VQQPTF OPX UIBU TQFDJFT XJUI MPX WBMVFT # BSF UIF POFT UIBU UFOE UP CF NJTTJOH ćJT DPVME IBQQFO GPS FYBNQMF JG TQFDJFT XJUI MPUT OFPDPSUFY BSF TUVEJFE FYBDUMZ GPS UIBU SFBTPO ćFSF XPVME CF NBOZ QSFDJTF NFBTVSFNFOUT PVU TVDI CSBJOT
CVU GFX QSFDJTF NFBTVSFNFOUT BCPVU CSBJOT XJUI MFTT OFPDPSUFY ćF %"( JHIU MPPL MJLF UIJT B B_obs K M R_B U JT JT B SFBM QSPCMFN /PX UIFSF JT B CBDLEPPS GSPN #PCT UISPVHI UIF NFDIBOJTN 3# BMM UIF BZ UP , ćFSF JT OP XBZ UP DMPTF UIJT CBDLEPPS
CFDBVTF XF DBOU DPOEJUJPO PO UIF DPNQMFUF SJBCMF # $POEJUJPOJOH PO . EPFTOU IFMQ *G XF DBO NPEFM UIF NJTTJOHOFTT NFDIBOJTN 3# FSF JT TUJMM IPQF #VU JO HFOFSBM UIFSF BSF OP HVBSBOUFFT IFSF ćJT TDFOBSJP JT TPNFUJNFT
Missing Not At Random Can also arise through unobserved variables (right). T TFDPOE UZQF
JU JT NBOEBUPSZ NJTTJOHOFTT JT B USVF UFSSPS 4VQQPTF OPX UIBU TQFDJFT XJUI MPX WBMVFT FOE UP CF NJTTJOH ćJT DPVME IBQQFO GPS FYBNQMF JG TQFDJFT XJUI MPUT E FYBDUMZ GPS UIBU SFBTPO ćFSF XPVME CF NBOZ QSFDJTF NFBTVSFNFOUT FX QSFDJTF NFBTVSFNFOUT BCPVU CSBJOT XJUI MFTT OFPDPSUFY ćF %"( B B_obs K M R_B U /PX UIFSF JT B CBDLEPPS GSPN #PCT UISPVHI UIF NFDIBOJTN 3# BMM UIF BZ UP DMPTF UIJT CBDLEPPS
CFDBVTF XF DBOU DPOEJUJPO PO UIF DPNQMFUF OH PO . EPFTOU IFMQ *G XF DBO NPEFM UIF NJTTJOHOFTT NFDIBOJTN 3#
JO HFOFSBM UIFSF BSF OP HVBSBOUFFT IFSF ćJT TDFOBSJP JT TPNFUJNFT DBMMFE ĺĶŀŀĶĻĴ ĻļŁ ĮŁ ĿĮĻıļĺ ./"3 * LOPXUIFTF UFSNT BSF GPDVT PO JT UIBU ./"3 BSJTFT XIFO UIFSF JT OP TFU PG WBSJBCMFT UP DPOEJ CBDLEPPST UISPVHI 3# -PUT PG EJČFSFOU HSBQIT DBO MFBE UP UIBU )FSF B B_obs K M R_B U1 U2 /PX JU JTOU UIF # WBMVFT UIFNTFMWFT UIBU QSPEVDF NJTTJOHOFTT 3BUIFS U WBSJBCMF 6 UIBU JOĘVFODFT CPUI # BOE NJTTJOHOFTT 6 DPVME CF GPS TJNJMBSJUZ UP IVNBOT )VNBOT IBWF BO VOSFBTPOBCMF BNPVOU PG OFPDPS QBZ BUUFOUJPO UP JUBOE PUIFS QSJNBUFT DMPTFMZ SFMBUFE UP VT BMTP UFOE UP *G UIPTF QSJNBUFT BSF TUVEJFE NPSF JOUFOTFMZ
# WBMVFT XJMM CF NJTTJOH N
Missing Not At Random • MNAR: K is unconditionally dependent on R_B • If you can model R_B, might be okay • No guarantees .*44*/( %"5" "/% 05)&3 0110356/*5*&4 DBMMFE ĺĶŀŀĶĻĴ ĻļŁ ĮŁ ĿĮĻıļĺ ./"3 * LOPXUIFTF UFSNT GPDVT PO JT UIBU ./"3 BSJTFT XIFO UIFSF JT OP TFU PG WBSJBCMFT UP DPO CBDLEPPST UISPVHI 3# -PUT PG EJČFSFOU HSBQIT DBO MFBE UP UIBU ) B B_obs K M R_B U1 U2 /PX JU JTOU UIF # WBMVFT UIFNTFMWFT UIBU QSPEVDF NJTTJOHOFTT 3BUI WBSJBCMF 6 UIBU JOĘVFODFT CPUI # BOE NJTTJOHOFTT 6 DPVME CF G TJNJMBSJUZ UP IVNBOT )VNBOT IBWF BO VOSFBTPOBCMF BNPVOU PG OFP QBZ BUUFOUJPO UP JUBOE PUIFS QSJNBUFT DMPTFMZ SFMBUFE UP VT BMTP UFOE *G UIPTF QSJNBUFT BSF TUVEJFE NPSF JOUFOTFMZ
# WBMVFT XJMM CF NJTTJO
MISSING COMPLETELY AT RANDOM MISSING AT RANDOM MISSING NOT AT RANDOM H* A D H H* A D H H* A D H DOG EATS ANY HOMEWORK DOG EATS STUDENTS’ HOMEWORK DOG EATS BAD HOMEWORK H: Homework H*: Homework with missing values A: Attribute of student D: Dog (missingness mechanism)
Milk energy MAR • Consider just neocortex variable: • Q: What is your best guess of each missing value? • A: Posterior distribution derived from remaining data neocortex.perc 1 55.16 2 NA 3 NA 4 NA 5 NA 6 64.54 7 64.54 8 67.64 9 NA 10 68.85 11 58.85 12 61.69 13 60.32 14 NA 15 NA 16 69.97 17 NA 18 70.41 19 NA 20 73.40 21 NA 22 67.53 23 NA 24 71.26 25 72.60 26 NA 27 70.24 28 76.30 29 75.49
Missing data • Can also impute discrete values, but need another technique (see text) • Extends to many model types: • Mark-recapture, occupancy (presence/absence) • Latent-state models (hidden Markov models)
Final Homework • A little imputation practice • Finish for complete sense of accomplishment EJČFSFODFT .BLF XIBUFWFS BEEJUJPOBM DBMDVMBUJPOT QVSTVJU PG BO BOTXFS #PT QSJNJHFOJVT
The Golem of Prague “Even the most perfect of Golem, risen to life to protect us, can easily change into a destructive force. Therefore let us treat carefully that which is strong, just as we bow kindly and patiently to that which is weak.” Rabbi Judah Loew ben Bezalel (1512–1609) From Breath of Bones: A Tale of the Golem
rmcelreath
asial_edu
signer
pylapp
yasslab
jonoalderson
juliawrobel
kfujita
pnuslide
vanstee
kneath
morganepeng
yeseniaperezcruz
3n
lauravandoore
sonatard
brianwarren
jlugia
denniskardys
lara
robhawkes