Work ow (1) De ne generative model of the sample (2) De ne a speci c estimand (3) Design a statistical way to produce estimate (4) Test (3) using (1) (5) Analyze sample, summarize
Generative model of the globe Begin conceptually: How do the variables in uence one another? OVNCFS PG HMPCF UPTTFT
/ ćJT JT DIPTFO CZ UIF FYQFSJNFOUFS OVNCFS PG XBUFS QPJOUT PCTFSWFE
8 OVNCFS PG MBOE QPJOUT PCTFSWFE
- EJBHSBN UIBU TIPXT UIFTF GPVS WBSJBCMFT BOE DPOOFDUT TPNF PG UIFN DBVTBM JOĘVFODF -FUT TUBSU XJUI B CMBOL EJBHSBN BOE BEE UIF BSSP Q / 8 - BEE TPNF BSSPXT "SSPXT JO UIFTF EJBHSBNT JOEJDBUF DBVTBM JOĘ proportion of water number of tosses water observations land observations
BEE TPNF BSSPXT "SSPXT JO UIFTF EJBHSBNT JOEJDBUF DBVTBM JO BCPVU XIBU iDBVTBM JOĘVFODFw NFBOT IFSF JT UP JNBHJOF DIBOHJOH IJDI PUIFS WBSJBCMFT BMTP DIBOHF BT B DPOTFRVFODF 'PS FYBNQMF
PG HMPCF UPTTFT /
UIFO CPUI 8 BOE - NJHIU DIBOHF #VU Q XPV BOE -
CVU OPU Q 8F ESBX UIBU MJLF UIJT Q / 8 - Generative model of the globe Begin conceptually: How do the variables in uence one another? N in uences W and L in uence in uence
4."-- 803-%4 "/% -"3(& 803-%4 TU
DIBOHJOH 8 BOE - EJSFDUMZGPS FYBNQMF CZ NBOJQVMBUJOH UIF PO Q PS / #VU DIBOHJOH Q
CZ GPS FYBNQMF FSBTJOH B SBOEPN DPO JOĘVFODF 8 BOE -
BU MFBTU PO BWFSBHF 4P XF OFFE UXP NPSF BSSP Q / 8 - S DBVTBM EJBHSBN PG UIF HMPCF UPTTJOH FYQFSJNFOU ćFSF BSF TPN Generative model of the globe Begin conceptually: How do the variables in uence one another?
Work ow (1) De ne generative model of the sample (2) De ne a speci c estimand (3) Design a statistical way to produce estimate (4) Test (3) using (1) (5) Analyze sample, summarize
Bayesian data analysis For each possible explanation of the sample, Count all the ways the sample could happen. Explanations with more ways to produce the sample are more plausible.
For each possible proportion of water on the globe, Count all the ways the sample of tosses could happen. Proportions with more ways to produce the sample are more plausible.
Probability Probability: Non-negative values that sum to one Suppose W=20, L=10. en p=0.5 has ways to produce sample. Better to convert to probability. 2W × 2L = 1,073,741,824
Work ow (1) De ne generative model of the sample (2) De ne a speci c estimand (3) Design a statistical way to produce estimate (4) Test (3) using (1) (5) Analyze sample, summarize
BEEJUJPO UP NBLJOH JU FBTJFS UP SFQFBU UIF DPEF
B GVODUJPO DBO BMTP NBLF UFTUJOH FBTJFS )FSFT B WFSZ TJNQMF GVODUJPO UIBU TJNVMBUFT TBNQMJOH OJOF UJNFT GSPN B HMPCF XJUI B XBUFS 3 DPEF
# function to toss a globe covered p by water N times sim_globe <- function( p=0.7 , N=9 ) { sample(c("W","L"),size=N,prob=c(p,1-p),replace=TRUE) } /PUIJOH IBQQFOT VOUJM XF DBMM UIF GVODUJPO CZ JUT OBNF 3 DPEF
sim_globe() [1] "L" "W" "W" "W" "L" "L" "L" "W" "L" 3FQFBU DBMMJOH UIF GVODUJPO UP TFF UIBU JU TJNVMBUFT B EJČFSFOU TBNQMF FBDI UJNF "OE CZ OBNJOH UIF QSPQPSUJPO PG XBUFS p BOE OVNCFS PG UPTTFT N JO UIF GVODUJPO EFĕOJUJPO
XF DBO FBTJMZ DIBOHF UIFTF WBMVFT XIFO XF DBMM UIF GVODUJPO Possible observations Number of tosses Probability of each possible observation
BEEJUJPO UP NBLJOH JU FBTJFS UP SFQFBU UIF DPEF
B GVODUJPO DBO BMTP NBLF UFTUJOH FBTJFS )FSFT B WFSZ TJNQMF GVODUJPO UIBU TJNVMBUFT TBNQMJOH OJOF UJNFT GSPN B HMPCF XJUI B XBUFS 3 DPEF
# function to toss a globe covered p by water N times sim_globe <- function( p=0.7 , N=9 ) { sample(c("W","L"),size=N,prob=c(p,1-p),replace=TRUE) } /PUIJOH IBQQFOT VOUJM XF DBMM UIF GVODUJPO CZ JUT OBNF 3 DPEF
sim_globe() [1] "L" "W" "W" "W" "L" "L" "L" "W" "L" 3FQFBU DBMMJOH UIF GVODUJPO UP TFF UIBU JU TJNVMBUFT B EJČFSFOU TBNQMF FBDI UJNF "OE CZ OBNJOH UIF QSPQPSUJPO PG XBUFS p BOE OVNCFS PG UPTTFT N JO UIF GVODUJPO EFĕOJUJPO
XF DBO FBTJMZ DIBOHF UIFTF WBMVFT XIFO XF DBMM UIF GVODUJPO
BEEJUJPO UP NBLJOH JU FBTJFS UP SFQFBU UIF DPEF
B GVODUJPO DBO BMTP NBLF UFTUJOH FBTJFS )FSFT B WFSZ TJNQMF GVODUJPO UIBU TJNVMBUFT TBNQMJOH OJOF UJNFT GSPN B HMPCF XJUI B XBUFS 3 DPEF
# function to toss a globe covered p by water N times sim_globe <- function( p=0.7 , N=9 ) { sample(c("W","L"),size=N,prob=c(p,1-p),replace=TRUE) } /PUIJOH IBQQFOT VOUJM XF DBMM UIF GVODUJPO CZ JUT OBNF 3 DPEF
sim_globe() [1] "L" "W" "W" "W" "L" "L" "L" "W" "L" 3FQFBU DBMMJOH UIF GVODUJPO UP TFF UIBU JU TJNVMBUFT B EJČFSFOU TBNQMF FBDI UJNF "OE CZ OBNJOH UIF QSPQPSUJPO PG XBUFS p BOE OVNCFS PG UPTTFT N JO UIF GVODUJPO EFĕOJUJPO
XF DBO FBTJMZ DIBOHF UIFTF WBMVFT XIFO XF DBMM UIF GVODUJPO
Code the estimator 26"-*5: "4463"/$& # function to compute posterior distribution compute_posterior <- function( the_sample , poss=c(0,0.25,0.5,0.75,1) ) { W <- sum(the_sample=="W") # number of W observed L <- sum(the_sample=="L") # number of L observed ways <- sapply( poss , function(q) (q*4)^W * ((1-q)*4)^L ) post <- ways/sum(ways) bars <- sapply( post, function(q) make_bar(q) ) data.frame( poss , ways , post=round(post,3) , bars ) } 5P VTF UIJT GVODUJPO
ZPV OFFE UP HJWF JU B TBNQMF "OE XF DBO KVTU FNCFE UIF QSFWJPVT TJNVMBUJPO GVODUJPO JOTJEF JU Ways for p to produce W,L = (4p)W × (4–4p)L
Code the estimator 26"-*5: "4463"/$& # function to compute posterior distribution compute_posterior <- function( the_sample , poss=c(0,0.25,0.5,0.75,1) ) { W <- sum(the_sample=="W") # number of W observed L <- sum(the_sample=="L") # number of L observed ways <- sapply( poss , function(q) (q*4)^W * ((1-q)*4)^L ) post <- ways/sum(ways) bars <- sapply( post, function(q) make_bar(q) ) data.frame( poss , ways , post=round(post,3) , bars ) } 5P VTF UIJT GVODUJPO
ZPV OFFE UP HJWF JU B TBNQMF "OE XF DBO KVTU FNCFE UIF QSFWJPVT TJNVMBUJPO GVODUJPO JOTJEF JU Ways for p to produce W,L = (4p)W × (4–4p)L
Code the estimator 26"-*5: "4463"/$& # function to compute posterior distribution compute_posterior <- function( the_sample , poss=c(0,0.25,0.5,0.75,1) ) { W <- sum(the_sample=="W") # number of W observed L <- sum(the_sample=="L") # number of L observed ways <- sapply( poss , function(q) (q*4)^W * ((1-q)*4)^L ) post <- ways/sum(ways) bars <- sapply( post, function(q) make_bar(q) ) data.frame( poss , ways , post=round(post,3) , bars ) } 5P VTF UIJT GVODUJPO
ZPV OFFE UP HJWF JU B TBNQMF "OE XF DBO KVTU FNCFE UIF QSFWJPVT TJNVMBUJPO GVODUJPO JOTJEF JU Ways for p to produce W,L = (4p)W × (4–4p)L
Code the estimator 26"-*5: "4463"/$& # function to compute posterior distribution compute_posterior <- function( the_sample , poss=c(0,0.25,0.5,0.75,1) ) { W <- sum(the_sample=="W") # number of W observed L <- sum(the_sample=="L") # number of L observed ways <- sapply( poss , function(q) (q*4)^W * ((1-q)*4)^L ) post <- ways/sum(ways) bars <- sapply( post, function(q) make_bar(q) ) data.frame( poss , ways , post=round(post,3) , bars ) } 5P VTF UIJT GVODUJPO
ZPV OFFE UP HJWF JU B TBNQMF "OE XF DBO KVTU FNCFE UIF QSFWJPVT TJNVMBUJPO GVODUJPO JOTJEF JU Ways for p to produce W,L = (4p)W × (4–4p)L
Code the estimator 26"-*5: "4463"/$& # function to compute posterior distribution compute_posterior <- function( the_sample , poss=c(0,0.25,0.5,0.75,1) ) { W <- sum(the_sample=="W") # number of W observed L <- sum(the_sample=="L") # number of L observed ways <- sapply( poss , function(q) (q*4)^W * ((1-q)*4)^L ) post <- ways/sum(ways) bars <- sapply( post, function(q) make_bar(q) ) data.frame( poss , ways , post=round(post,3) , bars ) } 5P VTF UIJT GVODUJPO
ZPV OFFE UP HJWF JU B TBNQMF "OE XF DBO KVTU FNCFE UIF QSFWJPVT TJNVMBUJPO GVODUJPO JOTJEF JU Ways for p to produce W,L = (4p)W × (4–4p)L
compute_posterior( rep("W",times=9) ) poss ways post bars 1 0.00 0 0.000 2 0.25 1 0.000 (1) Test the estimator where the answer is known (2) Explore di erent sampling designs (3) Develop intuition for sampling and estimation
In nite possibilities e globe is a polyhedron with an in nite number of sides e posterior probability of any “side” p is proportional to: Only trick is normalizing to probability. A er a little calculus: #":&4*"/ 61%"5*/( " DPOUJOVPVT TPMVUJPO *O UIJT FYBNQMF
JUT OPU IBSE UP EFSJWF UIF QPTUFSJPS EJTUSJ BO FYBDU DPOUJOVPVT EJTUSJCVUJPO GVODUJPO *U UVSOT PVU UIBU UIF QPTUFSJPS QSPCBCJ PTTJCMF QSPQPSUJPO PG XBUFS Q
CFUXFFO [FSP BOE POF
JT QSPQPSUJPOBM UP Q8 ( − Q)- 8 JT UIF OVNCFS PG XBUFS PCTFSWFE BOE - JT UIF OVNCFS PG MBOE PCTFSWFE 8I UIJT JT UIF FYBDU FYQSFTTJPO UIBU XF BMSFBEZ VTFE UP DBMDVMBUF UIF SFMBUJWF OVNCFS Z WBMVF Q DPVME QSPEVDF B TBNQMF XJUI 8 XBUFS BOE - MBOE *U JT B MPHJDBM JNQMJDBUJ BSEFO PG GPSLJOH QBUIT 8IFO XF VTFE UIJT FYQSFTTJPO PSJHJOBMMZ
XF NVMUJQMJFE Q IF TVN PG BMM OVNFSBUPST GPS FWFSZ QPTTJCMF Q 'PS B ĕOJUF OVNCFS PG E TVN ; = Q Q8(−Q)-
XIFSF UIF Q OPUBUJPO NFBOT UP FWBMVBUF UIFO BEE UIFN 'PS BO JOĕOJUF OVNCFS PG QPTTJCMF Q WBMVFT GSPN [FSP PVT QBSUOFS PG
BO JOUFHSBM ; = Q8 ( − Q)-EQ iGBNPVTw ZPV NFBO iLOPXOw :PV DBO MPPL JU VQ POMJOF *O UIF DBTF PG
ESB posterior probability 0 0.5 1 0 W L W W 0 0.5 1 0 W L W W W 0 0.5 1 0 W L W W W L proportion water (p) posterior probability 0 0.5 1 0 W L W W W L W proportion water (p) 0 0.5 1 0 W L W W W L W L proportion water (p) 0 0.5 1 0 W L W W W L W L W
0.0 1.0 2.0 proportion water density 0 0.5 1 (4) No one true interval Intervals communicate shape of posterior 95% is obvious superstition. Nothing magical happens at the boundary. 99%
Letters From My Reviewers “ e author uses these cute 89% intervals, but we need to see the 95% intervals so we can tell whether any of the e ects are robust.” at an arbitrary interval contains an arbitrary value is not meaningful. Use the whole distribution.
Work ow (1) De ne generative model of the sample (2) De ne a speci c estimand (3) Design a statistical way to produce estimate (4) Test (3) using (1) (5) Analyze sample, summarize
From Posterior to Prediction Implications of model depend upon entire posterior Must average any inference over entire posterior is usually requires integral calculus OR we can just take samples from the posterior
Sampling is Fun & Easy Sample from posterior, compute desired quantity for each sample, pro t Much easier than doing integrals Turn a calculus problem into a data summary problem MCMC produces only samples anyway
Bayesian data analysis For each possible explanation of the data, Count all the ways data can happen. Explanations with more ways to produce the data are more plausible.
Bayesian modesty No guarantees except logical Probability theory is a method of logically deducing implications of data under assumptions that you must choose Any framework selling you more is hiding assumptions
Misclassi cation estimator Use the intuition from the generative model to draw out the Garden of Forking Data, build a Bayesian estimator. Two stages: (1) true samples, (2) misclassi cation
Measurement matters When there is measurement error, better to model it than to ignore it Same goes for: missing data, compliance, inclusion, etc Good news: Samples do not need to be representative of population in order to provide good estimates of population What matters is why the sample di ers
rmcelreath
matleenalaakso
bernhardsvt
sapi_kawahara
signer
kfujita
oripsolob
cbtlibrary
senayakut
kevinliebholz
holman
mthomps
mongodb
tanoku
samlambert
colly
jlugia
dougneiner
jnunemaker
keathley
panda_program