Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Big data and reproducibility
Search
Jeff L.
June 28, 2014
Science
1
760
Big data and reproducibility
Talk at JHU summer institute.
Jeff L.
June 28, 2014
Tweet
Share
More Decks by Jeff L.
See All by Jeff L.
Data Science at JHSPH
jtleek
2
110
We are all statisticians now
jtleek
2
1.6k
Other Decks in Science
See All in Science
機械学習 - 決定木からはじめる機械学習
trycycle
PRO
0
970
KH Coderチュートリアル(スライド版)
koichih
1
41k
MCMCのR-hatは分散分析である
moricup
0
370
CV_5_3dVision
hachama
0
140
MoveItを使った産業用ロボット向け動作作成方法の紹介 / Introduction to creating motion for industrial robots using MoveIt
ry0_ka
0
500
データマイニング - ノードの中心性
trycycle
PRO
0
110
Hakonwa-Quaternion
hiranabe
1
110
局所保存性・相似変換対称性を満たす機械学習モデルによる数値流体力学
yellowshippo
1
280
深層学習を用いた根菜類の個数カウントによる収量推定法の開発
kentaitakura
0
160
創薬における機械学習技術について
kanojikajino
16
5.3k
証明支援系LEANに入門しよう
unaoya
0
1.2k
Lean4による汎化誤差評価の形式化
milano0017
1
240
Featured
See All Featured
Writing Fast Ruby
sferik
628
62k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.5k
How to Think Like a Performance Engineer
csswizardry
24
1.7k
Measuring & Analyzing Core Web Vitals
bluesmoon
7
500
Scaling GitHub
holman
459
140k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
35
2.4k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
The World Runs on Bad Software
bkeepers
PRO
69
11k
Designing for humans not robots
tammielis
253
25k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Optimizing for Happiness
mojombo
379
70k
Transcript
Big data and reproducibility
None
N = SAMPLE SIZE
N = ($ YOU HAVE) ($ PER SAMPLE)
Year $ per (human) Genome
rna-seq 2008 N≈2 2010 N≈70 2013 N≈900 PMIDS: 19056941, 20220758,
24092820
www.geni.com
http://erlichlab.wi.mit.edu/familinx/index.html
None
None
None
None
what went wrong? 2 things
what went wrong? transparency The data/code weren’t reproducible
what went wrong? transparency There was a lack of cooperation
what went wrong? expertise They used silly prediction rules (Pr(FEC)
= 5/8[Pr(F) + Pr(E) + Pr(C)] – ¼)
what went wrong? expertise They had study design problems (Batch
effects)
what went wrong? expertise Their predictions weren’t locked down Today:
Pr(FEC) = 0.8 Tomorrow: Pr(FEC) = 0.1
At the end of the day the Potti analysis was
fully reproducible The problem is that the analysis was wrong
1st Discussion Point: What is reproducibility?
The goal: a result that is reproducible (the code and
data can be used to recreate the results) and replicable (you can perform the experiment again and get the same answer)
The goal: a result that is reproducible (the code and
data can be used to recreate the results) and replicable (you can perform the experiment again and get the same answer)
Who Reproduces Research? The truth is A I
don’t care The truth is B The truth is not A Original InvesRgator Reproducers The truth is A ScienRsts General Public ??? Slide courtesy R. Peng
hVps://github.com/jtleek/datasharing
2nd Discussion Point: Statistical modeling is only part of the
process
What is Data Analysis? Raw Data Cleaning /
ValidaRon Pre-‐processing Exploratory data analysis StaRsRcal model development SensiRvity analysis Finalize results / report StaRsRcs! Slide courtesy R. Peng
3rd Discussion Point: Analysis is (often) an afterthought
hVp://bit.ly/OgW3xv
None
4th Discussion Point: Traditional statistics & epidemiology ideas still matter
for big data
association between shoe size and literacy
None
None
None
1. Reproducibility by data sharing 2. Big data is not
just statistics 3. Analysis is often an afterthought 4. Traditional ideas still matter
jhudatascience.org
9 classes 1 month long Every month
Cumulative Enrollment
jtleek.com/talks