Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Big data and reproducibility
Search
Jeff L.
June 28, 2014
Science
1
770
Big data and reproducibility
Talk at JHU summer institute.
Jeff L.
June 28, 2014
Tweet
Share
More Decks by Jeff L.
See All by Jeff L.
Data Science at JHSPH
jtleek
2
110
We are all statisticians now
jtleek
2
1.6k
Other Decks in Science
See All in Science
デジタルアーカイブの教育利用促進を目指したメタデータLOD基盤に関する研究 / Research on a Metadata LOD Platform for Promoting Educational Uses of Digital Archives
masao
0
100
Accelerated Computing for Climate forecast
inureyes
0
120
防災デジタル分野での官民共創の取り組み (1)防災DX官民共創をどう進めるか
ditccsugii
0
110
データベース02: データベースの概念
trycycle
PRO
2
920
AI(人工知能)の過去・現在・未来 —AIは人間を超えるのか—
tagtag
1
140
データベース08: 実体関連モデルとは?
trycycle
PRO
0
940
データベース06: SQL (3/3) 副問い合わせ
trycycle
PRO
1
640
データベース11: 正規化(1/2) - 望ましくない関係スキーマ
trycycle
PRO
0
950
機械学習 - 授業概要
trycycle
PRO
0
250
Optimization of the Tournament Format for the Nationwide High School Kyudo Competition in Japan
konakalab
0
100
学術講演会中央大学学員会府中支部
tagtag
0
310
コンピュータビジョンによるロボットの視覚と判断:宇宙空間での適応と課題
hf149
1
360
Featured
See All Featured
How to Ace a Technical Interview
jacobian
280
24k
Statistics for Hackers
jakevdp
799
220k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
9
860
Scaling GitHub
holman
463
140k
Unsuck your backbone
ammeep
671
58k
Rails Girls Zürich Keynote
gr2m
95
14k
Building Flexible Design Systems
yeseniaperezcruz
329
39k
It's Worth the Effort
3n
187
28k
Speed Design
sergeychernyshev
32
1.1k
A Tale of Four Properties
chriscoyier
160
23k
How STYLIGHT went responsive
nonsquared
100
5.8k
How to Think Like a Performance Engineer
csswizardry
27
2k
Transcript
Big data and reproducibility
None
N = SAMPLE SIZE
N = ($ YOU HAVE) ($ PER SAMPLE)
Year $ per (human) Genome
rna-seq 2008 N≈2 2010 N≈70 2013 N≈900 PMIDS: 19056941, 20220758,
24092820
www.geni.com
http://erlichlab.wi.mit.edu/familinx/index.html
None
None
None
None
what went wrong? 2 things
what went wrong? transparency The data/code weren’t reproducible
what went wrong? transparency There was a lack of cooperation
what went wrong? expertise They used silly prediction rules (Pr(FEC)
= 5/8[Pr(F) + Pr(E) + Pr(C)] – ¼)
what went wrong? expertise They had study design problems (Batch
effects)
what went wrong? expertise Their predictions weren’t locked down Today:
Pr(FEC) = 0.8 Tomorrow: Pr(FEC) = 0.1
At the end of the day the Potti analysis was
fully reproducible The problem is that the analysis was wrong
1st Discussion Point: What is reproducibility?
The goal: a result that is reproducible (the code and
data can be used to recreate the results) and replicable (you can perform the experiment again and get the same answer)
The goal: a result that is reproducible (the code and
data can be used to recreate the results) and replicable (you can perform the experiment again and get the same answer)
Who Reproduces Research? The truth is A I
don’t care The truth is B The truth is not A Original InvesRgator Reproducers The truth is A ScienRsts General Public ??? Slide courtesy R. Peng
hVps://github.com/jtleek/datasharing
2nd Discussion Point: Statistical modeling is only part of the
process
What is Data Analysis? Raw Data Cleaning /
ValidaRon Pre-‐processing Exploratory data analysis StaRsRcal model development SensiRvity analysis Finalize results / report StaRsRcs! Slide courtesy R. Peng
3rd Discussion Point: Analysis is (often) an afterthought
hVp://bit.ly/OgW3xv
None
4th Discussion Point: Traditional statistics & epidemiology ideas still matter
for big data
association between shoe size and literacy
None
None
None
1. Reproducibility by data sharing 2. Big data is not
just statistics 3. Analysis is often an afterthought 4. Traditional ideas still matter
jhudatascience.org
9 classes 1 month long Every month
Cumulative Enrollment
jtleek.com/talks