Selfish reasons to carry out
reproducible research
Dave Lunt
dave.lunt@gmail.com
@davelunt
https:/
/bit.ly/35yDvIG
Slide 2
Slide 2 text
What is
reproducibility?
Slide 3
Slide 3 text
first Why, then How
Slide 4
Slide 4 text
It's required
Why be reproducible?
Slide 5
Slide 5 text
McNutt M. Journals unite for reproducibility. Science. 2014;346: 679. doi:10.1126/science.aaa1724
Slide 6
Slide 6 text
No content
Slide 7
Slide 7 text
No content
Slide 8
Slide 8 text
RCUK – Statement of Expectations for
Postgraduate Training
Students should receive training in experimental design and
statistics appropriate to their disciplines, and in the
importance of ensuring research results are robust and
reproducible
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
No content
Slide 11
Slide 11 text
It's the right thing
to do
It's science
Slide 12
Slide 12 text
Ask not what you can
do for reproducibility,
but what
reproducibility can
do for you
Florian Markowetz
Slide 13
Slide 13 text
It will save you time and effort
It will advance your career
Selfish Reproducible Research
Slide 14
Slide 14 text
Who here has
tried to reproduce
a published
analysis?
Who is most likely
to reproduce your
work?
Slide 15
Slide 15 text
Do experiments work first
time for you?
Slide 16
Slide 16 text
“Future You” will be most likely
person to reproduce your work
Slide 17
Slide 17 text
Future You
Previous You
Previous You does not
respond to emails
Slide 18
Slide 18 text
It will
greatly help
“future you”
Selfish reasons to
carry out reproducible
research
Slide 19
Slide 19 text
How can we save time, effort?
eg: make figures from scripts
this is reproducible analysis
Slide 20
Slide 20 text
Your research will be
faster and easier (and
better)
Slide 21
Slide 21 text
The old way
Slide 22
Slide 22 text
No content
Slide 23
Slide 23 text
No content
Slide 24
Slide 24 text
Automated reproducible
Manual
Cumulative total effort
Number of repeats
Yes you will cross
this point
Slide 25
Slide 25 text
Reproducibility makes it easier
to write papers
and respond to
reviewers
Slide 26
Slide 26 text
Reproducible research will save
you time and effort Reuse and recycle data
generation and analysis
Slide 27
Slide 27 text
Errors are ubiquitous
Retractions will hurt you
Reproducibility helps your career
Slide 28
Slide 28 text
Reproducibility will help
your career
Reputation
Rigour
New collaborators
Rapid
Agile
Future-proof
Slide 29
Slide 29 text
Choose a collaborator
Rigorous, modern, open, with
future-proof methods.
Leading the way. Prepared
and shared many of the
methods you need already.
Slide 30
Slide 30 text
Projects are not unique.
How will you build your career?
Slide 31
Slide 31 text
Required
Helps “Future You”
Easier & faster, agile
Easier papers
Helps your next project
Builds your career
Avoid major screw-ups
Makes you a cool collaborator
Selfish reasons to
be reproducible
Slide 32
Slide 32 text
Pause
But what about ...?
Slide 33
Slide 33 text
I’d rather do real science than tidy my data
It's the way I’ve always done things, and I’ve got this far
Excel is just fine
My data and code are spread across
many computers, I couldn’t do this
I’ll sort this out at the end
My field is too competitive, I can’t slow
down to do this
Slide 34
Slide 34 text
I’m not a
computational
biologist
Slide 35
Slide 35 text
How?
Slide 36
Slide 36 text
1. relax, most problems are solved
Slide 37
Slide 37 text
2. think of it as training
Slide 38
Slide 38 text
3. celebrate the quick wins
Slide 39
Slide 39 text
Quick win:
Be part of a support
community
Slide 40
Slide 40 text
Make 1
figure from
a script
Quick win
Slide 41
Slide 41 text
Butterfly_project
- DATA
-raw_data
-fig1_data
- FIGURES
-fig1.pdf
-fig2.pdf
-table1.md
- RESULTS
-PCA
-lin_regr
- SCRIPTS
-fig1.py
- README.txt
Informative names
Structured
Text description of what is where
Spend 1 morning
to organise your
data
Quick win
=> Provenance and persistence
Slide 42
Slide 42 text
2. think of it as training
Slide 43
Slide 43 text
Wilkinson et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016;3:
160018. doi:10.1038/sdata.2016.18
Records, Coding, Workflows, &
Research Objects
Slide 44
Slide 44 text
Make data open with a doi
Findable
Accessible
Interoperable
Reusable
Quick win
zenodo.org
figshare.com
osf.io
For you (and for others)
Yes, data can be private until you’re ready
Slide 45
Slide 45 text
zenodo.org
Yes you can keep
data private until
publication
Slide 46
Slide 46 text
It's free osf.io
Slide 47
Slide 47 text
osf.io
Slide 48
Slide 48 text
No content
Slide 49
Slide 49 text
File storage
Integration of
GDrive, Box,
Dropbox, Git etc
OSF cloud
storage
Everything in
one place
Slide 50
Slide 50 text
Activity
All changes
recorded with
version control
Roll back to
previous
versions
Comments and
collaborations
Slide 51
Slide 51 text
Components
are folders
Structure and
backup
Robust sharing
and privacy
Can be
published with
doi
Slide 52
Slide 52 text
try osf.io
Easy to organise project
Easy to store & publish data
Easy to collaborate
Easy reproducibility
Slide 53
Slide 53 text
Making
labwork
reproducible
protocols.io
Slide 54
Slide 54 text
It's free
Slide 55
Slide 55 text
Quick win
METHODS SECTION
Experimental procedures are briefly described here for context,
and exact protocols and reagents are detailed in doi:1234567
and doi:987654
Slide 56
Slide 56 text
Summary
Slide 57
Slide 57 text
It will save you time & effort
Selfish reasons to be
reproducible
Write once and iterate, faster, helps with ms,
helps with reviewers, don’t start projects from
scratch- build on prior reproducibility
Slide 58
Slide 58 text
It will advance your career
Selfish reasons to be
reproducible
Fast, cutting edge, future-proof, you’ll look good,
more collaborators, extra citations, avoid
career-ending disasters, builds a group etc etc
Slide 59
Slide 59 text
Do not try to be
completely
reproducible!
Shocking finale
PTO...
Slide 60
Slide 60 text
Do not decide to be reproducible. Decide to be a bit more
reproducible, celebrate the small wins. Spread the word.
Take home message