How to write a reproducible paper

Presentation from the 2016 AGU Fall Meeting in San Francisco. A more detailed account of the content presented can be found at the following blog post:
https://drclimate.wordpress.com/2016/06/16/how-to-write-a-reproducible-paper/

Transcript

1. How to write a
   reproducible paper
   Damien Irving
   Postdoctoral Research Fellow
   CSIRO Oceans and Atmosphere
   Hobart, Australia
   @DrClimate
   https://drclimate.wordpress.com
   

   View Slide

2. Baker (2016). Nature 533, 452–4. doi:10.1038/533452a
   We don’t document our
   software and code, so it’s
   impossible to replicate the
   computational results
   presented in geoscience
   papers today
   

   View Slide

3. Wanted: A concrete example
   ž I set out to write a reproducible paper
   — My background
   ○ Software Carpentry instructor
   ○ CSIRO support scientist
   ○ Regular PyCon attendee
   ○ Data science blog: drclimate.wordpress.com
   — Lit review
   ○ Barriers to publishing code
   ○ Computational best practices
   

   View Slide

4. View Slide

5. View Slide

6. ž Brief overview of software packages
   ž Link to collection of supplementary materials
   — Code, software, processing steps
   — Host with Figshare / Zenodo
   Computation section
   http://dx.doi.org/10.6084/m9.figshare.1385387
   

   View Slide

7. Code
   ž Minimum standard
   — Snapshot of code library
   ž Other options
   — Version control that library
   — Link to an external hosting service
   ○ Allows readers to view updates & submit
   pull requests
   Simplest thing any author
   can do to ensure their
   results are reproducible
   Alternative and/or additional
   things that (some) authors
   could do to (potentially) make
   the reader’s life easier
   

   View Slide

8. Software
   ž Minimum standard
   — Manual listing: name, version number,
   release date, institution and DOI or URL
   ž Other options
   — Conda – “conda env create dirving/jclimate”
   — Docker / Nix
   

   View Slide

9. Processing steps
   ž Minimum standard:
   — Step-by-step account, raw data to result
   — Options depend on what software you’re using
   ○ Command line log file for each figure / result
   ○ Informative README file/s
   

   View Slide

10. Processing steps
    ž Other options
    — Makefile
    — Workflow
    management
    system
    VisTrails provides a graphical interface to drag and drop
    the various components of your workflow
    

    View Slide

11. Summary
    Minimum standard Other options
    Code Snapshot of code library* Version control*
    External hosting service*
    Software Name, version number, release date,
    institution and DOI or URL of each
    software package*
    Conda*
    Docker / Nix
    Processing
    steps
    Command line log file*
    README*
    Makefile*
    Workflow system
    *Skills can be picked up at a 2-day Software Carpentry workshop.
    

    View Slide

12. Questions?
    https://drclimate.wordpress.com/2016/06/16/
    how-to-write-a-reproducible-paper/
    Irving D, Simmonds I (2015). A novel approach to diagnosing
    Southern Hemisphere planetary wave activity and its influence
    on regional climate variability. Journal of Climate. 28, 9041-57.
    doi:10.1175/JCLI-D-15-0287.1
    Irving D (2016). A minimum standard for publishing
    computational results in the weather and climate sciences.
    Bulletin of the American Meteorological Society. 97, 1149-1158.
    doi:10.1175/BAMS-D-15-00010.1
    

    View Slide