Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reproducible Research: ideas & solutions for actually doing it

Dave Lunt
June 23, 2016

Reproducible Research: ideas & solutions for actually doing it

Labchat preentation given to EvoHull group on techniques for achieving reproducible research

Dave Lunt

June 23, 2016
Tweet

More Decks by Dave Lunt

Other Decks in Research

Transcript

  1. reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreprodu
    reproducible
    research
    ideas and solutions for actually doing it
    Dave Lunt June 2016
    [email protected]
    Labchat is reproducible, all resources are at:
    https://osf.io/cqb6x/

    View Slide

  2. http://xkcd.com/
    reproducible
    research is
    science

    View Slide

  3. View Slide

  4. View Slide

  5. Reproducibility
    makes your life
    much easier
    ‘future you’ will reproduce your work
    reproducibility gives you new
    experimental ‘powers’
    carrot not stick

    View Slide

  6. reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreprodu
    small wins
    remember past
    successes
    tolerate minor
    inconvenience
    iterate
    know your issues
    don’t reinvent the
    wheel
    do something now
    implementing
    reproducibility

    View Slide

  7. reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreprodu
    tolerate minor
    inconvenience
    remember
    past
    successes
    small wins
    don’t
    reinvent the
    wheel
    do
    something
    now
    know your
    issues
    iterate
    What do you
    want from
    reproducibility?

    View Slide

  8. reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreprodu
    tolerate minor
    inconvenience
    remember
    past
    successes
    small wins
    don’t
    reinvent the
    wheel
    do
    something
    now
    know your
    issues
    iterate
    What do you
    want from
    reproducibility?
    OK, now be more
    ambitious...

    View Slide

  9. ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC
    TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA
    TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG
    ACACGAACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATG
    ATCGACTACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATAC
    GATCGATCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAAT
    CGAGCGACACGAACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGA
    TCGATGATCGACTACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCG
    ATATACGATCGATCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGA
    TCGAATCGAGCGACACGA
    ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC
    TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA
    TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG
    ACACGA
    ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC
    TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA
    TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG
    ACACGA
    ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC
    TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA
    TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG
    ACACGA
    ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC
    TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA
    TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG
    ACACGA
    ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC
    TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA
    TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG
    ACACGA
    ACGTGACAGCAGCGCATAGGCTAGCGCATAGCATACGTACGATCGATGGCCANATCGATCGCGATTTAGATCGATGATCGAC
    TACGATCGATCATTCGATCATGCATGGCGATCTATGCGGCATTATCGAGCGATTATCGGGGATCGAGGCGATATACGATCGA
    TCGATCGCATCGATATCGATCGAGGATTCGACGATTGACTATCAGCGCGGCGCATATTACGATGCTACGATCGAATCGAGCG
    collaboration
    archiving
    credibility
    backup
    provenance
    ease
    publication

    View Slide

  10. reproducibility
    environments
    what you should use

    View Slide

  11. reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreprodu
    git
    osf
    docker jupyter
    figshare
    galaxy
    Protocols.io

    View Slide

  12. reproducibility
    environments
    Things that work for
    starting out
    reducing reproducibility friction

    View Slide

  13. reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreprodu
    git
    osf
    docker jupyter
    figshare
    galaxy
    Protocols.io

    View Slide

  14. https://osf.io

    View Slide

  15. View Slide

  16. http://osf.io
    OSF is imperfect
    but very workable
    small wins
    remember past
    successes
    tolerate minor
    inconvenience
    iterate
    know your issues
    don’t reinvent the
    wheel
    do something now

    View Slide

  17. http://osf.io
    OSF is imperfect
    but very workable
    small wins
    remember past
    successes
    tolerate minor
    inconvenience
    iterate
    know your issues
    don’t reinvent the
    wheel
    do something now
    Its pretty good actually, easy to use
    You could start today, achieve small wins

    View Slide

  18. https://osf.io/cqb6x/ live demo

    View Slide

  19. http://protocols.io
    Protocols are open cc-by
    Protocols have doi

    View Slide

  20. View Slide

  21. View Slide

  22. View Slide

  23. reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreproducibilityrep
    roducibilityreproducibilityreproducibilityreproducibilityreproducibility
    reproducibilityreproducibilityreproducibilityreproducibilityreproducib
    ilityreproducibilityreproducibilityreproducibilityreproducibilityreprod
    ucibilityreproducibilityreproducibilityreproducibilityreprodu
    small wins
    remember past
    successes
    tolerate minor
    inconvenience
    iterate
    know your issues
    don’t reinvent the
    wheel
    do something now
    know your issues, start small,
    and do something now
    [the end]

    View Slide