Reproducible Science Matteo Cantiello Kavli Institute for Theoretical Physics University of California Santa Barbara Authorea scientific advisor 

 Is Collaborative Science in the 3rd millennium 

 Is Collaborative  Produces / collects large Datasets Science in the 3rd millennium 

 Is Collaborative  Produces / collects large Datasets  Requires large Computations/complex Codes Science in the 3rd millennium 

 High publication pressure Science in the 3rd millennium 

“No research paper can ever be considered to be the final word, and the replication and corroboration of research results is key to the scientific process” http://www.nature.com/nature/focus/reproducibility/ Reproducibility is one pillar of the scientific method. It’s what distinguish science from subjective matters. Michael Kloran What about reproducibility? 

Emerging Bad Practices  Tendency to publish only positive results  Lack of duplication of “established” results  Increase number of retractions (misconduct?)  Clear cases of scientific misconducts  Clear cases of scientific frauds 

Data: Fanelli, D. (2010) Positive results in psychology [...] dominate most journals, which strive to present new, exciting research. Meanwhile, attempts to replicate those studies, especially when the findings are negative, go unpublished [...]“ There are some experiments that everyone knows don't replicate, but this knowledge doesn't get into the literature,” The publication barrier can be chilling: “I've seen students spending their entire PhD period trying to replicate a phenomenon, failing, and quitting academia because they had nothing to show for their time.” Ed Yong, Nature 485, 7398 ” “ 

Drivers / Players?  Publication Pressure  Role of journals / media  Peer review system  Dissemination format of scientific results not allowing full reproducibility 

“Central to this initiative is a checklist intended to prompt authors to disclose technical and statistical information in their submissions [...] To further increase transparency, we will encourage authors to provide tables of the data behind graphs and figures” http://www.nature.com/news/announcement-reducing-our-irreproducibility-1.12852 Nature, 2013 Reducing our Irreproducibility 

Is a fundamental problem not being tackled? 

written up on 20th century tools packaged in a 17th century format Scientists produce 21st century research 

Back to the founding fathers (of the scientific method) Galileo Galilei 1564-1642 

1610 

2014 

The scientific paper is a package that represents the ‘de-facto’ currency of scientific research 

Au thorea ... Demo