Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Reproducibility & Minimizing Bias

Reproducibility & Minimizing Bias

Hilda Bastian

June 14, 2018

More Decks by Hilda Bastian

Other Decks in Research


  1. Reproducibility & Minimizing Bias Research Reproducibility 2018: Short Course Science

    University of Utah, Salt Lake City 14 June 2018 Hilda Bastian @hildabast hildabastian.net
  2. “Statistical workers who fail to scrutinize the goodness of their

    observed data and carry through a satisfactory analysis upon poor observations, will end up with ridiculous conclusions which cannot be maintained”. Raymond Pearl, 1919 Via Doug Altman, Iveta Simera: http://www.jameslindlibrary.org/articles/a- history-of-the-evolution-of-guidelines-for-reporting-medical-research-the- long-road-to-the-equator-network/
  3. •  Methods reproducibility: enough detail to be able to repeat

    it •  Results reproducibility: it replicates •  Inferential reproducibility: similar conclusions drawn about it Defining research reproducibility Goodman, Fanelli, Ioannidis (2016). Science Translational Medicine; 8 (341) ps12 —  But even this still doesn’t cover everything in this debate
  4. Specialization & complexity has grown 1850 Beaver & Rosen (2005).

    Scientometrics; 1(3). http:// www.akademiai.com/doi/abs/10.1007/BF02016308 1st co-authored article (1870) 1st article with >1,000 authors (2004) INTERNET Wikipedia (2001) 40% of papers have co-authors (1940)
  5. “The idealized expert- generated, one-way, authoritative reign of science is

    over.” Debating science’s problems in 2016: now it’s everybody’s problem Sobo et al (2016). Medical Anthropology. 35 (6): 529-546.
  6. •  Research on scientific methods – across the full spectrum

    of activity and use •  Detailed registered protocols for as much research as possible
  7. •  The Yin & Yang of post- publication activity • 

    Valuing quality, replication and validation studies
  8. •  Strengthen culture of constructive criticism •  Consequences for authors

    of non-response to important questions & criticism •  More accountability and consequences for editors, reviewers, and journals (Südhof (2016). PLOS Biology 14(8): e1002547.)
  9. Systematic review •  Pre-specified/registered protocol •  Clear, precise question(s) • 

    Search strategy •  Inclusion/exclusion criteria •  Data extraction method •  Quality assessment •  Pre-specified analyses & methods
  10. Keep up with what are “questionable research practices” & how

    to avoid the pitfalls https://replicationindex.wordpress.com/2015/01/24/questionable-research- practices-definition-detect-and-recommendations-for-better-practices/
  11. Bias prevention beyond research design •  Diversity in participation and

    areas of study •  Cognitive biases e.g. confirmation bias, motivated reasoning
  12. “[We] have made far more progress in cataloguing cognitive biases

    than in finding ways to correct them.” Lilienfield et al (2009). Perspectives Psychol Sci. 4 (4): 390-398.