20200626-iqbio-intro.pdf

Transcript

1. Healthcare Innovation Replicathon 2020
   IQ BIO REU, UPR-RP
   Data Sciences, Dana-Farber Cancer Institute
   Biostatistics, Harvard TH Chan School of Public Health
   Patrick Kimes, Kelly Street, Jill Lundell, Mercedeh Movassagh
   Luli Zou, Ana Betty Villaseñor-Altamirano
   

   View Slide

2. HEALTHCARE
   INNOVATION
   REPLICATHON
   12 y 13 de julio de 2019
   

   View Slide

3. Hi!
   Ana Betty
   Patrick Jill
   Mercedeh
   Kelly
   Luli
   

   View Slide

4. I’d like to thank the Academy…
   Keegan
   Korthauer, PhD
   Alejandro
   Reyes, PhD
   for making this
   Patricia Ordóñez, PhD
   Juan S. Ramírez-Lugo, PhD
   Rafael Irizarry, PhD
   for making this possible
   

   View Slide

5. Code of Conduct
   https://www.pkimes.com/PR2020replicathon/code_of_conduct
   Assume competence in the people you interact with.
   There are no stupid questions.
   Be considerate in speech and actions, and actively seek
   to acknowledge and respect the boundaries of fellow
   community members.
   Take care of each other. Alert one of the organizers or
   facilitators if you notice a dangerous situation, someone
   in distress, or a potential violation of this Code of
   Conduct, even if it seems inconsequential.
   We do not tolerate harassment in any form.
   

   View Slide

6. part I. the crisis
   

   View Slide

7. http://jtleek.com/talks.html
   

   View Slide

8. View Slide

9. “many scientific studies are difficult or impossible to
   replicate or reproduce.”
   

   View Slide

10. https://doi.org/10.1371/journal.pmed.0020124
    

    View Slide

11. https://doi.org/10.1371/journal.pmed.0020124
    “Simulations show that for most study designs and settings,
    it is more likely for a research claim to be false than true.”
    

    View Slide

12. “… replications of 100 experimental
    and correlational studies …”
    https://www.ncbi.nlm.nih.gov/pubmed/26315443
    

    View Slide

13. “39% of effects were subjectively
    rated to have replicated the
    original results.”
    “… replications of 100 experimental
    and correlational studies …”
    https://www.ncbi.nlm.nih.gov/pubmed/26315443
    

    View Slide

14. 39%??
    https://www.ncbi.nlm.nih.gov/pubmed/26315443
    

    View Slide

15. http://jtleek.com/talks.html
    

    View Slide

16. http://jtleek.com/talks.html
    

    View Slide

17. http://jtleek.com/talks.html
    

    View Slide

18. http://jtleek.com/talks.html
    

    View Slide

19. http://jtleek.com/talks.html
    

    View Slide

20. http://jtleek.com/talks.html
    

    View Slide

21. http://jtleek.com/talks.html
    

    View Slide

22. what’s going on?
    

    View Slide

23. let’s clarify some language
    reproducibility
    replicability
    

    View Slide

24. let’s clarify some language
    reproducibility
    replicability
    the ability to take the original data
    and the computer code used to
    analyze the data and reproduce all of
    the numerical findings from the study
    https://simplystatistics.org/2016/08/24/replication-crisis/
    

    View Slide

25. let’s clarify some language
    reproducibility
    replicability
    the ability to take the original data
    and the computer code used to
    analyze the data and reproduce all of
    the numerical findings from the study
    https://simplystatistics.org/2016/08/24/replication-crisis/
    

    View Slide

26. let’s clarify some language
    reproducibility
    replicability
    the ability to repeat an entire study,
    independent of the original investigator
    without the use of original data
    https://simplystatistics.org/2016/08/24/replication-crisis/
    

    View Slide

27. let’s clarify some language
    reproducibility
    replicability
    the ability to repeat an entire study,
    independent of the original investigator
    without the use of original data
    https://simplystatistics.org/2016/08/24/replication-crisis/
    

    View Slide

28. let’s clarify some language
    reproducibility
    replicability
    

    View Slide

29. replicability
    

    View Slide

30. what’s going on?
    replicability
    

    View Slide

31. crisis:
    experiments are replicated.
    results not so much.
    replicability
    

    View Slide

32. should we expect
    scientific results to
    always replicate?
    replicability
    

    View Slide

33. Psychology studies reproducibility 2
    https://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html?
    “But the failure to replicate is not a cause for alarm;
    in fact, it is a normal part of how science works.”
    

    View Slide

34. https://simplystatistics.org/2016/08/24/replication-crisis/
    “the replication crisis in science is largely attributable to
    a mismatch in our expectations of how often findings
    should replicate and how difficult it is to actually
    discover true findings in certain fields.”
    

    View Slide

35. https://simplystatistics.org/2013/08/01/the-roc-curves-of-science/
    “…I argue that the rate of
    discoveries is higher in
    biomedical research than in
    physics. But, to achieve this higher
    true positive rate, biomedical
    research has to tolerate a higher
    false positive rate.”
    

    View Slide

36. should we expect
    scientific results to
    always replicate?
    not always
    replicability
    

    View Slide

37. crisis:
    experiments are replicated.
    results not so much.
    or maybe that’s science?
    replicability
    

    View Slide

38. 39%??
    

    View Slide

39. replicability crisis
    opportunity
    

    View Slide

40. part II. the question
    

    View Slide

41. our question
    https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905
    

    View Slide

42. https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905
    our question
    

    View Slide

43. drug sensitivity in
    cancer cell lines
    

    View Slide

44. drug sensitivity in
    cancer cell lines
    what is a cell line?
    

    View Slide

45. drug sensitivity in
    cancer cell lines
    cell line
    cell culture from a
    single cell that can
    grow indefinitely given
    appropriate conditions
    • easily grown
    • relatively inexpensive
    • amenable to high-
    throughput testing
    https://www.ncbi.nlm.nih.gov/pubmed/26248648
    

    View Slide

46. drug sensitivity in
    cancer cell lines
    what do we mean by
    drug sensitivity?
    

    View Slide

47. drug sensitivity in
    cancer cell lines
    viability
    relative measure of
    cell line abundance
    after treatment
    

    View Slide

48. drug sensitivity in
    cancer cell lines
    dose response curve
    model fit to viability
    as a function of drug
    concentration
    

    View Slide

49. drug sensitivity in
    cancer cell lines
    IC50
    IC50
    concentration at
    which cell growth is
    inhibited 50%
    

    View Slide

50. drug sensitivity in
    cancer cell lines
    AUC
    area under the activity
    curve
    AUC
    

    View Slide

51. drug sensitivity in
    cancer cell lines
    got it!
    … but why?
    

    View Slide

52. drug sensitivity in
    cancer cell lines
    clinical
    https://www.ncbi.nlm.nih.gov/pubmed/26248648
    

    View Slide

53. drug sensitivity in
    cancer cell lines
    important, impactful
    research!
    

    View Slide

54. https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905
    our question
    

    View Slide

55. https://www.ncbi.nlm.nih.gov/pubmed/24284626
    

    View Slide

56. https://www.ncbi.nlm.nih.gov/pubmed/24284626
    

    View Slide

57. https://www.ncbi.nlm.nih.gov/pubmed/27905415
    

    View Slide

58. View Slide

59. part III. the opportunity
    

    View Slide

60. crisis:
    experiments are replicated.
    results not so much.
    replicability
    opportunity
    

    View Slide

61. what does it mean for
    results to “replicate”?
    replicability
    

    View Slide

62. what does it mean for
    results to “replicate”?
    good question!!
    replicability
    

    View Slide

63. View Slide

64. crisis:
    experiments are replicated.
    results not so much.
    replicability
    opportunity
    

    View Slide

65. crisis:
    experiments are replicated.
    results not so much.
    or maybe yes?
    opportunity
    replicability
    

    View Slide

66. Healthcare Innovation Replicathon 2020
    IQ BIO REU, UPR-RP
    Data Sciences, Dana-Farber Cancer Institute
    Biostatistics, Harvard TH Chan School of Public Health
    Patrick Kimes, Kelly Street, Jill Lundell, Mercedeh Movassagh
    Luli Zou, Ana Betty Villaseñor-Altamirano
    

    View Slide