Upgrade to Pro — share decks privately, control downloads, hide ads and more …

20200626-iqbio-intro.pdf

 20200626-iqbio-intro.pdf

Patrick Kimes

June 26, 2020
Tweet

More Decks by Patrick Kimes

Other Decks in Education

Transcript

  1. Healthcare Innovation Replicathon 2020
    IQ BIO REU, UPR-RP
    Data Sciences, Dana-Farber Cancer Institute
    Biostatistics, Harvard TH Chan School of Public Health
    Patrick Kimes, Kelly Street, Jill Lundell, Mercedeh Movassagh
    Luli Zou, Ana Betty Villaseñor-Altamirano

    View Slide

  2. HEALTHCARE
    INNOVATION
    REPLICATHON
    12 y 13 de julio de 2019

    View Slide

  3. Hi!
    Ana Betty
    Patrick Jill
    Mercedeh
    Kelly
    Luli

    View Slide

  4. I’d like to thank the Academy…
    Keegan
    Korthauer, PhD
    Alejandro
    Reyes, PhD
    for making this
    Patricia Ordóñez, PhD
    Juan S. Ramírez-Lugo, PhD
    Rafael Irizarry, PhD
    for making this possible

    View Slide

  5. Code of Conduct
    https://www.pkimes.com/PR2020replicathon/code_of_conduct
    Assume competence in the people you interact with.
    There are no stupid questions.
    Be considerate in speech and actions, and actively seek
    to acknowledge and respect the boundaries of fellow
    community members.
    Take care of each other. Alert one of the organizers or
    facilitators if you notice a dangerous situation, someone
    in distress, or a potential violation of this Code of
    Conduct, even if it seems inconsequential.
    We do not tolerate harassment in any form.

    View Slide

  6. part I. the crisis

    View Slide

  7. http://jtleek.com/talks.html

    View Slide

  8. View Slide

  9. “many scientific studies are difficult or impossible to
    replicate or reproduce.”

    View Slide

  10. https://doi.org/10.1371/journal.pmed.0020124

    View Slide

  11. https://doi.org/10.1371/journal.pmed.0020124
    “Simulations show that for most study designs and settings,
    it is more likely for a research claim to be false than true.”

    View Slide

  12. “… replications of 100 experimental
    and correlational studies …”
    https://www.ncbi.nlm.nih.gov/pubmed/26315443

    View Slide

  13. “39% of effects were subjectively
    rated to have replicated the
    original results.”
    “… replications of 100 experimental
    and correlational studies …”
    https://www.ncbi.nlm.nih.gov/pubmed/26315443

    View Slide

  14. 39%??
    https://www.ncbi.nlm.nih.gov/pubmed/26315443

    View Slide

  15. http://jtleek.com/talks.html

    View Slide

  16. http://jtleek.com/talks.html

    View Slide

  17. http://jtleek.com/talks.html

    View Slide

  18. http://jtleek.com/talks.html

    View Slide

  19. http://jtleek.com/talks.html

    View Slide

  20. http://jtleek.com/talks.html

    View Slide

  21. http://jtleek.com/talks.html

    View Slide

  22. what’s going on?

    View Slide

  23. let’s clarify some language
    reproducibility
    replicability

    View Slide

  24. let’s clarify some language
    reproducibility
    replicability
    the ability to take the original data
    and the computer code used to
    analyze the data and reproduce all of
    the numerical findings from the study
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View Slide

  25. let’s clarify some language
    reproducibility
    replicability
    the ability to take the original data
    and the computer code used to
    analyze the data and reproduce all of
    the numerical findings from the study
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View Slide

  26. let’s clarify some language
    reproducibility
    replicability
    the ability to repeat an entire study,
    independent of the original investigator
    without the use of original data
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View Slide

  27. let’s clarify some language
    reproducibility
    replicability
    the ability to repeat an entire study,
    independent of the original investigator
    without the use of original data
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View Slide

  28. let’s clarify some language
    reproducibility
    replicability

    View Slide

  29. replicability

    View Slide

  30. what’s going on?
    replicability

    View Slide

  31. crisis:
    experiments are replicated.
    results not so much.
    replicability

    View Slide

  32. should we expect
    scientific results to
    always replicate?
    replicability

    View Slide

  33. Psychology studies reproducibility 2
    https://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html?
    “But the failure to replicate is not a cause for alarm;
    in fact, it is a normal part of how science works.”

    View Slide

  34. https://simplystatistics.org/2016/08/24/replication-crisis/
    “the replication crisis in science is largely attributable to
    a mismatch in our expectations of how often findings
    should replicate and how difficult it is to actually
    discover true findings in certain fields.”

    View Slide

  35. https://simplystatistics.org/2013/08/01/the-roc-curves-of-science/
    “…I argue that the rate of
    discoveries is higher in
    biomedical research than in
    physics. But, to achieve this higher
    true positive rate, biomedical
    research has to tolerate a higher
    false positive rate.”

    View Slide

  36. should we expect
    scientific results to
    always replicate?
    not always
    replicability

    View Slide

  37. crisis:
    experiments are replicated.
    results not so much.
    or maybe that’s science?
    replicability

    View Slide

  38. 39%??

    View Slide

  39. replicability crisis
    opportunity

    View Slide

  40. part II. the question

    View Slide

  41. our question
    https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905

    View Slide

  42. https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905
    our question

    View Slide

  43. drug sensitivity in
    cancer cell lines

    View Slide

  44. drug sensitivity in
    cancer cell lines
    what is a cell line?

    View Slide

  45. drug sensitivity in
    cancer cell lines
    cell line
    cell culture from a
    single cell that can
    grow indefinitely given
    appropriate conditions
    • easily grown
    • relatively inexpensive
    • amenable to high-
    throughput testing
    https://www.ncbi.nlm.nih.gov/pubmed/26248648

    View Slide

  46. drug sensitivity in
    cancer cell lines
    what do we mean by
    drug sensitivity?

    View Slide

  47. drug sensitivity in
    cancer cell lines
    viability
    relative measure of
    cell line abundance
    after treatment

    View Slide

  48. drug sensitivity in
    cancer cell lines
    dose response curve
    model fit to viability
    as a function of drug
    concentration

    View Slide

  49. drug sensitivity in
    cancer cell lines
    IC50
    IC50
    concentration at
    which cell growth is
    inhibited 50%

    View Slide

  50. drug sensitivity in
    cancer cell lines
    AUC
    area under the activity
    curve
    AUC

    View Slide

  51. drug sensitivity in
    cancer cell lines
    got it!
    … but why?

    View Slide

  52. drug sensitivity in
    cancer cell lines
    clinical
    https://www.ncbi.nlm.nih.gov/pubmed/26248648

    View Slide

  53. drug sensitivity in
    cancer cell lines
    important, impactful
    research!

    View Slide

  54. https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905
    our question

    View Slide

  55. https://www.ncbi.nlm.nih.gov/pubmed/24284626

    View Slide

  56. https://www.ncbi.nlm.nih.gov/pubmed/24284626

    View Slide

  57. https://www.ncbi.nlm.nih.gov/pubmed/27905415

    View Slide

  58. View Slide

  59. part III. the opportunity

    View Slide

  60. crisis:
    experiments are replicated.
    results not so much.
    replicability
    opportunity

    View Slide

  61. what does it mean for
    results to “replicate”?
    replicability

    View Slide

  62. what does it mean for
    results to “replicate”?
    good question!!
    replicability

    View Slide

  63. View Slide

  64. crisis:
    experiments are replicated.
    results not so much.
    replicability
    opportunity

    View Slide

  65. crisis:
    experiments are replicated.
    results not so much.
    or maybe yes?
    opportunity
    replicability

    View Slide

  66. Healthcare Innovation Replicathon 2020
    IQ BIO REU, UPR-RP
    Data Sciences, Dana-Farber Cancer Institute
    Biostatistics, Harvard TH Chan School of Public Health
    Patrick Kimes, Kelly Street, Jill Lundell, Mercedeh Movassagh
    Luli Zou, Ana Betty Villaseñor-Altamirano

    View Slide