Upgrade to Pro — share decks privately, control downloads, hide ads and more …

20190712-iqbio.pdf

Patrick Kimes
July 12, 2019
400

 20190712-iqbio.pdf

Patrick Kimes

July 12, 2019
Tweet

Transcript

  1. Replicability Hackathon 2019
    IQ BIO REU, UPR-RP
    Data Sciences, Dana-Farber Cancer Institute
    Biostatistics, Harvard TH Chan School of Public Health
    Patrick Kimes, PhD Kelly Street, PhD

    View full-size slide

  2. HEALTHCARE
    INNOVATION
    REPLICATHON
    12 y 13 de julio de 2019

    View full-size slide

  3. Hi, I’m Kelly

    View full-size slide

  4. Hi, I’m Patrick
    PhD, Statistics
    BA, Mathematics

    View full-size slide

  5. I’d like to thank the Academy…
    Keegan
    Korthauer, PhD
    Alejandro
    Reyes, PhD
    for making this
    Patricia Ordóñez, PhD
    Juan S. Ramírez-Lugo, PhD
    Rafael Irizarry, PhD
    for making this possible

    View full-size slide

  6. Code of Conduct
    https://github.com/pkimes/PR2019replicathon/blob/master/code_of_conduct.md
    Assume competence in the people you interact with.
    There are no stupid questions.
    Be considerate in speech and actions, and actively seek
    to acknowledge and respect the boundaries of fellow
    community members.
    Take care of each other. Alert one of the organizers or
    facilitators if you notice a dangerous situation, someone
    in distress, or a potential violation of this Code of
    Conduct, even if it seems inconsequential.
    We do not tolerate harassment in any form.

    View full-size slide

  7. Replicability Hackathon 2019
    IQ BIO REU, UPR-RP
    Data Sciences, Dana-Farber Cancer Institute
    Biostatistics, Harvard TH Chan School of Public Health
    Patrick Kimes, PhD Kelly Street, PhD

    View full-size slide

  8. http://jtleek.com/talks.html

    View full-size slide

  9. “many scientific studies are difficult or impossible to
    replicate or reproduce.”

    View full-size slide

  10. https://doi.org/10.1371/journal.pmed.0020124

    View full-size slide

  11. https://doi.org/10.1371/journal.pmed.0020124
    “Simulations show that for most study designs and settings,
    it is more likely for a research claim to be false than true.”

    View full-size slide

  12. “… replications of 100 experimental
    and correlational studies …”
    https://www.ncbi.nlm.nih.gov/pubmed/26315443

    View full-size slide

  13. “39% of effects were subjectively
    rated to have replicated the
    original results.”
    “… replications of 100 experimental
    and correlational studies …”
    https://www.ncbi.nlm.nih.gov/pubmed/26315443

    View full-size slide

  14. 39%??
    https://www.ncbi.nlm.nih.gov/pubmed/26315443

    View full-size slide

  15. http://jtleek.com/talks.html

    View full-size slide

  16. http://jtleek.com/talks.html

    View full-size slide

  17. http://jtleek.com/talks.html

    View full-size slide

  18. http://jtleek.com/talks.html

    View full-size slide

  19. http://jtleek.com/talks.html

    View full-size slide

  20. http://jtleek.com/talks.html

    View full-size slide

  21. http://jtleek.com/talks.html

    View full-size slide

  22. what’s going on?

    View full-size slide

  23. let’s clarify some language
    reproducibility
    replicability

    View full-size slide

  24. let’s clarify some language
    reproducibility
    replicability
    the ability to take the original data
    and the computer code used to
    analyze the data and reproduce all of
    the numerical findings from the study
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View full-size slide

  25. let’s clarify some language
    reproducibility
    replicability
    the ability to take the original data
    and the computer code used to
    analyze the data and reproduce all of
    the numerical findings from the study
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View full-size slide

  26. let’s clarify some language
    reproducibility
    replicability
    the ability to repeat an entire study,
    independent of the original investigator
    without the use of original data
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View full-size slide

  27. let’s clarify some language
    reproducibility
    replicability
    the ability to repeat an entire study,
    independent of the original investigator
    without the use of original data
    https://simplystatistics.org/2016/08/24/replication-crisis/

    View full-size slide

  28. let’s clarify some language
    reproducibility
    replicability

    View full-size slide

  29. replicability

    View full-size slide

  30. what’s going on?
    replicability

    View full-size slide

  31. replicability
    crisis:
    experiments are replicated.
    results not so much.

    View full-size slide

  32. should we expect
    scientific results to
    always replicate?
    replicability

    View full-size slide

  33. Psychology studies reproducibility 2
    https://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html?
    “But the failure to replicate is not a cause for alarm;
    in fact, it is a normal part of how science works.”

    View full-size slide

  34. https://simplystatistics.org/2016/08/24/replication-crisis/
    “the replication crisis in science is largely attributable to
    a mismatch in our expectations of how often findings
    should replicate and how difficult it is to actually
    discover true findings in certain fields.”

    View full-size slide

  35. https://simplystatistics.org/2013/08/01/the-roc-curves-of-science/
    “…I argue that the rate of
    discoveries is higher in
    biomedical research than in
    physics. But, to achieve this higher
    true positive rate, biomedical
    research has to tolerate a higher
    false positive rate.”

    View full-size slide

  36. should we expect
    scientific results to
    always replicate?
    replicability
    not always

    View full-size slide

  37. crisis:
    experiments are replicated.
    results not so much.
    or maybe that’s science?
    replicability

    View full-size slide

  38. replicability crisis
    opportunity

    View full-size slide

  39. Replicability Hackathon 2019
    IQ BIO REU, UPR-RP
    so what about …
    … ?

    View full-size slide

  40. our challenge
    https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905

    View full-size slide

  41. our challenge
    https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905

    View full-size slide

  42. drug sensitivity in
    cancer cell lines

    View full-size slide

  43. drug sensitivity in
    cancer cell lines
    what is a cell line?

    View full-size slide

  44. drug sensitivity in
    cancer cell lines
    cell line
    cell culture from a
    single cell that can
    grow indefinitely given
    appropriate conditions
    • easily grown
    • relatively inexpensive
    • amenable to high-
    throughput testing
    https://www.ncbi.nlm.nih.gov/pubmed/26248648

    View full-size slide

  45. drug sensitivity in
    cancer cell lines
    what do we mean by
    drug sensitivity?

    View full-size slide

  46. drug sensitivity in
    cancer cell lines
    viability
    relative measure of
    cell line abundance
    after treatment

    View full-size slide

  47. drug sensitivity in
    cancer cell lines
    dose response curve
    model fit to viability
    as a function of drug
    concentration

    View full-size slide

  48. drug sensitivity in
    cancer cell lines
    IC50
    IC50
    concentration at
    which cell growth is
    inhibited 50%

    View full-size slide

  49. drug sensitivity in
    cancer cell lines
    AUC
    area under the activity
    curve
    AUC

    View full-size slide

  50. drug sensitivity in
    cancer cell lines
    got it!
    … but why?

    View full-size slide

  51. drug sensitivity in
    cancer cell lines
    clinical
    https://www.ncbi.nlm.nih.gov/pubmed/26248648

    View full-size slide

  52. drug sensitivity in
    cancer cell lines
    important, impactful
    research!

    View full-size slide

  53. our challenge
    https://www.ncbi.nlm.nih.gov/pubmed/22460902
    https://www.ncbi.nlm.nih.gov/pubmed/22460905

    View full-size slide

  54. https://www.ncbi.nlm.nih.gov/pubmed/24284626

    View full-size slide

  55. https://www.ncbi.nlm.nih.gov/pubmed/24284626

    View full-size slide

  56. https://www.ncbi.nlm.nih.gov/pubmed/27905415

    View full-size slide

  57. “Upon careful analysis of the same data,
    we have come to quite different and
    much more positive conclusions.”
    https://www.ncbi.nlm.nih.gov/pubmed/27905415

    View full-size slide

  58. crisis:
    experiments are replicated.
    results not so much.
    or maybe yes?
    replicability
    opportunity

    View full-size slide

  59. enough with the what,
    on to the data

    View full-size slide

  60. our challenge
    https://www.github.com/pkimes/PR2019replicathon

    View full-size slide

  61. summarized data

    View full-size slide

  62. summarized data

    View full-size slide

  63. model
    summarized data

    View full-size slide

  64. our challenge
    datasets

    View full-size slide

  65. our challenge
    datasets

    View full-size slide

  66. our challenge
    datasets
    > setwd(‘path/to/PR2019replicathon’)
    > raw <- readRDS(‘data/rawPharmacoData.rds’)
    > ls()
    [1] “raw”

    View full-size slide

  67. our challenge
    template

    View full-size slide

  68. our challenge
    template

    View full-size slide

  69. our challenge
    template

    View full-size slide

  70. our challenge
    tutorials

    View full-size slide

  71. our challenge
    tutorials

    View full-size slide

  72. our challenge
    do these
    studies agree?

    View full-size slide

  73. Let’s Hack!

    View full-size slide

  74. getting setup with

    View full-size slide

  75. feel free to specify a
    different location

    View full-size slide

  76. ready to go!

    View full-size slide

  77. https://pkimes.github.io/PR2019replicathon/

    View full-size slide

  78. Let’s Hack!
    Gigaprobe
    learnlispandRTFM

    View full-size slide