20190712-iqbio.pdf

E1a375fbe8cc71e23307a519eb4848e9?s=47 Patrick Kimes
July 12, 2019
87

 20190712-iqbio.pdf

E1a375fbe8cc71e23307a519eb4848e9?s=128

Patrick Kimes

July 12, 2019
Tweet

Transcript

  1. Replicability Hackathon 2019 IQ BIO REU, UPR-RP Data Sciences, Dana-Farber

    Cancer Institute Biostatistics, Harvard TH Chan School of Public Health Patrick Kimes, PhD Kelly Street, PhD
  2. HEALTHCARE INNOVATION REPLICATHON 12 y 13 de julio de 2019

  3. Hi, I’m Kelly

  4. Hi, I’m Patrick PhD, Statistics BA, Mathematics

  5. I’d like to thank the Academy… Keegan Korthauer, PhD Alejandro

    Reyes, PhD for making this Patricia Ordóñez, PhD Juan S. Ramírez-Lugo, PhD Rafael Irizarry, PhD for making this possible
  6. Code of Conduct https://github.com/pkimes/PR2019replicathon/blob/master/code_of_conduct.md Assume competence in the people you

    interact with. There are no stupid questions. Be considerate in speech and actions, and actively seek to acknowledge and respect the boundaries of fellow community members. Take care of each other. Alert one of the organizers or facilitators if you notice a dangerous situation, someone in distress, or a potential violation of this Code of Conduct, even if it seems inconsequential. We do not tolerate harassment in any form.
  7. Replicability Hackathon 2019 IQ BIO REU, UPR-RP Data Sciences, Dana-Farber

    Cancer Institute Biostatistics, Harvard TH Chan School of Public Health Patrick Kimes, PhD Kelly Street, PhD
  8. http://jtleek.com/talks.html

  9. None
  10. “many scientific studies are difficult or impossible to replicate or

    reproduce.”
  11. https://doi.org/10.1371/journal.pmed.0020124

  12. https://doi.org/10.1371/journal.pmed.0020124 “Simulations show that for most study designs and settings,

    it is more likely for a research claim to be false than true.”
  13. “… replications of 100 experimental and correlational studies …” https://www.ncbi.nlm.nih.gov/pubmed/26315443

  14. “39% of effects were subjectively rated to have replicated the

    original results.” “… replications of 100 experimental and correlational studies …” https://www.ncbi.nlm.nih.gov/pubmed/26315443
  15. 39%?? https://www.ncbi.nlm.nih.gov/pubmed/26315443

  16. http://jtleek.com/talks.html

  17. http://jtleek.com/talks.html

  18. http://jtleek.com/talks.html

  19. http://jtleek.com/talks.html

  20. http://jtleek.com/talks.html

  21. http://jtleek.com/talks.html

  22. http://jtleek.com/talks.html

  23. what’s going on?

  24. let’s clarify some language reproducibility replicability

  25. let’s clarify some language reproducibility replicability the ability to take

    the original data and the computer code used to analyze the data and reproduce all of the numerical findings from the study https://simplystatistics.org/2016/08/24/replication-crisis/
  26. let’s clarify some language reproducibility replicability the ability to take

    the original data and the computer code used to analyze the data and reproduce all of the numerical findings from the study https://simplystatistics.org/2016/08/24/replication-crisis/
  27. let’s clarify some language reproducibility replicability the ability to repeat

    an entire study, independent of the original investigator without the use of original data https://simplystatistics.org/2016/08/24/replication-crisis/
  28. let’s clarify some language reproducibility replicability the ability to repeat

    an entire study, independent of the original investigator without the use of original data https://simplystatistics.org/2016/08/24/replication-crisis/
  29. let’s clarify some language reproducibility replicability

  30. replicability

  31. what’s going on? replicability

  32. replicability crisis: experiments are replicated. results not so much.

  33. should we expect scientific results to always replicate? replicability

  34. Psychology studies reproducibility 2 https://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html? “But the failure to replicate

    is not a cause for alarm; in fact, it is a normal part of how science works.”
  35. https://simplystatistics.org/2016/08/24/replication-crisis/ “the replication crisis in science is largely attributable to

    a mismatch in our expectations of how often findings should replicate and how difficult it is to actually discover true findings in certain fields.”
  36. https://simplystatistics.org/2013/08/01/the-roc-curves-of-science/ “…I argue that the rate of discoveries is higher

    in biomedical research than in physics. But, to achieve this higher true positive rate, biomedical research has to tolerate a higher false positive rate.”
  37. should we expect scientific results to always replicate? replicability not

    always
  38. crisis: experiments are replicated. results not so much. or maybe

    that’s science? replicability
  39. 39%??

  40. replicability crisis opportunity

  41. Replicability Hackathon 2019 IQ BIO REU, UPR-RP so what about

    … … ?
  42. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905

  43. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905

  44. drug sensitivity in cancer cell lines

  45. drug sensitivity in cancer cell lines what is a cell

    line?
  46. drug sensitivity in cancer cell lines cell line cell culture

    from a single cell that can grow indefinitely given appropriate conditions • easily grown • relatively inexpensive • amenable to high- throughput testing https://www.ncbi.nlm.nih.gov/pubmed/26248648
  47. drug sensitivity in cancer cell lines what do we mean

    by drug sensitivity?
  48. drug sensitivity in cancer cell lines viability relative measure of

    cell line abundance after treatment
  49. drug sensitivity in cancer cell lines dose response curve model

    fit to viability as a function of drug concentration
  50. drug sensitivity in cancer cell lines IC50 IC50 concentration at

    which cell growth is inhibited 50%
  51. drug sensitivity in cancer cell lines AUC area under the

    activity curve AUC
  52. drug sensitivity in cancer cell lines got it! … but

    why?
  53. drug sensitivity in cancer cell lines clinical https://www.ncbi.nlm.nih.gov/pubmed/26248648

  54. drug sensitivity in cancer cell lines important, impactful research!

  55. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905

  56. https://www.ncbi.nlm.nih.gov/pubmed/24284626

  57. https://www.ncbi.nlm.nih.gov/pubmed/24284626

  58. https://www.ncbi.nlm.nih.gov/pubmed/27905415

  59. “Upon careful analysis of the same data, we have come

    to quite different and much more positive conclusions.” https://www.ncbi.nlm.nih.gov/pubmed/27905415
  60. None
  61. crisis: experiments are replicated. results not so much. or maybe

    yes? replicability opportunity
  62. enough with the what, on to the data

  63. our challenge https://www.github.com/pkimes/PR2019replicathon

  64. raw data

  65. summarized data

  66. summarized data

  67. model summarized data

  68. our challenge datasets

  69. our challenge datasets

  70. our challenge datasets > setwd(‘path/to/PR2019replicathon’) > raw <- readRDS(‘data/rawPharmacoData.rds’) >

    ls() [1] “raw”
  71. our challenge template

  72. our challenge template

  73. our challenge template

  74. our challenge tutorials

  75. our challenge tutorials

  76. our challenge do these studies agree?

  77. Let’s Hack!

  78. getting setup with

  79. None
  80. None
  81. None
  82. feel free to specify a different location

  83. ready to go!

  84. https://pkimes.github.io/PR2019replicathon/

  85. Let’s Hack! Gigaprobe learnlispandRTFM