20190712-iqbio.pdf

Transcript

1. Replicability Hackathon 2019 IQ BIO REU, UPR-RP Data Sciences, Dana-Farber Cancer Institute Biostatistics, Harvard TH Chan School of Public Health Patrick Kimes, PhD Kelly Street, PhD
2. HEALTHCARE INNOVATION REPLICATHON 12 y 13 de julio de 2019
3. Hi, I’m Kelly
4. Hi, I’m Patrick PhD, Statistics BA, Mathematics
5. I’d like to thank the Academy… Keegan Korthauer, PhD Alejandro Reyes, PhD for making this Patricia Ordóñez, PhD Juan S. Ramírez-Lugo, PhD Rafael Irizarry, PhD for making this possible
6. Code of Conduct https://github.com/pkimes/PR2019replicathon/blob/master/code_of_conduct.md Assume competence in the people you interact with. There are no stupid questions. Be considerate in speech and actions, and actively seek to acknowledge and respect the boundaries of fellow community members. Take care of each other. Alert one of the organizers or facilitators if you notice a dangerous situation, someone in distress, or a potential violation of this Code of Conduct, even if it seems inconsequential. We do not tolerate harassment in any form.
7. Replicability Hackathon 2019 IQ BIO REU, UPR-RP Data Sciences, Dana-Farber Cancer Institute Biostatistics, Harvard TH Chan School of Public Health Patrick Kimes, PhD Kelly Street, PhD
8. http://jtleek.com/talks.html
10. “many scientific studies are difficult or impossible to replicate or reproduce.”
11. https://doi.org/10.1371/journal.pmed.0020124
12. https://doi.org/10.1371/journal.pmed.0020124 “Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true.”
13. “… replications of 100 experimental and correlational studies …” https://www.ncbi.nlm.nih.gov/pubmed/26315443
14. “39% of effects were subjectively rated to have replicated the original results.” “… replications of 100 experimental and correlational studies …” https://www.ncbi.nlm.nih.gov/pubmed/26315443
15. 39%?? https://www.ncbi.nlm.nih.gov/pubmed/26315443
16. http://jtleek.com/talks.html
17. http://jtleek.com/talks.html
18. http://jtleek.com/talks.html
19. http://jtleek.com/talks.html
20. http://jtleek.com/talks.html
21. http://jtleek.com/talks.html
22. http://jtleek.com/talks.html
23. what’s going on?
24. let’s clarify some language reproducibility replicability
25. let’s clarify some language reproducibility replicability the ability to take the original data and the computer code used to analyze the data and reproduce all of the numerical findings from the study https://simplystatistics.org/2016/08/24/replication-crisis/
26. let’s clarify some language reproducibility replicability the ability to take the original data and the computer code used to analyze the data and reproduce all of the numerical findings from the study https://simplystatistics.org/2016/08/24/replication-crisis/
27. let’s clarify some language reproducibility replicability the ability to repeat an entire study, independent of the original investigator without the use of original data https://simplystatistics.org/2016/08/24/replication-crisis/
28. let’s clarify some language reproducibility replicability the ability to repeat an entire study, independent of the original investigator without the use of original data https://simplystatistics.org/2016/08/24/replication-crisis/
29. let’s clarify some language reproducibility replicability
30. replicability
31. what’s going on? replicability
32. replicability crisis: experiments are replicated. results not so much.
33. should we expect scientific results to always replicate? replicability
34. Psychology studies reproducibility 2 https://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html? “But the failure to replicate is not a cause for alarm; in fact, it is a normal part of how science works.”
35. https://simplystatistics.org/2016/08/24/replication-crisis/ “the replication crisis in science is largely attributable to a mismatch in our expectations of how often findings should replicate and how difficult it is to actually discover true findings in certain fields.”
36. https://simplystatistics.org/2013/08/01/the-roc-curves-of-science/ “…I argue that the rate of discoveries is higher in biomedical research than in physics. But, to achieve this higher true positive rate, biomedical research has to tolerate a higher false positive rate.”
37. should we expect scientific results to always replicate? replicability not always
38. crisis: experiments are replicated. results not so much. or maybe that’s science? replicability
39. 39%??
40. replicability crisis opportunity
41. Replicability Hackathon 2019 IQ BIO REU, UPR-RP so what about … … ?
42. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905
43. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905
44. drug sensitivity in cancer cell lines
45. drug sensitivity in cancer cell lines what is a cell line?
46. drug sensitivity in cancer cell lines cell line cell culture from a single cell that can grow indefinitely given appropriate conditions • easily grown • relatively inexpensive • amenable to high- throughput testing https://www.ncbi.nlm.nih.gov/pubmed/26248648
47. drug sensitivity in cancer cell lines what do we mean by drug sensitivity?
48. drug sensitivity in cancer cell lines viability relative measure of cell line abundance after treatment
49. drug sensitivity in cancer cell lines dose response curve model fit to viability as a function of drug concentration
50. drug sensitivity in cancer cell lines IC50 IC50 concentration at which cell growth is inhibited 50%
51. drug sensitivity in cancer cell lines AUC area under the activity curve AUC
52. drug sensitivity in cancer cell lines got it! … but why?
53. drug sensitivity in cancer cell lines clinical https://www.ncbi.nlm.nih.gov/pubmed/26248648
54. drug sensitivity in cancer cell lines important, impactful research!
55. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905
56. https://www.ncbi.nlm.nih.gov/pubmed/24284626
57. https://www.ncbi.nlm.nih.gov/pubmed/24284626
58. https://www.ncbi.nlm.nih.gov/pubmed/27905415
59. “Upon careful analysis of the same data, we have come to quite different and much more positive conclusions.” https://www.ncbi.nlm.nih.gov/pubmed/27905415
61. crisis: experiments are replicated. results not so much. or maybe yes? replicability opportunity
62. enough with the what, on to the data
63. our challenge https://www.github.com/pkimes/PR2019replicathon
64. raw data
65. summarized data
66. summarized data
67. model summarized data
68. our challenge datasets
69. our challenge datasets
70. our challenge datasets > setwd(‘path/to/PR2019replicathon’) > raw <- readRDS(‘data/rawPharmacoData.rds’) > ls() [1] “raw”
71. our challenge template
72. our challenge template
73. our challenge template
74. our challenge tutorials
75. our challenge tutorials
76. our challenge do these studies agree?
77. Let’s Hack!
78. getting setup with
82. feel free to specify a different location
83. ready to go!
84. https://pkimes.github.io/PR2019replicathon/
85. Let’s Hack! Gigaprobe learnlispandRTFM