20190712-iqbio.pdf

Transcript

1. Replicability Hackathon 2019 IQ BIO REU, UPR-RP Data Sciences, Dana-Farber

   Cancer Institute Biostatistics, Harvard TH Chan School of Public Health Patrick Kimes, PhD Kelly Street, PhD

2. HEALTHCARE INNOVATION REPLICATHON 12 y 13 de julio de 2019

3. Hi, I’m Kelly

4. Hi, I’m Patrick PhD, Statistics BA, Mathematics

5. I’d like to thank the Academy… Keegan Korthauer, PhD Alejandro

   Reyes, PhD for making this Patricia Ordóñez, PhD Juan S. Ramírez-Lugo, PhD Rafael Irizarry, PhD for making this possible

6. Code of Conduct https://github.com/pkimes/PR2019replicathon/blob/master/code_of_conduct.md Assume competence in the people you

   interact with. There are no stupid questions. Be considerate in speech and actions, and actively seek to acknowledge and respect the boundaries of fellow community members. Take care of each other. Alert one of the organizers or facilitators if you notice a dangerous situation, someone in distress, or a potential violation of this Code of Conduct, even if it seems inconsequential. We do not tolerate harassment in any form.

7. Replicability Hackathon 2019 IQ BIO REU, UPR-RP Data Sciences, Dana-Farber

   Cancer Institute Biostatistics, Harvard TH Chan School of Public Health Patrick Kimes, PhD Kelly Street, PhD

8. http://jtleek.com/talks.html

9. None

10. “many scientific studies are difficult or impossible to replicate or

    reproduce.”

11. https://doi.org/10.1371/journal.pmed.0020124

12. https://doi.org/10.1371/journal.pmed.0020124 “Simulations show that for most study designs and settings,

    it is more likely for a research claim to be false than true.”

13. “… replications of 100 experimental and correlational studies …” https://www.ncbi.nlm.nih.gov/pubmed/26315443

14. “39% of effects were subjectively rated to have replicated the

    original results.” “… replications of 100 experimental and correlational studies …” https://www.ncbi.nlm.nih.gov/pubmed/26315443

15. 39%?? https://www.ncbi.nlm.nih.gov/pubmed/26315443

16. http://jtleek.com/talks.html

17. http://jtleek.com/talks.html

18. http://jtleek.com/talks.html

19. http://jtleek.com/talks.html

20. http://jtleek.com/talks.html

21. http://jtleek.com/talks.html

22. http://jtleek.com/talks.html

23. what’s going on?

24. let’s clarify some language reproducibility replicability

25. let’s clarify some language reproducibility replicability the ability to take

    the original data and the computer code used to analyze the data and reproduce all of the numerical findings from the study https://simplystatistics.org/2016/08/24/replication-crisis/

26. let’s clarify some language reproducibility replicability the ability to take

    the original data and the computer code used to analyze the data and reproduce all of the numerical findings from the study https://simplystatistics.org/2016/08/24/replication-crisis/

27. let’s clarify some language reproducibility replicability the ability to repeat

    an entire study, independent of the original investigator without the use of original data https://simplystatistics.org/2016/08/24/replication-crisis/

28. let’s clarify some language reproducibility replicability the ability to repeat

    an entire study, independent of the original investigator without the use of original data https://simplystatistics.org/2016/08/24/replication-crisis/

29. let’s clarify some language reproducibility replicability

30. replicability

31. what’s going on? replicability

32. replicability crisis: experiments are replicated. results not so much.

33. should we expect scientific results to always replicate? replicability

34. Psychology studies reproducibility 2 https://www.nytimes.com/2015/09/01/opinion/psychology-is-not-in-crisis.html? “But the failure to replicate

    is not a cause for alarm; in fact, it is a normal part of how science works.”

35. https://simplystatistics.org/2016/08/24/replication-crisis/ “the replication crisis in science is largely attributable to

    a mismatch in our expectations of how often findings should replicate and how difficult it is to actually discover true findings in certain fields.”

36. https://simplystatistics.org/2013/08/01/the-roc-curves-of-science/ “…I argue that the rate of discoveries is higher

    in biomedical research than in physics. But, to achieve this higher true positive rate, biomedical research has to tolerate a higher false positive rate.”

37. should we expect scientific results to always replicate? replicability not

    always

38. crisis: experiments are replicated. results not so much. or maybe

    that’s science? replicability

39. 39%??

40. replicability crisis opportunity

41. Replicability Hackathon 2019 IQ BIO REU, UPR-RP so what about

    … … ?

42. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905

43. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905

44. drug sensitivity in cancer cell lines

45. drug sensitivity in cancer cell lines what is a cell

    line?

46. drug sensitivity in cancer cell lines cell line cell culture

    from a single cell that can grow indefinitely given appropriate conditions • easily grown • relatively inexpensive • amenable to high- throughput testing https://www.ncbi.nlm.nih.gov/pubmed/26248648

47. drug sensitivity in cancer cell lines what do we mean

    by drug sensitivity?

48. drug sensitivity in cancer cell lines viability relative measure of

    cell line abundance after treatment

49. drug sensitivity in cancer cell lines dose response curve model

    fit to viability as a function of drug concentration

50. drug sensitivity in cancer cell lines IC50 IC50 concentration at

    which cell growth is inhibited 50%

51. drug sensitivity in cancer cell lines AUC area under the

    activity curve AUC

52. drug sensitivity in cancer cell lines got it! … but

    why?

53. drug sensitivity in cancer cell lines clinical https://www.ncbi.nlm.nih.gov/pubmed/26248648

54. drug sensitivity in cancer cell lines important, impactful research!

55. our challenge https://www.ncbi.nlm.nih.gov/pubmed/22460902 https://www.ncbi.nlm.nih.gov/pubmed/22460905

56. https://www.ncbi.nlm.nih.gov/pubmed/24284626

57. https://www.ncbi.nlm.nih.gov/pubmed/24284626

58. https://www.ncbi.nlm.nih.gov/pubmed/27905415

59. “Upon careful analysis of the same data, we have come

    to quite different and much more positive conclusions.” https://www.ncbi.nlm.nih.gov/pubmed/27905415
60. None

61. crisis: experiments are replicated. results not so much. or maybe

    yes? replicability opportunity

62. enough with the what, on to the data

63. our challenge https://www.github.com/pkimes/PR2019replicathon

64. raw data

65. summarized data

66. summarized data

67. model summarized data

68. our challenge datasets

69. our challenge datasets

70. our challenge datasets > setwd(‘path/to/PR2019replicathon’) > raw <- readRDS(‘data/rawPharmacoData.rds’) >

    ls() [1] “raw”

71. our challenge template

72. our challenge template

73. our challenge template

74. our challenge tutorials

75. our challenge tutorials

76. our challenge do these studies agree?

77. Let’s Hack!

78. getting setup with

79. None
80. None
81. None

82. feel free to specify a different location

83. ready to go!

84. https://pkimes.github.io/PR2019replicathon/

85. Let’s Hack! Gigaprobe learnlispandRTFM