Talking at the Applied Math Lab in Agrocampus Rennes

Personal thoughts around developing STATS software, as seen mainly from my experience within academia

Transcript

1. STATS software as seen from my academia experience Gaston Sanchez Agrocampus, Rennes. Nov-29-2013
2. Abstract Some personal thoughts around developing STATS software, as seen mainly from my experience within academia (i.e. doing research)
3. Hi
4. Gaston Sanchez www.gastonsanchez.com
5. Caveat
6. I don’t have a map
7. but I have a compass
8. Currently
9. Research Engineer Unité Sensométrie et Chimiométrie Oniris, Nantes
10. creating R packages multivariate methods
11. My Experience
12. Statistician
13. Statistician + Analyst
14. Statistician + Analyst + Programmer
15. Statistician + Analyst + Programmer + Researcher
16. Statistician + Analyst + Programmer + Researcher + Consultant
17. Statistician + Analyst + Programmer + Researcher + Consultant + Writer
18. Worked in
19. c Academia
20. c Academia Freelance
21. c Academia Industry Freelance
22. Academia Bias
23. teaching
24. teaching admin
25. admin teaching research
26. research me teaching admin
27. Research in Academia
28. research me teaching admin
29. scholars scientists professors practitioners grad students ... etc
30. www.phdcomics.com (by Jorge Cham)
36. STATS everywhere
37. STATS everywhere main analysis
38. Need for analysts
39. What about tools?
40. Data does NOT stop
44. Need for tools!
45. Typically in Academia
46. existing methods
47. idea! existing methods
48. sketching idea! existing methods
49. prototype sketching idea! existing methods
50. prototype sketching idea! iterations existing methods
51. prototype sketching idea! iterations formal existing methods
52. prototype paper sketching idea! iterations formal existing methods
53. prototype paper sketching idea! iterations formal tool existing methods ?
54. For instance
55. f(T) = g(T, T) g(T, W) = tr(T' A(w) T) g(T, W) < h(T, W) Maximize tr(T' A(w) T) T' T = 1 revolutionary breakthrough, statistical method
56. Very Fancy Method for Analyzing and Visualizing Multivariate Data Main Authora,b, Graduate Studentb, Assistantb a Renowned Statistical Research Center b Department of Stuffmetrics and Computational Methods, Famous School (Pretty City) Abstract This paper depicts a new methodology devoted to analyze complex data generated from extremely expensive experiments in the field of Stuffmetrics. The ultimate goal is to bla, bla, bla One or more publications
57. # routine XY fun_xy <- function(x, y) { n = nrow(x) p = ncol(x) q = ncol(y) mx = rowMeans(x) my = rowMeans(y) z = t(x) %*% y ... lots of functions and scripts
58. but…
59. but… NO available software
60. Why?
61. takes time
62. takes time little funding + other reasons
63. Developing Programs?
64. Scripts Programs
65. Reading
66. Studying
67. Struggling
68. Deciphering
69. Programming
70. Documenting
71. Communicating
72. Stuff I’ve Done in R
73. Lower Level Tools
74. if (mode(x) == ‘numeric’ && x > 0 && x %% 1 == 0) TRUE else FALSE
75. if (mode(x) == ‘numeric’ && x > 0 && x %% 1 == 0) TRUE else FALSE is_positive_integer(x)
76. is_positive_integer(x) has_missing(x) has_colnames(x) is_tabular(x) is_numeric_matrix(x) “tester”
77. Multivariate Tools
78. PLS Path Modeling PLS Regression Discrimination
79. Visualization Tools
80. “arcdiagram”
81. “colortools”
83. The way I see it
85. State of the Art
86. There’s always room for ...
87. Organization Enrichment Disruption
88. Organize
89. Enrich
90. Disrupt
92. What can we do?
93. Reduce friction
94. Reduce friction Save time
95. Reduce friction Save time Make it prettier
96. Reduce friction Save time Make it prettier Make it easier
97. Reduce friction Save time Make it prettier Make it easier Make it interesting
98. start small
99. gastonsanchez.com github.com/gastonstat
100. Slides at http://bit.ly/1giAA7U
101. Color Palette #ff5b1e #ff9900 #c8e717 #25d181 #45818e #d9d9d9