Scientific Software and the Open Collaborative Web

Practices vary between scientific domains but all too often the sharing of research software is done on an ad hoc basis between individuals and with little thought about the wider community. With code and computation routinely forming the backbone of many academic endeavours we need to focus on publishing *all* of the products of our research – papers, software, data and provenance. In this talk I will highlight some promising examples from across academia and discuss how software development in the sciences could benefit from the collaboration norms of a modern open source project.

Transcript

1. Scientific Software and the Open Collaborative Web Arfon Smith @arfon
2. http://www.flickr.com/photos/blachswan
3. http://www.flickr.com/photos/esoastronomy/
4. http://www.flickr.com/photos/esoastronomy/ http://www.flickr.com/photos/jamiegilbert
5. http://amandabauer.blogspot.com/
8. Diffraction grating Telescope Detector
14. 130 130 1 2048 189 189 258 258 480 562 378 378 493 521 390 397 851 851 247 274 319 319 304 580 493 511 610 636 188 188 228 228 > cat bad_pix_mask.txt
15. Wasteful
16. Wasteful 2 days work !
17. Wasteful 2 days work 3 observing runs/week !
18. Wasteful 2 days work 3 observing runs/week 52 weeks in year
19. Wasteful 2 days work 3 observing runs/week 52 weeks in year 15 year detector lifetime
20. Wasteful 2 days work 3 observing runs/week 52 weeks in year 15 year detector lifetime ! 2*3*52*15 = 4680 days (13 years)
21. Wasteful… but the norm 2 days work 3 observing runs/week 52 weeks in year 15 year detector lifetime ! 2*3*52*15 = 4680 days (13 years)
23. Why?
24. We don’t know any different
25. We’re taught to focus on research not tools
26. We don’t act any different
27. Products of research, not software valued
28. Not treating software as first class research objects
33. “publishing a paper about code is basically just advertising” David Donoho http://www.stanford.edu/~vcs/Video.html
34. How is the Open Source community doing it?
35. Culture of reuse
36. Low friction collaboration
38. The pull request
39. Code first, permission later
47. “open source is… reproducible by necessity” Fernando Perez http://blog.fperez.org/2013/11/an-ambitious-experiment-in-data-science.html
48. Better at collaborating because they have to be
49. The social web ⋆ ♥ likes props favs
50. The social web The collaborative web
51. GitHub is about helping people build software together
52. What’s happening today?
53. Collaboration around code
54. Unidata: geosciences
60. Astropy: astronomy
66. Collaborative authoring
69. Collaborative teaching
73. Collaborative data collection
76. Towards Collaborative Versioned Science
77. How do we make this behaviour the norm?
78. Incentive model
79. Credit
80. http://dx.doi.org/10.6084/m9.figshare.828487
81. http://dx.doi.org/10.6084/m9.figshare.828487
83. “publishing a paper about code is basically just advertising” David Donoho http://www.stanford.edu/~vcs/Video.html
85. Derive meaningful metrics from open contributions
86. “Academic environments of today do not reward tool builders” Ed Lazowska, OSTP event http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf
87. A VISION AND STRATEGY FOR SOFTWARE FOR SCIENCE, ENGINEERING, AND EDUCATION
88. Establish a virtuous cycle# •  6%working%groups,%each%with%# •  3:6%faculty%from%each%ins;tu;on# 14# http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf
89. http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf Software Tools, Environments, and Support# •  SoSware%environments%and%tools%are%crucial% •  Organic,#sustainable,#reusable,#extensible# •  Easy#to#translate#across#problem#domains# •  The#creaHon#and#usage#of#today’s#tools#and#sojware# environments#are#distracHng#from#the#science# •  Today's%academic%environments%do%not%reward%tool%builders# •  How#can#the#development,#hardening,#sustaining,#sharing,# and#integraHon#of#techniques#into#a#reusable#sojware# infrastructure#be#recognized#and#incenHvized?# 17# Example%approach:%Teams%of%soSware%architects,%engineers,% and%researchers%who%will%produce%data%science%tools%and%will%be% evaluated%on%the%impact%of%these%tools%
90. What can we do today?
91. Share more often
92. If you’re going to share it then you better put a licence on it
94. Share more often (no matter how small)
95. 130 130 1 2048 189 189 258 258 480 562 378 378 493 521 390 397 851 851 247 274 319 319 304 580 493 511 610 636 188 188 228 228 > cat bad_pix_mask.txt > git clone git@github.com:arfon/aat/pixel_masks
96. Thanks. arfon@github.com @arfon $