Scientific Software and the Open Collaborative Web

03e2e7de45b193cac192ae7ea071e5ff?s=47 Arfon Smith
November 17, 2013

Scientific Software and the Open Collaborative Web

Practices vary between scientific domains but all too often the sharing of research software is done on an ad hoc basis between individuals and with little thought about the wider community. With code and computation routinely forming the backbone of many academic endeavours we need to focus on publishing *all* of the products of our research – papers, software, data and provenance. In this talk I will highlight some promising examples from across academia and discuss how software development in the sciences could benefit from the collaboration norms of a modern open source project.

03e2e7de45b193cac192ae7ea071e5ff?s=128

Arfon Smith

November 17, 2013
Tweet

Transcript

  1. 6.
  2. 7.
  3. 9.
  4. 10.
  5. 11.
  6. 12.
  7. 13.
  8. 14.

    130 130 1 2048 189 189 258 258 480 562

    378 378 493 521 390 397 851 851 247 274 319 319 304 580 493 511 610 636 188 188 228 228 > cat bad_pix_mask.txt
  9. 15.
  10. 19.
  11. 20.

    Wasteful 2 days work 3 observing runs/week 52 weeks in

    year 15 year detector lifetime ! 2*3*52*15 = 4680 days (13 years)
  12. 21.

    Wasteful… but the norm 2 days work 3 observing runs/week

    52 weeks in year 15 year detector lifetime ! 2*3*52*15 = 4680 days (13 years)
  13. 22.
  14. 23.
  15. 29.
  16. 30.
  17. 31.
  18. 32.
  19. 33.

    “publishing a paper about code is basically just advertising” David

    Donoho http://www.stanford.edu/~vcs/Video.html
  20. 37.
  21. 40.
  22. 41.
  23. 42.
  24. 43.
  25. 44.
  26. 45.
  27. 46.
  28. 55.
  29. 56.
  30. 57.
  31. 58.
  32. 59.
  33. 61.
  34. 62.
  35. 63.
  36. 64.
  37. 65.
  38. 67.
  39. 68.
  40. 70.
  41. 71.
  42. 72.
  43. 74.
  44. 75.
  45. 79.
  46. 82.
  47. 83.

    “publishing a paper about code is basically just advertising” David

    Donoho http://www.stanford.edu/~vcs/Video.html
  48. 84.
  49. 86.

    “Academic environments of today do not reward tool builders” Ed

    Lazowska, OSTP event http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf
  50. 89.

    http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf Software Tools, Environments, and Support# •  SoSware%environments%and%tools%are%crucial% •  Organic,#sustainable,#reusable,#extensible#

    •  Easy#to#translate#across#problem#domains# •  The#creaHon#and#usage#of#today’s#tools#and#sojware# environments#are#distracHng#from#the#science# •  Today's%academic%environments%do%not%reward%tool%builders# •  How#can#the#development,#hardening,#sustaining,#sharing,# and#integraHon#of#techniques#into#a#reusable#sojware# infrastructure#be#recognized#and#incenHvized?# 17# Example%approach:%Teams%of%soSware%architects,%engineers,% and%researchers%who%will%produce%data%science%tools%and%will%be% evaluated%on%the%impact%of%these%tools%
  51. 93.
  52. 95.

    130 130 1 2048 189 189 258 258 480 562

    378 378 493 521 390 397 851 851 247 274 319 319 304 580 493 511 610 636 188 188 228 228 > cat bad_pix_mask.txt > git clone git@github.com:arfon/aat/pixel_masks