Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Web Scale Collaborations

03e2e7de45b193cac192ae7ea071e5ff?s=47 Arfon Smith
December 10, 2013
Science
0
62

Web Scale Collaborations

03e2e7de45b193cac192ae7ea071e5ff?s=128

Arfon Smith

December 10, 2013
Tweet

More Decks by Arfon Smith

See All by Arfon Smith
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
0
14
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
0
69
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
2
190
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
1
53
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
1
210
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
1
29
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
1
350
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
1
230
03e2e7de45b193cac192ae7ea071e5ff?s=48 arfon
1
150

Other Decks in Science

See All in Science
5473f177bf5562e68645e1a6e0428f9c?s=48 kuto5046
3
480
B099eaa1e1363ce79a24b4626dd930e7?s=48 smjyc
0
160
B57079e5c504d22b7fda28c5db13bcac?s=48 nkimoto
0
160
38e2af7f8bdad4f2087ab3d42b627e33?s=48 shuntaros
0
480
4fe5193960c3289769e3913281560dc4?s=48 taka88
0
280
A4d1023b1de7890c67a083d14573882d?s=48 ktgrstsh
3
400
C9ca88feec45d674cc85a7002674a0ce?s=48 ysawada
0
100
6402413ab839ccca6cce08c8bbd51827?s=48 terensarnic
0
220
E55f04d00716de969f69dd3c4475dc92?s=48 calpis10000
0
160
1e65da5f023dc428b8becced82bad823?s=48 takehikoihayashi
0
2.4k
38e2af7f8bdad4f2087ab3d42b627e33?s=48 shuntaros
0
470
753fba590507a4c2b0fa921f4bef68ab?s=48 ryoozawa
PRO
0
400

Featured

See All Featured
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
312
21k
027d1ebf66cc039a0bd3b55eeadbe75d?s=48 sachag
267
17k
170b760e0147792d0140e59ec78e9893?s=48 eitanlees
116
10k
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
779
240k
766b9746b59e6f09e280cd33cf4ed419?s=48 skipperchong
8
790
C9b18d9dff88a9dd2393364c2b3b21bd?s=48 colly
188
14k
F68cb25ae3e5c2106997454b13b7737d?s=48 brad_frost
157
6.5k
F57e2136eb9895eb95ff5dc8a1c02677?s=48 zakiwarfel
88
3.4k
B47b288784fda77a94aeb234ca743c24?s=48 keavy
108
14k
Ded01cdab114abe4ec13511e4c25b9bb?s=48 andyhume
64
3.8k
E76911cbe088e5b850d966de3fc7435b?s=48 robhawkes
53
2.9k
Dafc4723e9a1c067796c0fec6f509774?s=48 lemiorhan
629
48k

Transcript

  1. Web Scale Collaborations Arfon Smith @arfon

  2. Citizen Science

  3. Distributed Computing

  4. None

  5. Distributed Data Collection

  6. None
  7. None

  8. Distributed Analysis

  9. None
  10. None
  11. None
  12. None

  13. http://www.novacelestia.com

  14. None
  15. None
  16. None
  17. None
  18. None

  19. 0 250,000 500,000 750,000 1,000,000 Professor Paper PhD SDSS

  20. Classifications per hour 0 10,000 20,000 30,000 40,000 50,000 60,000

    70,000 Hours 0 6 12 18 24 30 36 42 48 1 Kevin months Fukugita et al. 2007
  21. None
  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. None
  29. None

  30. SDSS HST Starforming pea Narrow-line Seyfert pea

  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None

  38. Motivations

  39. None
  40. None
  41. None

  42. 1,000,000,000,000 hours / year

  43. Spectrum of cognitive surplus

  44. None
  45. None

  46. Begins with open data

  47. Open Source

  48. Not treating code and data as first class research objects

  49. GitHub

  50. What is a GitHub?

  51. None
  52. None
  53. None
  54. None

  55. Easier to work together than alone

  56. Open Source collaboration

  57. None
  58. None
  59. None
  60. None
  61. None
  62. None
  63. None
  64. None
  65. None
  66. None
  67. None
  68. None
  69. None
  70. None
  71. None
  72. None

  73. Open Public ≠

  74. Open (within your team, department or institution)

  75. Electronic

  76. Available

  77. Asynchronous

  78. Lock-free

  79. None
  80. None
  81. None

  82. Low friction collaboration

  83. “open source is… reproducible by necessity” Fernando Perez http://blog.fperez.org/2013/11/an-ambitious-experiment-in-data-science.html

  84. Better at collaborating because they have to be

  85. Towards Collaborative Versioned Science

  86. How do we make this behaviour the norm?

  87. Incentive model (it’s broken)

  88. Credit

  89. http://dx.doi.org/10.6084/m9.figshare.828487

  90. http://dx.doi.org/10.6084/m9.figshare.828487

  91. None
  92. None

  93. Derive meaningful metrics from open contributions

  94. “Academic environments of today do not reward tool builders” Ed

    Lazowska, OSTP event http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf

  95. A VISION AND STRATEGY FOR SOFTWARE FOR SCIENCE, ENGINEERING, AND

    EDUCATION

  96. What can we do today?

  97. Take data management plans seriously

  98. Try versioning your research

  99. Share more than just data

  100. If you’re going to share it then you better put

    a licence on it

  101. Thanks. arfon@github.com @arfon "