Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Web Scale Collaborations
Search
Arfon Smith
December 10, 2013
Science
0
79
Web Scale Collaborations
Arfon Smith
December 10, 2013
Tweet
Share
More Decks by Arfon Smith
See All by Arfon Smith
Generative AI is here: What are we going to do about it?
arfon
0
71
Five principles for building generative AI products
arfon
0
76
Five principles for building generative AI products
arfon
0
160
Learning from NASA's commitment to open
arfon
0
70
JOSS rOpenSci presentation
arfon
0
240
Five ways to use GitHub to automate scholarly work
arfon
0
94
Journal of Open Source Software: Bot-assisted community peer-review
arfon
0
85
A vision for the future of astronomical archives
arfon
0
130
Journal of Open Source Software: When collaborative open source meets peer review
arfon
2
340
Other Decks in Science
See All in Science
機械学習を支える連続最適化
nearme_tech
PRO
1
260
トラブルがあったコンペに学ぶデータ分析
tereka114
2
1.4k
03_草原和博_広島大学大学院人間社会科学研究科教授_デジタル_シティズンシップシティで_新たな_学び__をつくる.pdf
sip3ristex
0
140
20分で分かる Human-in-the-Loop 機械学習におけるアノテーションとヒューマンコンピューターインタラクションの真髄
hurutoriya
5
2.8k
ほたるのひかり/RayTracingCamp10
kugimasa
1
530
The thin line between reconstruction, classification, and hallucination in brain decoding
ykamit
1
1.2k
創薬における機械学習技術について
kanojikajino
16
5k
Machine Learning for Materials (Challenge)
aronwalsh
0
170
05_山中真也_室蘭工業大学大学院工学研究科教授_だてプロの挑戦.pdf
sip3ristex
0
160
証明支援系LEANに入門しよう
unaoya
0
660
Transformers are Universal in Context Learners
gpeyre
0
720
ガウス過程回帰とベイズ最適化
nearme_tech
PRO
1
190
Featured
See All Featured
Easily Structure & Communicate Ideas using Wireframe
afnizarnur
193
16k
For a Future-Friendly Web
brad_frost
176
9.6k
How STYLIGHT went responsive
nonsquared
98
5.4k
Side Projects
sachag
452
42k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
7
650
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
Making Projects Easy
brettharned
116
6k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
366
25k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
30
4.6k
Statistics for Hackers
jakevdp
797
220k
How To Stay Up To Date on Web Technology
chriscoyier
790
250k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
10
510
Transcript
Web Scale Collaborations Arfon Smith @arfon
Citizen Science
Distributed Computing
None
Distributed Data Collection
None
None
Distributed Analysis
None
None
None
None
http://www.novacelestia.com
None
None
None
None
None
0 250,000 500,000 750,000 1,000,000 Professor Paper PhD SDSS
Classifications per hour 0 10,000 20,000 30,000 40,000 50,000 60,000
70,000 Hours 0 6 12 18 24 30 36 42 48 1 Kevin months Fukugita et al. 2007
None
None
None
None
None
None
None
None
None
SDSS HST Starforming pea Narrow-line Seyfert pea
None
None
None
None
None
None
None
Motivations
None
None
None
1,000,000,000,000 hours / year
Spectrum of cognitive surplus
None
None
Begins with open data
Open Source
Not treating code and data as first class research objects
GitHub
What is a GitHub?
None
None
None
None
Easier to work together than alone
Open Source collaboration
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Open Public ≠
Open (within your team, department or institution)
Electronic
Available
Asynchronous
Lock-free
None
None
None
Low friction collaboration
“open source is… reproducible by necessity” Fernando Perez http://blog.fperez.org/2013/11/an-ambitious-experiment-in-data-science.html
Better at collaborating because they have to be
Towards Collaborative Versioned Science
How do we make this behaviour the norm?
Incentive model (it’s broken)
Credit
http://dx.doi.org/10.6084/m9.figshare.828487
http://dx.doi.org/10.6084/m9.figshare.828487
None
None
Derive meaningful metrics from open contributions
“Academic environments of today do not reward tool builders” Ed
Lazowska, OSTP event http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf
A VISION AND STRATEGY FOR SOFTWARE FOR SCIENCE, ENGINEERING, AND
EDUCATION
What can we do today?
Take data management plans seriously
Try versioning your research
Share more than just data
If you’re going to share it then you better put
a licence on it
Thanks.
[email protected]
@arfon "