Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Web Scale Collaborations
Search
Arfon Smith
December 10, 2013
Science
0
81
Web Scale Collaborations
Arfon Smith
December 10, 2013
Tweet
Share
More Decks by Arfon Smith
See All by Arfon Smith
Why Generative AI makes collaborative, versioned science more important than ever
arfon
0
26
Generative AI is here: What are we going to do about it?
arfon
0
120
Five principles for building generative AI products
arfon
0
93
Five principles for building generative AI products
arfon
0
180
Learning from NASA's commitment to open
arfon
0
82
JOSS rOpenSci presentation
arfon
0
260
Five ways to use GitHub to automate scholarly work
arfon
0
110
Journal of Open Source Software: Bot-assisted community peer-review
arfon
0
110
A vision for the future of astronomical archives
arfon
0
150
Other Decks in Science
See All in Science
地表面抽出の方法であるSMRFについて紹介
kentaitakura
1
770
03_草原和博_広島大学大学院人間社会科学研究科教授_デジタル_シティズンシップシティで_新たな_学び__をつくる.pdf
sip3ristex
0
510
02_西村訓弘_プログラムディレクター_人口減少を機にひらく未来社会.pdf
sip3ristex
0
520
生成AIと学ぶPythonデータ分析再入門-Pythonによるクラスタリング・可視化をサクサク実施-
datascientistsociety
PRO
4
1.6k
サイゼミ用因果推論
lw
1
7.4k
論文紹介 音源分離:SCNET SPARSE COMPRESSION NETWORK FOR MUSIC SOURCE SEPARATION
kenmatsu4
0
260
Ignite の1年間の軌跡
ktombow
0
140
LayerXにおける業務の完全自動運転化に向けたAI技術活用事例 / layerx-ai-jsai2025
shimacos
2
1.3k
安心・効率的な医療現場の実現へ ~オンプレAI & ノーコードワークフローで進める業務改革~
siyoo
0
270
SciPyDataJapan 2025
schwalbe10
0
240
mathematics of indirect reciprocity
yohm
1
160
科学で迫る勝敗の法則(電気学会・SICE若手セミナー講演 2024年12月) / The principle of victory discovered by science (Lecture for young academists in IEEJ-SICE))
konakalab
0
110
Featured
See All Featured
Adopting Sorbet at Scale
ufuk
77
9.5k
Documentation Writing (for coders)
carmenintech
72
4.9k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
760
Thoughts on Productivity
jonyablonski
69
4.7k
Making Projects Easy
brettharned
116
6.3k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Typedesign – Prime Four
hannesfritz
42
2.7k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
What's in a price? How to price your products and services
michaelherold
246
12k
Git: the NoSQL Database
bkeepers
PRO
431
65k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
Transcript
Web Scale Collaborations Arfon Smith @arfon
Citizen Science
Distributed Computing
None
Distributed Data Collection
None
None
Distributed Analysis
None
None
None
None
http://www.novacelestia.com
None
None
None
None
None
0 250,000 500,000 750,000 1,000,000 Professor Paper PhD SDSS
Classifications per hour 0 10,000 20,000 30,000 40,000 50,000 60,000
70,000 Hours 0 6 12 18 24 30 36 42 48 1 Kevin months Fukugita et al. 2007
None
None
None
None
None
None
None
None
None
SDSS HST Starforming pea Narrow-line Seyfert pea
None
None
None
None
None
None
None
Motivations
None
None
None
1,000,000,000,000 hours / year
Spectrum of cognitive surplus
None
None
Begins with open data
Open Source
Not treating code and data as first class research objects
GitHub
What is a GitHub?
None
None
None
None
Easier to work together than alone
Open Source collaboration
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Open Public ≠
Open (within your team, department or institution)
Electronic
Available
Asynchronous
Lock-free
None
None
None
Low friction collaboration
“open source is… reproducible by necessity” Fernando Perez http://blog.fperez.org/2013/11/an-ambitious-experiment-in-data-science.html
Better at collaborating because they have to be
Towards Collaborative Versioned Science
How do we make this behaviour the norm?
Incentive model (it’s broken)
Credit
http://dx.doi.org/10.6084/m9.figshare.828487
http://dx.doi.org/10.6084/m9.figshare.828487
None
None
Derive meaningful metrics from open contributions
“Academic environments of today do not reward tool builders” Ed
Lazowska, OSTP event http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf
A VISION AND STRATEGY FOR SOFTWARE FOR SCIENCE, ENGINEERING, AND
EDUCATION
What can we do today?
Take data management plans seriously
Try versioning your research
Share more than just data
If you’re going to share it then you better put
a licence on it
Thanks.
[email protected]
@arfon "