Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Web Scale Collaborations
Search
Arfon Smith
December 10, 2013
Science
0
83
Web Scale Collaborations
Arfon Smith
December 10, 2013
Tweet
Share
More Decks by Arfon Smith
See All by Arfon Smith
Why Generative AI makes collaborative, versioned science more important than ever
arfon
0
45
Generative AI is here: What are we going to do about it?
arfon
0
150
Five principles for building generative AI products
arfon
0
130
Five principles for building generative AI products
arfon
0
210
Learning from NASA's commitment to open
arfon
0
98
JOSS rOpenSci presentation
arfon
0
290
Five ways to use GitHub to automate scholarly work
arfon
0
140
Journal of Open Source Software: Bot-assisted community peer-review
arfon
0
130
A vision for the future of astronomical archives
arfon
0
160
Other Decks in Science
See All in Science
動的トリートメント・レジームを推定するDynTxRegimeパッケージ
saltcooky12
0
250
2025-05-31-pycon_italia
sofievl
0
140
高校生就活へのDA導入の提案
shunyanoda
1
6.2k
Accelerating operator Sinkhorn iteration with overrelaxation
tasusu
0
190
力学系から見た現代的な機械学習
hanbao
3
3.9k
20251212_LT忘年会_データサイエンス枠_新川.pdf
shinpsan
0
230
My Little Monster
juzishuu
0
550
baseballrによるMLBデータの抽出と階層ベイズモデルによる打率の推定 / TokyoR118
dropout009
2
760
Agent開発フレームワークのOverviewとW&B Weaveとのインテグレーション
siyoo
0
420
PPIのみを用いたAIによる薬剤–遺伝子–疾患 相互作用の同定
tagtag
PRO
0
160
コミュニティサイエンスの実践@日本認知科学会2025
hayataka88
0
120
やるべきときにMLをやる AIエージェント開発
fufufukakaka
2
1.1k
Featured
See All Featured
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
196
71k
Agile Actions for Facilitating Distributed Teams - ADO2019
mkilby
0
110
Leadership Guide Workshop - DevTernity 2021
reverentgeek
1
200
Building AI with AI
inesmontani
PRO
1
700
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
46
2.7k
A Soul's Torment
seathinner
5
2.3k
Stewardship and Sustainability of Urban and Community Forests
pwiseman
0
110
How Software Deployment tools have changed in the past 20 years
geshan
0
32k
The Illustrated Children's Guide to Kubernetes
chrisshort
51
51k
A Guide to Academic Writing Using Generative AI - A Workshop
ks91
PRO
0
210
Abbi's Birthday
coloredviolet
1
4.8k
The Cult of Friendly URLs
andyhume
79
6.8k
Transcript
Web Scale Collaborations Arfon Smith @arfon
Citizen Science
Distributed Computing
None
Distributed Data Collection
None
None
Distributed Analysis
None
None
None
None
http://www.novacelestia.com
None
None
None
None
None
0 250,000 500,000 750,000 1,000,000 Professor Paper PhD SDSS
Classifications per hour 0 10,000 20,000 30,000 40,000 50,000 60,000
70,000 Hours 0 6 12 18 24 30 36 42 48 1 Kevin months Fukugita et al. 2007
None
None
None
None
None
None
None
None
None
SDSS HST Starforming pea Narrow-line Seyfert pea
None
None
None
None
None
None
None
Motivations
None
None
None
1,000,000,000,000 hours / year
Spectrum of cognitive surplus
None
None
Begins with open data
Open Source
Not treating code and data as first class research objects
GitHub
What is a GitHub?
None
None
None
None
Easier to work together than alone
Open Source collaboration
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Open Public ≠
Open (within your team, department or institution)
Electronic
Available
Asynchronous
Lock-free
None
None
None
Low friction collaboration
“open source is… reproducible by necessity” Fernando Perez http://blog.fperez.org/2013/11/an-ambitious-experiment-in-data-science.html
Better at collaborating because they have to be
Towards Collaborative Versioned Science
How do we make this behaviour the norm?
Incentive model (it’s broken)
Credit
http://dx.doi.org/10.6084/m9.figshare.828487
http://dx.doi.org/10.6084/m9.figshare.828487
None
None
Derive meaningful metrics from open contributions
“Academic environments of today do not reward tool builders” Ed
Lazowska, OSTP event http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf
A VISION AND STRATEGY FOR SOFTWARE FOR SCIENCE, ENGINEERING, AND
EDUCATION
What can we do today?
Take data management plans seriously
Try versioning your research
Share more than just data
If you’re going to share it then you better put
a licence on it
Thanks.
[email protected]
@arfon "