Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Web Scale Collaborations
Search
Sponsored
·
Your Podcast. Everywhere. Effortlessly.
Share. Educate. Inspire. Entertain. You do you. We'll handle the rest.
→
Arfon Smith
December 10, 2013
Science
0
83
Web Scale Collaborations
Arfon Smith
December 10, 2013
Tweet
Share
More Decks by Arfon Smith
See All by Arfon Smith
Why Generative AI makes collaborative, versioned science more important than ever
arfon
0
44
Generative AI is here: What are we going to do about it?
arfon
0
140
Five principles for building generative AI products
arfon
0
130
Five principles for building generative AI products
arfon
0
210
Learning from NASA's commitment to open
arfon
0
96
JOSS rOpenSci presentation
arfon
0
290
Five ways to use GitHub to automate scholarly work
arfon
0
140
Journal of Open Source Software: Bot-assisted community peer-review
arfon
0
130
A vision for the future of astronomical archives
arfon
0
160
Other Decks in Science
See All in Science
Celebrate UTIG: Staff and Student Awards 2025
utig
0
710
データベース15: ビッグデータ時代のデータベース
trycycle
PRO
0
440
高校生就活へのDA導入の提案
shunyanoda
1
6.2k
タンパク質間相互作⽤を利⽤した⼈⼯知能による新しい薬剤遺伝⼦-疾患相互作⽤の同定
tagtag
PRO
0
140
データマイニング - グラフデータと経路
trycycle
PRO
1
270
Text-to-SQLの既存の評価指標を問い直す
gotalab555
1
170
次代のデータサイエンティストへ~スキルチェックリスト、タスクリスト更新~
datascientistsociety
PRO
2
27k
デジタルアーカイブの教育利用促進を目指したメタデータLOD基盤に関する研究 / Research on a Metadata LOD Platform for Promoting Educational Uses of Digital Archives
masao
0
150
Performance Evaluation and Ranking of Drivers in Multiple Motorsports Using Massey’s Method
konakalab
0
140
コミュニティサイエンスの実践@日本認知科学会2025
hayataka88
0
120
People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text
rudorudo11
0
190
My Little Monster
juzishuu
0
550
Featured
See All Featured
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
333
22k
Beyond borders and beyond the search box: How to win the global "messy middle" with AI-driven SEO
davidcarrasco
1
49
Testing 201, or: Great Expectations
jmmastey
46
8k
Stop Working from a Prison Cell
hatefulcrawdad
273
21k
[RailsConf 2023 Opening Keynote] The Magic of Rails
eileencodes
31
9.9k
Lightning talk: Run Django tests with GitHub Actions
sabderemane
0
110
Building Applications with DynamoDB
mza
96
6.9k
The Language of Interfaces
destraynor
162
26k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
12
1.4k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
10
1.1k
Why Your Marketing Sucks and What You Can Do About It - Sophie Logan
marketingsoph
0
72
Jess Joyce - The Pitfalls of Following Frameworks
techseoconnect
PRO
1
63
Transcript
Web Scale Collaborations Arfon Smith @arfon
Citizen Science
Distributed Computing
None
Distributed Data Collection
None
None
Distributed Analysis
None
None
None
None
http://www.novacelestia.com
None
None
None
None
None
0 250,000 500,000 750,000 1,000,000 Professor Paper PhD SDSS
Classifications per hour 0 10,000 20,000 30,000 40,000 50,000 60,000
70,000 Hours 0 6 12 18 24 30 36 42 48 1 Kevin months Fukugita et al. 2007
None
None
None
None
None
None
None
None
None
SDSS HST Starforming pea Narrow-line Seyfert pea
None
None
None
None
None
None
None
Motivations
None
None
None
1,000,000,000,000 hours / year
Spectrum of cognitive surplus
None
None
Begins with open data
Open Source
Not treating code and data as first class research objects
GitHub
What is a GitHub?
None
None
None
None
Easier to work together than alone
Open Source collaboration
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
None
Open Public ≠
Open (within your team, department or institution)
Electronic
Available
Asynchronous
Lock-free
None
None
None
Low friction collaboration
“open source is… reproducible by necessity” Fernando Perez http://blog.fperez.org/2013/11/an-ambitious-experiment-in-data-science.html
Better at collaborating because they have to be
Towards Collaborative Versioned Science
How do we make this behaviour the norm?
Incentive model (it’s broken)
Credit
http://dx.doi.org/10.6084/m9.figshare.828487
http://dx.doi.org/10.6084/m9.figshare.828487
None
None
Derive meaningful metrics from open contributions
“Academic environments of today do not reward tool builders” Ed
Lazowska, OSTP event http://lazowska.cs.washington.edu/MS/MS.OSTP.pdf
A VISION AND STRATEGY FOR SOFTWARE FOR SCIENCE, ENGINEERING, AND
EDUCATION
What can we do today?
Take data management plans seriously
Try versioning your research
Share more than just data
If you’re going to share it then you better put
a licence on it
Thanks.
[email protected]
@arfon "