Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GitHub Insights: Understanding Open Source
Search
Georgios Gousios
May 19, 2016
Technology
0
320
GitHub Insights: Understanding Open Source
Talk given at OSCON 2016
Georgios Gousios
May 19, 2016
Tweet
Share
More Decks by Georgios Gousios
See All by Georgios Gousios
NLP + SE = ❤️
gousiosg
0
250
The troubles of modern dependency management and what to do about them
gousiosg
0
470
Mining Repositories with Apache Spark
gousiosg
0
560
My adventures with open everything
gousiosg
0
240
Structure and Evolution of Package Dependency Networks
gousiosg
0
700
Mining Github for fun and profit
gousiosg
9
63k
Work Practices and Challenges in Pull-Based Development: The Contributor’s Perspective
gousiosg
0
880
Big Data in Software Engineering panel and Privacy: Should we care?
gousiosg
0
240
The #issue32 incident
gousiosg
2
15k
Other Decks in Technology
See All in Technology
CyberAgent 生成AI Deep Dive with Amazon Web Services / genai-aws
cyberagentdevelopers
PRO
1
480
分布で見る効果検証入門 / ai-distributional-effect
cyberagentdevelopers
PRO
4
700
小規模に始めるデータメッシュとデータガバナンスの実践
kimujun
3
590
Product Engineer Night #6プロダクトエンジニアを育む仕組み・施策
hacomono
PRO
1
470
Commitment vs Harrisonism - Keynote for Scrum Niseko 2024
miholovesq
6
1.1k
Autify Company Deck
autifyhq
1
39k
【若手エンジニア応援LT会】AWSで繋がり、共に成長! ~コミュニティ活動と新人教育への挑戦~
kazushi_ohata
0
180
使えそうで使われないCloudHSM
maikamibayashi
0
170
現地でMeet Upをやる場合の注意点〜反省点を添えて〜
shotashiratori
0
530
Automated Promptingを目指すその前に / Before we can aim for Automated Prompting
rkaga
0
110
Forget efficiency – Become more productive without the stress
ufried
0
150
Fargateを使った研修の話
takesection
0
120
Featured
See All Featured
Building a Scalable Design System with Sketch
lauravandoore
459
33k
Building a Modern Day E-commerce SEO Strategy
aleyda
38
6.9k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
41
2.1k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
37
1.8k
jQuery: Nuts, Bolts and Bling
dougneiner
61
7.5k
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
3
370
Java REST API Framework Comparison - PWX 2021
mraible
PRO
28
7.9k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
250
21k
Producing Creativity
orderedlist
PRO
341
39k
Building Applications with DynamoDB
mza
90
6.1k
Visualization
eitanlees
144
15k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
107
49k
Transcript
GitHub Insights Understanding Open Source @jeffmcaffer–Microsoft Georgios Gousios –Delft University
of Technology (TU Delft) Kevin Lewis – Microsoft
Snapshot overview
Inspire confidence
How open is a project? http://ghtorrent.org/pullreq-perf/
Commits (core vs community)
Commits (origin)
Comments (core vs community)
PR lifelines
Are we using git in a distributed way?
How may devs are there per country?
Insights
Business insights
Research insights
Cross-domain insights
Operational insights
Approach Data for the masses
GitHub by the numbers (Mid 2016)
Approach http://ghtorrent.org
How does it work? http://api.github.com/events
Example event (condensed) https://api.github.com/users/Cephei https://api.github.com/repos/PowerDMS/Owin.Scim https://api.github.com/repos/PowerDMS/Owin.Scim/commits/c751014f634d73e0b72f78a53c8cf137888b3 https://api.github.com/orgs/PowerDMS
Entities
GHTorrent architecture Github API Event Retrieval Commits Queue Project Events
Queue Events Data Retrieval Projects Commits evt.commit evt.watch evt.fork Data Retrieval Data Retrieval Data Retrieval Mirroring Cluster
GHTorrent by the numbers
Using the data You can do it too!
Using the data: Hosted http://ghtorrent.org
Using the data: Download
Using the data: Self-service https://github.com/ghtorrent/ghtorrent-webhook
Using the data: Azure Data Lake
Resources http://ghtorrent.org https://github.com/Microsoft/ghinsights @gousiosg @jeffmcaffer @kelewis