Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
GrimoireLab, a Python toolset for software development analytics - FOSDEM 2017
Search
Bitergia
February 05, 2017
Technology
0
160
GrimoireLab, a Python toolset for software development analytics - FOSDEM 2017
Slides for FOSDEM 2017 talk at Python DevRoom about GrimoireLab
Bitergia
February 05, 2017
Tweet
Share
More Decks by Bitergia
See All by Bitergia
Building and Supporting Open Source Communities through Metrics
bitergia
0
18
Defining the limits of Risk
bitergia
0
30
Present and Future of GrimoireLab
bitergia
0
23
InnerSource Commons
bitergia
0
66
Collaboration as Health Indicator
bitergia
0
83
La estrella de mi comunidad es un bot. ¿Dónde están los humanos?
bitergia
0
75
IoT Projects in FLOSS Foundations, a report based on community data
bitergia
0
75
Contributor Leaderboards to Incentivize Good Community Citizenship
bitergia
0
94
FreeScout: Cómo montar un departamento de soporte/atención al cliente con software libre
bitergia
0
210
Other Decks in Technology
See All in Technology
2023 Japan AWS Jr.Championsに選出されての振り返りとこれから
hiropy877
1
130
大規模なアジャイル開発の現場と技術負債 / Technical Debt
yoshiitaka
21
4.1k
VSCode上からSlackにメッセージを送る拡張機能を作っている話
ebarakazuhiro
0
120
Azureコストは水道代/The_47th_Tokyo_Jazug
aeonpeople
3
380
Elementaryを用いたデータ品質の可視化とデータ基盤の運用改善
10xinc
6
1.6k
サービスメッシュ環境における OpenTelemetry 活用 / OpenTelemetry in Service Mesh
k6s4i53rx
2
860
GitHub最新情報キャッチアップ 2024年3月
dzeyelid
17
3.2k
OpenTelemetry実践 はじめの一歩
taxin
0
320
継続的テストモデルを実現するためにスリーアミーゴスを用いた10Xでのシフトレフトの事例
nihonbuson
3
240
生成AI・LLM時代における 機械学習エンジニアとしてのキャリア戦略・開発戦略 / my-career-and-development-strategies-for-ml-engineer-2024
yuya4
4
850
技育祭2024春 LT Finatextホールディングス
kevinrobot34
1
170
中央集権体制からDataOpsへの転換 / centralized-to-dataops-transformation
pei0804
7
1.5k
Featured
See All Featured
Why Our Code Smells
bkeepers
PRO
330
56k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
657
120k
A designer walks into a library…
pauljervisheath
199
23k
Product Roadmaps are Hard
iamctodd
43
9.6k
Navigating Team Friction
lara
177
13k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
6
950
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
67
38k
Building an army of robots
kneath
300
41k
Designing on Purpose - Digital PM Summit 2013
jponch
109
6.4k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
39
4.3k
jQuery: Nuts, Bolts and Bling
dougneiner
57
7.1k
ParisWeb 2013: Learning to Love: Crash Course in Emotional UX Design
dotmariusz
101
6.6k
Transcript
GrimoireLab a Python toolset for software development analytics Jesus M.
Gonzalez-Barahona (URJC) @jgbarah jgb at bitergia dot com speakerdeck.com/bitergia FOSDEM 2017, Python devroom Brussels (Belgium) February 5th 2017
Outline Some context The software doing the magic Let’s be
practical Your turn
Some context
/me Like five years ago I was having coffees with
the gang of Bitergia founders Involved in the company since then bitergia.com I work at Universidad Rey Juan Carlos... ...researching about software development gsyc.es/~jgb My two hats:
Building Software Development Dashboards cauldron.io
The software doing the magic
Architecture Original Data Sources Retrieval Perceval Enrichment GrimoireELK Visualization Kibiter
(Kibana4/5) ElasticSearch GrimoireLab component
GrimoireLab grimoirelab.github.io
GrimoireLab grimoirelab.github.io
GrimoireLab Training Tutorial jgbarah.gitbooks.io/grimoirelab-training
Let’s be practical
Architecture Original Data Sources Retrieval Perceval Enrichment GrimoireELK Visualization Kibiter
(Kibana4/5) ElasticSearch GrimoireLab component
Retrieving data with Perceval In a Python3 environment… $ pip3
install perceval $ perceval git https://github.com/grimoirelab/perceval.git [2016-10-03 00:47:46,632] - Sir Perceval is on his quest. [2016-10-03 00:47:46,633] - Fetching commits: 'https://github.com/grimoirelab/perceval.git' git repository from 1970-01-01 00:00:00+00:00; all branches { "backend_name": "Git", "backend_version": "0.3.0", "data": { "Author": "Santiago Due\u00f1as <
[email protected]
>", "AuthorDate": "Tue Aug 18 18:08:27 2015 +0200", "Commit": "Santiago Due\u00f1as <
[email protected]
>", "CommitDate": "Tue Aug 18 18:08:27 2015 +0200", "commit": "dc78c254e464ff334892e0448a23e4cfbfc637a3", ….
Retrieving data with Perceval from Python from perceval.backends.core.github import GitHub
# GitHub object for owner / repo, repo_dir for cloning repo = GitHub(owner=owner, repository=repo, api_token=args.token) # fetch all issues/pull requests for item in repo.fetch(): if 'pull_request' in item['data']: kind = 'Pull request' else: kind = 'Issue' print(item['data']['number'], ':', kind)
Perceval Backends grimoirelab.github.io
Architecture Original Data Sources Retrieval Perceval Enrichment GrimoireELK Visualization Kibiter
(Kibana4/5) ElasticSearch GrimoireLab component
Creating indexes for a dashboard $ pip install grimoire-elk $
pip install grimoire-kidash $ p2o.py --enrich --index git_raw --index-enrich git \ -e http://localhost:9200 --no_inc --debug \ git https://github.com/grimoirelab/perceval.git $ kidash.py -e http://localhost:9200 \ --import git-dashboard.json Results: Raw index: git_raw Kibana dashboard Enriched index: git
Architecture Original Data Sources Retrieval Perceval Enrichment GrimoireELK Visualization Kibiter
(Kibana4/5) ElasticSearch GrimoireLab component
Querying ElasticSearch API $ curl -XGET "http://elasticsearch_url/git/_search/?size=1&pretty" {... "hits" :
{ "total" : 407, "hits" : [ { "_index" : "commits", "_type" : "summary", "_id" : "AVfPp9Po5xUyv5saVPKU", "_score" : 1.0, "_source" : { "hash" : "d1253dd9876bb76e938a861acaceaae95241b46d", "commit" : "Santiago Dueñas <
[email protected]
>", "author" : "Santiago Dueñas <
[email protected]
>", "author_date" : "Wed Nov 18 10:59:52 2015 +0100", "files_no" : 3, "commit_date" : "Wed Nov 18 14:41:21 2015 +0100" }} ] } }
Python scripting from elasticsearch import Elasticsearch from elasticsearch_dsl import Search
es = Elasticsearch(["elasticsearch_url”] s = Search(using=es, index=’git’) s = s.filter('range', files={'gt':0}) s = s.filter('range', author_date={'gt': datetime(2016, 7, 1)}) s.aggs.metric('commits', 'cardinality', field='hash') s.aggs.bucket('histogram', 'date_histogram', field='author_date', interval='quarter') by_q = s.execute() for quarter in by_q.to_dict()['aggregations']['histogram']['buckets']: print("Unique commits for quarter starting on ", Quarter['key_as_string'], ": ", quarter['doc_count'])
More details: GrimoireLab Training Tutorial jgbarah.gitbooks.io/grimoirelab-training
Try it live: The Cauldron cauldron.io
Your turn Enjoy! http://grimoirelab.github.io https://jgbarah.gitbooks.io/grimoirelab-training http://caludron.io