Experiences analyzing OSS ecosystems

Inner Source Commons @ Boston, 2016

Transcript

1. Experiences analyzing OSS ecosystems Daniel Izquierdo, CDO dizquierdo@bitergia.com @dizquierdo speakerdeck.com/bitergia
2. Outline What we do Use cases Methodology (similar method and different goals in inner source) What we have learned/Achievements Summary
3. Goal Share experience analyzing OSS communities Bring that knowledge to Inner Source Help to build trustable communities
4. /me Bitergia co-founder (CDO) Free Software Engineering PhD ‘Buggy human being activity patterns’ Q-reports, diversity analysis, ad hoc dashboards
5. What we do
6. What we do “Software analytics for your peace of mind” Aggregate info. from usual OSS data sources Enrich data about those OSS ecosystems Outcomes: Data API, actionable dashboards, quarterly reports, training, customizations
7. What we do Grimoire* toolchain helps with this Supported data sources: Git, Gerrit, Github (enterprise), mailing lists, IRC channels, Slack, Twitter, Jira, Bugzilla, Reps, Meetup, Stackoverflow (Stackexchange), Redmine, Phabricator... Grimoirelab, Metrics Grimoire @ github.com
8. What we do Metrics: dozens of them per data source Tools to merge info: eg. projects or developers across several data sources Panels with several focuses (others can be done)
9. What we do: examples Eclipse Foundation: eclipse.biterg.io OPNFV: opnfv.biterg.io CoreOS: coreos.biterg.io Xen Code Review: xen.biterg.io
10. What we do: Overview panel
11. What we do: Data Sources
12. What we do: Backlog and Eng.
13. What we do: features Projects/Developers/other layer visualizations Tooling to agg. Info per project/developer Drill down/enrichment process/data API...
14. Use Cases
15. Use case Attraction of developers and companies Marketing, transparency and neutrality Eg: who are the relevant members of this community?
16. Use case Development cycle From user stories to deployment Eg: How fast are we implementing requirements? Eg: How long does it take each of the phases of the development? Feature request -> backlog -> developing -> reviewing process -> CI -> entering into master -> more CI -> deployed in customer
17. Use case Engadgement Attraction and retention of new members Eg: How good are we retaining developers? And compared to others?
18. Use case Contributors funnel From first traces to core developer Eg: What % of basic contributors are actual developers nowadays? Eg: First traces: bug report, email, feature need -> first pull request -> first accepted pull request -> active in the project/repository
19. Use case Social Network Analysis Knowledge evolution Eg: From hidden knowledge to shared knowledge Eg: Who’s this developer working with? Areas of knowledge?
20. Use case Mentorship Helping newcomers Eg: Who are those helping others to understand the current product? Eg: Who are the newcomers? How fast are newcomers becoming mentors?
21. Methodology
22. Method Understand the process How people work, what rules they follow
23. Method Looking for data sources Communication infrastructure: Git, mailing lists, bug tracking system, code review, CI and others
24. Method Tools Data retrieval, cleaning, massaging... We use Grimoire* toolchain (100% OSS) + ELK for viz. There are others!
25. (Inner Source) Method We believe it’s quite straightforward from the OSS world Different goals, similar data sources, similar method, probably same tools to analyze repositories.
26. (Inner Source) Method Goals (some detailed) Increase collaboration Increase documentation Increase the number of people working in some technologies and at different layers
27. (Inner Source) Method Data Sources: Those like in OSS projects! GitHub [Enterprise], Mailing lists, IRC channels, Slack, Gerrit, etc
28. (Inner Source) Method Not that usual metrics in the OSS world: Productivity Human Resources Priorities
29. Lessons learned
30. Lessons learned: starting Building dashboards and fitting this to requirements take time
31. Lessons learned: starting Really understand the process You say you follow a process, but is it 100% true?
32. Lessons learned: define goals First stage goals are different from mature goals Initially: understand and check Then: new questions and definitions of KPIs Finally: definition of alerts and actions
33. Lessons learned: infrastructure Infrastructure is similar in most of the OSS projects Even when they use different tools (mailing lists vs Gerrit) Although some tools help a lot
34. Lessons learned: inner source It’s feasible to bring that experience and toolchain to the inner source world Metrics are similar but with different purposes
35. Lessons learned: You bring the knowledge of your community and how to proceed We measure if this is being successful
36. Lessons learned: cultural issues Metrics may lead to undesired situations People don’t like to be tracked Gamification may not be appealing to devs. Having metrics may change people's behaviour to be good at those metrics
37. Lessons learned: tooling Several layers help the ‘Inner Source Team’ Data API -> Data Scientists Q reports -> Management Code Review panels -> Developers Demographics panel -> Recruiting
38. Lessons learned: tooling Whatever you use (Dashboard, reports, …) try to involve your community! They’ll feel it’s part of their process They’ll help improving that tooling (inner source project) It’s all about transparency and data driven
39. Successful Use Cases (probably of interest for inner source)
40. OpenStack Code Review Perception of a decay in the code review. Study: time waiting for a submitter or reviewer action (Gerrit) Outcome: training for newcomers and more focused reviewers and quarterly reports to follow those metrics
41. Xen Code Review Perception of a decay in the code review. Study: time to merge, cycle time Outcome: definition of user stories and creation of dashboard to follow those metrics
42. Ceph Goal: aggregate info about machines and performance Study: analysis of logs left by those machines Outcome: dashboard to follow all of those metrics
43. Mediawiki Goal: engineering/community metrics focused on volunteers (foster their participation) Study: analysis of those developers Outcome: dashboard to follow all of those metrics
44. Eclipse Foundation Goal: have a dashboard for generic tracking purposes Outcome: generic dashboard to follow all of those metrics
45. Summary
46. Summary Experts measuring OSS communities from several perspectives Infrastructure similar to inner source needs Bring that experience to inner source, we already have the tools! We’d love to learn together
47. Last but not least You apply inner source, but how do you measure success? Indeed, how do you know you’re measuring the right thing?
48. Thanks! Daniel Izquierdo, CDO dizquierdo@bitergia.com @dizquierdo