Share
Embed
Experiences analyzing OSS ecosystems
Inner Source Commons @ Boston, 2016
Bitergia
September 29, 2016
Transcript
48 slides
-
Outline What we do Use cases Methodology (similar method and
different goals in inner source) What we have learned/Achievements Summary -
Goal Share experience analyzing OSS communities Bring that knowledge to
Inner Source Help to build trustable communities -
/me Bitergia co-founder (CDO) Free Software Engineering PhD ‘Buggy human
being activity patterns’ Q-reports, diversity analysis, ad hoc dashboards -
What we do “Software analytics for your peace of mind”
Aggregate info. from usual OSS data sources Enrich data about those OSS ecosystems Outcomes: Data API, actionable dashboards, quarterly reports, training, customizations -
What we do Grimoire* toolchain helps with this Supported data
sources: Git, Gerrit, Github (enterprise), mailing lists, IRC channels, Slack, Twitter, Jira, Bugzilla, Reps, Meetup, Stackoverflow (Stackexchange), Redmine, Phabricator... Grimoirelab, Metrics Grimoire @ github.com -
What we do Metrics: dozens of them per data source
Tools to merge info: eg. projects or developers across several data sources Panels with several focuses (others can be done) -
What we do: examples Eclipse Foundation: eclipse.biterg.io OPNFV: opnfv.biterg.io CoreOS:
coreos.biterg.io Xen Code Review: xen.biterg.io -
What we do: features Projects/Developers/other layer visualizations Tooling to agg.
Info per project/developer Drill down/enrichment process/data API... -
Use case Attraction of developers and companies Marketing, transparency and
neutrality Eg: who are the relevant members of this community? -
Use case Development cycle From user stories to deployment Eg:
How fast are we implementing requirements? Eg: How long does it take each of the phases of the development? Feature request -> backlog -> developing -> reviewing process -> CI -> entering into master -> more CI -> deployed in customer -
Use case Engadgement Attraction and retention of new members Eg:
How good are we retaining developers? And compared to others? -
Use case Contributors funnel From first traces to core developer
Eg: What % of basic contributors are actual developers nowadays? Eg: First traces: bug report, email, feature need -> first pull request -> first accepted pull request -> active in the project/repository -
Use case Social Network Analysis Knowledge evolution Eg: From hidden
knowledge to shared knowledge Eg: Who’s this developer working with? Areas of knowledge? -
Use case Mentorship Helping newcomers Eg: Who are those helping
others to understand the current product? Eg: Who are the newcomers? How fast are newcomers becoming mentors? -
Method Looking for data sources Communication infrastructure: Git, mailing lists,
bug tracking system, code review, CI and others -
Method Tools Data retrieval, cleaning, massaging... We use Grimoire* toolchain
(100% OSS) + ELK for viz. There are others! -
(Inner Source) Method We believe it’s quite straightforward from the
OSS world Different goals, similar data sources, similar method, probably same tools to analyze repositories. -
(Inner Source) Method Goals (some detailed) Increase collaboration Increase documentation
Increase the number of people working in some technologies and at different layers -
(Inner Source) Method Data Sources: Those like in OSS projects!
GitHub [Enterprise], Mailing lists, IRC channels, Slack, Gerrit, etc -
(Inner Source) Method Not that usual metrics in the OSS
world: Productivity Human Resources Priorities -
Lessons learned: starting Really understand the process You say you
follow a process, but is it 100% true? -
Lessons learned: define goals First stage goals are different from
mature goals Initially: understand and check Then: new questions and definitions of KPIs Finally: definition of alerts and actions -
Lessons learned: infrastructure Infrastructure is similar in most of the
OSS projects Even when they use different tools (mailing lists vs Gerrit) Although some tools help a lot -
Lessons learned: inner source It’s feasible to bring that experience
and toolchain to the inner source world Metrics are similar but with different purposes -
Lessons learned: You bring the knowledge of your community and
how to proceed We measure if this is being successful -
Lessons learned: cultural issues Metrics may lead to undesired situations
People don’t like to be tracked Gamification may not be appealing to devs. Having metrics may change people's behaviour to be good at those metrics -
Lessons learned: tooling Several layers help the ‘Inner Source Team’
Data API -> Data Scientists Q reports -> Management Code Review panels -> Developers Demographics panel -> Recruiting -
Lessons learned: tooling Whatever you use (Dashboard, reports, …) try
to involve your community! They’ll feel it’s part of their process They’ll help improving that tooling (inner source project) It’s all about transparency and data driven -
OpenStack Code Review Perception of a decay in the code
review. Study: time waiting for a submitter or reviewer action (Gerrit) Outcome: training for newcomers and more focused reviewers and quarterly reports to follow those metrics -
Xen Code Review Perception of a decay in the code
review. Study: time to merge, cycle time Outcome: definition of user stories and creation of dashboard to follow those metrics -
Ceph Goal: aggregate info about machines and performance Study: analysis
of logs left by those machines Outcome: dashboard to follow all of those metrics -
Mediawiki Goal: engineering/community metrics focused on volunteers (foster their participation)
Study: analysis of those developers Outcome: dashboard to follow all of those metrics -
Eclipse Foundation Goal: have a dashboard for generic tracking purposes
Outcome: generic dashboard to follow all of those metrics -
Summary Experts measuring OSS communities from several perspectives Infrastructure similar
to inner source needs Bring that experience to inner source, we already have the tools! We’d love to learn together -
Last but not least You apply inner source, but how
do you measure success? Indeed, how do you know you’re measuring the right thing?