Are you succeeding when InnerSourcing? Defining a metrics strategy

https://conferences.oreilly.com/oscon/oscon-tx/public/schedule/detail/61401

InnerSource is a new approach to deal with usual issues in large organizations to increase development velocity and improve developer engagement, but it is still unclear how this is measured—or if this process is even actually succeeding. Daniel Izquierdo explores the concepts and tools you need from an analytics perspective and explains how they can help you make decisions.

Monitoring should be one of the key aspects when applying InnerSource concepts within any organization. It requires expertise in the field, a detailed methodology and in general a strategy around measurements focused on awareness, process improvement, and motivational actions: awareness helps to understand the current software development stage; process improvement helps to detect issues and understand and fix the root error cause; and motivational actions push developers to reach some goals.

Metrics provide an understanding of the current structure and methodology of a software development team and how far an organization is from an ideal InnerSource situation. Indeed, it is possible to benchmark the team with respect to open source projects, such as the Apache Software Foundation, usually taken as example of where the InnerSource path leads to.

Topics include:

Goals using metrics
Areas of analysis
The Goal-Question-Metric approach
Strategy when using metrics
Some examples

Transcript

1. Defining a Metrics Strategy Daniel Izquierdo, CDO dizquierdo@bitergia.com @dizquierdo speakerdeck.com/bitergia
2. /me Founder of Bitergia, CDO Retrieve, massage, visualize More at bitergia.com/customers
3. Datasets are everywhere Overwhelming task What’s the right metric to use?
4. GrimoireLab data sources grimoirelab.github.io
6. Why?
7. Why? Why do we need metrics?
8. Why do we need metrics? Awareness
9. Why do we need metrics? Awareness Lead a change
10. Why do we need metrics? Awareness Lead a change Motivational
11. In Inner Source we may face any of those three stages at any point in our trip
12. What?
13. What? What can we measure?
14. What can we measure? Activity
15. What can we measure? Activity Community
16. What can we measure? Activity Community Process
17. What can we measure? Activity Community Process Code
18. How?
19. How? How can we measure?
20. How can we measure? Method Goal Question Metric Approach Objective and Key Results
21. How can we measure? Method Documentation Documentation Documentation Documentation Documentation
22. How can we measure? Method Strategy
23. How can we measure? Method Strategy C-level Middle management Contributors
24. How can we measure? Method Strategy For how long is a metric useful? => link to goals
25. How can we measure? Method Strategy For how long is a metric useful? => link to goals For how long is a goal useful?
26. How can we measure? Method Strategy Infra
27. When?
28. When? When do we start?
29. When do we start? Mature Org Transparency Aligned Interests
30. Bonus
31. Bonus: Gitbook GitBook/GitHub (CC-BY-SA) Looking for public use cases http://bit.ly/isbook-html Please take a copy!
32. Bonus: Gitbook
33. Bonus: Inner Source Patterns Those related to metrics https://github.com/paypal/InnerSourcePatterns We’ll see a great talk later about this
34. Bonus: Inner Source Commons The place to be http://paypal.github.io/InnerSourceCommons/ Great working group environment Useful insights
35. Summary
36. Summary Why What When How
37. Decisions based on data!
38. Defining a Metrics Strategy [Booth #429] Daniel Izquierdo, CDO dizquierdo@bitergia.com @dizquierdo speakerdeck.com/bitergia
39. Use Cases
40. Use case Mentorship Helping newcomers Eg: Who are those helping others to understand the current product? Eg: Who are the newcomers? How fast are newcomers becoming mentors?
41. Use case Attraction of developers and companies Marketing, transparency and neutrality Eg: who are the relevant members of this community?
42. Use case Development cycle From user stories to deployment Eg: How fast are we implementing requirements? Eg: How long does it take each of the phases of the development? Feature request -> backlog -> developing -> reviewing process -> CI -> entering into master -> more CI -> deployed in customer
43. Use case Engagement Attraction and retention of new members Eg: How good are we retaining developers? And compared to others?
44. Use case Contributors funnel From first traces to core developer Eg: What % of basic contributors are actual developers nowadays? Eg: First traces: bug report, email, feature need -> first pull request -> first accepted pull request -> active in the project/repository
45. Use case Social Network Analysis Knowledge evolution Eg: From hidden knowledge to shared knowledge Eg: Who’s this developer working with? Areas of knowledge?
46. Lessons learned: starting Building dashboards and fitting this to requirements take time
47. Lessons learned: starting Really understand the process You say you follow a process, but is it 100% true?
48. Lessons learned: define goals First stage goals are different from mature goals Initially: understand and check Then: new questions and definitions of KPIs Finally: definition of alerts and actions
49. Lessons learned: infrastructure Infrastructure is similar in most of the OSS projects Even when they use different tools (mailing lists vs Gerrit) Although some tools help a lot
50. Lessons learned: inner source It’s feasible to bring that experience and toolchain to the inner source world Metrics are similar but with different purposes
51. Lessons learned: cultural issues Metrics may lead to undesired situations People don’t like to be tracked Gamification may not be appealing to devs. Having metrics may change people's behaviour to be good at those metrics
52. Lessons learned: cultural issues Try to build comfortable environments when using metrics Developers should agree that ‘such metric’ is important for the project Metric projects with community traction are great, those are inclusive
53. Lessons learned: tooling Several layers help the ‘Inner Source Team’ Data API -> Data Scientists Q reports -> Management Code Review panels -> Developers Demographics panel -> Recruiting
54. Lessons learned: tooling Whatever you use (Dashboard, reports, …) try to involve your community! They’ll feel it’s part of their process They’ll help improving that tooling (inner source project) It’s all about transparency and data driven