Why is this so Bob Killen @mrbobbytables [email protected] Conveying the Business Value of Open Source HARD!

● Program Manager @ Google OSPO ● Previously Academia (16 years) ● Kubernetes Steering Committee Member ● Former Chair of Kubernetes SIG Contributor Experience Bob Killen @mrbobbytables 👋 Hi.

👋 Hi. ● Program Manager @ Google OSPO ● Previously Academia (16 years) ● Kubernetes Steering Committee Member ● Former Chair of Kubernetes SIG Contributor Experience Corporate Contributor Maintainer Hobby Contributor End User Bob Killen @mrbobbytables

Why is this so Conveying the Business of Open Source HARD! Value Bob Killen @mrbobbytables [email protected]

CNCF Contributions CNCF Devstats - Companies Table (All Projects - 1yr) 95.8% Affiliated 4.2% Independent Other Organizations (not VMware)

CNCF Contributions CNCF Devstats - Companies Table (All Projects - 1yr) 95.8% Affiliated 4.2% Independent Contributions Maintainers 94.7% Affiliated 5.3% Unaffiliated CNCF project-maintainers.csv Other Organizations (not VMware)

52% Contributed in both personal and professional time. 30% Contributed in professional time only. What brings you to open source? - August 2023 Personal or Professional? 18% Contributed in personal time only.

TL;DR MOST contributors are contributing on behalf of an organization. MOST contributors LIKE and WANT to contribute more to OSS.

TL;DR MOST contributors are contributing on behalf of an organization. MOST contributors LIKE and WANT to contribute more to OSS. And more importantly… To get PAID for it!

Explaining the Business Value Trying to make a bunch of random connections about influence, reducing risk of use, and trying to attract/retain talent… Can Feel Like…

Explaining the Business Value Trying to make a bunch of random connections about influence, reducing risk of use, and trying to attract/retain talent… Can Feel Like… ...but that frequently doesn’t hold up when pressed for why you should contribute vs. focusing on other initiatives.

Data-Enablement Communication Ensuring the right data is available to support OSS investments. Providing context in a human-friendly manner & telling the story of the data.

Data Enablement aka label all the things

Insufficient Data The Problem with

Insufficient Data 50% Labeled Issues Consistently 17% Labeled PRs Consistently 58% Used Milestones Source: ● 12 random CNCF projects ● 4 from maturity tier

The Importance of Labels Labels can ● Provide context ● Prioritization ● Scope ● State ● … Answer Questions & Track Trends

Examples Question: How many bugs were in the 1.30 Kubernetes release? Query: is:pr label:kind/bug milestone:v1.30 Answer: 180 Question: How are we progressing on tech-debt in SIG-Node? Query: label:kind/cleanup label:sig/node milestone:v1.30 Answer: 71

Example: Bug Trends is:pr label:kind/bug milestone:v1.XX Question: Has Kubernetes Stability improved over the past 6 releases? (2 years)

Example: Bug Trends is:pr label:kind/bug label:kind/regression milestone:v1.XX Question: Have there been fewer regressions in newer Kubernetes releases?

Example: Bug Trends is:pr label:kind/bug label:kind/api-change milestone:v1.XX Question: Have there been fewer regressions in newer Kubernetes releases?

Example: Bug Trends is:pr label:kind/bug label:area/kube-proxy milestone:v1.XX Question: Have there been fewer regressions in newer Kubernetes releases?

Which is easier?

Milestones Milestones offer an incredibly convenient way to associate issues/PRs into a logical grouping (e.g. releases). milestone:v1.30 created:2024-03-01..2024-03-05 VS Items opted into milestone Glob, queries everything between date range

Why Projects Don’t Triage Why are so many projects inconsistent with labels and milestones? ● Unaware of GitHub Triage role and believe write access is still required to apply labels & milestones. ● Don’t think they actually matter and are just another piece of “overhead” ● Too small a project to matter or warrant use.

Helpful Triage Actions Label Actions ● Copy Issue Label - Sync issue label to PR following “fixed”, “solved” etc. ● Pull Request Labeler - Applies labels to PRs based on glob patterns. ● Label Syncer - Declarative way to ensure labels are consistent across repos/orgs Project Actions ● Add To GitHub Projects - Automatically adds Issues / PRs to project boards based on labels used.

Triage With Prow Prow Prow fundamentally encourages label use, and can auto-apply labels via OWNERS files. Prow-github-actions Utilities action that mimics Prow functionality. Example Functions ● /milestone ● /kind ● /area ● /label ● /approve & /lgtm

Data Conclusions Consistent data from labels & milestones is crucial for surfacing the state of a project and enabling both contributors and businesses to evaluate the impact of investment. Data provides the evidence to back up your messaging to leadership. Recruiting people to help with triage and using tools to automate the process can ensure things are properly labeled and milestoned.

Communication Back to first principles

OSS Enthusiast ● Knows how to navigate OSS Ecosystem & projects well ● Does not mind diving deep into technical details ● May be participating as a hobby or as part of their job duties

OSS Enthusiast (Maintainer) ● Knows how to navigate OSS Ecosystem well projects well ● Does not mind diving deep into technical details ● May be participating as a hobby or as part of their job duties ● Burned Out, does not understand why more people don’t contribute ● Does “Hero” work to keep project(s) going ● …does not like to do project management / non-code type work …generally

Leadership (VPs, SVPs, C*Os) Concerned with resources, health and growth of company. Focused on Health of business, opportunities & risks. Prioritizes Return of Investment. Least technical. Product Managers Prioritizes making the best product. Skilled at conveying technical details in user friendly fashions. Desires customer/user feedback. End User Prioritizing how tools, features, etc can benefit them / their organization. Diverse range of technical skill. Bombarded by options. Managers / Leads Concerned about employees and meeting org objectives. Better understanding of tech and knowledge of what team is working on. Intermediary between team and leadership.

Limiting growth If it takes unnecessary additional cognitive overhead to: - Understand in basic terms what that feature does - Find your roadmap or the features & initiatives you’re working on - Know what parts of the project/feature/initiative are at risk You are losing potential users, future contributors, and making it more difficult to justify further investment of resources (e.g. headcount)

Example 1 Before: “This KEP proposes an automatic topology aware hinting mechanism that would provide a way for EndpointSlice producers to indicate where consumers should use specific endpoints.”

Example 1 Before: “This KEP proposes an automatic topology aware hinting mechanism that would provide a way for EndpointSlice producers to indicate where consumers should use specific endpoints.”

Example 1 After: “Topology Aware Routing adjusts routing behavior to prefer keeping traffic in the zone it originated from. In some cases this can help reduce costs or improve network performance.” Before: “This KEP proposes an automatic topology aware hinting mechanism that would provide a way for EndpointSlice producers to indicate where consumers should use specific endpoints.”

Example 2 Before: “This KEP proposes to allow mutating spec.completions for Indexed job if spec.completions equals to spec.parallelism before and after the update, meaning that spec.completions is mutable only in tandem with spec.parallelism.”

Example 2 After: “Enables autoscaling for Indexed Jobs” Before: “This KEP proposes to allow mutating spec.completions for Indexed job if spec.completions equals to spec.parallelism before and after the update, meaning that spec.completions is mutable only in tandem with spec.parallelism.” I can get behind that!

Surfacing Information Information regarding roadmaps, features, and their state should be easily accessible and written for a wide audience. ● Regular Project Reports / Blog posts ● Roadmap on website ● GitHub Project Board

Project Boards ● Project Boards are highly automatable via graphQL API ● Support for multiple views ● Exportable as tsv Kubernetes Release Team Board Automation Scripts

Status & Roadmap Issue Description Status Notes Target Tracking Issue - “Friendly” short description. - Parsed from tracking issue body - Alpha/Beta/Stable - Red/Yellow/Green - Optional notes to describe status - Parsed from tracking issue body Target Release ● Discoverable and easy to link to ● Provides just enough information to convey feature / initiative state & target release ● Status field can be used to signal things that are at risk ● Easily mappable to internal tracking information

Talking to Leadership ● How critical is the project? Can you associate $$ or level of risk with it. ● How many SWE hours are being spent? ● What is the business getting in return for those SWE hours? What happens if you stop? ○ List features, bugs - both driven by you, and ones that your org uses that you influenced, and others that were completely developed by others. ○ Try to associate with anything internal and $$ Leadership (VPs, SVPs, C*Os) Concerned with resources, health and growth of company. Focused on Health of business, opportunities & risks. Prioritizes Return of Investment. Least technical.

Managers & Users ● What is the roadmap? ● What is the status of items on the roadmap? And the things directly impacting us. ● What features / things are at risk? ○ If at risk, what is needed? ● What are the blockers / bugs? ● Managers: what is my team involved in?

Communication Summary How you communicate value is just as important as the metrics you use to back it up. They are complimentary, and should be used together. Understanding your audience and communicating with them in terms & language they are accustomed to can help you maximize the impact of what you are presenting. GitHub project boards are an excellent tool to use to for triage and sharing information about the status of enhancements, or used as a roadmap.

Based on True Events

Based on True Events The Troubled End User

The Troubled End User ● CNCF End User Company with historically very pro Open Source stance. ● Company’s Software Engineers & SysAdmins actively contribute multiple Open Source Projects. ● Portion of staff are maintainers of several projects that are used in the Company’s stack. ● Recently undergoing significant pressure from leadership to drop / cut back OSS to focus internal initiatives to deliver more “impact” to organization.

Understanding the problem Managers/Tech Leads generated a report for leadership about their open source presence. ● Overview of what was used internally in their stack. ● Included stats about influence and “work” done such as: % contributions, # of maintainers, commits. ● Included information about what features being developed upstream were being driven by company.

Understanding the problem Managers/Tech Leads generated a report for leadership about their open source presence. ● Overview of what was used internally in their stack. ● Included stats about influence and “work” done such as: % contributions, # of maintainers, commits. ● Included information about what features being developed upstream were being driven by company. Not giving leadership what they want

Understanding the problem - What is the criticality of the projects they are maintaining? - What are we getting out of contributing to the project? - How much SWE time is ACTUALLY being spent on maintaining the projects?

Bug Fix Stats (prev. year) Bug Statistics (prev. year) Total bugs 55 Submitted by Company members 11 Company bugs fixed by Company members 6 Mean time to fix Company member bugs 3~ days GitHub Query: is:issue closed:2023-01-01..2023-12-31 label:kind/bug ● Filter for company contributors: author:...author: ● Used GitHub API to query created_at and closed_at for each issue created by compantry contributors. ● Scriptable to generate regular report using GitHub GraphQL API.

Company Activity (prev. year) Company SWEs: 6 Average SWE time allocation: 10% Total SWE allocation: .6 Company Activity vs Others Project Committers: 54 Total Project Contributions: 4663 Company Contributions: 633 % Company Contributions: 14%

Outcomes & Lessons Learned Presenting information to leadership in ways they understand and align with the kind of decisions they were making, worked very well. Tying projects to their stack, and what services it enabled. Let leadership understand the criticality of those projects, and something to “weigh” the SWE investment.

Based on True Events The Struggling Maintainer

Struggling Maintainer ● Struggling maintainer was the only active maintainer left, others all drifted away from project. ● Could not adequately stay on top of issues/PRs. ● Pressure from $dayjob to spend time elsewhere. Project

Understanding the Problem ● No easily discernible way to see state of project. ● No easy way to follow up if someone could commit serious resources.

Understanding the Problem Not communicating the problem! ● No easily discernible way to see state of project. ● No easy way to follow up if someone could commit serious resources.

Actions Taken ● Updated Issue/PR template, and repo README communicating project is down to 1 maintainer and is at risk of going unmaintained. ● Reached out to known users/companies that have a dependency on project and highlighted risks & opportunities. ● Offered mentoring to those that could commit resources to become future maintainers.

Focus: Outreach Reached out to known users/companies that have a dependency on project and highlighted risks & opportunities. ● # of open bugs + rate of being addressed ● Looked at open feature requests ● Drafted “job description” for maintainer role ● Estimated weekly SWE hours for “maintenance” once onboarded ● Proposed mentorship to onboard if they can commit resources.

Focus: Mentoring Offered mentoring to those that could commit resources to become future maintainers. ● Convinced maintainer’s employer to let them spend more time on the project for a short period of time to on-ramp others to de-risk their continued usage of the project and let them spend more time on other things long-term. ● 3 month mentor cohort of 8~ people that met “job description” from various vendors and users with the goal of getting 3~ people who could take on maintainer duties.

Outcomes & Lessons Learned Framing the problem around what’s important to the company, and conveying it in a way those not “close” to the project understand was essential to getting buy-in. Providing some base metrics (bugs, issues etc) as a starting point, ensured everyone had a way to follow progress and track success.

Putting it all Together There is significant unrealized value in Open Source Software. Projects can begin to surface that value by following good triage practices and surfacing information about their state in more accessible ways to support the maintainers and the businesses behind them to justify their investments.

Thanks for listening to me ramble for 30min <3 Want to follow up? [email protected] @mrbobbytables https://mrbobbytabl.es Plz may I have some feedback 🙏