Stéfane Fermigier Founder & CEO, Abilian - Enterprise Social Software Corporate open source development (When you’re not an open source software vendor) 26 May 2019

About me • PhD in mathematics • Free / Open Source user since 1988 • Linux user since 1991 • Python developer since 1996 • PyData Paris / PyParis founder / (co)organiser (2014-2018) • Co-founded AFUL, EuroLinux Alliance, GTLL, CNLL • Founded 3 companies: Linbox (1999), Nuxeo (2000), Abilian (2013) • Launched a few open source projects: Nuxeo CPS, Nuxeo EP, Apache Stanbol, Apache Chemistry, Eclipse Apricot, Abilian SBE, OlaPy… • “Expert” in open source R&D projects evaluation

Before you start

Why open source your software (as a company)? Depends on the kind of product you are considering open sourcing: • B2B or B2C • Business or technical • Core or not • Strategic or not

Why open source your software (as a company)? Many possible reasons: • It’s a core, strategic asset and you have an open source business model (i.e. monetization strategy) for it. Ex: MongoDB, Nuxeo, etc. • It’s a non-core, strategic asset you are using in a play against competitors (ex: Google w/ Android) • It’s a non-core, non-strategic asset (ex: Yahoo w/ Hadoop, Facebook w/ React…) and you just want your technology to become the standard (instead of someone else’s) • You’re going bankrupt (ex: RethinkDB) • …

Why open source your software (as a company)? In this talk, we will assume it’s for non-core non-strategic (i.e. non revenue producing) software aimed primarily at software developers (library / framework)

Costs and benefits Benefits • Improve your software quality • Get feedback • Get contributors • Share the burden of maintenance with others • Positive image for the company • Recrutement • Internal transformation Costs • Initial costs: Setting up the whole thing • Ongoing costs: • Maintenance (specially for use cases that are not yours) • Communication • It’s a long-term commitment (ex: security)

What’s an open source community? • Shared values, goals, vision & rules • People > Code • But overall goal is to create quality software (paradox) • Natural progression (funnel / ladder) PROSPECT / LURKER USER (PASSIVE -> ENGAGED) CONTRIBUTOR / DEVELOPER CORE DEV

Transparency Openness Consensus Open source code is freely available, one can “look inside the box”. But more importantly, the development process should be fully transparent too (no closed- door decisions, for instance). Open source is open to use, but also should be open to participation. Deliberate processes, tools and mindsets should be designed to help grow an healthy community. Decisions should be taken by consensus, with all the stakeholders being part of the process, and the needs of the majority (not just the sponsoring organisation) taken into account. Open source values

Main challenges Internal • Shared objectives and vision (internally) • IP / legal • Organisational and technical (process & tools) External • Indifference • Dealing with users with highly variable environments, tools, needs, experience, culture, etc. • Conflicts (opening the project to potentially toxic people)

Legal issues: Licensing and CLA • License choice means maximising two different objectives: community engagement and the benefits for your company • Main license types • Copyleft or not • Patent clauses or not • Probably easier for a non-strategic project • Choose the license that will maximise community engagement as long as you can use the software as intended) • CLA or not CLA ? (Answer: avoid if possible) • Third-party libraries (license compatibility)

The Architecture of Participation (O’Reilly, 2003) • Make it as easy as possible to get involved as a developer, by e.g.: • Making easy or at least properly documenting how to set up a development or testing environment (➔ Virtualenv, Vagrant, Docker…) • Providing lists of “easy tickets” for newcomers • Welcoming and possibly mentor newcomers, thanking contributors… • Make it hard to derail the project (resilience)

Daily life of an open source development team

Development Packaging & Distribution Marketing & Support Collaboration Making great software Making software easy to run Getting more users and contributors Working with people outside your organisations Missions of an OSS project team

Tools to supports these missions • “Forge” (GitHub ? Gitlab ?) • CI / CD • Automated metrics and quality audit + Human code reviews • Web site / Documentation site • Communication tools

Development, packaging and distribution Building & shipping great software

Specific tools for Python projects • Virtual env management: Virtualenvwrapper, Pipenv… • Test harness: pytest, tox… • CI : Travis, Gitlab CI, Azure Pipelines… • Documentation: Sphinx + Read the docs • Formatting: black • Linting: flake8 + plugins, mypy, pylint • Packaging: Setuptools, Poetry…

Releasing and packaging • Versioning: semantic vs. time-based ? • Preserve forward compatibility (up to a point) • Deprecation, maintenance of old versions (LTS?), security updates • Relationship with downstream distributors: artefact repositories (PyPi, NPM, Maven, etc.), Linux distributions, Docker images, etc.

Communication & Collaboration

Communication: towards users Some tools • Marketing (logo, swag…) (vs / and) documentation • Pitch your project in 1, 5, 60 minutes • Roadmap and changes • Videos: screencasts and conference talks (don’t forget to link the slide decks too) • Jupyter notebooks • Sample projects & data, SDKs, IDE plugins… • Link to external resources: blog posts, projects using your project… Goal: make it easy for people to discover, evaluate and use your software

Key documentation artefacts or parts In the source code • README (.txt / .md / .rst) • LICENSE (of course!) • NEWS / CHANGES • ROADMAP / TODO • CONTRIBUTING / COC • CONTRIBUTORS Outside the source code • Pitch / Vision (functional & technical) • Tutorial • Reference (API) • Mostly autogenerated from source (e.g. Sphinx) • Include some of the resources on the left column See: https://tom.preston-werner.com/2010/08/23/readme-driven-development.html 
 & https://www.writethedocs.org/

Communication: with users • Main goals: • Provide support to users with minimal effort from the development team • Try to grow users into contributors • Online tools: • Mailing list (+ archives!) & forums for announcements, discussions • Ticket trackers for support (Issue templates can be useful) • Synchronous tools (chat) ? - IRC, Jabber, Discord, Gitter… • Participation in outside communities (ex: StackOverflow, Reddit, LinuxFr…) • Also IRL: conferences, meetups, hackathons

Collaboration: with contributors / developers • Main goals: • Make it easy to contribute for people with different expertise, schedule, etc. • Remember the 3 core values (openness, transparency & consensus) • Tools • Same as previous slide, plus… • A DCVS and associated tools • Pull requests with comments, templates, etc. • Issue tracker used as task tracker (w/ milestone, etc.) • Wiki for collaborative elaboration of long-term plans • Real life meetings (must be open to all) or on a chat platform

Organisational considerations

Managing an open source community • Dealing with humans may not be a natural skills for some technical people • “Architecture of participation” ↝ Developer Experience (DX) et Developer Relations (DevRel) • Governance: should conform to the three open source values (transparency, openness, consensus) ➔ Clarity of: vision + decision process

Useful tools • Contributing guide • Example: How We Work: the Abilian Developer Guide • Code of Conduct • Release process (i.e. milestones) • Formalised enhancement process (à la PEP, JCP…) • Remember: no decisions behind closed doors • Project templates if you plan to create many small(ish) projects

KPIs / (Vanity?) Metrics • Usage: number of downloads, usage tracker… • Source code: number of “stars”, contributors, active contributors, contributions over time, time to close a PR… • Support: number of tickets, time to close a ticket… • External engagement: distribution of contributors between internal and external "If you can’t measure it, you can’t improve it.” - Lord Kelvin (wrongly attributed to P. Drucker)

Conclusion

Key takeaways • Going open source is, for a company, a business decision with objectives and resources that should be defined clearly from the onset, and managed throughout the life of the project • A corporate-sponsored open source project should align the interests of internal and external stakeholders, and must adhere to the open source values to be successful • Strive to learn and adhere to commonly established (and evolving) best practices • Have fun!

Questions ?

Annexes (not shown during presentation)

Some references • Open Source values (2007) by Dana Blankenhorn • Antipatterns (2010): LCA: How to destroy your community • Jono Bacon: The Art of Community (2012) • Kathy Sierra: Badass Users book — Making users awesome (2015) • The struggles of an open source maintainer - (2019)

The 10 antipatterns by Josh Berkus 1. Make the project depend as much on difficult tools 2. Encourage the presence of poisonous people 3. Provide no documentation 4. Make decisions in closed-door meetings 5. Employ large amounts of legalese 6. Choose the wrong leaders / liaisons 7. Governance obfuscation 8. Screw around with licensing 9. Do not allow anybody outside the company to have commit access 10. Silence

Credits & License License for this presentation • Creative Commons CC-BY-SA Image credits • https://pixabay.com/photos/raindrops-transparency-green-nature-1532618/ • https://pixabay.com/illustrations/hands-finger-touch-team-teamwork-1691221/ • https://pixabay.com/photos/illuminated-light-neon-light-sign-1853924/ • https://pixabay.com/illustrations/megaphone-loud-speak-communication-1725694/ • https://pixabay.com/vectors/box-packaging-2071537/ • https://pixabay.com/photos/team-team-building-success-computer-3373638/ • https://commons.wikimedia.org/wiki/File:HAL9000.svg