Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Open Source at Commercial Organisations

Open Source at Commercial Organisations

Deck presented at Devox UK 2022 and NDC Porto 2022.
Video from Devoxx UK: https://youtu.be/LDd-mzegvqg

Nikos Katirtzis

May 25, 2022
Tweet

More Decks by Nikos Katirtzis

Other Decks in Technology

Transcript

  1. E X P E D I A G R O

    U P Open Source at Commercial Organisations N i k o s K a t i r t z i s – D a n i e l A l b u q u e r q u e
  2. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S Agenda SECTION CONTENT E X P E D I A G R O U P 01 | Timeline 02 | Recommendations 03 | Projects Evolution of Open Source On-road experience Good practices ideas Our Projects 2
  3. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P 3 • Evolution of Open Source at Expedia Group • On-road Experience 01 Timeline
  4. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline Styx Open Source Foundation Hotels.com’s Open Source group 2017 2018 2019 4 ExpediaGroup on github.com 2020 2021 2022 Centralised Secret management Centralised User management Internal documentation InnerSource rollout Opt-in to OSCI GitHub-based process Internal Slack New project template Decommissioning organisations Showcase website > 100 projects ~300 community
  5. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P • One of our first projects • Took ~1 year to open source • Required CEO approval • Lack of processes is just as bad as too many • Do not be a perfectionist, ship the project and iterate • Avoid internal forks, prefer working in the open https://github.com/ExpediaGroup/styx Timeline - How it Started Lessons Learned What 📋 5
  6. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P • Creation of the Open Source and InnerSource Foundation • Mission; To foster an Open Source and InnerSource culture across the company Timeline - Expedia Group Open Source Foundation • A team of individuals with previous experience in Open Source can kickstart this Lessons Learned What 📋 6
  7. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Having multiple orgs: • Adds maintenance overhead • Hinders discoverability • Poses security challenges Timeline - Expedia Group on github.com https://github.com/ExpediaGroup Lessons Learned What 📋 7
  8. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P • How-to: Create Open Source Software • How-to: Use Open Source Software • … • How-to: Publish artifacts Timeline - Internal Documentation Lessons Learned What 📋 • A central space with documentation is of utmost importance • Use a memorable URL (e.g.: https://company/opensource) • Build a community in which individuals contribute to the documentation 8
  9. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - InnerSource Rollout 📋 Lessons Learned What • Presentation to engineering teams and to leadership • InnerSource checklist • Do not enforce InnerSource, it will come naturally as longs as the organisational structures allow it • Be aware of Conway’s Law • The InnerSource checklist can help set the expectations and guide teams 9
  10. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline – Internal Slack 📋 Lessons Learned What • Make employees aware of the communication channels • Provide a single place and reduced friction to reach out to open source champions • Internal Slack for Open Source • Used for: • Questions • Announcements • Promotion of projects • Reaching out to maintainers 10
  11. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - GitHub-based Process for Releasing Projects 📋 Lessons Learned What • Avoid emails & tickets • Introduce and document a clear and easy to follow process • Try to reuse the same process (for Contributor License Agreements, etc.) • Unified GitHub-based process for releasing projects to the open • More than 30 projects have been released using this process so far 11 Open Source at Expedia Group https://medium.com/expedia-group-tech
  12. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P • The owners of the project raise a Pull Request (PR) in our internal repository • The PR includes a short business justification, and a link to its internal repository • An Open Source champion picks up the PR and shepherds it through the release process Timeline - GitHub-based Process for Releasing Projects 12
  13. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P • Labels are used for each of the stages (technical review, security review, legal review) • GitHub Projects allow us to move projects across different stages • GitHub Teams enable us to tag stakeholders and add them as reviewers • A similar process is used for Contributor License Agreements (CLAs) Timeline - GitHub-based Process for Releasing Projects 13
  14. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P 📋 Lessons Learned What • Avoid emails and tickets • Can you use the same process for other approvals such as Contributor License Agreements (CLAs)? • Unified GitHub-based process for approving Contributor License Agreements (CLAs) • More than 10 CLAs have been approved using this process so far Timeline - GitHub-based Process for CLAs 14
  15. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - Centralised Secrets Management 📋 Lessons Learned What • Centralised management of Secrets using organisation Secrets on GitHub • Centralise Secret management to reduce maintenance overhead • Document available Secrets 15
  16. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - Centralised User Management 📋 Lessons Learned What • Centralise User management to reduce maintenance overhead • Automate addition and removal of members • Centralised and automated management of users using LDAP and GitHub APIs 16
  17. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - Decommissioning organisations Lessons Learned What • From 10+ GitHub organisations to a single one • Moved active repos, archived inactive ones 📋 • Consolidation is hard - think before you create more organisations • Consolidation is possible • Engage with your company's Comms/PR teams 17
  18. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - Open Source Website 📋 Lessons Learned What • A showcase website can boost the company’s profile and attract talent • Work with designers and your Comms counterparts • Creating these websites is a good learning opportunity for engineers Expedia Group’s Open Source website https://opensource.expediagroup.com/ Building the Expedia Group Open Source Site https://medium.com/expedia-group-tech 18
  19. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - Open Source Contributor Index Opt-in 📋 Lessons Learned What • OSCI helps you track contributions to open source projects from your organisation • Ensure you add all the email domains associated with your company • Remember - this is just a number • Opt-in to the Open Source Contributor Index (OSCI) 19 https://opensourceindex.io
  20. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - New Project Template 📋 Lessons Learned What • A template for new projects reduces time-to- market • It also ensures legal and security compliance • Template for new open source projects • Includes common files; license, code of conduct, templates for PRs and issues, etc. https://github.com/ExpediaGroup/new-project 20
  21. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Timeline - Now 📋 Lessons Learned What • Creating a community takes time • Open Source at commercial organisations requires constant investment • ~100 projects, ~30 active • ~90 active contributors, ~300 community • Over the last year we have 2X our community in terms of contributors 21
  22. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P 22 • Good Practices • Ideas 02 Recommendations
  23. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - On-road Experience GitHub Actions GitHub Secrets GitHub Apps GitHub Packages Creating an on-road experience makes it easier for you to manage and for teams to open source projects 23
  24. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Dependency Updates • Use tools for automated dependency updates • They also provide security mitigations at scale 24
  25. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Dogfood • Seek for internal contributions, not only external ones • This is a great opportunity to introduce engineers to open source 25
  26. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Work with Security • Work closely with your Security counterparts • Git hosting services provide a plethora of security tools nowadays: • Token scanning • CodeQL • Dependabot security alerts 26
  27. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Work with Legal • Work closely with your Legal counterparts • Licenses, logos, trademarks • Especially important for public companies ⚖ 27
  28. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Employee Departures • Convert any ex-employees to external collaborators if they still need access • Do not count on employees leaving the company maintaining projects • Communicate lack of maintenance 🛫 28
  29. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - CI/CD • Use your Git hosting service as much as possible • Travis, CircleCI, GitHub Actions • Attempt to offer a single solution for all the repositories 29
  30. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Documentation 📄 ü Keep it simple and concise ü Keep it up-to-date ü Know your audience ü Prefer documentation as code ü Consider documentation reviews • README.md • Markdown files • GitHub Pages • Wiki • External 30
  31. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Discoverability 🔎 • Make your repositories public • Documentation • Use GitHub topics • GitHub search 31
  32. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Promotion • Promote internally and externally • Open Source website • Conference talks • Blogposts • Swag 📣 32
  33. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Working in the Open https://github.com/DataDog/chaos-controller Happy Birthday, Backstage: Spotify’s Biggest Open Source Project Grows Up Fast https://engineering.atspotify.com 33 Chaos Engineering at Expedia Group https://medium.com/expedia-group-tech
  34. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Recommendations - Buy vs Build vs Open Source 34 Source: Photo by Alan O'Rourke - https://workcompass.com/
  35. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Key Takeaways • Introduce simple processes • Document expectations • Involve legal and security Build a structure around Open Source • Showcase your work • Avoid reinventing the wheel • Work in the open Leverage Open Source • Provide an on-road experience for open source • Address fragmentation and duplication as soon as you can • Centralise users and secrets management Consolidation is key 35
  36. O P E N S O U R C E

    A T C O M M E R C I A L O R G A N I S A T I O N S E X P E D I A G R O U P Expedia Group Projects https://github.com/ExpediaGroup/graphql-kotlin https://github.com/ExpediaGroup/insights-explorer https://github.com/ExpediaGroup/bull https://github.com/ExpediaGroup/beekeeper https://github.com/ExpediaGroup/mittens Libraries for running GraphQL in Kotlin A tool to catalogue and present analytical & research work Bean Utils Light Library Service for automatically managing and cleaning up unreferenced data Warm-up routine for HTTP applications over REST and gRPC 36