Consumption is Fractal: Open Source Sustainability (with notes)

Transcript

2. Let's get started. That's me in all of the places. I'm an infrastructure engineer at Ridecell and today I'm talking about open source sustainability.
3. Before we talk about sustainability, let's lay out some scope
4. There are some specific opinions on what exactly open source software is, but I'm going to take a more general stance further away from the specifics of what precisely makes something open.
5. I'm also not going to wade into the open-source vs. free-software debate.
6. So with that out of the way, let's look at a few types of projects.
7. The majority of projects by the numbers, one person writing some code to do a thing they need and posting it to GitHub because why not. This is how I got my start in open source, writing plugins for Trac.
8. But the more visible projects are the big ones. Projects so vast and omnipresent it's easy to think of them as part of the fabric of the universe.
9. Some of those big projects have formalized themselves under dedicated legal entities, usually those that are a partnership between multiple large companies.
10. A newer venue for open source projects has been as part of a VC-backed startup, with the company acting as a steward for their open source project and selling related stuff.
11. What I call mirage projects are an offshoot of that "feels like the fabric of the universe" kind of thing, except unlike some of the bigger projects, these often have frighteningly few core maintainers. We saw this with OpenSSL after Heartbleed, with most of the world unaware that such a critical library was maintained by a single digit number of volunteers.
12. Some older projects, especially big linux packagers tend to feel a bit stuffy compared to the simpler structures around the GitHub generation. But these projects also usually have very important history, having been built to stick around for the long haul even as individual developers come and go.
13. And some open source projects can only be described as crazy moonshots.
14. In my very wide view of open source are plenty of non-software projects too. Some of the legal specifics differ but the overall dynamics are basically the same no matter the medium.
15. This is clearly not a complete list of every open source project, but that's at least a taste of the kind of breadth and depth that open source occupies.
16. So that's two words from the title, what about the third?
17. Literally speaking, sustainability is ensuring that a project which exists today will continue to exist tomorrow, and the tomorrow after that, and so on.
18. Money and time are important, and we'll get to talking about those soon, but the fundamental limiting factor on most open-source projects is the rate of burnout of the contributors and maintainers. Most projects fail because people don't want to work on them anymore.
19. The most common symptoms of burnout in open source are these. A cycle of feeling guilty about letting down your users, leading to dread about working on the project, and finally bitterness towards the project itself or sometimes even the users of it.
20. Open source projects have truly taken over the world. In almost every facet of technology, communal resources have pushed us farther and faster. If we want this pace of progress to continue we need these shared resources, but with this cross-pollination comes inter-dependence. While open licensing helps ensure the possibility of someone picking up a fallow project, that doesn't help us if there are no new maintainers left to pick up a fallen banner.
21. And if I can digress for a moment on why I think open source itself is important, I think together we can build things better than any one team or organization. With a lot of problems in software, the value to the market is small or hard to define such that no one company can easily build a solution. But collectively we all can, and we are all better off for it.
22. So open source rules the world and that's great, but clearly I think there are problems with the system or I wouldn't be giving this talk. How did we end up with a worldwide ecosystem built on the backs of a relatively tiny number of volunteers?
23. If we want a globally simplistic answer, infrastructure investment has been lagging everywhere. Putting time and money into things like bridges and dams rarely shows returns immediately, if a return can even be quantified. If your focus is only on the earnings for the next quarter, it's easy to let things slip.
24. But a deeper, structural answer is in how we make things. I can only put my brain towards building one thing at a time, but even while doing that I am almost certainly making use of hundreds of other projects and tools. Even in the best scenarios there will always be more users than contributors, and that imbalance has spiraled far out of control.
25. Even a simple project like a Ruby on Rails web app sits on top of a mountain of other projects and libraries. As we build better and better abstractions, it becomes easier to forget the shoulders of the giants on which we stand.
26. But why all the interest in sustainability lately?
27. Way back when open-source was just some crazy college students, basically everyone that consumed open-source software also produced. Maybe I worked on gcc and you worked on bash, but we both benefited from each other.
28. Over time, this balance eroded. More people pull value out of the system than are putting it back in. As the third wave of open-source developers is reaching middle age and deciding that other pursuits are more important, often there are no longer fresh faces waiting to take up the banner. And so, here we are.
29. It's hard to pick a patient zero, but it was probably either the LAMP stack (PHP, MySQL, and Apache) or Ruby on Rails that really got the ball rolling on corporate use of open-source. VC-funded startups realized they could switch to free as in beer tools and save money. And since VCs hadn't learned about this one cool trick yet, the checks coming in stayed just as big.
30. If I'm being cynical, those early startups just pocketed the difference. Or if I'm being nice they used it to double-down on product development and figuring out how this whole "web" thing was going to work.
31. But all good bubbles must come to an end. By now startups and big companies alike know that zero-cost software stacks exist and many are very high quality, and budgets around the world have adapted to that reality. For many young startups, using free stacks is not just a technical choice, they often can't afford anything else.
32. At the end of the day, open-source and capitalism are at loggerheads. Capitalism requires privatization, and open-source is the opposite of that. As long as the bulk of companies continue to draw more value from the commons than they put back in, we'll always have a sustainability problem.
33. Much of the interaction between capitalism and open- source is in the form of IP. I did a whole talk on that last year so I'll just refer you to that if you want more info on how IP shapes open-source.
34. If that's the general answer to what went wrong, what about some specific projects that were ultimately unsustainable in one way or another.
35. Two that are close to my heart, Riak and RethinkDB. Both were originally built by VC-backed startups but as open source projects from the start. One of the founders of RethinkDB has a fantastic post-mortem on what he thinks went wrong which I think should be required reading for all open-source users, but roughly it's a very saturated market in both finance terms and on mind-share. Whatever your feelings on MySQL and Postgres, they have set a high bar for what is available for free.
36. A big player in the Android open-source world is Cyanogen and their former OS distribution CyanogenMod. In 2016, after a big management shakeup, they decided the benefits of remaining open were simply not making up for the huge costs in user support and lost potential revenue from people that would pay for the OS if it wasn't free.
37. Two notable individual cases are Why The Lucky Stiff and Mark Pilgrim. I say this not as a callout or critique of either, more of a cautionary tale of how putting the burden of a big community on one person rarely works out well for either side.
38. And finally a more general problem many projects struggle with. On one hand, drive-by contributors are often bringing important fixes and features, and can sometimes convert into regular contributor or maintainers. On the other, it often takes a lot of time and mentorship to shepherd a first contribution and doing that every time for a dozen drive-bys is a big load itself. There is no good answer, but at least be aware of the issues and know when to step back from mentoring first-time contributors if you don't have the energy.
39. So if that's things that detract from sustainability, how about things to improve it? A big part of any project is how it is run, and that can have a similarly large effect on sustainability.
40. A benevolent dictator for life is where most projects start from. The project is created by one person, and even as more contributors and maintainers join, they remain in control of overall direction. This can work well for a young project, but having to run all decisions through one person can quickly become a bottleneck which leads to frustration.
41. Some projects that originally started with a BDFL have slowly fragmented into a small number of BDFL- like maintainers all exchanging patches with each other and the community. This helps reduce friction in some ways, but can be intimidating for a new contributor.
42. Some projects have a more rigid hierarchy for leadership decisions. This makes it easy for maintainers to work out disagreements but often brings a lot of social overhead.
43. Others use a federated council approach which each group having final say over their area of expertise.
44. Sometimes when a contributor base is stable enough over time, you can forgo most of the rules and run purely on rough consensus. This can feel great for long-time contributors but can easily result in hidden power structures that are incomprehensible to new folks.
45. And from there are many other models.
46. As with the project types, no project is only one of these. Each project pulls together a (hopefully) coherent set of these ideas and does their best to tweak as needed for their specific circumstances and people
47. Conway's Law states that the output of any team is going to eventually look a lot like the org chart of that team. This is intended to be a bit tongue-in-cheek but is still remarkably accurate. A choice of leadership and decision making structure affects not just the humans, but the software those humans produce.
48. Unfortunately it can sometimes be hard to tell which direction to move in to improve sustainability. For some projects, more structure will help unload overburdened maintainers, but in others the increase in bureaucracy and paperwork will push them further into burnout. When playing with governance structures always make sure to listen to the people it impacts and be ready to change gears if you got it wrong.
49. I mentioned before that open licensing is an important component of open source. It's not directly related to the governance model used, but one often influences the other. A good license ensures that even if all the contributors from a project move on, at least the source code will remain available. But what about cases where it can be more than just a backstop?
50. While the full intricacies of licensing are best left to the other talk I mentioned before, the GPL family does have a goal of sustainability in its own way.
51. To start off, the GPL doesn't care about money. We'll talk about dual licensing later, but the promises of the GPL are around code access. Exactly how it works depends on which flavor you use, but overall the idea is that changes to the code should be made available to users, which means they are also available to you as the maintainer in most cases.
52. One failure mode of big projects using permissive licenses is that someone can fork your code, make some kind of change to it, keep that change to themselves, and then turn around and sell the whole thing. This could, in theory, pull resources away from your project to their fork. So if you make your code available under a copyleft license, you can head off this whole cycle, again, in theory.
53. But really how often has this happened? It's definitely not zero, big projects like the Linux kernel continually have issues with proprietary extensions causing friction for both users and maintainers. But it's by far the exception. My personal view is that copyleft causes more problems than it solves, but perhaps your project is different.
54. But okay, really in the end maintaining open source projects is about time, and time is money.
55. The simplest funding model is to not have one, to be a volunteer project. And those are great, some people enjoy coding for its own sake, or want to build up their professional portfolio, or just really need a problem fixed. But always remember that volunteering your time is an act of altruism. I speak from personal experience that it's easy to end up stuck in that cycle of guilt and bitterness I mentioned before until you never want to see another computer again.
56. One step up is working on open source during your normal work hours. This might be a project that belongs to the company that you are improving, or contributing a bug fix back to a project you use. This is a great option when you can find it, but many employers don't yet see why they should pay someone to give a product away for free.
57. And then next step up from just one person using company time is everyone using company time. This isn't a long-term solution so you'll need to explore other options to eventually repay your investors, but VC funding can be a way to get a new or nascent project off the ground.
58. So now we enter the realm of projects that are attempting to be self-sustaining, to bring in enough income to support at least some of the time spent on it. By far the most popular method is to sell support for your project. This can be very hit-or-miss though, many small companies can't afford anything that takes away from their launch runway, and big companies often think "we'll just support it ourselves". Examples include MySQL and Java.
59. Sometimes the project itself or the company that stewards the project aren't set up to offer commercial services, so maintainers set up smaller commercial entities specifically to sell support services or similar stuff, but with the goal still being to sustainably fund their work. This is the dominant model in the Postgres community.
60. After selling support, the most common models are all based around having an "open core" and selling related closed-source software on top of that to add additional functionality that caters to some specific use case, often things that are only needed for large enterprise customers.
61. Sometimes there are clear places to add these paid addons, like SAML login support or cross-team reporting. But other times it can be hard to find ways to sell these features without making life difficult for your open-source users. This has been popular in the DevOps space, Chef, Puppet, Salt, and Ansible all offer commercial add-ons.
62. Dual licensing is a special case where you just sell the same software they could get for free, but under difference license terms. The common case is a project available under a copyleft license, a company wants to modify it without sharing the patches so they purchase a license that doesn't include the copyleft restrictions. Not very common these days but still an option. Qt is probably the biggest example still using this model.
63. Similar to selling support contracts is selling professional services. This is usually some mix of consulting to help them best use or deploy the project, customization to make any custom plugins or modifications they need, or running a hosted version if the project has a server component. Examples are too numerous to count, but Pivotal Software is a big one, providing services around CloudFoundry among others.
64. Grants have been a mainstay of the academic world since time immemorial, and we've slowly seen more available for software projects. Like with other kinds of grants, you write a list of things you want to do, why they're important, and how long you think it will take, and use that to come up with a dollar amount to ask for.
65. I'm sure the PSF wasn't the first to offer open-source grants but they were the first I encountered. A lot of other groups in the Python ecosystem have taken up this mantle as well, offering project-specific grants.
66. The MOSS grants program, and Mozilla grants in general, are another example. Rather than being ecosystem-specific, they aim a bit more broadly for projects that help with Mozilla's core mission to improve the open web.
67. And one step yet broader are places like Google's open source programs office, with a mandate to support projects used by Google, a very wide net.
68. A big problem with grants, like with VC funding or any other non-renewable income stream is it is highly dependent on how good your time estimates were. If it turns out you missed some complex requirement and that takes you an extra 6 weeks, that might end up coming out of your own pocket. And we all know that engineers are super great at estimation.
69. And so we reach the option most people go with. "If you like this, how about sending me a few bucks on paypal". This is so common I want to dive into some specifics here.
70. The simplest solution is a straight up tip jar. This is easy to set up, but rarely very useful. It can be heartening to get $10 every now and then, but if you aren't already okay with working for free, I doubt those random rewards will help you keep away the burn out very much. As a maintainer, I would much rather a nice thank-you email than $5 from one person.
71. A slightly more stable solution is some kind of recurring donation, allowing you to plan out how much income you'll have to work with and allocating time appropriately. Patreon currently mostly owns this market, and is very easy to get set up with, but they haven't done much to make in-roads with the software world.
72. And both one-time and recurring donations often share the same problem, friends all paying each other while the companies deriving the economic benefit from the projects do nothing. I can only speak for myself, but I don't want individual users to ever feel like they owe me for my code. What I want most of the time is for some of the places making money with things I write to let those resources get back to me in some form so we can both benefit and continue improving things.
73. But okay, some other donation options. I did actually run a Kickstarter to fund some open-source development and it was successful, but it was very stressful and I wouldn't really recommend it. In the same way as I mentioned with grants, I ended up mis- estimating my timeline by about 3 months, but the money you have is what it is.
74. Some mid-size projects looking to tempt donations from companies rather than people look to get non- profit status so those donations can be tax deductible. Creating a non-profit from scratch is rather hard, so often they join existing umbrella groups that can accept donations and distribute them out to projects.
75. I especially want to highlight the great work of the Software Freedom Conservancy, which has helped dozens of projects set up this arrangement. If you're interested in being able to accept donations like this, I highly recommend talking to them.
76. Outside of the simple logistical issues, a common problem with donations is working out how to divide them if there is more than one maintainer. Sometimes this can be sidestepped by using donations to pay for project-level stuff (hosting, swag, whatnot) but it can lead to hurt feelings and perverse incentives very easily.
77. While money makes the world go 'round, many donations take the form of discounted service or free credits. For example Rackspace has graciously hosted part of the CI infrastructure for my Chef cookbooks for many years by providing a discounted Rackspace Cloud account. It can often be easier to ask for a company to donate these kinds of resources rather than cash.
78. But okay, enough about donations, on to some less common but interesting funding strategies. Various projects have tried funding themselves through book sales, notably the Lua language. This can often be a difficult line to walk between making the free documentation good enough, but still having something left to sell. More often the book content is made available for free on the website but physical copies are available for purchase, though this will bring in much less revenue.
79. Selling ad space can theoretically be a lucrative option, ad sales basically run the world these days, right? Setting up google ads is very easy, but also doesn't usually bring in much revenue. Negotiating your own ad deals gives a lot more control over the kinds of ads you get, but is also time-consuming. And all of this might be for naught if your highly-technical audience all immediately blocks the ads anyway.
80. Bug bounties are most common in the infosec world as a tool to try and incentivize people away from selling attacks to the bad guys, but they do sometimes pop up for normal feature requests and bugs. Outside of infosec it's still too rare to be a sustainable funding source, but maybe in the future this could improve.
81. Worker co-ops are a deep rabbit hole, but we're starting to see a few based around open source development like Snow Drift and Tidelift. Time will tell if they are successful.
82. And so we look out to the future. With science, medicine, and the arts we have many national and international organizations the provide public funding for things like basic research, projects that won't reap commercial rewards directly, but form the basis for future work. This sounds a lot like many open source projects, so maybe one day we could see a National Institute of Software.
83. And if I can dream really big, universal basic income would allow people to spend time building things they think are important even if the market doesn't directly value it.
84. So hopefully by now I've convinced that sustainable software is important and you've got some ideas brewing for how we can all move in that direction, but in the end, what do we do now?
85. As I stressed before, the enemy of open source is most often burnout, not technical failings. Practice empathy when both making and using open source and it can help a lot.
86. If you make money using open source, which is almost every company these days, be on the look out for ways you can help the projects you depend on.
87. And most of all, always be ready for changes. Maybe the only maintainer on a library you use will have a child and decide that's more important, maybe you'll get a new job and realize that utility you made just isn't fun to work on anymore, or maybe something totally unexpected will change how open source works. We've all only been doing this for a few decades, we still have a lot to learn about how to build software the best way.
88. Thank you very much.
89. There are too many people I want to thank for their advice and wisdom over the years about how to run sustainable projects, but I would like to extend an extra special thanks to Nadia, Russell, and Eric as true luminaries. I highly recommend all of their writings if you're interested in diving deeper.
90. Any questions?