My talk from #oscon 2015. Video at https://www.youtube.com/watch?v=Zpnncakrelk
Leaping the chasm from
proprietary to open:
A survivor’s guide
Open source: In the beginning
• In the beginning, it was all open: software shipped with its source
• IBM led the move to proprietary enterprise software with their
move to an “object-only” model in 1983
• The rise of the personal computer in the 1980s led to the broader
rise of proprietary software, with the tone set by Bill Gates’
infamous 1976 letter, “An Open Letter to Hobbyists”
• By the mid-1990s, software was commercial, proprietary — and
entirely ossiﬁed, with many domains (e.g., languages, operating
systems, databases) considered “done”
Open source: The internet cometh
• The rise of the internet (and especially, Perl and Apache) brought
with it an ethos of sharing source to collaboratively develop it
• Free-and-open dominated among de novo non-commercial
systems software: languages (Perl, Python) databases (MySQL,
Postgres), web servers (Apache), OSs (Linux, BSD)
• Proprietary software remained, but with cost often driven to $0
• The early internet can be seen as a growing chasm between free-
but-proprietary (e.g. Java) and free-and-open (e.g. Perl)
Open source in the 2000s
• In the Dot Com Bust, economics dictated that open source (and
commodity hardware!) be used wherever possible
• Companies would occasionally contribute back the changes they
made to the software they used, but only rarely did they open
source software that they themselves invented
• Counter-examples were in infrastructural software domains in
which open source had become a hard requirement...
Leaping the chasm: OpenSolaris
• Sun — alone among the proprietary Unix vendors — chose the
arduous path of open sourcing its OS to assure its vitality
• Solaris was open sourced in June 2005 — and was showcased
by Sun engineers at OSCON in August (a decade ago!)
• To balance community needs with IHV wants, Sun chose a weak
copy-left (CDDL) that amounted to a cleaned-up MPL
• After being open sourced, Solaris was developed entirely in the
open, becoming perhaps the largest single software project to
completely transition from proprietary to open
Oracle pulls OpenSolaris back into the abyss
• After Oracle acquired Sun, it was clear that there was no interest
in open source — and zero appreciation of its social contract
• On Friday, August 13th an internal memo closed OpenSolaris:
We will distribute updates to approved CDDL or other open source-licensed code following
full releases of our enterprise Solaris operating system. In this manner, new technology
innovations will show up in our releases before anywhere else. We will no longer
distribute source code for the entirety of the Solaris operating system in real-time
while it is developed, on a nightly basis.
• Oracle’s depraved (and cowardly!) act foreshadowed even more
diabolical behavior: the assertion that APIs can be copyrighted!
The rise of illumos
• Oracle couldn’t actually close OpenSolaris; they could only fork it
• A new community, illumos, formed from the ashes of OpenSolaris,
aided by (but much larger than!) the Solaris diaspora
• Five years later, illumos is thriving with an established track
record for innovation (e.g., ZFS, DTrace, containers), new blood,
and multiple distributions (e.g., OmniOS, Joyent’s SmartOS)
• See http://illumos.org, http://omnios.omniti.com, http://smartos.org
Open sourcing shrink-wrapped software
• The OpenSolaris experience can be viewed as an object lesson
in open sourcing traditional, licensed, shrink-wrapped software
• Leaping the chasm from proprietary to open is (ultimately) a
business decision, and you will need to make the business case
• You need to move past fear to opportunity: think of open source
not as endangering revenue but rather as driving adoption
• Reminder: those that will operate and self-support your software
weren’t going to pay you anything anyway!
• And no, your competitors aren’t going to steal it
Open sourcing shrink-wrapped software, cont.
• Don’t be forkaphobic; the easier it is to fork your software, the
harder it is to fork the community!
• Don’t fetishize governance and/or process — for most open
source projects, core team + consensus is just ﬁne (there are
exceptions to this; stay tuned!)
• Prefer permissive (or weak copy-left) licenses over strong copy-
left ones — license incompatibility impedes adoption
• Elevate your people! Organizations don’t innovate, people do —
and with open source, you can provide that transparency!
Open source and the cloud
• With the rise of cloud computing in the late-2000s, open source
became ever more entrenched, with more professionally-written
software being born in the open
• Trend was both accelerated and reﬂected by the rise of GitHub,
which greatly reduced the cost of open sourcing software
• Large companies began open sourcing important de novo
software e.g. Yahoo’s Hadoop, Google’s V8, Facebook’s Hive
• This has accelerated so much that it is now an engineering
challenge to determine which open source project to use!
SaaS: Proprietary software strikes back!
• But the cloud also represented a new kind of proprietary software:
proprietary software made available as a service
• Lock-in from services — especially platforms-as-a-service — can
be even more acute than that from shrink-wrapped software!
• Adding insult to injury, these proprietary services are often
(quietly) built on open source components — without so much as
patches being offered back!
• The new vector for proprietary software reveals GPL’s copy-left to
be an obviated notion — and that coercion is the wrong approach
to encourage contribution
Leaping the chasm: SmartDataCenter
• At Joyent, our core technologies — SmartOS and node.js — have
always been open source, but the distributed systems and
services that we built and operated based on them were not
• First among these is SmartDataCenter, our SmartOS- and
container-based cloud orchestration software that we run
ourselves as the Joyent Public Cloud — and sell to others
• Open sourcing software that runs your service is technically hard;
it is easy for implicit operational dependencies to creep in
• Risk is higher than with open sourcing software you merely use,
as code (and histories!) must be carefully scrubbed of secrets
Leaping the chasm: Manta
• But wait, it gets worse: proprietary systems and services can
easily build dependencies on other proprietary services
• We had built a signiﬁcant dependency on a new, container-based
storage service that we had built, Manta
• We couldn’t meaningfully open source SmartDataCenter without
also open sourcing Manta, complicating the effort signiﬁcantly
• If it needs to be said: it is much easier to be born open than to
become open — all software should be built assuming that it is
Leaping the chasm: SDC + Manta
• For many years, we had wanted to open source SDC and Manta
• It took much longer than one would like, in part because open
sourcing proprietary software is so easy to veto or (worse!) defer
• In making the case for open sourcing SDC + Manta, we made the
familiar arguments for open sourcing shrink-wrapped software:
open source as lead generation, as early evaluation program, etc.
• But two other arguments ultimately carried the day…
Infrastructure software must be open source!
• For infrastructure software (e.g., operating systems, databases,
runtimes), open source is now a constraint on the problem
• In this regard, open source has unequivocally won
• This trend is continuing upstack — software used as foundation
and/or component is overwhelmingly biased to be open in the
limit: increased adoption feeds network effects
• Docker is a particularly strong testament to the transformative
power of open sourcing infrastructure software...
Open source is our best vector for hiring
• Joyent — like most companies — struggles to ﬁnd the right talent
• University hiring has reasonable ﬁdelity, but is increasingly
expensive/competitive — and can result in a monoculture
• Open source communities self-identify and self-select, attracting
those who are drawn to the mission
• Participation in our open source communities is a much better
analogue for the work that we actually do!
• Our open source communities are our highest ﬁdelity, lowest cost
vector for hiring — it’s our farm system!
Leaping the chasm: SDC + Manta!
• On November 6, 2014, we open sourced SDC and Manta!
• We moved to become an all-open company; the only proprietary
software is that which contains true secrets (e.g., billing software)
• e.g., Triton is our new service for running Docker containers
directly on the metal — and was born open
• We have already found that our technologies are being used in
places and by people where it wouldn’t have been otherwise
• And our ﬁrst hire because of open source happened mere weeks
after we open sourced it! (And others have followed…)
SDC + Manta: Licensing!
• Based on our previous experiences, we:
• Love permissive licensing, but hate license incompatibilities
• Love contributors, but hate contributor license agreements
• Based on this, we selected the Mozilla Public License 2.0
• For a commercial entity, the MPL has some nice CYA attributes:
e.g., explicit warranting of original work, patent troll protection
• The MPL is explicitly compatible with other open source licenses
• The MPL is a great license that merits broader consideration!
SDC + Manta: Governance!
• Most open source projects can be entirely consensus driven —
the patch rate is low enough and the number of contributors small
enough that everyone can effectively look at everything
• If projects become more popular, the model can be extended with
a core team that can review work and guide contributors
• Most projects — including (for the moment) SDC and Manta —
can stop there: GitHub is all the governance one needs
• So while there is generally no need for elections or foundations or
any other such madness, there are, of course, exceptions...
Open source: When to consider a foundation
• Some (very few!) open source projects become so popular that
multiple, new commercial entities are created around them
• For these projects, no corporate steward is sufﬁciently neutral:
commercial tensions will threaten to rip a project apart
• In these rare cases, projects may be better served in a foundation
• This can be very hard to internalize — especially to a corporate
steward that believes that they are acting (and have acted) in the
best interests of the community!
• Yes, we learned this the hard way with node.js!
The limitations of foundations
• Foundations should be used and created only sparingly!
• Foundations pose more questions than they answer:
• How are board seats determined? (Pay-to-play? Community
elections? Code contributions?)
• How are conﬂicts resolved? (By consensus? By voting?)
• What are the principles that guide the work? (Does the mission
of the foundation go beyond mere self-preservation?)
• Foundations are only better than the alternatives
Leaping the chasm
• While there is still a chasm between proprietary and open source
software, open source is broadly winning — especially in
• It is much easier to open source when a project is still young —
and being open may take the project in interesting directions
• For commercial entities, challenges have shifted from “when do I
open source?” to “when does this need to be in a foundation?”
• Participating in and leading open source efforts as a commercial
entity is an ongoing education; check back at OSCON 2025!
• SmartDataCenter: https://github.com/joyent/sdc
• Manta: https://github.com/joyent/manta
• Thanks to the many Joyent engineers who made open source
SDC and Manta happen, especially @trentmick, @rmustacc,
@dapsays, @jmclulow, Keith Wesolowski, @pfmooney,
@joshwilsdon, Jerry Jelinek, @kusor, @notmatt and @orlandov
• And did I mention that we’re hiring? ;) (Remote-friendly!)