Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Leaping the chasm from proprietary to open: A survivor's guide

Leaping the chasm from proprietary to open: A survivor's guide

My talk from #oscon 2015. Video at https://www.youtube.com/watch?v=Zpnncakrelk

Bryan Cantrill

July 24, 2015
Tweet

More Decks by Bryan Cantrill

Other Decks in Technology

Transcript

  1. Leaping the chasm from
    proprietary to open:
    A survivor’s guide
    CTO
    Bryan Cantrill
    @bcantrill

    View Slide

  2. Open source: In the beginning
    • In the beginning, it was all open: software shipped with its source
    • IBM led the move to proprietary enterprise software with their
    move to an “object-only” model in 1983
    • The rise of the personal computer in the 1980s led to the broader
    rise of proprietary software, with the tone set by Bill Gates’
    infamous 1976 letter, “An Open Letter to Hobbyists”
    • By the mid-1990s, software was commercial, proprietary — and
    entirely ossified, with many domains (e.g., languages, operating
    systems, databases) considered “done”

    View Slide

  3. Open source: The internet cometh
    • The rise of the internet (and especially, Perl and Apache) brought
    with it an ethos of sharing source to collaboratively develop it
    • Free-and-open dominated among de novo non-commercial
    systems software: languages (Perl, Python) databases (MySQL,
    Postgres), web servers (Apache), OSs (Linux, BSD)
    • Proprietary software remained, but with cost often driven to $0
    • The early internet can be seen as a growing chasm between free-
    but-proprietary (e.g. Java) and free-and-open (e.g. Perl)

    View Slide

  4. Open source in the 2000s
    • In the Dot Com Bust, economics dictated that open source (and
    commodity hardware!) be used wherever possible
    • Companies would occasionally contribute back the changes they
    made to the software they used, but only rarely did they open
    source software that they themselves invented
    • Counter-examples were in infrastructural software domains in
    which open source had become a hard requirement...

    View Slide

  5. Leaping the chasm: OpenSolaris
    • Sun — alone among the proprietary Unix vendors — chose the
    arduous path of open sourcing its OS to assure its vitality
    • Solaris was open sourced in June 2005 — and was showcased
    by Sun engineers at OSCON in August (a decade ago!)
    • To balance community needs with IHV wants, Sun chose a weak
    copy-left (CDDL) that amounted to a cleaned-up MPL
    • After being open sourced, Solaris was developed entirely in the
    open, becoming perhaps the largest single software project to
    completely transition from proprietary to open

    View Slide

  6. Oracle pulls OpenSolaris back into the abyss
    • After Oracle acquired Sun, it was clear that there was no interest
    in open source — and zero appreciation of its social contract
    • On Friday, August 13th an internal memo closed OpenSolaris:
    We will distribute updates to approved CDDL or other open source-licensed code following
    full releases of our enterprise Solaris operating system. In this manner, new technology
    innovations will show up in our releases before anywhere else. We will no longer
    distribute source code for the entirety of the Solaris operating system in real-time
    while it is developed, on a nightly basis.
    • Oracle’s depraved (and cowardly!) act foreshadowed even more
    diabolical behavior: the assertion that APIs can be copyrighted!

    View Slide

  7. The rise of illumos
    • Oracle couldn’t actually close OpenSolaris; they could only fork it
    • A new community, illumos, formed from the ashes of OpenSolaris,
    aided by (but much larger than!) the Solaris diaspora
    • Five years later, illumos is thriving with an established track
    record for innovation (e.g., ZFS, DTrace, containers), new blood,
    and multiple distributions (e.g., OmniOS, Joyent’s SmartOS)
    • See http://illumos.org, http://omnios.omniti.com, http://smartos.org

    View Slide

  8. Open sourcing shrink-wrapped software
    • The OpenSolaris experience can be viewed as an object lesson
    in open sourcing traditional, licensed, shrink-wrapped software
    • Leaping the chasm from proprietary to open is (ultimately) a
    business decision, and you will need to make the business case
    • You need to move past fear to opportunity: think of open source
    not as endangering revenue but rather as driving adoption
    • Reminder: those that will operate and self-support your software
    weren’t going to pay you anything anyway!
    • And no, your competitors aren’t going to steal it

    View Slide

  9. Open sourcing shrink-wrapped software, cont.
    • Don’t be forkaphobic; the easier it is to fork your software, the
    harder it is to fork the community!
    • Don’t fetishize governance and/or process — for most open
    source projects, core team + consensus is just fine (there are
    exceptions to this; stay tuned!)
    • Prefer permissive (or weak copy-left) licenses over strong copy-
    left ones — license incompatibility impedes adoption
    • Elevate your people! Organizations don’t innovate, people do —
    and with open source, you can provide that transparency!

    View Slide

  10. Open source and the cloud
    • With the rise of cloud computing in the late-2000s, open source
    became ever more entrenched, with more professionally-written
    software being born in the open
    • Trend was both accelerated and reflected by the rise of GitHub,
    which greatly reduced the cost of open sourcing software
    • Large companies began open sourcing important de novo
    software e.g. Yahoo’s Hadoop, Google’s V8, Facebook’s Hive
    • This has accelerated so much that it is now an engineering
    challenge to determine which open source project to use!

    View Slide

  11. SaaS: Proprietary software strikes back!
    • But the cloud also represented a new kind of proprietary software:
    proprietary software made available as a service
    • Lock-in from services — especially platforms-as-a-service — can
    be even more acute than that from shrink-wrapped software!
    • Adding insult to injury, these proprietary services are often
    (quietly) built on open source components — without so much as
    patches being offered back!
    • The new vector for proprietary software reveals GPL’s copy-left to
    be an obviated notion — and that coercion is the wrong approach
    to encourage contribution

    View Slide

  12. Leaping the chasm: SmartDataCenter
    • At Joyent, our core technologies — SmartOS and node.js — have
    always been open source, but the distributed systems and
    services that we built and operated based on them were not
    • First among these is SmartDataCenter, our SmartOS- and
    container-based cloud orchestration software that we run
    ourselves as the Joyent Public Cloud — and sell to others
    • Open sourcing software that runs your service is technically hard;
    it is easy for implicit operational dependencies to creep in
    • Risk is higher than with open sourcing software you merely use,
    as code (and histories!) must be carefully scrubbed of secrets

    View Slide

  13. Leaping the chasm: Manta
    • But wait, it gets worse: proprietary systems and services can
    easily build dependencies on other proprietary services
    • We had built a significant dependency on a new, container-based
    storage service that we had built, Manta
    • We couldn’t meaningfully open source SmartDataCenter without
    also open sourcing Manta, complicating the effort significantly
    • If it needs to be said: it is much easier to be born open than to
    become open — all software should be built assuming that it is
    already open!

    View Slide

  14. Leaping the chasm: SDC + Manta
    • For many years, we had wanted to open source SDC and Manta
    • It took much longer than one would like, in part because open
    sourcing proprietary software is so easy to veto or (worse!) defer
    • In making the case for open sourcing SDC + Manta, we made the
    familiar arguments for open sourcing shrink-wrapped software:
    open source as lead generation, as early evaluation program, etc.
    • But two other arguments ultimately carried the day…

    View Slide

  15. Infrastructure software must be open source!
    • For infrastructure software (e.g., operating systems, databases,
    runtimes), open source is now a constraint on the problem
    • In this regard, open source has unequivocally won
    • This trend is continuing upstack — software used as foundation
    and/or component is overwhelmingly biased to be open in the
    limit: increased adoption feeds network effects
    • Docker is a particularly strong testament to the transformative
    power of open sourcing infrastructure software...

    View Slide

  16. Open source is our best vector for hiring
    • Joyent — like most companies — struggles to find the right talent
    • University hiring has reasonable fidelity, but is increasingly
    expensive/competitive — and can result in a monoculture
    • Open source communities self-identify and self-select, attracting
    those who are drawn to the mission
    • Participation in our open source communities is a much better
    analogue for the work that we actually do!
    • Our open source communities are our highest fidelity, lowest cost
    vector for hiring — it’s our farm system!

    View Slide

  17. Leaping the chasm: SDC + Manta!
    • On November 6, 2014, we open sourced SDC and Manta!
    • We moved to become an all-open company; the only proprietary
    software is that which contains true secrets (e.g., billing software)
    • e.g., Triton is our new service for running Docker containers
    directly on the metal — and was born open
    • We have already found that our technologies are being used in
    places and by people where it wouldn’t have been otherwise
    • And our first hire because of open source happened mere weeks
    after we open sourced it! (And others have followed…)

    View Slide

  18. SDC + Manta: Licensing!
    • Based on our previous experiences, we:
    • Love permissive licensing, but hate license incompatibilities
    • Love contributors, but hate contributor license agreements
    • Based on this, we selected the Mozilla Public License 2.0
    • For a commercial entity, the MPL has some nice CYA attributes:
    e.g., explicit warranting of original work, patent troll protection
    • The MPL is explicitly compatible with other open source licenses
    • The MPL is a great license that merits broader consideration!

    View Slide

  19. SDC + Manta: Governance!
    • Most open source projects can be entirely consensus driven —
    the patch rate is low enough and the number of contributors small
    enough that everyone can effectively look at everything
    • If projects become more popular, the model can be extended with
    a core team that can review work and guide contributors
    • Most projects — including (for the moment) SDC and Manta —
    can stop there: GitHub is all the governance one needs
    • So while there is generally no need for elections or foundations or
    any other such madness, there are, of course, exceptions...

    View Slide

  20. Open source: When to consider a foundation
    • Some (very few!) open source projects become so popular that
    multiple, new commercial entities are created around them
    • For these projects, no corporate steward is sufficiently neutral:
    commercial tensions will threaten to rip a project apart
    • In these rare cases, projects may be better served in a foundation
    • This can be very hard to internalize — especially to a corporate
    steward that believes that they are acting (and have acted) in the
    best interests of the community!
    • Yes, we learned this the hard way with node.js!

    View Slide

  21. The limitations of foundations
    • Foundations should be used and created only sparingly!
    • Foundations pose more questions than they answer:
    • How are board seats determined? (Pay-to-play? Community
    elections? Code contributions?)
    • How are conflicts resolved? (By consensus? By voting?)
    • What are the principles that guide the work? (Does the mission
    of the foundation go beyond mere self-preservation?)
    • Foundations are only better than the alternatives

    View Slide

  22. Leaping the chasm
    • While there is still a chasm between proprietary and open source
    software, open source is broadly winning — especially in
    infrastructure software
    • It is much easier to open source when a project is still young —
    and being open may take the project in interesting directions
    • For commercial entities, challenges have shifted from “when do I
    open source?” to “when does this need to be in a foundation?”
    • Participating in and leading open source efforts as a commercial
    entity is an ongoing education; check back at OSCON 2025!

    View Slide

  23. Thank you!
    • SmartDataCenter: https://github.com/joyent/sdc
    • Manta: https://github.com/joyent/manta
    • Thanks to the many Joyent engineers who made open source
    SDC and Manta happen, especially @trentmick, @rmustacc,
    @dapsays, @jmclulow, Keith Wesolowski, @pfmooney,
    @joshwilsdon, Jerry Jelinek, @kusor, @notmatt and @orlandov
    • And did I mention that we’re hiring? ;) (Remote-friendly!)

    View Slide