Upgrade to Pro — share decks privately, control downloads, hide ads and more …

What can servicing Kubernetes tell us about the future of open source in the cloud?

What can servicing Kubernetes tell us about the future of open source in the cloud?

Enabling, integrating, releasing and contributing to open source have been the four pillars of Microsoft's approach to open source in the cloud. How can an organization like Microsoft learn from developers, customers and partners that use Kubernetes in our platform or with our products? And what does an operational consideration such as the servicing of a Kubernetes cluster tell us about the changing preferences and upcoming challenges with open source in the Enterprise? This talk was given at the "Road to FOSDEM" Meetup in Rotterdam on January 2020.

José Miguel Parrella

January 29, 2020
Tweet

More Decks by José Miguel Parrella

Other Decks in Technology

Transcript

  1. What can servicing Kubernetes
    tell us about the future of open
    source in the cloud?
    Jose Miguel Parrella
    Office of the Azure CTO,
    Microsoft
    @bureado

    View full-size slide

  2. Open source at Microsoft: a cultural change driven
    by demographics and leadership affinity
    Phase I: 2000-
    2005
    • "Shared
    Source"
    • "Accidental"
    product
    truths
    (Interix)
    Phase II: 2005-
    2010
    • CodePlex
    • "Insular"
    product
    truths (PHP
    on Windows,
    but also Linux
    on Hyper-V)
    Phase III: 2010-
    2015
    • "Trying too
    hard"
    • Microsoft
    Open
    Technologies
    • Node.js,
    TypeScript
    Phase IV: 2015-
    2020
    • Collaborative
    • Linux:
    Canonical,
    Red Hat
    • Hadoop:
    Hortonworks,
    Cloudera
    Phase V:
    Tomorrow
    • Innovative
    • Docker &
    Kubernetes
    • Rust &
    Golang
    Windows
    Azure
    Microsoft
    Azure

    View full-size slide

  3. https://dev.to/bureado/
    open-source-
    engagement-snow-
    tracks-2lkh

    View full-size slide

  4. Open
    Source
    Tools &
    artifacts
    Roadmap
    Design
    Dialogue
    Community

    View full-size slide

  5. How we are learning from Kubernetes
    • We run it as a service (AKS)
    • You can run it too (AKS Engine, Arc)
    • We brought Windows to it
    • ...and SQL Server
    • We believe in a great developer experience for it
    • We run on it
    • ...Xbox's Xcloud
    • Kubernetes is a driver of Linux usage, cloud-native development
    practices and deep community interaction for teams at Microsoft

    View full-size slide

  6. Kubernetes servicing blows my mind
    • I'm a 15+ year Linux user that grew up in Latin America:
    • I lived the CFEngine to Puppet transition
    • I embraced (deb) packages as atomic units of delivery
    • Things like Corosync, Ultramonkey, etc., were basic clustering building blocks
    • Databases and NFS were best delegated elsewhere
    • We had maintenance windows
    • In Kubernetes, a node is the servicing unit – sometimes even the
    whole cluster. (this is what modern compute and network buys!)
    • In AKS, fresh nodes are added to the pool, then a node is cordoned and
    drained. If your state and ingress are outside your cluster, you might be able
    to just deploy a new cluster and point stuff at it.

    View full-size slide

  7. Kubernetes servicing blows my mind
    • This means that Enterprise customers are now:
    • Adopting open source software at a much faster cadence than their regular
    enterprise open source vendors
    • Willing to take supply chain risks as final nodes are assembled from bits that are
    often coming from many different sources
    • Incurring the cost of replacing nodes even for trivial changes to achieve the benefits
    of immutability
    • Willing to introduce tools incompatibility as cluster API leaps forward (and in a
    DevOps world, "tools" can be anything from CD logic to operational sensors)
    • Seeing Kubernetes multiple personalities: scheduler, state driver, app plat...
    • Note: Kubernetes versions n-4, n-5 and n-6 are highly prevalent
    • Lawrence Hecht on Twitter: most popular versions are 10+ months old, "...users of
    hosted Kubernetes services are usually presented with a default, stable version at
    cluster launch...version 1.13 was still the default for GKE and AKS, whereas EKS
    provided version 1.14 by default."

    View full-size slide

  8. We all think Kubernetes is "going away" but
    it's hard to tell what it will look like in 3 years
    • Will it be defined by the application model?
    • Will it be defined by the application packaging and delivery?
    • Will it be defined by the service mesh?
    • Will it be driven by paradigms such as event-driven, dataflow, etc.?
    • Will it be driven by where the applications and data live?
    • Will it be driven by innovation in the underlying virtualization and
    container technologies?
    • Will it just be "servicefied" by a handful of service providers and
    that's it?

    View full-size slide

  9. Predictions in the wild
    • Part-time developers such as business analysts, data analysts and data
    scientists to double in five years (as a reference, there's 40M people on
    GitHub, but only 25M devs according to Evans Data...)
    • By 2024, low-code application development will be responsible for more
    than 65% of application development activity (Gartner)
    • Use of open source software from public repositories used to
    build enterprise apps to double, and...
    • By 2023, 60% of the G2000 enterprises will have created their own software
    ecosystem, while 90% will use publicly accessible community repositories to access
    reusable code components (IDC)
    • By 2023, 60% of the G2000 will have a digital developer ecosystem with thousands of
    developers; half of those enterprises will drive 20%+ of digital revenue through their
    digital ecosystem/platform (also read)

    View full-size slide

  10. Stuff I'm watching...
    • We can still do better at persisting state
    • From operators et al to File systems unfit as distributed storage backends
    • Confidential computing and enclaves
    • sgx-lkl
    • Software supply chain security and Kubernetes
    • How data flows in and out of pods, licensing, model pedigree...
    • Kubernetes operators that are bringing learnings to Linux
    • Yelp's ClusterMan, Canonical's Dqlite, HashiCorp's Nomad, Kong's Kuma,
    Puppet's Bolt, GitHub's GLB, Clockwork-Scheduler (DynamoDB)

    View full-size slide

  11. PSI
    ◦ Quantifies lost wall clock
    time due to resource
    contention
    ◦ Exported via /proc
    ◦ Works with cgroup2
    ◦ In kernel 4.20 and newer
    ◦ Full introduction of PSI in
    the kernel mailing lists
    Jose Miguel Parrella | @bureado

    View full-size slide

  12. Stuff I'll be watching during FOSDEM
    • Containers and security
    • Containers devroom
    • Storage devroom
    • Lazy distribution of container images
    • Linux memory management at scale
    • Do Linux Distributions Still Matter with Containers?
    • Demo: SGX-LKL
    • Ephemeral Environments For Developers In Kubernetes
    • Immutable deployments: the new classic way for service deployment
    • Kubernetes of Things
    • Linux Kernel Library
    • PostgreSQL on K8S at Zalando: Two years in production

    View full-size slide

  13. ...all of this isn't limited to Kubernetes!
    • It's happening with Linux...
    • ...for storage: io_uring, bcachefs
    • ...for servicing: Yocto in automotive/embedded
    • It's happening in the container ecosystem at large with software
    supply chain with things like TUF, in-toto and more...
    • It's happening in the Node.js, PyPI and Golang ecosystems for things
    like software supply chain...
    • It's happening with the explosion of innovation in open source
    databases, including multi-model DBs and licensing experiments...

    View full-size slide

  14. Debian at Microsoft
    APT is (possibly) the most prevalent package manager in the
    Microsoftecosystem
    Microsoft distributes software for Debian
    Microsoft runs Debian for top-of-rack networking in all datacenters
    Microsoft runs Debian (on Azure) for all the Skype relays
    Microsoft builds a minimal set of Debian packages for reproducibility
    Jose Miguel Parrella | @bureado

    View full-size slide

  15. [email protected]
    @bureado
    speakerdeck.com/bureado

    View full-size slide