Kubic - openSUSEs Container Starship

Containers, Containers, Containers - The world seems to be going container crazy. openSUSE has a strong heritage with containers, being one of the first distributions to support lxc, but as time has moved on it's become more apparent that traditional, general purpose, operating systems provide as many problems for running containers as they solve.

Container admins need an operating system platform that can provide their container runtime (Docker, Cri-O, etc) and their container orchestration tooling (Kubernetes, etc), and then generally get out of their way, so they can focus on the containers and their contents.

Requirements include absolutely fully automatic updates,while updating far more often than a traditional enterprise distribution (to get the latest from this fast moving ecosystem), all the while never breaking, or at least automatically rolling back to a working state if it does.

Impossible? Some say so, CoreOS and Red Hat have taken the route of re-inventing wheels with their Container Linux & Red Hat Atomic systems.

openSUSE takes a different route. This talk introduces Kubic, details how it is the perfect operating system for running container workloads, explore how it leverages our existing expertise in rolling releases, btrfs, snapshots, and rollbacks, and explains how it forms the base of SUSE's commercial Container as a Service Platform.

Transcript

1. Richard Brown
   openSUSE Chairman
   rbrown@opensuse.org
   Kubic Project
   openSUSE’s Container Starship, going where no
   distribution has gone before
   

   View Slide

2. Topics
   ●
   What is the Kubic Project?
   ●
   The Problem with Traditional Distributions
   ●
   Transactional Updates In Detail
   ●
   MicroOS – *SUSE enters the Atomic-Era
   ●
   Cool Things you can do with MicroOS Today
   ●
   Just the Beginning – Kubic as a Container Service Platform
   

   View Slide

3. Kubic Project
   ●
   Founded in May 2017
   ●
   Sub-Project of the openSUSE Project
   ●
   Focused on Container Technologies incl:
   – MicroOS, Tumbleweed-based Cluster Host OS
   – Velum, Kubernetes Cluster Bootstrapper & MicroOS Cluster
   Dashboard
   ●
   Upstream of SUSE Container as a Service Platform (CaaSP)
   

   View Slide

4. Why?
   ●
   Application delivery is changing
   – Containers
   – Flatpak/snaps/AppImage
   

   View Slide

5. Why?
   ●
   Micro-Services are fun
   – Why bother with complex installations when
   someone else's solution is a docker pull away?
   – VMs are comparably resource and management
   heavy
   – Lots of positive momentum behind containers,
   but tools are lacking
   

   View Slide

6. Why?
   ●
   Containers are disruptive
   – Diferent requirements from the OS
   – Faster delivery expectations
   – Portability increases support demands
   – Containerized clusters want to run at scale
   

   View Slide

7. Why?
   ●
   Setting up a container host should be easy
   – No point saving time with containers just to
   waste it administering a machine
   – Containers should take care of themselves, why
   can’t the host system?
   

   View Slide

8. The Problem with
   Traditional Distributions
   

   View Slide

9. “I NEVER want to touch a running system”
   - Every SysAdmin, ever
   

   View Slide

10. Upgrading a Running System Is DANGEROUS
    ●
    Services are running
    ●
    Users are doing things
    ●
    Sofware changes things (sometimes on purpose!)
    ●
    Not all packages are packaged properly (sorry)
    

    View Slide

11. Reality does not make things easy
    Rolling Releases bring larger challenges
    ●
    Intrusive Updates (SysV init → systemd)
    ●
    Major version updates of complex stacks (GNOME, KDE)
    ●
    What should I do if the update breaks my system?
    

    View Slide

12. Reality does not make things easy
    Critical Mission Systems are even worse
    ●
    Update should not interrupt services
    Service interruption more expensive than regular reboots
    ●
    Updates need to be fully applied perfectly, or no changes made
    System state should never be undefined/questionable
    RPM post-installation scripting can lead to an undefined state
    

    View Slide

13. Traditional Snapshots – An Imperfect Solution
    curr.
    root ro
    ro ro ro
    

    View Slide

14. Traditional Snapshots – An Imperfect Solution
    curr.
    root ro
    ro ro ro ro
    ro-Clone 1
    

    View Slide

15. Traditional Snapshots – An Imperfect Solution
    curr.
    root ro
    ro ro ro ro
    ro-Clone 1
    zypper up
    2
    

    View Slide

16. Transactional Updates
    

    View Slide

17. What is a Transactional Update?
    An Update that:
    ●
    Is Atomic
    – Either fully applied, or not at all
    – Update does not influence the running system
    ●
    Can be rolled back
    – A failed or incompatible update can be quickly discarded to
    restore the previous system conditions
    

    View Slide

18. Other Solutions
    ●
    Several Diferent Partitions
    – Waste of Disk Space
    ●
    Special package formats (rpm-ostree, snap, etc)
    – Need to learn something new
    – Can’t re-use existing packages
    → And you need to redesign systems, tools & company
    policies
    

    View Slide

19. All you need
    ●
    btrfs root filesystem with snapshots/rollback enabled
    ●
    snapper
    ●
    zypper (or package manager of your choice)
    ●
    btrfsprogs
    

    View Slide

20. Traditional Snapshots
    curr.
    root ro
    ro ro ro
    

    View Slide

21. Transactional Updates
    ro
    ro ro ro
    curr.
    root
    

    View Slide

22. ro
    Transactional Updates
    ro
    ro ro ro
    curr.
    root
    1
    ro-Clone
    

    View Slide

23. Transactional Updates
    ro
    ro ro ro
    curr.
    root rw
    1
    ro-Clone
    2
    Change rw
    

    View Slide

24. Transactional Updates
    ro
    ro ro ro
    curr.
    root rw
    1
    ro-Clone
    2
    Change rw
    3
    zypper up
    

    View Slide

25. Transactional Updates
    ro
    ro ro ro
    curr.
    root ro
    1
    ro-Clone
    2
    Change rw
    3
    zypper up
    Change ro 4
    

    View Slide

26. Transactional Updates
    ro
    ro ro ro
    curr.
    root
    New
    root
    1
    ro-Clone
    2
    Change rw
    3
    zypper up
    Change ro
    5
    “Rollforward” btrfs subvol set-default
    4
    

    View Slide

27. Transactional Updates – Afer Reboot
    ro
    ro ro ro
    curr.
    root
    ro
    

    View Slide

28. Transactional Updates – without Reboot
    ro
    ro ro ro
    curr.
    root
    ro ro
    New
    root
    

    View Slide

29. How to do transactional updates
    ●
    transactional-update [up|dup|patch]
    ●
    transactional-update pkg [install|remove|update] $PKG1..$PKGN
    – Install anything you’d like
    ●
    BONUS: transactional-update shell
    

    View Slide

30. Transactional Updates – Rollback
    ro
    ro ro ro
    curr.
    root
    New
    root
    1
    Rollback btrfs subvol set-default
    

    View Slide

31. Advantages
    ●
    btrfs:
    – Snapshots with CoW are space eficient
    – Deduplication possible
    – btrfs send/receive – update with golden images
    ●
    rpm:
    – Existing packages can be used
    – No need to learn new tools, policies remain valid
    

    View Slide

32. Advantages, cont.
    ●
    Normal boot time
    ●
    Very quick rollback
    

    View Slide

33. Disadvantages
    ●
    Currently limited to btrfs only
    ●
    Special requirements of RPMs
    – Care needs to be made that files are in the right place
    – User data & configuration should be converted on use, not
    during package install
    ●
    Reboot needed to activate changes
    

    View Slide

34. Dealing with Configuration Files
    ●
    rootfs snapshots always contain package-provided files for /etc
    ●
    overlayfs used to transparently store/overlay user-provided
    files for /etc
    

    View Slide

35. MicroOS
    

    View Slide

36. openSUSE MicroOS
    ●
    openSUSE Tumbleweed-derivative utilising transactional-
    updates & read-only rootfs
    ●
    Docker installed and enabled by default (for now)
    ●
    Download/Install from
    https://sofware.opensuse.org/distributions/tumbleweed
    

    View Slide

37. Diferences from Tumbleweed
    ●
    Update using transactional-update dup
    ●
    Install/Remove packages using transactional-update pkg
    install|remove
    

    View Slide

38. MicroOS Today
    

    View Slide

39. MicroOS Example Use Cases
    ●
    Containers
    – docker
    – runc
    – lxc
    – cri-o
    

    View Slide

40. MicroOS Example Use Cases
    

    View Slide

41. MicroOS Example Use Cases
    ●
    Rolling Server – Use the latest of everything with no risk
    – Web (esp. NodeJS, Ruby, etc)
    – Virtualisation host
    – Media Server
    

    View Slide

42. MicroOS on Raspberry Pi3
    

    View Slide

43. Kubic Desktop
    

    View Slide

44. Kubic Desktop
    https://rootco.de/2017-11-16-hackweek-2017-conclusion/
    

    View Slide

45. Kubic Container Service
    Platform
    

    View Slide

46. Why?
    ●
    Setting up a container host should be easy
    – No point saving time with containers just to
    waste it administering a machine
    – Containers should take care of themselves, why
    can’t the host system?
    

    View Slide

47. What’s Next?
    ●
    Setting up a container cluster should be easy
    – No point saving time with containers just to
    waste it administering a cluster of machines
    – Containers should take care of themselves, why
    can’t the cluster?
    – Cluster admins should be able to seemlessly
    bootstrap, monitor and upgrade whole clusters
    at once
    

    View Slide

48. Kubic Container Service Architecture
    Cluster Nodes
    (etcd + Kubernetes Master)
    Log Server
    Administration Node
    

    View Slide

49. Current Status
    ●
    “Unconfigured Cluster Node” - fully supported system-role in
    Kubic
    ●
    Can be used to setup a Kubernetes Cluster Manually
    ●
    https://kubernetes.io/docs/getting-started-guides/scratch/
    #bootstrapping-the-cluster
    

    View Slide

50. Current Status
    ●
    Velum & related “Container-as-a-Service Platform” containers are not
    working yet
    ●
    Contributions welcome
    ●
    https://github.com/kubic-project/container-images
    ●
    Alternatives being investigated
    – OpenShif
    – kubeadm
    

    View Slide

51. Join Us at www.opensuse.org
    

    View Slide

52. License
    This slide deck is licensed under the Creative Commons Attribution-ShareAlike 4.0 International license.
    It can be shared and adapted for any purpose (even commercially) as long as Attribution is given and any
    derivative work is distributed under the same license.
    Details can be found at https://creativecommons.org/licenses/by-sa/4.0/
    General Disclaimer
    This document is not to be construed as a promise by any participating organisation to develop, deliver, or
    market a product. It is not a commitment to deliver any material, code, or functionality, and should not be
    relied upon in making purchasing decisions. openSUSE makes no representations or warranties with respect
    to the contents of this document, and specifically disclaims any express or implied warranties of
    merchantability or fitness for any particular purpose. The development, release, and timing of features or
    functionality described for openSUSE products remains at the sole discretion of openSUSE. Further,
    openSUSE reserves the right to revise this document and to make changes to its content, at any time,
    without obligation to notify any person or entity of such revisions or changes. All openSUSE marks
    referenced in this presentation are trademarks or registered trademarks of SUSE LLC, in the United States
    and other countries. All third-party trademarks are the property of their respective owners.
    Credits
    Template
    Richard Brown
    rbrown@opensuse.org
    Design & Inspiration
    openSUSE Design Team
    http://opensuse.github.io/branding-
    guidelines/
    

    View Slide