Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Operations without the Operating System

Operations without the Operating System

Talk from the Operability conference in London about fundamental problems with the evolution of infrastructure, and how new operating systems and unikernels might provide a solutions.

Some truth and lots of speculation about the future of operations.

Gareth Rushgrove

September 25, 2015
Tweet

More Decks by Gareth Rushgrove

Other Decks in Technology

Transcript

  1. Operations without the
    Operating System
    Puppet Labs
    Gareth Rushgrove
    Ops People and Future Technology

    View full-size slide

  2. Gareth Rushgrove
    @garethr

    View full-size slide

  3. Gareth Rushgrove

    View full-size slide

  4. First, some quotes

    View full-size slide

  5. These systems had never
    been designed with security in
    mind and now were becoming
    hugely complex
    Gareth Rushgrove

    View full-size slide

  6. The dense complexity and
    increasing length of computer
    code as computers where asked
    to do more and more would make
    malicious code increasingly hard
    to find
    Gareth Rushgrove

    View full-size slide

  7. Written in 2014
    Gareth Rushgrove

    View full-size slide

  8. Not about Heartbleed
    Gareth Rushgrove

    View full-size slide

  9. In a history book about the 1970s
    Gareth Rushgrove
    Intercept: The Secret History of Computers and Spies
    by Gordon Corera. Specifically about the Anderson Report, 1972

    View full-size slide

  10. Lots of new operating systems
    have appeared in the last year
    Gareth Rushgrove

    View full-size slide

  11. Speculation Klaxon

    View full-size slide

  12. The age of the general purpose
    operating system is over
    Gareth Rushgrove

    View full-size slide

  13. What does that mean for
    operators?
    Gareth Rushgrove

    View full-size slide

  14. A New Breed of OS
    Smaller, faster, more productive?

    View full-size slide

  15. What server operating systems
    are you running?
    Gareth Rushgrove

    View full-size slide

  16. Gareth Rushgrove
    I’m guessing most people said:

    View full-size slide

  17. Gareth Rushgrove
    Someone will try and be smart and say:

    View full-size slide

  18. Gareth Rushgrove
    And someone will hide the fact they’re running:

    View full-size slide

  19. But elsewhere on the internet…
    Gareth Rushgrove

    View full-size slide

  20. Gareth Rushgrove
    From RedHat, uses
    RPM-OSTree under
    the hood

    View full-size slide

  21. Gareth Rushgrove
    Built-in service-discovery
    and clustering support

    View full-size slide

  22. Gareth Rushgrove
    Snappy from Ubuntu,
    basically replaces apt-get
    with containers

    View full-size slide

  23. Gareth Rushgrove
    Docker running on
    Docker running as PID 1

    View full-size slide

  24. Gareth Rushgrove
    Nano is a tiny alternative
    to Windows Server
    6

    View full-size slide

  25. Gareth Rushgrove
    Bonneville provides
    containers that are really
    virtual machines

    View full-size slide

  26. Gareth Rushgrove
    Clear Linux provides
    hardware isolation
    guarantees to containers

    View full-size slide

  27. Common themes
    Gareth Rushgrove

    View full-size slide

  28. Cluster native
    Read-only file systems
    Transactional updates
    Integrated with containers
    Gareth Rushgrove

    View full-size slide

  29. We’ve seen some of this
    story before
    Gareth Rushgrove

    View full-size slide

  30. Gareth Rushgrove

    View full-size slide

  31. ESXi, XenServer
    Gareth Rushgrove

    View full-size slide

  32. Gareth Rushgrove

    View full-size slide

  33. What plays out the same
    and what different is
    interesting
    Gareth Rushgrove
    Note, everyone wants to be the VMware of containers

    View full-size slide

  34. Why The Interest in New
    Operating Systems
    And why now?

    View full-size slide

  35. Lots of homogeneous workloads
    Gareth Rushgrove

    View full-size slide

  36. Security is front page news
    Gareth Rushgrove

    View full-size slide

  37. I just want somewhere to run my
    containers
    Gareth Rushgrove

    View full-size slide

  38. Size as a proxy for complexity
    Gareth Rushgrove
    Can you name the 184 default packages in Ubuntu?

    View full-size slide

  39. Utilisation matters
    Gareth Rushgrove
    Only to large organisations who pay for software mind

    View full-size slide

  40. Increasingly interacting with
    higher level abstractions anyway
    Gareth Rushgrove
    Mesos, Kubernetes, Cloud Foundry

    View full-size slide

  41. Piling up Abstractions
    A brief history of infrastructure

    View full-size slide

  42. Gareth Rushgrove
    Operating system
    Hardware
    Application
    Once upon a time…

    View full-size slide

  43. Gareth Rushgrove
    Operating system
    Hardware
    Runtime
    Application
    But your application might
    need a runtime so lets
    add that

    View full-size slide

  44. Gareth Rushgrove
    Operating system
    Hardware
    Runtime
    Application
    Application Application
    Application
    Application Application
    Runtime
    But hardware is
    expensive so lets run
    multiple applications

    View full-size slide

  45. Gareth Rushgrove
    Operating system
    Hypervisor
    Hardware
    Runtime
    Application
    Applications need
    isolating so lets use
    virtualisation

    View full-size slide

  46. Gareth Rushgrove
    Operating system
    Hypervisor
    Hardware
    Runtime
    Application
    Operating system
    Runtime
    Application
    Operating system
    Runtime
    Application
    Run multiple virtual
    machines! Each with
    there own copy of the OS

    View full-size slide

  47. Gareth Rushgrove
    Operating system
    Hypervisor
    Hardware
    Runtime
    Application
    Operating system
    Runtime
    Application Application
    Application Application
    But the overhead of
    virtualisation is expensive
    so run multiple apps per VM

    View full-size slide

  48. Gareth Rushgrove
    Operating system
    Hypervisor
    Hardware
    Container
    Operating system
    Runtime
    Application
    Container runtime
    I heard you like
    containers

    View full-size slide

  49. Gareth Rushgrove
    Operating system
    Hypervisor
    Hardware
    Container
    A different OS
    Runtime
    Application
    Container runtime
    Container
    Operating system
    Runtime
    Application
    Container
    Even more OS
    Runtime
    Application

    View full-size slide

  50. Hypervisor
    Hardware
    Gareth Rushgrove
    Operating system
    Container
    Static binary
    Container runtime
    Container
    Static binary
    Container
    Static binary
    Cool folks use static
    binaries and scratch
    containers

    View full-size slide

  51. Gareth Rushgrove
    Operating system
    Hardware
    Container
    Static binary
    Container runtime
    Container
    Static binary
    Container
    Static binary
    Don’t need
    virtualisation isolation
    guarantees?

    View full-size slide

  52. Unikernels
    A library operating system

    View full-size slide

  53. What if there is no OS
    above the hypervisor?
    Gareth Rushgrove

    View full-size slide

  54. Gareth Rushgrove

    View full-size slide

  55. Gareth Rushgrove
    Unikernel
    Hypervisor
    Hardware
    Unikernels compile your
    application to a kernel, which
    can run on a hypervisor

    View full-size slide

  56. Compile your application
    into a Kernel
    Gareth Rushgrove
    No userspace

    View full-size slide

  57. Only include the capabilities/
    libraries you need
    Gareth Rushgrove
    No bash/ssh/sh/apt unless you explicitly include it

    View full-size slide

  58. Hypervisor/hardware isolation
    Smaller attack surface area
    Running less code
    Enforced immutability
    No default remote access
    Gareth Rushgrove

    View full-size slide

  59. The promise of containers, but
    without needing to pretend the
    intermediary OS doesn’t exist
    Gareth Rushgrove

    View full-size slide

  60. Gareth Rushgrove
    MirageOS

    View full-size slide

  61. Gareth Rushgrove
    HaLVM

    View full-size slide

  62. Gareth Rushgrove
    Rump Kernel

    View full-size slide

  63. Gareth Rushgrove
    LING

    View full-size slide

  64. Gareth Rushgrove

    View full-size slide

  65. Gareth Rushgrove

    View full-size slide

  66. What Happens to
    Operators?
    Technical operators anyway, service

    management folks are safe for now

    View full-size slide

  67. Hypervisor as the platform
    Gareth Rushgrove
    1

    View full-size slide

  68. Separate team running a
    hypervisor as a service
    Gareth Rushgrove

    View full-size slide

  69. Separate team company
    running a hypervisor as a
    service
    Gareth Rushgrove

    View full-size slide

  70. How many AWS users are
    Xen experts?
    Gareth Rushgrove

    View full-size slide

  71. It happened with Type 1
    hypervisors, I posit it will
    happen for Type 2 as well
    Gareth Rushgrove

    View full-size slide

  72. Everything else as an application
    Gareth Rushgrove
    2

    View full-size slide

  73. Firewalls as an application
    Gareth Rushgrove

    View full-size slide

  74. Proxies as an application
    Gareth Rushgrove

    View full-size slide

  75. Network switches as an application
    Gareth Rushgrove

    View full-size slide

  76. Intrusion detection as an
    application
    Gareth Rushgrove

    View full-size slide

  77. Remote shell as an application
    Gareth Rushgrove

    View full-size slide

  78. Everyone not running
    the hypervisor is an
    application developer
    Gareth Rushgrove

    View full-size slide

  79. Standards, standards and
    defacto standards
    Gareth Rushgrove
    3

    View full-size slide

  80. We need to start agreeing
    Gareth Rushgrove
    I don’t have time here for a rant about the contributors dilemma,
    initiation bias and how prototypes never die on GitHub

    View full-size slide

  81. Platforms
    Gareth Rushgrove

    View full-size slide

  82. Containers
    Gareth Rushgrove

    View full-size slide

  83. Monitoring
    Gareth Rushgrove
    ?

    View full-size slide

  84. Gareth Rushgrove
    StatsD

    View full-size slide

  85. Gareth Rushgrove
    Metrics 2.0

    View full-size slide

  86. Publish more schemas and fewer
    incompatible duplicate
    implementations
    Gareth Rushgrove
    We can get to actual Standards after some agreement

    View full-size slide

  87. Infrastructure is code
    Gareth Rushgrove
    4

    View full-size slide

  88. Infrastructure as algorithms
    Gareth Rushgrove

    View full-size slide

  89. Schedulers
    Gareth Rushgrove

    View full-size slide

  90. Back pressure
    Gareth Rushgrove

    View full-size slide

  91. Autoscaling
    Gareth Rushgrove

    View full-size slide

  92. Composition/configuration
    Gareth Rushgrove

    View full-size slide

  93. Configuration with static data
    won’t be enough
    Gareth Rushgrove

    View full-size slide

  94. Learn to programme. This time
    we mean it
    Gareth Rushgrove
    If you want this future we need to build it. Probably with
    static types and safer programming languages

    View full-size slide

  95. Revolution not evolution
    Gareth Rushgrove
    5

    View full-size slide

  96. The distance between old
    infrastructure and new will
    be huge
    Gareth Rushgrove

    View full-size slide

  97. Moving between the paradigms
    will be bigger than moving
    between Linux and Windows
    today
    Gareth Rushgrove

    View full-size slide

  98. Not about tools. About models of
    interaction and the skills required
    to operate
    Gareth Rushgrove

    View full-size slide

  99. We either fix our tech, or the
    security challenge will result in
    laws and policy that try and fix
    it for us
    Gareth Rushgrove
    I remember when it was legal to run software I wrote myself
    without cyber insurance. Ah, happy days.

    View full-size slide

  100. Gareth Rushgrove
    Remember. Blade
    Runner is a movie about
    the Government running
    around fixing bugs in
    software

    View full-size slide

  101. Conclusions
    If all you remember from this talk is…

    View full-size slide

  102. We still have fundamental
    problems that date back 40 years
    Gareth Rushgrove

    View full-size slide

  103. We can build better infrastructure
    Gareth Rushgrove

    View full-size slide

  104. Doing so might mean going down
    different evolutionary paths
    Gareth Rushgrove

    View full-size slide

  105. We know many of the patterns
    we want, but we’re mainly
    working backwards
    Gareth Rushgrove
    Stripping down our current operating systems isn’t sufficient

    View full-size slide

  106. This means throwing away things
    we care about
    Gareth Rushgrove
    What would you say if I said we don’t need Linux for the general case

    View full-size slide

  107. Collaborate on hard problems,
    rather than marvelling about how
    easy it is to build your own
    Docker PaaS
    Gareth Rushgrove

    View full-size slide

  108. Questions?
    And thanks for listening

    View full-size slide