Past, Present and Future of Containers

Slide 1

Slide 1 text

Past, Present and Future of Containers Timothy Chen

Slide 2

Slide 2 text

About me: - Lead Distributed Systems Engineer at Mesosphere - Apache Mesos, Drill PMC - Maintains Spark on Mesos, Docker Swarm

Slide 3

Slide 3 text

Outline: Why containers? What is a container? How is it implemented now? Where is it going?

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

VMs

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

The Matrix From Hell django web frontend ? ? ? ? ? ? node.js async API ? ? ? ? ? ? background workers ? ? ? ? ? ? SQL database ? ? ? ? ? ? distributed DB, big data ? ? ? ? ? ? message queue ? ? ? ? ? ? my laptop your laptop QA staging prod on cloud VM prod on bare metal

Slide 10

Slide 10 text

Another Matrix from Hell ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

Slide 11

Slide 11 text

Solution: the intermodal shipping container

Slide 12

Slide 12 text

Solved!

Slide 13

Slide 13 text

Solution to the deployment problem: the Linux container

Slide 14

Slide 14 text

Linux containers... Units of software delivery (ship it!) ● run everywhere – regardless of kernel version – regardless of host distro – (but container and host architecture must match*) ● run anything – if it can run on the host, it can run in the container – i.e., if it can run on a Linux kernel, it can run *Unless you emulate CPU with qemu and binfmt

Slide 15

Slide 15 text

CFO, CIO, CTO, ... ● LESS overhead! ● MOAR consolidation! ● MOAR agility! ● LESS costs!

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

1998

Slide 18

Slide 18 text

Solaris Zones 2002

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

High level approach: it's a lightweight VM ● own process space ● own network interface ● can run stuff as root ● can have its own /sbin/init (different from the host)

Slide 23

Slide 23 text

Low level approach: it's chroot on steroids ● can also not have its own /sbin/init ● container = isolated process(es) ● share kernel with host ● no device emulation (neither HVM nor PV)

Slide 24

Slide 24 text

Separation of concerns: Dave the Developer ● inside my container: – my code – my libraries – my package manager – my app – my data

Slide 25

Slide 25 text

Separation of concerns: Oscar the Ops guy ● outside the container: – logging – remote access – network configuration – monitoring

Slide 26

Slide 26 text

How does it work? Isolation with namespaces ● pid ● mnt ● net ● uts ● ipc ● user

Slide 27

Slide 27 text

How does it work? Isolation with cgroups ● memory ● cpu ● blkio ● devices

Slide 28

Slide 28 text

Efficiency: almost no overhead ● processes are isolated, but run straight on the host ● CPU performance = native performance ● memory performance = a few % shaved off for (optional) accounting ● network performance = small overhead; can be optimized to zero overhead

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

https://www.youtube.com/watch?v=JHqM_5X3MBU

Slide 31

Slide 31 text

Docker

Slide 32

Slide 32 text

What's Docker? ● Open Source engine to commoditize LXC ● using copy-on-write for quick provisioning ● allowing to create and share images ● propose a standard format for containers

Slide 33

Slide 33 text

Yes, but... ● « I don't need Docker; I can do all that stuff with LXC tools, rsync, some scripts! » ● correct on all accounts; but it's also true for apt, dpkg, rpm, yum, etc. ● the whole point is to commoditize, i.e. make it ridiculously easy to use

Slide 34

Slide 34 text

Docker: authoring images ● you can author « images » – either with « run+commit » cycles, taking snapshots – or with a Dockerfile (=source code for a container) – both ways, it's ridiculously easy ● you can run them – anywhere – multiple times

Slide 35

Slide 35 text

No content

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

Dockerfile example FROM ubuntu RUN apt-get -y update RUN apt-get install -y g++ RUN apt-get install -y erlang-dev erlang-manpages erlang-base-hipe ... RUN apt-get install -y libmozjs185-dev libicu-dev libtool ... RUN apt-get install -y make wget RUN wget http://.../apache-couchdb-1.3.1.tar.gz | tar -C /tmp -zxf- RUN cd /tmp/apache-couchdb-* && ./configure && make install RUN printf "[httpd]\nport = 8101\nbind_address = 0.0.0.0" > /usr/local/etc/couchdb/local.d/docker.ini EXPOSE 8101 CMD ["/usr/local/bin/couchdb"]

Slide 38

Slide 38 text

Docker: sharing images ● you can push/pull images to/from a registry (public or private) ● you can search images through a public index ● dotCloud maintains a collection of base images (Ubuntu, Fedora...) ● satisfaction guaranteed or your money back

Slide 39

Slide 39 text

Docker: not sharing images ● private registry – for proprietary code – or security credentials – or fast local access

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

Efficiency: storage-friendly ● unioning filesystems (AUFS, overlayfs) ● snapshotting filesystems (BTRFS, ZFS) ● copy-on-write (thin snapshots with LVM or device-mapper) This wasn't part of LXC at first; but you definitely want it!

Slide 43

Slide 43 text

Efficiency: storage-friendly ● provisioning now takes a few milliseconds ● … and a few kilobytes ● creating a new base/image/whateveryoucallit takes a few seconds