Slide 1

Slide 1 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINERS, FTW! 1 Michael Hausenblas | 2016-04-22 | ProgSCon, London

Slide 2

Slide 2 text

© 2015 Mesosphere, Inc. All Rights Reserved. sys admin devops developer architect QA/testing other

Slide 3

Slide 3 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER 101 3

Slide 4

Slide 4 text

© 2016 Mesosphere, Inc. All Rights Reserved. LINUX
 CONTAINERS 4 The why and the what: • Containers vs VMs • app-level dependency management • lightweight (startup time, footprint, average runtime) • isolation & security

Slide 5

Slide 5 text

© 2016 Mesosphere, Inc. All Rights Reserved. LINUX
 CONTAINERS 5 • namespaces • Isolate PIDs between processes • Isolate process to network resources • Isolate the hostname to fake it out (UTS) • Isolate the filesystem mount points (chroot) • Isolate inter process communication (IPC) • Isolate specific users to specific processes • cgroups
 https://sysadmincasts.com/episodes/14-introduction-to-linux-control-groups-cgroups

Slide 6

Slide 6 text

© 2016 Mesosphere, Inc. All Rights Reserved. DOCKER 6

Slide 7

Slide 7 text

© 2016 Mesosphere, Inc. All Rights Reserved. DOCKER 7 Registries • Docker Hub
 https://hub.docker.com/ • Google Cloud
 https://cloud.google.com/tools/container-registry/ • AWS
 https://aws.amazon.com/ecr/ • Run your own
 https://docs.docker.com/registry/deploying/

Slide 8

Slide 8 text

© 2015 Mesosphere, Inc. All Rights Reserved. ARE CONTAINERS SECURE? 8

Slide 9

Slide 9 text

© 2015 Mesosphere, Inc. All Rights Reserved. ARE CONTAINERS SECURE? 9 • containers share same kernel (!) • namespaces (user NS: UID 0 recently introduced) • lock down networking (ICC, for example) • apply common sense when handling credentials

Slide 10

Slide 10 text

© 2015 Mesosphere, Inc. All Rights Reserved. ARE CONTAINERS SECURE? 10 Please, don't bake credentials into images … rather do: $ docker run -d -e API_TOKEN=SECRET somedatabase $ docker run -d -v $(pwd):/fsecret:/fsecret:ro somedatabase → even better: use a key-value in-memory store such as Square's KeyWhiz, HashiCorp's Vault, or Crypt or native solutions such as Kubernetes Secrets for credentials.

Slide 11

Slide 11 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER OPERATIONS 11

Slide 12

Slide 12 text

© 2016 Mesosphere, Inc. 12 https://dcos.io

Slide 13

Slide 13 text

© 2016 Mesosphere, Inc. All Rights Reserved. DC/OS 101 13 Any Service on DC/OS Any Infrastructure DC/OS Your favorite services, container formats, and those yet to come Build apps once on DCOS, and run it anywhere Runs distributed apps anywhere 
 as simply as running apps on your laptop

Slide 14

Slide 14 text

© 2016 Mesosphere, Inc. All Rights Reserved. DC/OS ARCHITECTURE 14

Slide 15

Slide 15 text

© 2016 Mesosphere, Inc. All Rights Reserved. LOCAL OS
 VS
 DISTRIBUTED OS 15

Slide 16

Slide 16 text

© 2015 Mesosphere, Inc. All Rights Reserved. QUICK INTERLUDE: APACHE MESOS 16

Slide 17

Slide 17 text

© 2016 Mesosphere, Inc. All Rights Reserved. APACHE MESOS 17 • A top-level ASF project • A cluster resource negotiator • Scalable to 10,000s of nodes but also useful for a handful of nodes • Fault-tolerant, battle-tested • An SDK for distributed apps • Native Docker support mesos.apache.org

Slide 18

Slide 18 text

© 2016 Mesosphere, Inc. All Rights Reserved. MESOS
 ARCHITECTURE 18 http://mesos.berkeley.edu/mesos_tech_report.pdf Marathon
 scheduler Spark
 scheduler Spark
 executor Marathon
 executor Spark
 executor Marathon
 executor Mesos Agent Mesos Agent Mesos Agent Mesos
 Master Standby
 Master Standby
 Master Framework

Slide 19

Slide 19 text

© 2015 Mesosphere, Inc. All Rights Reserved. RESOURCES 19 • resource: anything a task consumes to do its work • standard resources: cpu
 mem
 disk
 ports • Dominant Resource Fairness (DRF) algorithm guarantees fair allocation across resource types

Slide 20

Slide 20 text

© 2015 Mesosphere, Inc. All Rights Reserved. … BACK TO DC/OS 20

Slide 21

Slide 21 text

© 2016 Mesosphere, Inc. All Rights Reserved. DC/OS BENEFITS 21 • One cluster for • stateless services such as Web servers & app servers (via Marathon) • stateful services like PostgreSQL, MemSQL, Kafka, Cassandra, etc. • elastic data processing via Spark, Akka, etc. • CI/CD, for example Jenkins+Marathon • Dynamic partitioning of your cluster, depending on your needs • Increased utilization (10% → 80%+)

Slide 22

Slide 22 text

© 2015 Mesosphere, Inc. All Rights Reserved. 22

Slide 23

Slide 23 text

© 2015 Mesosphere, Inc. All Rights Reserved. 23

Slide 24

Slide 24 text

© 2016 Mesosphere, Inc. All Rights Reserved. MARATHON 24 The DC/OS 'init system' • starts instances of a long-running services • restarts the instances if they crash • provides composition primitives • supports health checks • supports rolling upgrades

Slide 25

Slide 25 text

© 2015 Mesosphere, Inc. All Rights Reserved. MARATHON
 APP SPEC 25 { "id": "/webserver", "instances": 5, "cpus": 0.1, "mem": 128, "container": { "type": "DOCKER", "docker": { "image": "nginx:1.9.14", "network": "BRIDGE", "portMappings": [ { "containerPort": 80, "hostPort": 0 } ] } }, "upgradeStrategy": { "minimumHealthCapacity": 0.9, "maximumOverCapacity": 0.2 }, "acceptedResourceRoles": [ "slave_public" ] }

Slide 26

Slide 26 text

© 2016 Mesosphere, Inc. All Rights Reserved. MARATHON
 GROUPS 26 • Groups can contain one or more apps/groups • Dependency management • Scaling https://mesosphere.com/blog/2015/06/21/web-application-analytics-using-docker-and-marathon/

Slide 27

Slide 27 text

© 2015 Mesosphere, Inc. All Rights Reserved. MARATHON UI/UX 27

Slide 28

Slide 28 text

© 2016 Mesosphere, Inc. All Rights Reserved. CI/CD AND DEPLOYMENT STRATEGIES 28

Slide 29

Slide 29 text

© 2016 Mesosphere, Inc. All Rights Reserved. CONTAINER LIFE CYCLE 29

Slide 30

Slide 30 text

© 2016 Mesosphere, Inc. All Rights Reserved. CI/CD WITH DC/OS 30

Slide 31

Slide 31 text

© 2016 Mesosphere, Inc. All Rights Reserved. HANDS-ON 31

Slide 32

Slide 32 text

© 2016 Mesosphere, Inc. All Rights Reserved. LEARNING RESOURCES 32

Slide 33

Slide 33 text

© 2015 Mesosphere, Inc. All Rights Reserved. 33 WHERE CAN
 I LEARN MORE? http://shop.oreilly.com/product/9781939902184.do 33 http://shop.oreilly.com/product/0636920035671.do

Slide 34

Slide 34 text

© 2015 Mesosphere, Inc. All Rights Reserved. 34 WHERE CAN
 I LEARN MORE? 34 https://www.nginx.com/resources/library/docker-networking/

Slide 35

Slide 35 text

© 2015 Mesosphere, Inc. All Rights Reserved. 35 WHERE CAN
 I LEARN MORE? http://shop.oreilly.com/product/0636920039952.do https://manning.com/books/mesos-in-action 35

Slide 36

Slide 36 text

© 2016 Mesosphere, Inc. All Rights Reserved. Q & A 36 • @mhausenblas • mhausenblas.info • [email protected] https://dcos.io