Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Next generation infrastructure with Docker and Mesos

Next generation infrastructure with Docker and Mesos

Inspired by Google, organizations are building out warehouse-scale infrastructure. In this model, applications packaged in lightweight containers run on large pools of computers to achieve high levels of flexibility and utilization. Web applications run on the same servers that are processing batch big data workloads and stream processing.

In this talk we'll cover two building blocks of this new infrastructure: Docker and Apache Mesos.

Docker allows organizations to easily package, run and share server applications. Since Docker's release over a year ago, it has become one of the fastest growing open source projects in history.

Mesos is a open-source distributed scheduler that is similar to the Omega scheduler that powers Google. Mesos is the original computation engine for Apache Spark, but has added the ability to run Docker containers, Hadoop and applications like Storm.


Steven Borrelli

June 18, 2014


  1. D O C K E R + M E S

    O S S T L H A D O O P J U N E 2 0 1 4 S T E V E N B O R R E L L I @stevendborrelli steve@aster.is
  2. A B O U T M E 2 0 1

    4 F O U N D E D : O R G A N I Z E R O F S T L M A C H I N E L E A R N I N G A N D D O C K E R S T L F I R S T C O M P U T E R : @ S Y S T E M S E N G I N E E R I N G , H P C , B I G D A TA & C L O U D
  3. Let’s talk about

  4. I N F R A S T R U C

    T U R E T H E N E X T G E N E R A T I O N O F
  5. Why should I care?

  6. I N F R A S T R U C

    T U R E I S T H E F O U N D AT I O N O F T H E I N F O R M AT I O N E C O N O M Y
  7. – M A R C A N D R E

    E S S E N “We are in the middle of a dramatic and broad technological and economic shift in which software companies are poised to take over large swathes of the economy.”
  8. I N F R A S T R U C

    T U R E I S A C O M P E T I T I V E A D VA N TA G E
  9. ƭ W R I T E T E S T

    D E P L O Y S P E E D + Q U A L I T Y O F S O F T WA R E D E V E L O P M E N T H A S B E C O M E C R I T I C A L

  11. WA R E H O U S E S C

    A L E C O M P U T I N G
  12. I N S P I R E D B Y

  13. GFS (2003) MapReduce (2004) Chubby (2006) BigTable (2006)

  14. H O W D O E S G O O

    G L E R U N ?

    E S G O O G L E R U N ?
  16. B U I L D I N G B L

    O C K S
  17. C O N TA I N E R S PA

    C K A G E A P P L I C AT I O N S I N L I G H T W E I G H T V I RT U A L M A C H I N E S
  18. D I S T R I B U T E

    D S C H E D U L I N G R U N W O R K L O A D S A C R O S S M U LT I P L E S E R V E R S

  20. Let’s start with containers

  21. 1.0 released last week!

  22. W H Y D O C K E R ?

    Docker makes it easy to: Package Deploy Share Containerized Server Applications
  23. Think: java -jar vs. ./configure; make install !

  24. What’s the difference between containers and virtual machines (VMs)? !

  25. V I RT U A L I Z AT I

    O N
  26. 1966-1972 IBM CP/CMS 1989 Insignia SoftPC 1997 Connectix VirtualPC 1999

    VMWare Workstation 2001 IBM AIX LPAR 2002 Xen 2006 Amazon EC2 2007 Sun Logical Domains 2007 Linux KVM 2007 InnoTek VirtualBox 2008 Microsoft Hyper-V H A R D WA R E V I RT U A L I Z AT I O N
  27. C O N TA I N E R S

  28. P R O C E S S V I RT

    U A L I Z AT I O N 1979-1982 UNIX Chroot 1998 FreeBSD Jail 2001 Parallels Virtuozzo 2001 Linux-VServer 2005 Solaris Zones 2005 OpenVZ 2008 Linux LXC 2007+ PAAS: Heroku, Joyent, CloudFoundry 2013 Docker

    O C K E R A R C H I T E C T U R E
  30. N A M E S PA C E S V

    S . C G R O U P S Namespaces provide isolation: • pid (processes) • net (network interfaces, routing...) • ipc (System V IPC) • mnt (mount points, filesystems) • uts (hostname) • user (UIDs) Control groups control resources: • cpu (CPU shares) • cpusets (limit processes to a CPU) • memory (swap, dirty pages) • blockio (throttle reads/writes) • devices • net_cls, net_prio: control packet class and priority

  31. Differences between containers and virtual machines ! • Weaker isolation

    in containers • Containers run near-native speed for CPU/IO • Cached containers launch in around 0.1 second (libcontainer) • Less memory overhead

  33. R U N N I N G A C O

    N TA I N E R Start a container: Mount host filesystems:
  34. Example: run Zookeeper + Exhibitor Host ports will be dynamically

    allocated by docker Port 2181 on host will be mapped to 2181 on container M A P P I N G P O RT S
  35. New in 0.11, allows a container to access host adapters:

    D I R E C T H O S T N E T W O R K Port 8080 on the container is 8080 on the host:
  36. It’s easy to build containers

  37. J AVA D O C K E R F I

    L E
  38. No we can build a Tomcat container from the Java

  39. T O M C AT D O C K E

    R F I L E
  40. L AY E R E D F S I S

    A G R A P H
  41. It’s easy to share containers

  42. D O C K E R R E G I

    S T RY S E A R C H P U S H P U L L
  43. D O C K E R S U M M

    A RY Easy to build, run & share containers Rapidly expanding ecosystem Better performance vs. VMs Layered filesystem gives us git-like control of images. Reduces complexity of system builds
  44. We’ve packaged up all our apps into containers! ! Now

    how do we run them?
  45. D I S T R I B U T E

    D S C H E D U L I N G
  46. Launches 2 Billion containers/week ~3,300/second Powered by:

  47. Can I implement Google-style computing?

  48. None
  49. None
  50. B E R K E L E Y D ATA

    A N A LY T I C S S TA C K ( B D A S )
  51. Fault Tolerant Massively Scalable Mix Diverse Workloads Efficient/Optimal Resource Utilization

    D E S I R E D C H A R A C T E R I S T I C S
  52. S TAT I C PA R T I T I

    O N I N G C U R R E N T LY, A L M O S T E V E RY O N E U S E S
  53. M I X W O R K L O A

    D S W I T H M E S O S
  54. I N C R E A S E U T

    I L I Z AT I O N W I T H M E S O S
  55. Mesos uses Zookeeper for High Availability

  56. H I G H LY AVA I L A B

    L E A R C H I T E C T U R E
  57. Mesos is a Two-Level Scheduler

  58. Frameworks Manage Jobs Mesos Tracks and Provides Resources

  59. R U N M U LT I P L E

    F R A M E W O R K S App-specific: Generic:
  60. R U N M U LT I P L E

    F R A M E W O R K S
  61. +

  62. Package Apps into Containers Run Containers at Scale N AT

    U R A L C O M P L E M E N T S
  63. M E S O S D O C K E

    R I N T E G R AT I O N Mesos Uses Cgroups for Isolation Fall 2013: External Script to shell out to Docker June 2014 0.19: Native Container Support Two Docker executors:
  64. ! {! "cmd": "env && sleep 300",! "constraints": [! ["attribute",

    "OPERATOR", "value"]! ],! "container": {! "image": "docker:///dockeruser/oracle-java7",! "options": ["-v", "/etc/config:/etc/config:ro"]! },! "cpus": 2,! "env": {! "LD_LIBRARY_PATH": "/usr/local/lib/myLib"! }! } L A U N C H I N G D O C K E R C O N TA I N E R S W I T H M A R AT H O N
  65. D O C K E R S U P P

    O RT F O R M E S O S Prototype Mesos Backend Planned Support
  66. T O O L S C A M B R

    I A N E X P L O S I O N
  67. Docker has a rapidly growing ecosystem Existing software is being

    rewritten to support Docker containers D O C K E R I S D R I V I N G N E W T O O L I N G
  68. Docker-centric Big Data Pre-Docker PAAS D O C K E

    R S U P P O R T I N G I N F R A S T R U C T U R E
  69. Service Discovery Dynamic Proxies Linking Containers Storage N E W

    M O D E L S E M E R G I N G
  70. N E X T S T E P S Dockerize

    your applications, run them using simple scripts Use Mesos as a backend for Spark Start running apps via Marathon & Chronos Integrate containers into your build process
  71. None
  72. Docker/Mesos/AWS Professional Services ! Jenkins Plugin for Docker & Mesos

    ! Prebuilt Amazon AMI ! API to spin up clusters on AWS http://aster.is
  73. I M A G E C R E D I

    T S • "U.S. Army Photo", number 163-12-62. Patsy Simmers, et. al holding computer parts: http://ftp.arl.army.mil/ ftp/historic-computers/ • “U.S. Army Photo”, Two women operating the ENIAC’s: http://ftp.arl.army.mil/ftp/historic-computers/png/ eniac7.png • Factory: http://commons.wikimedia.org/wiki/File:Float_Glass_Unloading.jpg • Distributed Scheduling: http://s1.reutersmedia.net/resources/r/? m=02&d=20120619&t=2&i=620575938&w=&fh=&fw=&ll=700&pl=390&r=2012-06-19T114519Z_02_GM1E86I1 B6M01_RTRRPP_0_CHINA • Mesos Paper: http://static.usenix.org/event/nsdi11/tech/full_papers/Hindman_new.pdf • Static Workloads/Mix Workloads/Mix Utilization, Ben Hindman, http://www.slideshare.net/dotCloud/high- speed-shipping-lanes-how-containers-are-revolutionizing-distributed-computing-at-scale • Cambrian Explosion: http://www.bio.miami.edu/dana/pix/cambrian_explosion.jpg