Docker & Linux Containers

Presentation I gave for my team at Microsoft Azure.

This does not have the animations in them. For the slides with animations, check out: https://speakerdeck.com/alp/docker-and-linux-containers-with-animations

Follow me on twitter: https://twitter.com/ahmetalpbalkan

Transcript

1. Linux  Containers  &  Docker   Ahmet  Alp  Balkan   8/14/2014   NOT  MICROSOFT  CONFIDENTIAL  
2. ‘Real’  containers  
3. What  is  special  about  containers?  
4. Power  of  Containers   •  Standard  format  everybody  agrees  on   •  Once  you  seal  the  box,  it  gets  shipped  “as  is”   •  SeparaVon  of  concerns   – Building  ó  Packaging  ó  Shipping  
5. Containers  vs.  VMs  
6. Type  1  Hypervisor     Hardware   VM   VM   OperaVng  System   OperaVng  System   bins/libs   bins/libs   App   App   App   App   Hypervisor   Kernel  
7. Type  2  Hypervisor     Hardware   Hypervisor  So[ware   VM   VM   OperaVng  System   OperaVng  System   bins/libs   bins/libs   App   App   App   App   OperaVng  System   App   App   App  
8. Linux  Containers   Hardware   OperaVng  System  (Linux)   bins/libs   Container   Container   bins/libs   App   App   App  
9. lxc:  Linux  Containers   •  OS-­‐Level  virtualizaVon,  no  Hypervisor   •  Namespacing   –  PIDs   –  user  IDs   –  network  tables   –  mounted  drivers   –  …   •  Started  in  2006.   •  Developed  by  a  team  of  4   –  v1.0  in  2014.  
10. cgroups:  Control  Groups   •  Fair  resource  usage  sharing  for  Linux  kernel   – CPU   – Memory   – I/O   •  Started  in  2007.   – by  2  Google  Engineers  
11. Distributed  ApplicaVon  Model   Front-­‐end  (web)   App  services   Background  workers   DB   Cache  Servers  
12. Advantages  of  containers   •  Only  one  kernel  runs  on  the  machine   •  No  Hypervisor  overhead.   –  Apps  directly  run  on  CPU   –  Rack  space  &  power  &  more  free  CPU  cycles   •  Run  different  versions  of  OS  on  same  server   –  Run  different  OSes  on  the  same  server   •  Process  isolaVon  &  security  
13. Use  of  Linux  Containers   •  Borg   –  Google’s  “secret  sauce”   –  Everything  runs  in  containers   –  Not  open  source     •  Mesos   –  Developed  by  Twiger   –  Open  sourced   “IT’S  A  WAY  OF  STITCHING  TOGETHER  A   COLLECTION  OF  MACHINES  INTO,  BASICALLY,   A  BIG  COMPUTER.”  
15. …is  a  Linux  container  engine  
16. …is  Open  Source  
17. …is  also  a  company.  
18. Why  was  Docker  born?   •  Shipping  code  to  servers  is  hard   •  It’s  hard  to  glue  the  technology   – LXC  +  cgroups   – lmcky:  “Let  me  contain  that  for  you”   •  open  source  version  of  Google’s  container  stack     •  There  is  no  standard  image  format  
19. Another  moVve:  eliminaVng  Ops   •  So[ware  used  to  be  deployed  by  ops  teams   •  Docker  might  be  “the  first  true  DevOps  tool”   – Increased  shipment  cadence   – Fast  moving  engineering  culture   •  Sysadmins  no  more.   – A  great  abstracVon  over  infrastructure.  
20. Why  is  Docker  special?   •  The  real  value  is  not  technology.   •  It’s  “gelng  people  to  agree  on  something”.   – Packaging  and  delivery  method.  
21. Cloud  as  one  big  machine   docker  client   $  docker  run  …   libswarm  
22. Demo  Vme   Linux  VM  on  Azure   docker  daemon   docker  client   Container   Container   App   App  
23. Docker  Image  Layers   ubuntu   apache+php   ADD    /my/applicaIon   ADD    /my/configuraIon   200  MB   250  MB   260  MB   261  MB   50  MB  diff   10  MB  diff   1  MB  diff  
24. Layers  &  Reuse   Base  Linux  DistribuVon  image   Google  configuraVon   mapreduce   crawler   web  frontend   video  encoder  
25. Dockerfile   FROM ubuntu:latest MAINTAINER Ahmet <ahmetb@microsoft.com> RUN apt-get install nginx mysql python ADD /my/app /var/www/app ENTRYPOINT /var/www/app/server.py $ docker build
26. Dockerfile  inheritance   FROM microsoft/aspnet-vnext MAINTAINER Bob <bob@microsoft.com> ADD /my/app /var/www/app ENTRYPOINT /var/www/app/Server.cs Reuse  
27. Docker  use  cases   •  Just  like  VMs  in  producVon   –  SpoVfy,  eBay   •  One  off  tasks   –  Build  automaVon   –  ConVnuous  integraVon   –  TesVng   –  Bug  repros   •  Making  the  cloud  look  like  “one  big  machine”    
28. …  is  a  minimal  host  OS  (Linux)  for  Docker.  
29. …is  an  open  source  project  
30. …is  also  a  company  
31. …is  auto  self-­‐updaVng.  
32. …has  rollbacks.     (updates  are  atomic)  
33. Clustering   •  Kubernetes  (by  Google)   –  for  clustering  Docker  containers  across  nodes   –  Inspired  by  Google’s  internal  systems  like  Borg/ Omega   –  Open  source   –  10+  years  of  experience  in  running  containers   –  ContribuVons  by  Microso[  &  IBM  etc.   •  fleet  (by  CoreOS)   –  for  clustering  CoreOS  nodes   –  Open  source    
34. Cons  of  Docker   •  Linux  only   – Probably  will  never  support  Windows.     •  Security  risks     •  Same  pricing  as  VMs   •  SVll  developing  ecosystem  
35. Further  reading   •  Return  of  the  Borg  (Wired)   •  Inside  the  Borg  and  Mesos  (Verge)   •  Google  Open  Sources  Its  Secret  Weapon  in  Cloud   CompuVng:  Kubernetes  (Wired)   •  Docker:  hgp://docker.com   •  CoreOS:  hgp://coreos.com   •  Running  Docker  on  Windows  (docker.com)   •  Docker  on  Azure  (azure.com)   •  Omega:  flexible,  scalable  schedulers  for  large  compute   clusters  SIGOPS  EuroSys  2013,  ACM,  pp.  351-­‐364     (Google  Research)  
36. Thanks.   QuesVons?