Clocker - The Docker Cloud Maker

Talk at DockerCon Europe, Amsterdam, December 2014

Transcript

1. Clocker   The  Docker  Cloud  Maker  
2. Andrew  Kennedy  @grkvlt   Clocker   The  Docker  Cloud  Maker  
3. @grkvlt   Introduc/on   •  Andrew  Kennedy   –  So6ware  Engineer   –  Open  Source  and  Distributed  Systems   –  github.com/grkvlt   •  Cloudso6  Corpora/on   –  Open  Source  Applica/on  Management  Specialists   –  We’re  Hiring…  
4. @grkvlt   Agenda   •  Overview   •  Components   •  Deep  Dive   •  Demonstra/on   •  Roadmap  
5. Clocker  Overview  
6. @grkvlt   What  does  it  do?   1.  Spins  up  and  Manages  Docker   Clouds   2.  Serves  up  Containers  on  Demand   3.  Manages  Composite  Applica/on   Deployments  on  Docker  
7. @grkvlt   What  does  it  provide?   •  Mul/  Host  and  Mul/  Container  Applica/ons   •  Seamless  Networking   – Communica/on  Between  Services   •  Orchestra/on  and  Clustering   – Control  of  Containers   – Container  Management  
8. @grkvlt   Who  is  using  it?   •  Tes/ng  and  Proof  of  Concept  Stage   –  Financial  Services   –  Insurance   •  Produc/on   –  Mul/-­‐tenant  Applica/on  Trial   –  Container  per  service   –  Via  IBM  BlueMix  
9. @grkvlt   Where  can  I  find  it?   •  Open  Source   •  Apache  2.0  Licensed   •  hZp://clocker.io     •  Status   •  0.7.0  Beta  Release   •  0.8.0-­‐SNAPSHOT  Development  
10. Clocker  Components  
11. @grkvlt   Clocker  and  Brooklyn   •  What  is  it?   – Brooklyn  Applica/on   – Brooklyn  Loca/on   •  What  does  it  provide?   – First  Class  Docker  Support  in  Brooklyn   – Op/mized  Brooklyn  Blueprints  for  Docker  
12. Apache  Brooklyn   •  Apache  Brooklyn   – Open  Source   – Java   – Donated  to  the  Apache  So6ware  Founda/on   – ASF  Incubator  Status  
13. @grkvlt   Apache  Brooklyn   •  Applica/on  Management  Pla^orm   •  Deploy,  Manage  and  Monitor  Blueprints   •  Provisioning,  Installa/on  and  Customiza/on   •  Management   – AutoScaling,  Resilience,  Performance,  Security  
14. Apache  jclouds   •  Java  Cloud  Library   •  API  Agnos/c   – CloudStack,  OpenStack,  AWS  EC2,  GCE…   •  Create  Virtual  Machines   – Return  SSH  Endpoint   – Manage  Proper/es  
15. @grkvlt   Apache  jclouds   •  Drivers  for  REST  APIs   •  Docker  Driver   –  WriZen  by  @turlinux   •  Virtual  Container   –  Using  SSH  Daemon   –  Same  Endpoint  Type  as  VM   –  Composi/on  on  any  Image  or  Dockerfile  
16. @grkvlt   Docker  Networking  101   •  Separate  Container  Network   •  RFC  1918  Private  Addresses   •  Non  Routable   •  Bridged  to  Host   •  Port  Forwarding   •  Mul/ple  Endpoints  for  Services  
17. Docker  Topology  
18. @grkvlt   So6ware  Defined  Networking   •  Overlay  Networks   •  Kernel  or  User  Space   •  Many  Op/ons   –  Pipework   –  SocketPlane   –  Flannel   –  Open  vSwitch   –  Weave  
19. Weave  Topology  
20. @grkvlt   Weave   •  Ethernet  Switch   –  User  Space   –  Docker  Container   •  Mesh  Topology   •  Routes  TCP  Traffic   –  Sniffs  on  Host   –  Forwards  over  TCP  
21. Weave  
22. Clocker  Deep  Dive  
23. Clocker  Architecture  
24. Clocker  Architecture  
25. @grkvlt   Clocker  Features   •  Applica/on  Deployment   –  Oasis  CAMP  YAML  Blueprint   –  TOSCA  in  Development   –  Same  as  Core  Brooklyn   •  Mixed  Des/na/ons   –  Some  Virtual  Machines   –  Some  Bare  Metal   –  Some  Containers  
26. @grkvlt   Clocker  Features   •  Docker  Extensions   –  Container  or  Image   –  Placement  Strategy   –  Dockerfiles   •  Manages  Docker   –  Cloud  Deployment   –  Configura/on   –  Run/me  Control  
27. @grkvlt   Container  Management   •  Sources   –  Docker  Image  Defini/on   –  Docker  Hub   –  Dockerfile   –  Brooklyn  En/ty  Defini/on   •  Create  Image  Automa/cally   –  Commit  or  Push  for  Reuse  
28. @grkvlt   Container  Management   •  Installa/on  of  Services   –  Defined  by  Brooklyn  or  Dockerfile   –  Common  to  all  En/ty  Instances   •  Commit  Image   –  Available  for  next  En/ty   •  Push  Image   –  Available  for  all  Hosts  
29. Container  Management   id:  dockerfile-­‐mysql   name:  "Docker  Hub  MySQL  Application"   origin:  "https://registry.hub.docker.com/_/mysql/"   locations:   -­‐  my-­‐docker-­‐cloud   services:   -­‐  serviceType:  brooklyn.entity.container.docker.application.DockerfileApplication      id:  mysql      name:  "MySQL"      brooklyn.config:          docker.dockerfile.url:              "https://s3-­‐eu-­‐west-­‐1.amazonaws.com/brooklyn-­‐clocker/mysql-­‐5.6.tgz"          docker.container.environment:              MYSQL_ROOT_PASSWORD:  "s3cr3t”    
30. @grkvlt   Container  Placement   •  Demand   –  Adding  an  Applica/on   –  Scaling  exis/ng  Applica/on   •  Requirements   –  Host  Loca/on   –  Service  Resources   –  CPU,  Memory    
31. @grkvlt   Container  Placement   •  Supply   – Choose  a  Host  from  available   – Create  new  Host   •  Start  Container   –  Set  CPU  and  Memory   –  AZach  to  Network  
32. @grkvlt   Container  Placement   •  Placement  Strategies   –  Filter  Hosts   –  Sort  Hosts   •  Informa/on  from   –  Docker   –  Underlying  Machine   –  Cloud  Environment   –  En/ty  or  Service  
33. @grkvlt   Container  Placement   •  Placement  Strategies   –  Random,  Depth  or  Breadth  First   –  CPU  or  Memory  Usage   –  Memory,  CPU  or  Container  Limits   –  Geographic  Constraints   •  User  Defined   –  Java  Predicate  
34. Placement  Strategy   •  Determinis/c   •  Simple   – Predicate  and  Comparator   docker.container.strategies:      -­‐  $brooklyn:object:              type:  "brooklyn.location.docker.strategy.MaxContainersPlacementStrategy”              brooklyn.config:                  maxContainers:  16      -­‐  $brooklyn:object:              type:  "brooklyn.location.docker.strategy.CpuUsagePlacementStrategy”              brooklyn.config:                  maxCpu:  0.75  
35. @grkvlt   Container  Provisioning   •  Provisioning  Strategy   –  New  Docker  Host  Loca/on   •  Constraints   –  Docker  Infrastructure  Constraints   –  En/ty  or  Applica/on  Constraints   •  User  Defined  Strategies   •  Intelligent  Container  Orchestra/on  
36. @grkvlt   Container  Provisioning   •  Headroom   –  Ensure  resources  available   –  Based  on  MaxContainers  strategy  limit   –  Or  CPU  and  RAM  alloca/on   •  Scale  Docker  Host  Cluster  Automa/cally     –  Add  new  Docker  hosts   –  Remove  empty  Docker  hosts  
37. @grkvlt   Networking   •  Shared  Weave  LAN   – Common  to  All  Containers   – Private  (Link  Local)  Addresses   •  Clocker  Controls  IP  Alloca/on   – Applica/ons  Segmented  by  CIDR   •  Docker  Port  Forwarding  Access  
38. @grkvlt   Networking   •  Work  in  Progress   •  Name  Resolu/on   – BIND  and  DNSmasq  or  Weave  DNS   – Needed  for  JMX  et  al   – Enables  Many  More  En//es   •  Adding  More  Supported  Blueprints  
39. Demonstra>on  
43. @grkvlt   Roadmap   •  Latest  Docker  Feature  Support   •  Swarm   –  Same  API   –  Can  re-­‐use  our  jclouds  Driver   •  Machine   –  Cloud  agnos/c  provisioning  via  jclouds  already   –  Could  support  as  an  op/on  
44. @grkvlt   Roadmap   •  Networking   – DNS  and  DNSmasq  Integra/on   – Rancher  Integra/on   – Open  vSwitch  Integra/on   •  Gelng  Started   – Clocker  Dockerfile  
45. @grkvlt   Roadmap   •  Repositories   –  Docker  Repository   –  Bintray,  Ar/factory,  Quay.io   –  Private  Repositories   •  Applica/on  Defini/on   –  TOSCA   –  Fig  
46. @grkvlt   Roadmap   •  Plugin  API  may  be  helpful?   •  Provide  services  to  Docker  this  way   •  Orchestra/on   •  Policies   •  Consume  wider  range  of  services   •  Networking  
47. @grkvlt   Summary   •  Clocker   –  Brooklyn  +  jclouds  +  Docker  +  Weave  +  …   –  @clockercentral   •  Solves   –  Composite  Applica/on  Management   –  Docker  Cloud  Networking   –  Container  Placement  and  Provisioning  
48. @grkvlt   Audience  Ques/ons?   1.  What  networking  features  would   make  Docker  easier  to  use?   2.  What  other  features  would  be  most   useful  to  enhance  Docker  usability?   3.  Will  you  be  tes/ng  Swarm  and   Machine  or  would  you  prefer  an   external  orchestra/on  tool?  
49. @grkvlt   Web  Resources   http://clocker.io/   http://brooklyn.io/   http://docker.io/   http://github.com/zettio/weave/   http://abstractvisitorpattern.co.uk/  
50. 50! Thanks!   Ques>ons?