Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Clocker - The Docker Cloud Maker

Clocker - The Docker Cloud Maker

Talk at DockerCon Europe, Amsterdam, December 2014

Avatar for Andrew Kennedy

Andrew Kennedy

December 05, 2014
Tweet

More Decks by Andrew Kennedy

Other Decks in Technology

Transcript

  1. @grkvlt   Introduc/on   •  Andrew  Kennedy   –  So6ware

     Engineer   –  Open  Source  and  Distributed  Systems   –  github.com/grkvlt   •  Cloudso6  Corpora/on   –  Open  Source  Applica/on  Management  Specialists   –  We’re  Hiring…  
  2. @grkvlt   Agenda   •  Overview   •  Components  

    •  Deep  Dive   •  Demonstra/on   •  Roadmap  
  3. @grkvlt   What  does  it  do?   1.  Spins  up

     and  Manages  Docker   Clouds   2.  Serves  up  Containers  on  Demand   3.  Manages  Composite  Applica/on   Deployments  on  Docker  
  4. @grkvlt   What  does  it  provide?   •  Mul/  Host

     and  Mul/  Container  Applica/ons   •  Seamless  Networking   – Communica/on  Between  Services   •  Orchestra/on  and  Clustering   – Control  of  Containers   – Container  Management  
  5. @grkvlt   Who  is  using  it?   •  Tes/ng  and

     Proof  of  Concept  Stage   –  Financial  Services   –  Insurance   •  Produc/on   –  Mul/-­‐tenant  Applica/on  Trial   –  Container  per  service   –  Via  IBM  BlueMix  
  6. @grkvlt   Where  can  I  find  it?   •  Open

     Source   •  Apache  2.0  Licensed   •  hZp://clocker.io     •  Status   •  0.7.0  Beta  Release   •  0.8.0-­‐SNAPSHOT  Development  
  7. @grkvlt   Clocker  and  Brooklyn   •  What  is  it?

      – Brooklyn  Applica/on   – Brooklyn  Loca/on   •  What  does  it  provide?   – First  Class  Docker  Support  in  Brooklyn   – Op/mized  Brooklyn  Blueprints  for  Docker  
  8. Apache  Brooklyn   •  Apache  Brooklyn   – Open  Source  

    – Java   – Donated  to  the  Apache  So6ware  Founda/on   – ASF  Incubator  Status  
  9. @grkvlt   Apache  Brooklyn   •  Applica/on  Management  Pla^orm  

    •  Deploy,  Manage  and  Monitor  Blueprints   •  Provisioning,  Installa/on  and  Customiza/on   •  Management   – AutoScaling,  Resilience,  Performance,  Security  
  10. Apache  jclouds   •  Java  Cloud  Library   •  API

     Agnos/c   – CloudStack,  OpenStack,  AWS  EC2,  GCE…   •  Create  Virtual  Machines   – Return  SSH  Endpoint   – Manage  Proper/es  
  11. @grkvlt   Apache  jclouds   •  Drivers  for  REST  APIs

      •  Docker  Driver   –  WriZen  by  @turlinux   •  Virtual  Container   –  Using  SSH  Daemon   –  Same  Endpoint  Type  as  VM   –  Composi/on  on  any  Image  or  Dockerfile  
  12. @grkvlt   Docker  Networking  101   •  Separate  Container  Network

      •  RFC  1918  Private  Addresses   •  Non  Routable   •  Bridged  to  Host   •  Port  Forwarding   •  Mul/ple  Endpoints  for  Services  
  13. @grkvlt   So6ware  Defined  Networking   •  Overlay  Networks  

    •  Kernel  or  User  Space   •  Many  Op/ons   –  Pipework   –  SocketPlane   –  Flannel   –  Open  vSwitch   –  Weave  
  14. @grkvlt   Weave   •  Ethernet  Switch   –  User

     Space   –  Docker  Container   •  Mesh  Topology   •  Routes  TCP  Traffic   –  Sniffs  on  Host   –  Forwards  over  TCP  
  15. @grkvlt   Clocker  Features   •  Applica/on  Deployment   – 

    Oasis  CAMP  YAML  Blueprint   –  TOSCA  in  Development   –  Same  as  Core  Brooklyn   •  Mixed  Des/na/ons   –  Some  Virtual  Machines   –  Some  Bare  Metal   –  Some  Containers  
  16. @grkvlt   Clocker  Features   •  Docker  Extensions   – 

    Container  or  Image   –  Placement  Strategy   –  Dockerfiles   •  Manages  Docker   –  Cloud  Deployment   –  Configura/on   –  Run/me  Control  
  17. @grkvlt   Container  Management   •  Sources   –  Docker

     Image  Defini/on   –  Docker  Hub   –  Dockerfile   –  Brooklyn  En/ty  Defini/on   •  Create  Image  Automa/cally   –  Commit  or  Push  for  Reuse  
  18. @grkvlt   Container  Management   •  Installa/on  of  Services  

    –  Defined  by  Brooklyn  or  Dockerfile   –  Common  to  all  En/ty  Instances   •  Commit  Image   –  Available  for  next  En/ty   •  Push  Image   –  Available  for  all  Hosts  
  19. Container  Management   id:  dockerfile-­‐mysql   name:  "Docker  Hub  MySQL

     Application"   origin:  "https://registry.hub.docker.com/_/mysql/"   locations:   -­‐  my-­‐docker-­‐cloud   services:   -­‐  serviceType:  brooklyn.entity.container.docker.application.DockerfileApplication      id:  mysql      name:  "MySQL"      brooklyn.config:          docker.dockerfile.url:              "https://s3-­‐eu-­‐west-­‐1.amazonaws.com/brooklyn-­‐clocker/mysql-­‐5.6.tgz"          docker.container.environment:              MYSQL_ROOT_PASSWORD:  "s3cr3t”    
  20. @grkvlt   Container  Placement   •  Demand   –  Adding

     an  Applica/on   –  Scaling  exis/ng  Applica/on   •  Requirements   –  Host  Loca/on   –  Service  Resources   –  CPU,  Memory    
  21. @grkvlt   Container  Placement   •  Supply   – Choose  a

     Host  from  available   – Create  new  Host   •  Start  Container   –  Set  CPU  and  Memory   –  AZach  to  Network  
  22. @grkvlt   Container  Placement   •  Placement  Strategies   – 

    Filter  Hosts   –  Sort  Hosts   •  Informa/on  from   –  Docker   –  Underlying  Machine   –  Cloud  Environment   –  En/ty  or  Service  
  23. @grkvlt   Container  Placement   •  Placement  Strategies   – 

    Random,  Depth  or  Breadth  First   –  CPU  or  Memory  Usage   –  Memory,  CPU  or  Container  Limits   –  Geographic  Constraints   •  User  Defined   –  Java  Predicate  
  24. Placement  Strategy   •  Determinis/c   •  Simple   – Predicate

     and  Comparator   docker.container.strategies:      -­‐  $brooklyn:object:              type:  "brooklyn.location.docker.strategy.MaxContainersPlacementStrategy”              brooklyn.config:                  maxContainers:  16      -­‐  $brooklyn:object:              type:  "brooklyn.location.docker.strategy.CpuUsagePlacementStrategy”              brooklyn.config:                  maxCpu:  0.75  
  25. @grkvlt   Container  Provisioning   •  Provisioning  Strategy   – 

    New  Docker  Host  Loca/on   •  Constraints   –  Docker  Infrastructure  Constraints   –  En/ty  or  Applica/on  Constraints   •  User  Defined  Strategies   •  Intelligent  Container  Orchestra/on  
  26. @grkvlt   Container  Provisioning   •  Headroom   –  Ensure

     resources  available   –  Based  on  MaxContainers  strategy  limit   –  Or  CPU  and  RAM  alloca/on   •  Scale  Docker  Host  Cluster  Automa/cally     –  Add  new  Docker  hosts   –  Remove  empty  Docker  hosts  
  27. @grkvlt   Networking   •  Shared  Weave  LAN   – Common

     to  All  Containers   – Private  (Link  Local)  Addresses   •  Clocker  Controls  IP  Alloca/on   – Applica/ons  Segmented  by  CIDR   •  Docker  Port  Forwarding  Access  
  28. @grkvlt   Networking   •  Work  in  Progress   • 

    Name  Resolu/on   – BIND  and  DNSmasq  or  Weave  DNS   – Needed  for  JMX  et  al   – Enables  Many  More  En//es   •  Adding  More  Supported  Blueprints  
  29. @grkvlt   Roadmap   •  Latest  Docker  Feature  Support  

    •  Swarm   –  Same  API   –  Can  re-­‐use  our  jclouds  Driver   •  Machine   –  Cloud  agnos/c  provisioning  via  jclouds  already   –  Could  support  as  an  op/on  
  30. @grkvlt   Roadmap   •  Networking   – DNS  and  DNSmasq

     Integra/on   – Rancher  Integra/on   – Open  vSwitch  Integra/on   •  Gelng  Started   – Clocker  Dockerfile  
  31. @grkvlt   Roadmap   •  Repositories   –  Docker  Repository

      –  Bintray,  Ar/factory,  Quay.io   –  Private  Repositories   •  Applica/on  Defini/on   –  TOSCA   –  Fig  
  32. @grkvlt   Roadmap   •  Plugin  API  may  be  helpful?

      •  Provide  services  to  Docker  this  way   •  Orchestra/on   •  Policies   •  Consume  wider  range  of  services   •  Networking  
  33. @grkvlt   Summary   •  Clocker   –  Brooklyn  +

     jclouds  +  Docker  +  Weave  +  …   –  @clockercentral   •  Solves   –  Composite  Applica/on  Management   –  Docker  Cloud  Networking   –  Container  Placement  and  Provisioning  
  34. @grkvlt   Audience  Ques/ons?   1.  What  networking  features  would

      make  Docker  easier  to  use?   2.  What  other  features  would  be  most   useful  to  enhance  Docker  usability?   3.  Will  you  be  tes/ng  Swarm  and   Machine  or  would  you  prefer  an   external  orchestra/on  tool?  
  35. @grkvlt   Web  Resources   http://clocker.io/   http://brooklyn.io/   http://docker.io/

      http://github.com/zettio/weave/   http://abstractvisitorpattern.co.uk/