Clocker - The Docker Cloud Maker

Transcript

1. Clocker   The  Docker  Cloud  Maker  

2. Andrew  Kennedy  @grkvlt   Clocker   The  Docker  Cloud  Maker

    

3. @grkvlt   Introduc/on   •  Andrew  Kennedy   –  So6ware

    Engineer   –  Open  Source  and  Distributed  Systems   –  github.com/grkvlt   •  Cloudso6  Corpora/on   –  Open  Source  Applica/on  Management  Specialists   –  We’re  Hiring…  

4. @grkvlt   Agenda   •  Overview   •  Components  

   •  Deep  Dive   •  Demonstra/on   •  Roadmap  

5. Clocker  Overview  

6. @grkvlt   What  does  it  do?   1.  Spins  up

    and  Manages  Docker   Clouds   2.  Serves  up  Containers  on  Demand   3.  Manages  Composite  Applica/on   Deployments  on  Docker  

7. @grkvlt   What  does  it  provide?   •  Mul/  Host

    and  Mul/  Container  Applica/ons   •  Seamless  Networking   – Communica/on  Between  Services   •  Orchestra/on  and  Clustering   – Control  of  Containers   – Container  Management  

8. @grkvlt   Who  is  using  it?   •  Tes/ng  and

    Proof  of  Concept  Stage   –  Financial  Services   –  Insurance   •  Produc/on   –  Mul/-­‐tenant  Applica/on  Trial   –  Container  per  service   –  Via  IBM  BlueMix  

9. @grkvlt   Where  can  I  find  it?   •  Open

    Source   •  Apache  2.0  Licensed   •  hZp://clocker.io     •  Status   •  0.7.0  Beta  Release   •  0.8.0-­‐SNAPSHOT  Development  

10. Clocker  Components  

11. @grkvlt   Clocker  and  Brooklyn   •  What  is  it?

      – Brooklyn  Applica/on   – Brooklyn  Loca/on   •  What  does  it  provide?   – First  Class  Docker  Support  in  Brooklyn   – Op/mized  Brooklyn  Blueprints  for  Docker  

12. Apache  Brooklyn   •  Apache  Brooklyn   – Open  Source  

    – Java   – Donated  to  the  Apache  So6ware  Founda/on   – ASF  Incubator  Status  

13. @grkvlt   Apache  Brooklyn   •  Applica/on  Management  Pla^orm  

    •  Deploy,  Manage  and  Monitor  Blueprints   •  Provisioning,  Installa/on  and  Customiza/on   •  Management   – AutoScaling,  Resilience,  Performance,  Security  

14. Apache  jclouds   •  Java  Cloud  Library   •  API

     Agnos/c   – CloudStack,  OpenStack,  AWS  EC2,  GCE…   •  Create  Virtual  Machines   – Return  SSH  Endpoint   – Manage  Proper/es  

15. @grkvlt   Apache  jclouds   •  Drivers  for  REST  APIs

      •  Docker  Driver   –  WriZen  by  @turlinux   •  Virtual  Container   –  Using  SSH  Daemon   –  Same  Endpoint  Type  as  VM   –  Composi/on  on  any  Image  or  Dockerfile  

16. @grkvlt   Docker  Networking  101   •  Separate  Container  Network

      •  RFC  1918  Private  Addresses   •  Non  Routable   •  Bridged  to  Host   •  Port  Forwarding   •  Mul/ple  Endpoints  for  Services  

17. Docker  Topology  

18. @grkvlt   So6ware  Defined  Networking   •  Overlay  Networks  

    •  Kernel  or  User  Space   •  Many  Op/ons   –  Pipework   –  SocketPlane   –  Flannel   –  Open  vSwitch   –  Weave  

19. Weave  Topology  

20. @grkvlt   Weave   •  Ethernet  Switch   –  User

     Space   –  Docker  Container   •  Mesh  Topology   •  Routes  TCP  Traffic   –  Sniffs  on  Host   –  Forwards  over  TCP  

21. Weave  

22. Clocker  Deep  Dive  

23. Clocker  Architecture  

24. Clocker  Architecture  

25. @grkvlt   Clocker  Features   •  Applica/on  Deployment   – 

    Oasis  CAMP  YAML  Blueprint   –  TOSCA  in  Development   –  Same  as  Core  Brooklyn   •  Mixed  Des/na/ons   –  Some  Virtual  Machines   –  Some  Bare  Metal   –  Some  Containers  

26. @grkvlt   Clocker  Features   •  Docker  Extensions   – 

    Container  or  Image   –  Placement  Strategy   –  Dockerfiles   •  Manages  Docker   –  Cloud  Deployment   –  Configura/on   –  Run/me  Control  

27. @grkvlt   Container  Management   •  Sources   –  Docker

     Image  Defini/on   –  Docker  Hub   –  Dockerfile   –  Brooklyn  En/ty  Defini/on   •  Create  Image  Automa/cally   –  Commit  or  Push  for  Reuse  

28. @grkvlt   Container  Management   •  Installa/on  of  Services  

    –  Defined  by  Brooklyn  or  Dockerfile   –  Common  to  all  En/ty  Instances   •  Commit  Image   –  Available  for  next  En/ty   •  Push  Image   –  Available  for  all  Hosts  

29. Container  Management   id:  dockerfile-­‐mysql   name:  "Docker  Hub  MySQL

     Application"   origin:  "https://registry.hub.docker.com/_/mysql/"   locations:   -­‐  my-­‐docker-­‐cloud   services:   -­‐  serviceType:  brooklyn.entity.container.docker.application.DockerfileApplication      id:  mysql      name:  "MySQL"      brooklyn.config:          docker.dockerfile.url:              "https://s3-­‐eu-­‐west-­‐1.amazonaws.com/brooklyn-­‐clocker/mysql-­‐5.6.tgz"          docker.container.environment:              MYSQL_ROOT_PASSWORD:  "s3cr3t”    

30. @grkvlt   Container  Placement   •  Demand   –  Adding

     an  Applica/on   –  Scaling  exis/ng  Applica/on   •  Requirements   –  Host  Loca/on   –  Service  Resources   –  CPU,  Memory    

31. @grkvlt   Container  Placement   •  Supply   – Choose  a

     Host  from  available   – Create  new  Host   •  Start  Container   –  Set  CPU  and  Memory   –  AZach  to  Network  

32. @grkvlt   Container  Placement   •  Placement  Strategies   – 

    Filter  Hosts   –  Sort  Hosts   •  Informa/on  from   –  Docker   –  Underlying  Machine   –  Cloud  Environment   –  En/ty  or  Service  

33. @grkvlt   Container  Placement   •  Placement  Strategies   – 

    Random,  Depth  or  Breadth  First   –  CPU  or  Memory  Usage   –  Memory,  CPU  or  Container  Limits   –  Geographic  Constraints   •  User  Defined   –  Java  Predicate  

34. Placement  Strategy   •  Determinis/c   •  Simple   – Predicate

     and  Comparator   docker.container.strategies:      -­‐  $brooklyn:object:              type:  "brooklyn.location.docker.strategy.MaxContainersPlacementStrategy”              brooklyn.config:                  maxContainers:  16      -­‐  $brooklyn:object:              type:  "brooklyn.location.docker.strategy.CpuUsagePlacementStrategy”              brooklyn.config:                  maxCpu:  0.75  

35. @grkvlt   Container  Provisioning   •  Provisioning  Strategy   – 

    New  Docker  Host  Loca/on   •  Constraints   –  Docker  Infrastructure  Constraints   –  En/ty  or  Applica/on  Constraints   •  User  Defined  Strategies   •  Intelligent  Container  Orchestra/on  

36. @grkvlt   Container  Provisioning   •  Headroom   –  Ensure

     resources  available   –  Based  on  MaxContainers  strategy  limit   –  Or  CPU  and  RAM  alloca/on   •  Scale  Docker  Host  Cluster  Automa/cally     –  Add  new  Docker  hosts   –  Remove  empty  Docker  hosts  

37. @grkvlt   Networking   •  Shared  Weave  LAN   – Common

     to  All  Containers   – Private  (Link  Local)  Addresses   •  Clocker  Controls  IP  Alloca/on   – Applica/ons  Segmented  by  CIDR   •  Docker  Port  Forwarding  Access  

38. @grkvlt   Networking   •  Work  in  Progress   • 

    Name  Resolu/on   – BIND  and  DNSmasq  or  Weave  DNS   – Needed  for  JMX  et  al   – Enables  Many  More  En//es   •  Adding  More  Supported  Blueprints  

39. Demonstra>on  

40. None
41. None
42. None

43. @grkvlt   Roadmap   •  Latest  Docker  Feature  Support  

    •  Swarm   –  Same  API   –  Can  re-­‐use  our  jclouds  Driver   •  Machine   –  Cloud  agnos/c  provisioning  via  jclouds  already   –  Could  support  as  an  op/on  

44. @grkvlt   Roadmap   •  Networking   – DNS  and  DNSmasq

     Integra/on   – Rancher  Integra/on   – Open  vSwitch  Integra/on   •  Gelng  Started   – Clocker  Dockerfile  

45. @grkvlt   Roadmap   •  Repositories   –  Docker  Repository

      –  Bintray,  Ar/factory,  Quay.io   –  Private  Repositories   •  Applica/on  Defini/on   –  TOSCA   –  Fig  

46. @grkvlt   Roadmap   •  Plugin  API  may  be  helpful?

      •  Provide  services  to  Docker  this  way   •  Orchestra/on   •  Policies   •  Consume  wider  range  of  services   •  Networking  

47. @grkvlt   Summary   •  Clocker   –  Brooklyn  +

     jclouds  +  Docker  +  Weave  +  …   –  @clockercentral   •  Solves   –  Composite  Applica/on  Management   –  Docker  Cloud  Networking   –  Container  Placement  and  Provisioning  

48. @grkvlt   Audience  Ques/ons?   1.  What  networking  features  would

      make  Docker  easier  to  use?   2.  What  other  features  would  be  most   useful  to  enhance  Docker  usability?   3.  Will  you  be  tes/ng  Swarm  and   Machine  or  would  you  prefer  an   external  orchestra/on  tool?  

49. @grkvlt   Web  Resources   http://clocker.io/   http://brooklyn.io/   http://docker.io/

      http://github.com/zettio/weave/   http://abstractvisitorpattern.co.uk/  

50. 50! Thanks!   Ques>ons?