Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Docker Container Orchestration Devopscon 2015

Docker Container Orchestration Devopscon 2015

**Docker orchestration talk**

Here you can find my docker slide container decks from my docker orchestration talks:

- [DevOpsCon Berlin 2015 - CONTAINER ORCHESTRATION WITH THE DOCKER ECOSYSTEM ][8].
- open slidefire presentation container with `docker run -d -p 8000:80 rossbachp/docker-orchestration:devopscon-2015`
- pdf [Docker Orchestration][9] is now avaiable!
- [JAX Finance London - CONTAINER ORCHESTRATION WITH THE DOCKER ECOSYSTEM ][6].
- open slidefire presentation container with `docker run -d -p 8000:80 rossbachp/docker-orchestration:jaxfinance-2015`
- pdf [Docker Orchestration][7] is now avaiable!
- [Microservice Meetup Berlin - Docker Orchestration ][3].
- open slidefire presentation container with `docker run -d -p 8000:80 rossbachp/docker-orchestration:mberlin201502`
- pdf [Docker Orchestration][4] is now avaiable!
- [Docker Meetup Frankfurt - Docker Orchestration ][1].
- open slidefire presentation container with `docker run -d -p 8000:80 rossbachp/docker-orchestration:mfrankfurt-2015`
- pdf [Docker Orchestration][2] is now avaiable!

Open slidefire presentation container with your browser `http :8000/docker-orchestration`

Feedback welcome

[Peter Rossbach][5]

[1]: http://www.meetup.com/Docker-Frankfurt/events/219160756/
[2]: https://speakerdeck.com/rossbachp/docker-meetup-frankfurt-2015-docker-orchestration
[3]: http://www.meetup.com/Microservices-Meetup-Berlin/events/219979690/
[4]: https://speakerdeck.com/rossbachp/microservice-meetup-berlin-2015-docker-orchestration
[5]: http://twitter.com/PRossbach
[6]: http://jax-finance.com/2015/session/container-orchestration-docker-ecosystem/
[7]: https://speakerdeck.com/rossbachp/container-orchestration-with-the-docker-ecosystem
[8]: http://devopsconference.de/2015/de/sessions/docker-container-orchestrierung-leicht-gemacht
[9]: https://speakerdeck.com/rossbachp/docker-container-orchestration-devopscon-2015

Peter Rossbach

June 05, 2015
Tweet

More Decks by Peter Rossbach

Other Decks in Programming

Transcript

  1. 5 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Make software faster available Produce higher quality Make your services globally available Improve your service with customer feedback Reduce your cost
  2. 6 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Work together (DevOps) Automate your Delivery Pipeline Use Orchestration Tools Use a DataCenter Operating System Migrate Services to the Cloud Integrate more monitoring into your apps
  3. 10 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Docker works locally and in the cloud with the same binary with the same functionality with the same version with the same filesystem layout with the same dependencies but with different Linux kernels! This dream is now reality... => Infrastructur First
  4. 13 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Use Linux at bare metal, instead a virtual machine provider
  5. 14 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    but there exists a lot of competitors CoreOS Rocket | pure LXC | Ubuntu joyent | SmartOS | ...
  6. 15 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Docker Engine 1.6 Docker Machine 0.2 Docker Swarm 0.2 Docker Compose (fig.sh) 1.2 && 1.3 RC1 Docker Network Docker Plugins with Powerstrip Mesos and Marathon
  7. 16 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Management Docker Host Native Windows Docker Client available Fast Image Loading ( Registry 2.0) Daemon add IPv6 support Resource Limit Management Access Container stats
  8. 19 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Create and manage machines Use libmachine framework for easier driver integration Docker swarm support Regenerate Certs after IP-address is changed Use current Docker 1.6 engine release Docker engine upgrade support First presentation at second docker hackday end of Oct- 2014 Product offering at DockerConf Amsterdam Dec-2014 Second release v0.2 Apr-2015 is avaliable https://github.com/docker/machine
  9. 20 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    https://github.com/docker/machine/releases $ URL=https://github.com/docker/machine/releases/download $ curl -L ${URL}/v0.2.0/docker-machine_darwin-amd64 \ >~/bin/docker-machine_darwin-amd64 $ cd ~/bin $ ln -s docker-machine_darwin-amd64 docker-machine $ chmod +x docker-machine_darwin-amd64 $ export PATH=~/bin:$PATH
  10. 21 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ git clone https://github.com/docker/machine.git $ cd machine $ boot2docker start $ eval $(boot2docker shellinit) $ script/build -osarch="darwin/amd64" $ cp docker-machine_darwin_amd64 ~/bin/ $ cd ~/bin $ ln -s docker-machine_darwin-amd64 docker-machine $ chmod +x docker-machine_darwin-amd64 Crosscompile with golang, inside a build container!
  11. 22 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ cd ~/bin $ docker-machine ls INFO[0000] Creating CA: /Users/peter/.docker/machine/certs/ca.pem INFO[0001] Creating client certificate: /Users/peter/.docker/machine/certs/c NAME ACTIVE DRIVER STATE URL $ ls -a .docker/machine/certs ca-key.pem ca.pem cert.pem key.pem server-key.pem server.pem Build your own docker-machine CA!
  12. 23 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ time docker-machine create -d virtualbox dev INFO[0000] Downloading boot2docker.iso to /Users/peter/.docker/images/boot2d INFO[0019] Creating SSH key... INFO[0020] Creating VirtualBox VM... INFO[0027] Starting VirtualBox VM... INFO[0027] Waiting for VM to start... INFO[0061] "dev" has been created and is now the active machine INFO[0061] To point your Docker client at it, run this in your shell: eval real 1m1.937s user 0m6.904s sys 0m6.231s
  13. 24 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ time docker-machine create -d virtualbox dev2 INFO[0000] Creating SSH key... INFO[0000] Creating VirtualBox VM... INFO[0008] Starting VirtualBox VM... INFO[0008] Waiting for VM to start... INFO[0042] "dev2" has been created and is now the active machine. INFO[0042] To point your Docker client at it, run this in your shell: eval real 0m42.850s user 0m6.031s sys 0m5.960s Use Image-Cache
  14. 25 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ docker-machine ls NAME ACTIVE DRIVER STATE URL dev virtualbox Running tcp://192.168.99.100:2376 dev2 * virtualbox Running tcp://192.168.99.101:2376 $ docker-machine config dev2 --tlsverify --tlscacert="/Users/peter/.docker/machine/machines/dev2/ca.pem" --tlscert="/Users/peter/.docker/m $ docker $(docker-machine config dev) ps Start a Container $ docker $(docker-machine config dev2) run \ --rm busybox /bin/sh -c "echo hello machine $(hostname)" Unable to find image 'busybox:latest' locally busybox:latest: The image you are pulling has been verified 511136ea3c5a: Pull complete df7546f9f060: Pull complete ea13149945cb: Pull complete 4986bf8c1536: Pull complete Status: Downloaded newer image for busybox:latest hello machine 117ade53e1d6 $ docker $(docker-machine config dev2) ps -a REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE busybox latest 4986bf8c1536 5 weeks ago 2.433 MB
  15. 27 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ docker-machine ssh dev ## . ## ## ## == ## ## ## ## ## === /"""""""""""""""""\___/ === ~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ / ===- ~~~ \______ o __/ \ \ __/ \____\_______/ _ _ ____ _ _ | |__ ___ ___ | |_|___ \ __| | ___ ___| | _____ _ __ | '_ \ / _ \ / _ \| __| __) / _` |/ _ \ / __| |/ / _ \ '__| | |_) | (_) | (_) | |_ / __/ (_| | (_) | (__| < __/ | |_.__/ \___/ \___/ \__|_____\__,_|\___/ \___|_|\_\___|_| Boot2Docker version 1.6.0, build master : a270c71 - Thu Apr 16 19:50:36 UTC 2015 Docker version 1.6.0, build 47496 Really easy peasy...
  16. 28 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Works now as proposed... Generating a machine in a minute... A lot of pull requests exists to support more providers CenturyLink, CloudStack Parallels, QCloud, Gandi, Linode...:-) Swarm support Manage your machine infrastructure Simple CLI Missing a machine description file
  17. 30 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    https://github.com/docker/swarm Scheduling docker containers at multiple machines Maintain docker machine cluster Support multiple discovery services Support filtering Open API and first batteries included First presentation at second docker hackday end of October 2014 Product offering at DockerConf Amsterdam 2014 Second Release Apr 2015 Current Release 0.2.0
  18. 33 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ docker pull swarm $ docker run --rm swarm create <SWARM_TOKEN> $ docker run -d -p 3376:2376 swarm manage \ <TLS CONFIG> token://<SWARM_TOKEN> $ docker run -d swarm join \ --addr=<node_ip:2376> token://<cluster_id>
  19. 34 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    docker run docker create docker inspect docker kill docker logs docker start docker info docker ps
  20. 35 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    docker hub swarm discovery simple file or cmdline nodes agent based consul etcd zookeeper http://discovery-stage.hub.docker.com https://github.com/docker/swarm/tree/master/discovery
  21. 36 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Spread Bin Packing Random Rescheduling container is planned! http://en.wikipedia.org/wiki/Bin_packing_problem
  22. 37 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Memory based docker run -m 1g ... CPU CORE based docker run -c 1 ... Port based docker run -p 80:80 ...
  23. 38 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ docker run -c 1 -p 9082:8080 -d tomcat:8.0 35d50a29ed0fde780c0826ef16ff24d2582609f877ee092c780d9aeb6f27fc75 $ docker ps CONTAINER ID IMAGE COMMAND CREATED \ STATUS PORTS NAMES 35d50a29ed0f tomcat:8.0 "catalina.sh run" 16 minutes ago\ Up Less than a second 192.168.33.89:9082->8080/tcp \ dockerhost1/insane_brattain 9c79ad1c4958 nginx:latest "nginx -g 'daemon of 5 hours ago\ Up 5 hours 443/tcp, 192.168.33.90:8080->80/tcp \ dockerhost2/multiple-machine_nginx_1421067200 $ docker run -c 1 -p 9083:8080 -d tomcat:8.0 f5bc6a274e82426df98eaca5e88fb01f43e03455dfebdb9ee4a3b00a7eab202e $ docker ps CONTAINER ID IMAGE COMMAND CREATED\ STATUS PORTS NAMES f5bc6a274e82 tomcat:8.0 "catalina.sh run" 16 minutes ago\ Up Less than a second 192.168.33.90:9083->8080/tcp \ dockerhost2/stupefied_albattani 35d50a29ed0f tomcat:8.0 "catalina.sh run" 17 minutes ago\ Up Less than a second 192.168.33.89:9082->8080/tcp \ dockerhost1/insane_brattain 9c79ad1c4958 nginx:latest "nginx -g 'daemon of 5 hours ago\ Up 5 hours 443/tcp, 192.168.33.90:8080->80/tcp \ dockerhost2/multiple-machine_nginx_1421067200 ...
  24. 39 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    lsb_release -a DOCKER_OPTS="--label region=de-berlin \ --label storage=ssd-vdi \ --label role=loadbalancer" docker -d ${DOCKER_OPTS} --label stage=prod Usage of the label with constraints docker run -e "constraint:operatingsystem==fedora" ... docker run -e "constraint:storagedriver==aufs" ... docker run -e "constraint:region==de-berlin" ... docker run -e "constraint:node!=dev2" ... Provision to loaded image or named container docker run -d -p 80:80 --name front nginx docker run -e "affinity:container==front" logger docker run -e "affinity:image==redis" redis
  25. 40 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Shared volumes: --volumes-from=dependency Links: --link=dependency:alias Shared network stack: --net=container:dependency
  26. 41 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    DOT=XYZ docker pull swarm SWARM_TOKEN=$(docker run --rm swarm create) echo $SWARM_TOKEN >token.txt docker-machine create \ --driver digitalocean \ --digitalocean-region fra1 \ --digitalocean-size 512mb \ --digitalocean-access-token $DOT \ --digitalocean-private-networking \ --swarm \ --swarm-master \ --swarm-discovery token://$SWARM_TOKEN \ jax-swarm docker-machine create \ --driver digitalocean \ --digitalocean-region fra1 \ --digitalocean-size 2gb \ --digitalocean-access-token $DOT \ --digitalocean-private-networking \ --swarm \ --swarm-discovery token://$SWARM_TOKEN \ jax-swarm-001
  27. 42 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Container Scheduler Docker Proxy Supports TLS Up and Running Nice docker-machine swarm support exists Like It!
  28. 44 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Integration with Swarm - multi host support Applications spanning multiple teams/hubs An even better tool for development environments Extend configs
  29. 45 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    command description build Build or rebuild services help Get help on a command kill Kill containers logs View output from containers port Print the public port for a port binding ps List containers pull Pulls service images rm Remove stopped containers run Run a one-off command scale Set number of containers for a service start Start services stop Stop services restart Restart services up Create and start containers
  30. 46 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    common.yml tomcat: image: infrabricks/tomcat:8 ports: - "8080" volumes: - tomcat-users.xml:/opt/tomcat/conf/tomcat-users.xml environment: constraint: zone==dev constraint: disk==ssd SERVICE_8080_NAME: status-http SERVICE_REGION: bee42.1 SERVICE_8080_CHECK_HTTP: /status/ping.jsp SERVICE_8080_CHECK_INTERVAL: 30s
  31. 47 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    status: image: infrabricks/status tomcatblue: extends: file: common.yml service: tomcat volumes_from: - status environment: SERVICE_TAGS: tomcat,blue
  32. 48 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    status: image: infrabricks/status tomcatgreen: extends: file: common.yml service: tomcat volumes_from: - status environment: SERVICE_TAGS: tomcat,green
  33. 49 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    shipyardrethinkdbdata: image: shipyard/rethinkdb entrypoint: /bin/bash command: -l rethinkdb: image: shipyard/rethinkdb expose: - "8080" - "28015" - "29015" volumes_from: - shipyardrethinkdbdata shipyard: image: shipyard/shipyard ports: - "5080:8080" links: - rethinkdb:rethinkdb
  34. 51 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ eval $(docker-machine env dev2) $ docker-compose up -d $ open http://$(docker-machine ip dev2):5080 $ docker run --rm -it shipyard/shipyard-cli Default Credentials: admin/shipyard More about the project http://shipyard-project.com/
  35. 53 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    crane maestro-ng rancher-os shipyard dockerui decking CoreOS fleet Kubernetes Marathon vagrant saltstack ansible puppet chef
  36. 55 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    You can write your own proprietary clients. Add tracking Add your own auth control Add functionality like weave or flocker Split up the big docker daemon project container management network storage and image management ...
  37. 56 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ cat > powerstrip-demo/adapters.yml <<EOF version: 1 endpoints: "POST /*/containers/create": pre: [slowreq] adapters: slowreq: http://slowreq/slowreq-adapter EOF
  38. 57 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    $ docker run -d --name powerstrip-slowreq \ --expose 80 \ clusterhq/powerstrip-slowreq:v0.0.1 $ docker run -d --name powerstrip \ -v /var/run/docker.sock:/var/run/docker.sock \ -v $PWD/powerstrip-demo/adapters.yml:/etc/powerstrip/adapters.yml \ --link powerstrip-slowreq:slowreq \ -p 2375:2375 \ clusterhq/powerstrip:v0.0.1 https://github.com/ClusterHQ/powerstrip-slowreq.git
  39. 58 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Use Local unix socket RPC's to extend go daemon functionality See my powerstrip TLS experiment https://github.com/docker/swarm/pull/347 https://github.com/chanwit/swarm/tree/plugin-system https://github.com/infrabricks/powerstrip-demo
  40. 60 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    ipv6 support at docker 1.5 socketplane weave de-wire pipework flannel openVSwitch https://speakerdeck.com/aschmidt75/docker-networking https://speakerdeck.com/aschmidt75/docker-network- playground http://www.infrabricks.de
  41. 62 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Distributed key value store Service Registry Multi-Datacenter Support ACL HTTP and DNS Discovery support Simple: curl'able user facing API (HTTP+JSON) Secure: optional SSL client cert authentication Reliable: properly distributed using Raft Written in go Use the Find more info at website RAFT Protocol Consul
  42. 63 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Registrator automatically registers/deregisters services for Docker containers published ports published metadata from the container environment Consul (Tags, Healthcheck) Blog explain registrator Github registrator Consul ETCD
  43. 66 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Twitter AirBnB Apple (Siri) Google Cloud Plattform Amazon AWS Digital Ocean Sloppy.io
  44. 67 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Tasks-Scheduler Docker Support Deployment Groups Auto Rescheduler Schedule Groups of Apps Swarm Support planned!
  45. 75 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    View the slides as a docker container $ docker run -d -p 8000:80 rossbachp/docker-orchestration:devopscon-2015 $ open http://<docker host>:8000/docker-orchestration Slides at speakerdeck https://speakerdeck.com/rossbachp Available at 2.06.2015* [email protected] @PRossbach Infrabricks Blog bee42
  46. 76 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    DevOpsCon 1-3.6 2015 - Berlin DockerConf 20-23.6 2015 - San Fransisco DevOpsDays at 2-6.11 WJax 2015 - Munich Find my training offerings at http://www.bee42.com/training/ http://devops-training.de/2015
  47. 80 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    Service benefit Lab: use my IT-brick knowledge to be faster informed Consulting: use my expertise to start smarter Training: let me help you to get a docker-brick expert Products: use my knowledge of the docker ecosystem [email protected]
  48. 82 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    bee42 solutions gmbh starts the implementation of an Infrabricks line! Peter Rossbach DevOps, Docker-enthusiast and infra-coder with passion Systemarchitect Java Developer Apache Tomcat Committer Member of the Apache Software Foundation Founder of the bee42 solutions gmbh Author starts writing the german docker microservice handbook! We're hiring!
  49. 84 / 84 © 2015 <[email protected]>, @PRossbach, DevOpsCon Berlin 2015

    DockerConf Amsterdam 2014 videos https://blog.docker.com/2014/12/dockercon-eu-keynote-on-orchestration/ http://de.slideshare.net/d0cent/orchestrating-docker-containersatscale https://github.com/docker/machine https://github.com/docker/swarm https://github.com/docker/docker/issues/9459 http://de.slideshare.net/Docker/docker-swarm-by-victor-vieux http://blog.docker.com/2014/12/announcing-docker-machine-swarm-and- compose-for-orchestrating-distributed-apps/ https://community.runabove.com/kb/en/instances/docker-in-5-minutes-on- runabove-with-docker-machine.html https://github.com/docker/machine/pull/73 http://blogs.vmware.com/cloudnative/docker-machine-deployment-vm-fusion- vsphere-vcloud-air/ https://github.com/cloudnativeapps/machine/releases/tag/vmw_tech_preview http://technolo-g.com/intro-to-docker-swarm-pt1-overview/ https://github.com/technolo-g/docker-swarm-demo http://technolo-g.com/intro-to-docker-swarm-pt2-config-options-requirements/