Building a Docker Based Platform at Pantalk

My talk at the Pantalk meetup on June 18th in Zurich, Switzerland about building a Docker based platform and its advantages. Highlighting service discovery with Consul and routing with nginx.

https://www.cloudgear.net
http://www.meetup.com/de/Pantalks-tech-non-tech-talks-Panter-AG-Zurich/events/223096569/

Transcript

1. Building a Docker Based Platform Georg Kunz, CloudGear 18. June 2015
2. About Me Ruby and infrastructure freelancer and founder of CloudGear, previous web architect at Local.ch Interests: Rails/Ruby/Go/Cloud/Docker/PaaaS/Infrastructure Web: http://georgkunz.com https://www.cloudgear.net Twitter: @geku Email: georg@cloudgear.net
3. Why? a Platform CC BY-SA 2.0 http://bit.ly/1GgCb8H
4. Because ...it’s hip! ...it’s fun to build!
5. Reality Check What we would like to ship What we have to ship Images by Derell Licht https://flic.kr/p/pnSzL licensed under CC BY-ND 2.0 and Chris https://flic.kr/p/5TtPFZ licensed under CC BY 2.0
6. standardize simplifies operations
7. speed up time from dev to production CC BY 2.0 http://bit.ly/1L2dEdY
8. • microservices • polyglot landscape • continuous delivery Buzzwords CC BY 2.0 http://bit.ly/1IQNpVK
9. How? CC BY 2.0 http://bit.ly/1Gu9SqD
10. First Things First We still need servers CC BY 2.0 http://bit.ly/1Guan42
11. Container Runtime
12. Isolation, isolation, isolation Immutability Container Runtime CC BY 2.0 http://bit.ly/1epJ7uu
13. Run Container $ docker run -d -p 5000 geku/go-app:0.1 d79e4898f8afc9e4de85c1fa8aafe70f7069dff44f9f42511b7b9c73fb9777ad $ docker ps CONTAINER ID IMAGE COMMAND PORTS d79e4898f8af geku/go-app:0.1 "/app/go-app" 0.0.0.0:49154->5000/tcp $ curl localhost:49154/json {"hostname":"d79e4898f8af","env":["HOSTNAME=d79e4898f8af","HOME=/root"]}
14. Routing • Expose services on port 80/443 • Domain based routing • Load balancing (HTTP & TCP) CC BY 2.0 http://bit.ly/1MNLEcC
15. Port 80 Node a.com container c.com container Routing Port 80 Node A a.com container b.com container Internet Node B
16. Port 80 Node a.com 34205 c.com 33789 Routing Port 80 Node A a.com 32781 b.com 36491 Node B nginx reverse proxy config: upstream a-com { server nodeA:32781; server nodeB:34205; } server { listen 80; server_name a.com www.a.com; location / { proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For ... proxy_set_header Host $http_host; proxy_pass http://a-com; } }
17. Discovery Inventory and Source of truth Provides lookup table CC BY 2.0 http://bit.ly/1CeVe2C
18. Discovery a.com: nodeA:32506 nodeB:33555 nodeB:34912 nodeC:38144 b.com: nodeA:36707 .... Node Node A a.com container b.com container 0.com container Node Node B a.com container a.com container 0.com container Node Node C e.com container a.com container f.com container register/deregister container
19. Consul $ http PUT 192.168.59.103:8500/v1/agent/service/register \ ID=redis.1 Name=redis Address="192.168.59.103" Port:=9500 $ http 192.168.59.103:8500/v1/catalog/services { "consul": [], "redis": [] } $ http 192.168.59.103:8500/v1/catalog/service/redis [ { "Address": "192.168.59.103", "Node": "n1", "ServiceID": "redis.1", "ServiceName": "redis", "ServicePort": 9500, }, { "Address": "192.168.59.103", "Node": "n1", "ServiceID": "redis.2", "ServiceName": "redis", "ServicePort": 8500, } ] 1. register 2. list services 3. list instances
20. Routing & Discovery Node Node A a.com container b.com container b.com container 2. register new container 1. add container for b.com watch services backend 1 3. send event “new instance” 4. backend 2 gets added
21. Deploy Pipeline • git push or callback • Docker build with Dockerfile • Image storage • Instance updates CC BY 2.0 http://bit.ly/1QFnF5E
22. Deploy Pipeline GIT client repo 1. git push Builder docker build ... docker push image bare GIT repo Docker Registry Repository 2. build Image 3. push image to registry Node A a.com v1 0.com Node B a.com v1 0.com a.com v2 Image 4. replace running instances
23. Recap Port 80 Node Registrator c.com container Port 80 Node A Registrator b.com container Internet Node B GIT repo Builder Node C Docker Registry Developer
24. • Images • use few base images (security updates) • Understand what’s inside! • Start today step by step, e.g. • Consul • Dev or Staging with Docker & compose CC BY 2.0 http://bit.ly/1eqbk4j
25. Final Words You made it! CC BY 2.0 http://bit.ly/1eqdwsI
26. Final Words • Ask me questions anytime • Please follow @cloudgear_net or sign up on the mailing list • Support us with open source development CC BY 2.0 http://bit.ly/1eqdwsI
27. ? CC BY 2.0 http://bit.ly/1JWCVae
28. Resources • Docker Compose (https://github.com/docker/compose) • Consul (https://www.consul.io/) • Registrator (https://github.com/gliderlabs/registrator) • gitreceive (https://github.com/progrium/gitreceive) • CloudGear (https://www.cloudgear.net)
29. Backup Slides
30. Clustering • Nodes are aware of each other • Use Consul • runs agent on every node • health checks CC BY 2.0 http://bit.ly/1Im3WhE
31. Consul Features • Key/Value store: • Persist service attributes and definition • Bindings, e.g. router • Health Checks • Node or service level