Deploy and monitor an application with Docker

How to use docker, docker machine and docker compose to ease the deployment of an application.

Transcript

1. Deploy and monitor an application with Docker Julián Espinel 2016-11-16
2. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
3. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
4. 1. What is Docker? Is an operating-system-level virtualization method for running multiple isolated Linux systems (containers) on a control host using a single Linux kernel.
5. 1. What is Docker? Isolate pid: process net: networking ipc: interprocess communication mnt: filesystem mount points uts: unix timesharing system (kernel and version ids) Namespaces
6. 1. What is Docker? Single Linux kernel Share the part of the operating system responsible for: • memory management • network management • device driver • file management • process management
7. 1. What is Docker?
8. 1. What is Docker? Docker concepts Image Container blueprint
9. 1. What is Docker? Docker concepts Container Isolated workspace Described by an image
10. 1. What is Docker? Docker concepts Registry Place to store and distribute images
11. 1. What is Docker? Docker engine
12. 1. What is Docker?
13. 1. What is Docker? Platform for running multiple isolated systems (containers) on a control host using a single kernel.
14. 1. What is Docker? Code example: Dockerfile FROM ubuntu:latest MAINTAINER Julian Espinel "julianespinel@gmail.com" RUN apt-get update -y RUN apt-get install -y python3-pip python3-dev build-essential COPY . /app WORKDIR /app RUN pip3 install -r requirements.txt EXPOSE 5000
15. 1. What is Docker? Code example: How to run an image from a Dockerfile? Run the following commands in your terminal 1. docker build -t "stockreader:dockerfile" . -> Build the docker image from a Dockerfile (tag the image) 2. docker images -> Check the image has been generated. 3. docker run -it stockreader:dockerfile python3 src/stockreader.py config-standalone.toml -> Run the a docker container based on the image.
16. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
17. 2. What is Docker Machine? Automates • Provisioning of hosts (local or remote) • Instal docker engine on those hosts • Configure docker client to talk to those docker engines
18. 2. What is Docker Machine?
19. Code: Run the following commands in your terminal 1. docker-machine create -d amazonec2 --amazonec2-vpc-id <your-vpc-id> stockreader -> Create a new EC2 instance in AWS. 2. docker-machine env stockreader -> Print export commands. 3. eval $(docker-machine env stockreader) -> point docker machine to the remote node. 2. What is Docker Machine?
20. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
21. 3. What is Docker Compose? Define and run multi-container applications • Define application services in a single file • Manage your application with a single command
22. api: build: . command: python3 src/stockreader.py config-docker.toml ports: - "5000:5000" links: - mongodb - influxdb mongodb: image: mongo:3 ports: - "27017:27017" volumes: - /data/db/:/data/db 3. What is Docker Compose? docker-compose.yml
23. 3. What is Docker Compose? Code: Run the following commands in your terminal 1. docker-compose build -> Build the images. 2. docker-compose up -d -> Run the services.
24. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
25. 4. How to dockerize an application? Three different approaches: 1. Dockerize a single service 2. Dockerize a multi-container application (set of services) 3. Scaling with docker? (out of the scope of this talk)
26. 4. How to dockerize an application? Sample application: Stockreader API MongoDB InfluxDB Application boundary
27. 4. How to dockerize an application? Stockreader: https://github.com/julianespinel/stockreader 1. Single service Start stand alone 1. Install MongoDB 2. Install InfluxDB 3. Install Grafana 4. Install API Start with docker 1. Install MongoDB 2. Install InfluxDB 3. Install Grafana 4. docker build -t "stockreader:dockerfile" . 5. docker run -it stockreader:dockerfile python3 src/stockreader.py config-standalone.toml
28. 4. How to dockerize an application? Stockreader: https://github.com/julianespinel/stockreader 2. Multi-container application Start stand alone 1. Install MongoDB 2. Install InfluxDB 3. Install Grafana 4. Install API dependency libraries Start with docker-compose 1. docker-compose build 2. docker-compose up
29. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
30. 5. How to deploy an application to AWS using Docker? Code: Run the following commands in your terminal 1. eval $(docker-machine env stockreader) -> point docker machine to the remote node. 2. docker-compose build -> build the images you defined. 3. docker-compose up -d -> run the services you defined.
31. Deploy without docker EC2 host OS API MongoDB InfluxDB Grafana 5. How to deploy an application to AWS using Docker? Deploy with docker EC2 host OS Docker engine API MongoDB InfluxDB Grafana
32. 5. How to deploy an application to AWS using Docker? Almost the same, then why use docker?
33. 5. How to deploy an application to AWS using Docker? Almost the same, then why use docker?
34. 5. How to deploy an application to AWS using Docker? Almost the same, then why use docker? Microservices deployment is not trivial, docker compose does not solve this problem but it helps.
35. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
36. 6. How to monitor a dockerized application? Mon·i·tor Verb to watch, observe, listen to, or check (something) for a special purpose over a period of time
37. 6. How to monitor a dockerized application? Two types of monitoring 1. Application monitoring • Is the system dead or alive? Monit • How is the system doing? Collectd + InfluxDB + Grafana 2. Docker container monitoring https://github.com/google/cadvisor
38. 6. How to monitor a dockerized application? 1. Application monitoring Is the system dead or alive? Monit https://mmonit.com/monit
39. 6. How to monitor a dockerized application? 1. Application monitoring Is the system dead or alive? Monit
40. 6. How to monitor a dockerized application? 1. Application monitoring Is the system dead or alive? Monit
41. 6. How to monitor a dockerized application? 1. Application monitoring How is the system doing? Collectd + InfluxDB + Grafana 1. Collectd: daemon to collect system metrics 2. InfluxDB: time series database 3. Grafana: metrics and time series visualization tool Collectd 1 Collectd n InfluxDB Grafana
42. 6. How to monitor a dockerized application? 1. Application monitoring How is the system doing? Collectd + InfluxDB + Grafana http://54.166.202.203:3000/dashboard/db/stockreader-node
43. 6. How to monitor a dockerized application? 2. Docker container monitoring
44. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor (Container Advisor) It is a running daemon that: Information about running containers 1. Collects 2. Aggregates 3. Processes 4. Exports
45. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor (Container Advisor) Web UI
46. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/
47. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
48. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
49. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
50. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
51. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
52. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
53. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
54. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor web UI -> http://localhost:8080/docker/:container-id
55. 6. How to monitor a dockerized application? 2. Docker container monitoring cAdvisor (Container Advisor) REST API
56. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
57. 7. Next steps: how to scale an application in AWS using Docker? What if we need to scale our deployment from 2 nodes to 5 nodes? • How to connect containers between different nodes? What if we need to scale only one (docker compose) service? • How to expose the new service containers to the internet? • Load balancing? What if we need to autoscale our docker deployment? • How should be responsible for monitoring and take action?
58. 7. Next steps: how to scale an application in AWS using Docker? Possible solutions
59. Going back to: then why use docker? Scaling? Imagine just typing: docker scale <service> <instances> 7. Next steps: how to scale an application in AWS using Docker?
60. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
61. 8. Q&A
62. Agenda 1. What is Docker? 2. What is Docker Machine? 3. What is Docker Compose? 4. How to dockerize an application? 5. How to deploy an application to AWS using Docker? 6. How to monitor a dockerized application? 7. Next steps: how to scale an application in AWS using Docker? 8. Q&A 9. References
63. 9. References https://en.wikipedia.org/wiki/LXC https://blog.docker.com/2016/04/docker-engine-1-11-runc/ https://docs.docker.com/engine/understanding-docker/ http://stackoverflow.com/questions/16047306/how-is-docker-different-from-a-normal-virtual-machine https://runc.io/ https://blog.docker.com/2016/04/docker-engine-1-11-runc/ https://speakerdeck.com/anildigital/docker https://techcrunch.com/2016/10/16/wtf-is-a-container/ http://unix.stackexchange.com/questions/254956/what-is-the-difference-between-docker-lxd-and-lxc http://superuser.com/questions/889472/docker-containers-have-their-own-kernel-or-not http://stackoverflow.com/questions/25444099/why-docker-has-ability-to-run-different-linux-distribution
64. 9. References https://blog.risingstack.com/operating-system-containers-vs-application-containers/ https://en.wikipedia.org/wiki/Linux_kernel http://www.makeuseof.com/tag/linux-kernel-explanation-laymans-terms/ https://www.digitalocean.com/community/tutorials/the-docker-ecosystem-an-overview-of-containerization https://www.docker.com/products/docker-machine https://docs.docker.com/machine/reference/ https://docs.docker.com/compose/reference/ http://anders.janmyr.com/2015/03/a-not-very-short-introduction-to-docker.html Monitor. (n.d.). Retrieved November 13, 2016, from http://www.merriam-webster.com/dictionary/monitor https://technologyconversations.com/2016/11/07/collecting-metrics-and-monitoring-the-cluster/