[deSymfony] Docker on every environment

Transcript

4. Containers. How do we transport containers from development to production?
5. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.
6. Docker provides a beautiful API that hides the real complexity.
7. Why do we call them containers?
8. ISO standards for containers were published between 1968 and 1970.
9. Software containers try to be the standard for Applications distribution Applications runtime
10. Development Workflow Dev Prod CI / CD
11. CI / CD Development Workflow Dev Prod
13. Distribution: Container Images
14. Packaging applications for distribution is not a new problem.
15. ❏ Disk images .iso ❏ VMware .vdmk ❏ Vagrant .box ❏ Amazon Machine Images AMI Systems Packaging
16. ❏ zip/tgz ❏ Java jar/war ❏ Debian deb ❏ RedHat rpm Application Packaging
17. Container images combine both system and application packaging. Old best practices still apply.
18. Container images can be easily built using manifests.
19. Building the same manifest twice could produce different images.
20. The difference between how you think something works and how it actually works risks hard-to-debug production issues. Gareth Rushgrove @garethr
22. ❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container
23. ❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container
24. Operating System Which Alpine? FROM  alpine CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]
25. Operating System Is this better? FROM  alpine:3.4 CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]
26. Operating System Tags can be overwritten! 3.4 won’t be the same in two weeks, probably FROM  alpine:3.4 CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]
27. ❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container
28. Packages Which pip? FROM  alpine:3.4 RUN  apk  add  -­‐-­‐update  py-­‐pip CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]
29. Versions Specify the version FROM  alpine:3.4 RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 CMD  [“echo”,  “Knock”,  ”Knock”,  “Neo”]
30. ❏ Which OS is it based on? ❏ Which packages are installed? ❏ What application is running inside? Giving a running container
31. Application Which version of our application? FROM  alpine:3.4 RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 COPY  app.py  /app.py CMD  [“python”,  “/app.py”]
32. Metadata Use Docker Labels for application metadata FROM  alpine:3.4 ARG  vcs_ref="Unknown" ARG  build_date="Unknown" RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 LABEL  org.label-­‐schema.vcs-­‐ref=$vcs_ref  \ org.label-­‐schema.build-­‐date=$build_date COPY  app.py  /app.py CMD  [“python”,  “/app.py”]
33. Metadata Use Docker Labels for application metadata FROM  alpine:3.4 ARG  vcs_ref="Unknown" ARG  build_date="Unknown" RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 LABEL  org.label-­‐schema.vcs-­‐ref=$vcs_ref  \ org.label-­‐schema.build-­‐date=$build_date COPY  app.py  /app.py CMD  [“python”,  “/app.py”]
34. Standard for Docker labels
35. Use labels to extract info
36. Metadata Use Docker Labels for application metadata FROM  alpine:3.4 ARG  vcs_ref="Unknown" ARG  build_date="Unknown" RUN  apk  add  -­‐-­‐update  py-­‐pip=8.1.2-­‐r0 LABEL  org.label-­‐schema.vcs-­‐ref=$vcs_ref  \ org.label-­‐schema.build-­‐date=$build_date COPY  app.py  /app.py CMD  [“python”,  “/app.py”]
37. Metadata Calculate the values for the labels $  docker  build  \ -­‐-­‐build-­‐arg  vcs_ref=`git  rev-­‐parse  HEAD`  \ -­‐-­‐build-­‐arg  date=`date  -­‐u  +  "%Y-­‐%m-­‐%dT%H:%MZ"`  \ -­‐t  your_image_name  .
38. Open Source Docker Registries
39. Docker Hub
40. Official Docker Registry
41. Harbor (VMware)
42. Port.us (Suse)
43. Paid Docker Registries
44. Docker DataCenter
45. AWS ECR
46. JFrog Artifactory
47. Development Workflow Dev CI / CD Prod
48. Building our images
49. Building the same manifest twice could produce different images.
50. Build once. Promote images to different environments.
51. Jenkins Workflow 1. Detect merge to repository
52. Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry
53. Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry 3. Deploy to pre environment
54. Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry 3. Deploy to pre environment 4. If tests pass, push image to pro registry
55. Jenkins Workflow 1. Detect merge to repository 2. If tests pass, build image and push it to pre production registry 3. Deploy to pre environment 4. If tests pass, push image to pro registry 5. Deploy to production
56. Keep In Mind ❏ Be clear on which versions of docker/docker-compose you allow ❏ Use Jenkins build number or timestamp as image tag ❏ Seek a Generic Build process ❏ Clean old images/containers
57. Clean old images/containers
58. CI / CD Development Workflow Dev Prod
59. Container Runtime
60. Container vs VM comparison
61. It’s better to think about containers as regular unix processes
62. Run only one process
63. ❏ Easier debugging ❏ Simplified setup of networks, ports... ❏ Logs have only one format ❏ Resources sharing: CPU vs memory Running one process
64. Processes inside containers are started and stopped the same way a unix system would do it.
65. ❏ How they start? ❏ How they stop? Containers and processes share a similar lifecycle
66. Unix Processes: the chosen PID1 (the One, got it? :D)
67. When you run the container, the init (PID1) process is started, which is responsible for starting other processes, creating a tree-like hierarchy.
68. $  ps -­‐e  -­‐o  user,pid,ppid,args -­‐-­‐forest UID   PID   PPID   CMD root   1   0   init root   205   1 /sbin/udevd -­‐-­‐daemon root    1113    205 \_  /sbin/udevd -­‐-­‐daemon root    1114    205   \_  /sbin/udevd -­‐-­‐daemon root    1199   1 crond -­‐f  -­‐d  8 root    1216   1 VBoxService root    1241   1 /usr/local/sbin/acpid root    1249   1 /sbin/udhcpc -­‐b  -­‐i eth1...
69. Every process has a parent process, except for the init process.
70. When a process ends, its entry in the process table remains, keeping the process in a defunct or zombie status.
71. Only after the parent read the child’s exit status to know what happened, the zombie is removed. This is called reaping.
72. The init process also adopts orphans. An orphan is a process that is still executing but whose parent has died.
74. If your process is running as PID1, it’s probably expecting a init process to properly adopting and reaping processes.
75. Running our process as a subcommand of bash would solve this problem. But bash doesn’t handle unix signals.
76. Unix Signals
77. A signal is an asynchronous notification sent to a process in order to notify it of an event that occurred.
78. SIGINT Interrupt from keyboard SIGKILL Kill process immediately SIGTERM Request nice process termination
79. Signals sent by the Docker engine to the containers are handled by the init process.
80. Unless a process has registered a custom signal handler for SIGTERM, the kernel will fall back sending a SIGKILL signal.
81. For PID1, though, the kernel won’t even fall back to SIGKILL. If your process hasn’t registered a handler, SIGTERM will have no effect on the process.
83. By default, docker stop sends a SIGTERM and waits 10 seconds for the container to stop. After that, it sends a SIGKILL.
84. nginx expects a SIGQUIT to do a graceful shutdown. Apache expects a SIGWINCH.
85. Options for your container entrypoint ❏ shell form
86. Options for your container entrypoint ❏ shell form ❏ exec form
87. Options for your container entrypoint ❏ shell form ❏ exec form ❏ Using a proper init process
88. shell form ❏ ENTRYPOINT  command  param1  param2 ❏ The process will be started as a subcommand of /bin/sh -c ❏ Remember, bash does not pass signals
89. exec form ❏ ENTRYPOINT  ["executable",  "param1"] ❏ The process will be PID1 init process inside the container ❏ Your process should adopt and take care of reaping
90. Using a proper init process ❏ A init process that pass signals, adopt orphans and takes care of reaping. ❏ There are several init processes for containers ❏ tini ❏ dumb-init
91. Demo
92. Wrapping Up
93. We need formal pipelines to promote images from development to production. Build once and promote.
94. Without a proper init system ❏ Zombie processes could become a problem in the system. ❏ Signals are not passed to your process as you may expect, not being able to gracefully stop a process.
95. We need to understand the container lifecycle to deploy those images.
96. Your Kubernetes cluster on raspberry pi is not production ready.
98. @fiunchinho Schibsted Spain Jose Armesto