Butter bei die Fische. Ein Jahr Entwicklung und Produktion mit Docker

Vor etwa einem Jahr wurde bei der JUGH die damals sehr junge Technologie Docker vorgestellt, Basics und Start gezeigt. In der Zwischenzeit haben Johannes Unterstein und Patrick Busch ein Jahr intensiv mit Docker gearbeitet und auch Anwendungen damit in Produktion geschoben.

In diesen Slides teilen sie ihre Erkenntnisse: wie sie Docker einsetzen und welche positiven und negativen Erfahrungen sie dabei bereits gemacht haben.

Dabei gehen sie auf sinnvolle Anordnung von Docker-Befehlen ein, auf sinnvolle Docker-Registries, auf Staging und Verlinkung von Containern über Hardwaregrenzen hinweg, auf Continuous Deployment und all das andere lustige Zeug, was sie so mit Docker machen.

Transcript

1. BUTTER BEI DIE FISCHE Ein Jahr Entwicklung und Produktion mit Docker Johannes Unterstein und Patrick Busch
2. AGENDA How can we make a single tenant system suitable for multitenancy and scalable without changing the whole system?
3. AGENDA • Introduction • Dockerizing the Application • Dockerizing the Infrastructure • Best Practices • Lessons Learned
4. INTRODUCTION • One existing application • Have it available for several different legal entities • No way to implement multitenancy the existing application
5. INTRODUCTION SOAP SQL STORED PROCEDURES NO CACHING flens/web flens/burg flens/db
6. INTRODUCTION SOAP SQL STORED PROCEDURES NO CACHING SOAP SQL STORED PROCEDURES NO CACHING SOAP SQL STORED PROCEDURES NO CACHING etc. PROXY A B C
7. DOCKERIZING THE APPLICATION
8. VARIANT 1
9. VARIANT 1 • Pros • Easy to understand/build/run/host • Cons • Separation • Scalability • Updates
10. VARIANT 2
11. VARIANT 2 • Pros • Scalability • Separation • Updates • Cons • Advanced connection between containers needed
12. VARIANT 2 • Connection between containers • Not via docker linkage • Via /etc/host entry and environment variable • Interpreting startup shell script in container
13. VARIANT 2 docker run -dP --env flensburgHost=someHost --env flensburgPort=1234 flens/web:1.23
14. DOCKERIZING THE INFRASTRUCTURE
15. VARIANT A
16. VARIANT A FRONTEND MW/MANAGER REGISTRY EXECUTOR
17. VARIANT A FRONTEND MW/MANAGER REGISTRY EXECUTOR builds the image stores the image runs the container
18. VARIANT A • Classic approach • Running applications on the metal • Physical servers, each needs to be configured for the application • One server that runs the application containers
19. VARIANT B
20. VARIANT B • More flexibility • Every physical server is basically the same • Installation done via script in a few minutes each • Containers can then be run on any server • Images contain all the needed configuration
21. VARIANT C
22. VARIANT C *n
23. VARIANT C • Multiple servers for the application containers • Better load distribution • Improved security
24. VARIANT D PROXY *n PROXY PROXY ? FIREWALL FIREWALL FIREWALL
25. VARIANT D • Configure containers to point to proxies • Proxies manage certificates • Proxies pass through containers • Allows multiple containers per system to run in parallel while they can be addressed on their own
26. FULL SCALE *n BUILD IMAGES PROXY PROXY FIREWALL FIREWALL FIREWALL
27. FULL SCALE *n BUILD IMAGES STORE IMAGES PUSH PROXY PROXY FIREWALL FIREWALL FIREWALL
28. RUN CONTAINERS FULL SCALE *n BUILD IMAGES STORE IMAGES PUSH PULL *n *n *n PROXY PROXY FIREWALL FIREWALL FIREWALL
29. ADDED BENEFITS • Self Service • Flexibility • Scalability • Security • A/B-Switching
30. STAGING *n PUSH PULL *n PUSH PULL PRODUCTION STAGE
31. STAGING *n PUSH PULL *n PUSH PULL PRODUCTION STAGE SHARED REGISTRY FOR INFRASTRUCTURE IMAGES
32. STAGING • Easily duplicated environment • Use docker registry for infrastructure images • Release versioned images • Script checks that versions cannot be overwritten • Stage first approach
33. BEST PRACTICES
34. COMMON BASE IMAGES • Common stuff in common base image • As much as possible in base image • Define versions of tools explicitly • Lowers registry size
35. GROUP COMMANDS • Try to combine commands with „&&“ • Less intermediate containers • Increases build performance • Lowers registry size
36. GROUP COMMANDS RUN chmod u+x /home/app/start.sh RUN chown app:app /home/app/start.sh
37. GROUP COMMANDS RUN chmod u+x /home/app/start.sh && \ chown app:app /home/app/start.sh
38. ORDER COMMANDS • Stable commands as early as possible • ADD commands as late as possible • Caching increases build performance • Lowers registry size
39. USE SCRIPTS docker run -d \ --read-only \ -p 127.0.0.1:30022:22 \ -p 127.0.0.1:38080:8080 \ -v /docker/data/nginx:/var/lib/nginx \ -v /docker/logs/nginx:/var/log/nginx \ -v /docker/tmp:/tmp \ -v /docker/run:/var/run \ --name flens_web \ repository_host_name:8888/flens/web:1.0
40. USE SCRIPTS flens web run 1.0
41. USE SCRIPTS • Running containers can be complicated on the console • Scripts can improve readability and memorability • Improved speed and less failures • Reusability
42. BUILD CONTINUOUS • Use scripts in continuous integration server as well • We use „Execute shell command“ jobs • e.g.: flens web build && flens web rerun
43. USE PROXIES • Proxy on the physical machines (e.g. nginx) • Containers listen only to localhost device • Nginx handles incoming requests and passes on • Nginx handles security • More than one container of a given type • By symlinking nginx config files you can switch from one slot to another
44. USE VOLUMES • Volumes are directories mounted from the physical host • Files in a volume are visible from inside the container (and writeable) • Useful for logging, syncing data, etc…
45. READ-ONLY CONTAINERS • A read only container cannot write to its own file system • Can only write to volumes • Perfectly immutable containers are easily interchangable! • Build and distribute containers even more freely • No unexpected states, defined income -> defined outcome
46. MAKE YOUR CONTAINERS FLEXIBLE • Use /etc/hosts defined hostnames instead of IP addresses • Use environment variables at startup (--env)
47. LESSONS LEARNED
48. QUIRKS OF DOCKERFILES • COPY vs ADD • ADD can be a URL, ADD extracts tar.gz files automatically • ENTRYPOINT vs CMD • CMD can be overwritten at startup, ENTRYPOINT cannot • Both are possible in a single Dockerfile • ENTRYPOINT/CMD syntax • determines if the executable is started directly or in a shell
49. QUIRKS OF DOCKERFILES • COPY vs ADD • ADD can be a URL, ADD extracts tar.gz files automatically • ENTRYPOINT vs CMD • CMD can be overwritten at startup, ENTRYPOINT cannot • Both are possible in a single Dockerfile - this combines them! • ENTRYPOINT/CMD syntax • determines if the executable is started directly or in a shell
50. CMD AND ENTRYPOINT CMD ping localhost => /bin/sh -c ‘ping localhost’ CMD[“ping”,”localhost”] => ping localhost ENTRYPOINT[“ping”] CMD [“localhost”] => ping localhost $ docker run container_name www.flens.de => ping www.flens.de
51. TRUST YOUR OWN SKILLS • Young technology, many tutorials, everybody else knows it better • Linking is fine, but not for us • Configuring /etc/hosts at startup works wonders • Try to use your own solution
52. DON’T USE LINKAGE • Not possible over real machine boundaries • Often leads to problems during startup • Use /etc/hosts and environment parameters
53. DOCKER IN DOCKER • Our infrastructure builds docker images dynamically • Our infrastructure is dockerized • Do we need „docker in docker?“
54. DOCKER IN DOCKER •Docker in docker is possible • docker run -- privileged flens/mw:1.23 •Container runs inside flens/mw •Problems during update of outer app
55. DOCKER IN DOCKER •We used client/server docker communication •Client = flens/mw •Server = Docker of host system •Similar to boot2docker •All container runs on host system
56. IT’S CHEAPER • We can use off the shelf servers • We can use virtualized servers • We can distribute easily over different server providers • Easily scalable
57. IT’S BETTER • Release on touch of a button • Deployment on touch of a button • Transparent versioning of all apps • Transparency of OS environment running the apps • Environment is now part of dev process and versionable
58. THANKS Cheers