DockerCon 2020 - Simplify all the Things with Docker Compose

As you probably know by now, containers have revolutionized the software industry. But, once you have a container, then what? How do you run it? How do you help someone else run it? There are so many flags and options to remember, ports to configure, volume mappings to remember, and don't even get me started with networking containers together! While it's possible to do all of this through the command line, don't do it that way! With Docker Compose, you can create an easily shareable file that makes all of this a piece of cake. And once you fully adopt containers in your dev environment, it lets you setup your code repos to allow the simplest dev onboarding experience imaginable: 1) git clone; 2) docker-compose up; 3) write code.

In this talk, we'll talk about several tips to help make all of this a reality. We'll start with a few Docker Compose basics, but then quickly move into several advanced topics. We'll even talk about how to use the same Dockerfile for dev and prod (we've all been there by having two separate files)! As an added bonus, we'll look at how to use Docker Compose in our CI/CD pipelines to perform automated tests of the container images built earlier in the pipeline! We'll have a few slides (because we have to explain a few things), lots of live demos (show it in action!), and maybe a few other surprises as well! Let's have some fun and help simplify all the things with Docker Compose!

Transcript

1. Michael Irwin Application Architect, Virginia Tech @mikesir87
2. • Working at Virginia Tech since 2011 ◦ Leading efforts to build a Common Platform ◦ Adjunct Faculty with VT CS Dept since S16 • Community Involvement ◦ Docker Captain and Community Leader ◦ Traefik Ambassador • Husband and dad to four girls with a fifth kiddo on the way! About Me @mikesir87
3. Introducing our Dream Team! Alice Bob @mikesir87
4. Our App! (Look familiar?) @mikesir87
5. TERMINAL # Build and start the app! > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app # Make some changes to app code. Rebuild image. Start up new container. > docker build -t my-app . > docker stop my-app > docker rm my-app > docker run --rm -tip 3000:3000 -n my-app my-app # Open browser to localhost:3000. Check out the changes. # Find it’s still not quite right. Make changes. Rebuild image. Start new container. And so on... Does this sound familiar? @mikesir87
6. Use Docker Compose to replace run commands you’re using over and over. @mikesir87
7. What actually is Docker Compose? @mikesir87
8. Docker Architecture • The Docker Daemon is responsible for: ◦ Pulling images and starting containers ◦ Managing volumes, networks, and DNS • The REST API provides access to the daemon ◦ Available locally at /var/run/docker.sock • The Docker CLI is simply making API requests @mikesir87
9. Docker Compose • Higher-abstraction tool that… ◦ Handles multiple containers at a time ◦ Uses YAML to define an application stack (it can be version controlled!!!!!!!!) ◦ Makes spinning up dev environments a piece of cake! ◦ Is an open specification! @mikesir87
10. Docker Compose Commands • docker-compose ◦ up - spin up all of the services! ◦ down - tear it all down ◦ logs - view all logs in a single stream @mikesir87
11. Back to Alice and Bob... @mikesir87
12. docker-compose.yml version: "3.8" Making our Compose file TERMINAL > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app Start with a version number. Typically just use the latest version @mikesir87
13. docker-compose.yml version: "3.8" services: app: build: ./ Making our Compose file TERMINAL > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app Define a service (a container to run). It’s image will come from the Dockerfile in the current directory @mikesir87
14. docker-compose.yml version: "3.8" services: app: build: ./ ports: - 3000:3000 Making our Compose file TERMINAL > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app Let’s define the port mapping, pulling it directly from the CLI. Compose does offer a long form that’s more descriptive. @mikesir87
15. docker-compose.yml version: "3.8" services: app: build: ./ ports: - 3000:3000 Our Final Compose File TERMINAL > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app @mikesir87
16. TERMINAL # Build and start the app! > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app > docker-compose up --build # Make some changes to app code. Rebuild. Start up. > docker-compose up --build # Open browser to localhost:3000. Check out the changes. # Find it’s still not quite right. Make changes. Rebuild… Updated Workflow @mikesir87
17. Don’t build a container every time you make a change! Where possible, mount source code into a dev container that responds to file changes. @mikesir87
18. Creating Dev Containers • Explore what dev tooling exists for your frameworks • Build a container image to support that dev tooling • Mount in your source code • Victory! @mikesir87
19. docker-compose.yml version: "3.8" services: app: image: node:lts-alpine command: sh -c "npm install && npm run dev" working_dir: /app volumes: - nodemodules:/app/node_modules - ./:/app ports: - 3000:3000 volumes: nodemodules: Creating a Dev Container docker-compose.yml version: "3.8" services: app: build: ./ ports: - 3000:3000 @mikesir87
20. Demo Time! (compose tag) @mikesir87
21. But… now we have two locations describing our environments. Isn’t that a problem? @mikesir87
22. Multi-stage Images to the Rescue! • Multi-stage images are often used to separate build-time and runtime dependencies (copy from a previous stage) • A stage can build on a previous stage • Compose can build an image and specify a target stage @mikesir87
23. Using Multi-Stage Dockerfile Dockerfile FROM node:lts-alpine AS base WORKDIR /app COPY package.json yarn.lock ./ RUN yarn install --production && yarn cache clean COPY src ./src CMD ["node", "/app/src/index.js"] Dockerfile FROM node:lts-alpine AS base WORKDIR /app COPY package.json yarn.lock ./ FROM base AS dev RUN yarn install && yarn cache clean CMD ["npm", "run", "dev"] FROM base AS prod RUN yarn install --production && yarn cache clean COPY src ./src CMD ["node", "/app/src/index.js"] @mikesir87
24. docker-compose.yml version: "3.8" services: app: build: context: . target: dev volumes: - nodemodules:/app/node_modules - ./:/app ports: - 3000:3000 volumes: nodemodules: Using Multi-Stage in Compose docker-compose.yml version: "3.8" services: app: image: node:lts-alpine command: sh -c "npm install && npm run dev" working_dir: /app volumes: - nodemodules:/app/node_modules - ./:/app ports: - 3000:3000 volumes: nodemodules: @mikesir87
25. Demo Time! (compose-multi-stage tag) @mikesir87
26. Thanks! I feel much better about that now! @mikesir87
27. We want to add a database to our app. How? @mikesir87
28. The Twelve-Factor methodology specifies configuration should be stored in the environment around an application. @mikesir87
29. Configuring Apps • There are two main ways to configure an application ◦ Environment variables - great for non-secret data ◦ Files - great for any type of configuration • Often, you see both used together ◦ An env var set that points to the file location @mikesir87
30. Configuring Local Dev • Dev environments should use dummy credentials whenever possible • If you can use dummy credentials, you can… ◦ Create files for each of the configuration options ◦ Mount them into the container @mikesir87
31. docker-compose.yml mysql: image: mysql:5.7 volumes: - ./config:/config - mysql-data:/var/lib/mysql environment: MYSQL_USER_FILE: /config/MYSQL_USER MYSQL_PASSWORD_FILE: /config/MYSQL_PASSWORD MYSQL_DATABASE_FILE: /config/MYSQL_DB MYSQL_RANDOM_ROOT_PASSWORD: "true" volumes: nodemodules: mysql-data: Adding MySQL docker-compose.yml version: "3.8" services: app: build: context: . target: dev volumes: - nodemodules:/app/node_modules - ./:/app - ./config:/config environment: MYSQL_HOST: mysql MYSQL_USER_FILE: /config/MYSQL_USER MYSQL_PASSWORD_FILE: /config/MYSQL_PASSWORD MYSQL_DB_FILE: /config/MYSQL_DB @mikesir87
32. Demo Time! (mysql tag) @mikesir87
33. Great! And now everyone can run it locally without worrying too much! @mikesir87
34. Hi team! I noticed our React app a single file. Can we make it a standard React app? @mikesir87
35. Add other services • Split different toolchains into separate services ◦ Let the backend watch its files and restart ◦ Let webpack watch and transpile the frontend • Use proxies to handle traffic routing ◦ Send API requests to the backend ◦ Let the frontend handle everything else @mikesir87
36. Container-Aware Proxies • Traefik is a great reverse proxy with minimal setup • It configures itself based on the environment • The Docker provider uses labels to provide configuration for what services receive what traffic Frontend Backend @mikesir87
37. Adding the Proxy docker-compose.yml version: "3.8" services: proxy: image: traefik:v2.2 command: --providers.docker ports: - 80:80 volumes: - /var/run/docker.sock:/var/run/docker.sock app: ... labels: traefik.http.routers.backend.rule: Host(`localhost`) && PathPrefix(`/items`) traefik.http.services.backend.loadbalancer.server.port: 3000 @mikesir87
38. Demo Time! (split-frontend tag) @mikesir87
39. Great! This will help us build better apps and hire qualified frontend developers! @mikesir87
40. While we’re at it, can we add some end-to-end tests? @mikesir87
41. Running end-to-end tests • Use the tool(s) that makes sense for your organization ◦ Create container(s) that bundle those tools together ◦ Selenium has images with bundled Firefox/Chrome browsers • You can launch it all with Docker Compose ◦ The --exit-code-from flag will tell Compose to watch one service, tear the stack down when it exits, and pass along the exit code (GREAT for CI builds!!!) @mikesir87
42. Sample Test Architectures App Database Tests @mikesir87
43. Something more sophisticated... Proxy Front End Client Selenium Container w/ Browser API Selenium Tests Database Mock CAS @mikesir87
44. Demo Time! (api-tests tag) @mikesir87
45. Wrapping Up • Docker Compose can be used to simplify dev environments a. git clone b. docker-compose up c. Write code! • With Compose, you can easily add services as needs arise • Compose can be used for automated testing in CI pipelines @mikesir87
46. Thank you! Questions? @mikesir87