Upgrade to Pro — share decks privately, control downloads, hide ads and more …

DockerCon 2020 - Simplify all the Things with Docker Compose

DockerCon 2020 - Simplify all the Things with Docker Compose

As you probably know by now, containers have revolutionized the software industry. But, once you have a container, then what? How do you run it? How do you help someone else run it? There are so many flags and options to remember, ports to configure, volume mappings to remember, and don't even get me started with networking containers together! While it's possible to do all of this through the command line, don't do it that way! With Docker Compose, you can create an easily shareable file that makes all of this a piece of cake. And once you fully adopt containers in your dev environment, it lets you setup your code repos to allow the simplest dev onboarding experience imaginable: 1) git clone; 2) docker-compose up; 3) write code.

In this talk, we'll talk about several tips to help make all of this a reality. We'll start with a few Docker Compose basics, but then quickly move into several advanced topics. We'll even talk about how to use the same Dockerfile for dev and prod (we've all been there by having two separate files)! As an added bonus, we'll look at how to use Docker Compose in our CI/CD pipelines to perform automated tests of the container images built earlier in the pipeline! We'll have a few slides (because we have to explain a few things), lots of live demos (show it in action!), and maybe a few other surprises as well! Let's have some fun and help simplify all the things with Docker Compose!

0a0f7378a8339f07ccc09b4485c003d1?s=128

Michael Irwin

May 28, 2020
Tweet

Transcript

  1. Michael Irwin Application Architect, Virginia Tech @mikesir87

  2. • Working at Virginia Tech since 2011 ◦ Leading efforts

    to build a Common Platform ◦ Adjunct Faculty with VT CS Dept since S16 • Community Involvement ◦ Docker Captain and Community Leader ◦ Traefik Ambassador • Husband and dad to four girls with a fifth kiddo on the way! About Me @mikesir87
  3. Introducing our Dream Team! Alice Bob @mikesir87

  4. Our App! (Look familiar?) @mikesir87

  5. TERMINAL # Build and start the app! > docker build

    -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app # Make some changes to app code. Rebuild image. Start up new container. > docker build -t my-app . > docker stop my-app > docker rm my-app > docker run --rm -tip 3000:3000 -n my-app my-app # Open browser to localhost:3000. Check out the changes. # Find it’s still not quite right. Make changes. Rebuild image. Start new container. And so on... Does this sound familiar? @mikesir87
  6. Use Docker Compose to replace run commands you’re using over

    and over. @mikesir87
  7. What actually is Docker Compose? @mikesir87

  8. Docker Architecture • The Docker Daemon is responsible for: ◦

    Pulling images and starting containers ◦ Managing volumes, networks, and DNS • The REST API provides access to the daemon ◦ Available locally at /var/run/docker.sock • The Docker CLI is simply making API requests @mikesir87
  9. Docker Compose • Higher-abstraction tool that… ◦ Handles multiple containers

    at a time ◦ Uses YAML to define an application stack (it can be version controlled!!!!!!!!) ◦ Makes spinning up dev environments a piece of cake! ◦ Is an open specification! @mikesir87
  10. Docker Compose Commands • docker-compose ◦ up - spin up

    all of the services! ◦ down - tear it all down ◦ logs - view all logs in a single stream @mikesir87
  11. Back to Alice and Bob... @mikesir87

  12. docker-compose.yml version: "3.8" Making our Compose file TERMINAL > docker

    build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app Start with a version number. Typically just use the latest version @mikesir87
  13. docker-compose.yml version: "3.8" services: app: build: ./ Making our Compose

    file TERMINAL > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app Define a service (a container to run). It’s image will come from the Dockerfile in the current directory @mikesir87
  14. docker-compose.yml version: "3.8" services: app: build: ./ ports: - 3000:3000

    Making our Compose file TERMINAL > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app Let’s define the port mapping, pulling it directly from the CLI. Compose does offer a long form that’s more descriptive. @mikesir87
  15. docker-compose.yml version: "3.8" services: app: build: ./ ports: - 3000:3000

    Our Final Compose File TERMINAL > docker build -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app @mikesir87
  16. TERMINAL # Build and start the app! > docker build

    -t my-app . > docker run --rm -tip 3000:3000 -n my-app my-app > docker-compose up --build # Make some changes to app code. Rebuild. Start up. > docker-compose up --build # Open browser to localhost:3000. Check out the changes. # Find it’s still not quite right. Make changes. Rebuild… Updated Workflow @mikesir87
  17. Don’t build a container every time you make a change!

    Where possible, mount source code into a dev container that responds to file changes. @mikesir87
  18. Creating Dev Containers • Explore what dev tooling exists for

    your frameworks • Build a container image to support that dev tooling • Mount in your source code • Victory! @mikesir87
  19. docker-compose.yml version: "3.8" services: app: image: node:lts-alpine command: sh -c

    "npm install && npm run dev" working_dir: /app volumes: - nodemodules:/app/node_modules - ./:/app ports: - 3000:3000 volumes: nodemodules: Creating a Dev Container docker-compose.yml version: "3.8" services: app: build: ./ ports: - 3000:3000 @mikesir87
  20. Demo Time! (compose tag) @mikesir87

  21. But… now we have two locations describing our environments. Isn’t

    that a problem? @mikesir87
  22. Multi-stage Images to the Rescue! • Multi-stage images are often

    used to separate build-time and runtime dependencies (copy from a previous stage) • A stage can build on a previous stage • Compose can build an image and specify a target stage @mikesir87
  23. Using Multi-Stage Dockerfile Dockerfile FROM node:lts-alpine AS base WORKDIR /app

    COPY package.json yarn.lock ./ RUN yarn install --production && yarn cache clean COPY src ./src CMD ["node", "/app/src/index.js"] Dockerfile FROM node:lts-alpine AS base WORKDIR /app COPY package.json yarn.lock ./ FROM base AS dev RUN yarn install && yarn cache clean CMD ["npm", "run", "dev"] FROM base AS prod RUN yarn install --production && yarn cache clean COPY src ./src CMD ["node", "/app/src/index.js"] @mikesir87
  24. docker-compose.yml version: "3.8" services: app: build: context: . target: dev

    volumes: - nodemodules:/app/node_modules - ./:/app ports: - 3000:3000 volumes: nodemodules: Using Multi-Stage in Compose docker-compose.yml version: "3.8" services: app: image: node:lts-alpine command: sh -c "npm install && npm run dev" working_dir: /app volumes: - nodemodules:/app/node_modules - ./:/app ports: - 3000:3000 volumes: nodemodules: @mikesir87
  25. Demo Time! (compose-multi-stage tag) @mikesir87

  26. Thanks! I feel much better about that now! @mikesir87

  27. We want to add a database to our app. How?

    @mikesir87
  28. The Twelve-Factor methodology specifies configuration should be stored in the

    environment around an application. @mikesir87
  29. Configuring Apps • There are two main ways to configure

    an application ◦ Environment variables - great for non-secret data ◦ Files - great for any type of configuration • Often, you see both used together ◦ An env var set that points to the file location @mikesir87
  30. Configuring Local Dev • Dev environments should use dummy credentials

    whenever possible • If you can use dummy credentials, you can… ◦ Create files for each of the configuration options ◦ Mount them into the container @mikesir87
  31. docker-compose.yml mysql: image: mysql:5.7 volumes: - ./config:/config - mysql-data:/var/lib/mysql environment:

    MYSQL_USER_FILE: /config/MYSQL_USER MYSQL_PASSWORD_FILE: /config/MYSQL_PASSWORD MYSQL_DATABASE_FILE: /config/MYSQL_DB MYSQL_RANDOM_ROOT_PASSWORD: "true" volumes: nodemodules: mysql-data: Adding MySQL docker-compose.yml version: "3.8" services: app: build: context: . target: dev volumes: - nodemodules:/app/node_modules - ./:/app - ./config:/config environment: MYSQL_HOST: mysql MYSQL_USER_FILE: /config/MYSQL_USER MYSQL_PASSWORD_FILE: /config/MYSQL_PASSWORD MYSQL_DB_FILE: /config/MYSQL_DB @mikesir87
  32. Demo Time! (mysql tag) @mikesir87

  33. Great! And now everyone can run it locally without worrying

    too much! @mikesir87
  34. Hi team! I noticed our React app a single file.

    Can we make it a standard React app? @mikesir87
  35. Add other services • Split different toolchains into separate services

    ◦ Let the backend watch its files and restart ◦ Let webpack watch and transpile the frontend • Use proxies to handle traffic routing ◦ Send API requests to the backend ◦ Let the frontend handle everything else @mikesir87
  36. Container-Aware Proxies • Traefik is a great reverse proxy with

    minimal setup • It configures itself based on the environment • The Docker provider uses labels to provide configuration for what services receive what traffic Frontend Backend @mikesir87
  37. Adding the Proxy docker-compose.yml version: "3.8" services: proxy: image: traefik:v2.2

    command: --providers.docker ports: - 80:80 volumes: - /var/run/docker.sock:/var/run/docker.sock app: ... labels: traefik.http.routers.backend.rule: Host(`localhost`) && PathPrefix(`/items`) traefik.http.services.backend.loadbalancer.server.port: 3000 @mikesir87
  38. Demo Time! (split-frontend tag) @mikesir87

  39. Great! This will help us build better apps and hire

    qualified frontend developers! @mikesir87
  40. While we’re at it, can we add some end-to-end tests?

    @mikesir87
  41. Running end-to-end tests • Use the tool(s) that makes sense

    for your organization ◦ Create container(s) that bundle those tools together ◦ Selenium has images with bundled Firefox/Chrome browsers • You can launch it all with Docker Compose ◦ The --exit-code-from flag will tell Compose to watch one service, tear the stack down when it exits, and pass along the exit code (GREAT for CI builds!!!) @mikesir87
  42. Sample Test Architectures App Database Tests @mikesir87

  43. Something more sophisticated... Proxy Front End Client Selenium Container w/

    Browser API Selenium Tests Database Mock CAS @mikesir87
  44. Demo Time! (api-tests tag) @mikesir87

  45. Wrapping Up • Docker Compose can be used to simplify

    dev environments a. git clone b. docker-compose up c. Write code! • With Compose, you can easily add services as needs arise • Compose can be used for automated testing in CI pipelines @mikesir87
  46. Thank you! Questions? @mikesir87