Local development with Docker and traefik.io

A short introduction to what we learned over the last couple of months while developing with Docker, which problems we faced and how we used traefik.io to solved one of them.

Transcript

1. Local development with Docker and traefik.io 2018-06-05, trivago Architecture Meeting Leipzig
2. Email: torsten.heinrich@trivago.com Twitter: https://twitter.com/heinrichtorsten Senior Software Engineer trivago Leipzig Torsten Heinrich
3. Docker
4. !4 Docker • Platform to develop, deploy and run applications with containers • Package software into standardised units for development, shipment and deployment • Containers run natively as a discrete process, sharing the kernel of the host machine • Tool support includes Docker Machine, Compose, Swarm • Available for Linux, macOS and Windows • Available as CE (stable, edge) 18.03 and EE 17.06 • Community-driven and open source, see https://www.opencontainers.org or https://mobyproject.org
5. !5 Docker • The Dockerfile defines the production-ready version of the software • Every Dockerfile defines one ENTRYPOINT • Every entry point defines all the CMD available (serve, develop or migrate) • No shared base images • No separate build containers or multi-stage builds yet • File system performance in macOS can be bad, use cached or delegated flags for volumes to tune configurations
6. !6 Environment variables • Everything that varies between environments is stored in environment variables, see https:// 12factor.net/config • The ARG instruction defines variables at build-time, passed by calling docker build … —build-arg xyz=123 • The ENV instruction defines environment variables, set at run-time by calling docker run … -e xyz=123 • .env.application file in the project folder for production-ready variables • .env file copied to the project folder for local development variables
7. !7 Docker FROM node:8-alpine ENV APP_DIR=/app ENV HOST=0.0.0.0 ENV PORT=3000 ENV NODE_ENV=production ... COPY package.json package-lock.json* ./ RUN npm install --quiet ADD ./docker-entrypoint.sh /docker-entrypoint.sh ENTRYPOINT ["/bin/sh", "/docker-entrypoint.sh"] CMD ["serve"]
8. !8 Problem • How do you run all the dependencies of a service locally? • Do you run a separate stack in AWS? • Do you start them all manually? • Can we simplify this somehow?
9. !9 Docker Compose • The docker-compose.yml file defines the local development environment • Use a separate docker-compose.override.yml file for configuration overrides • Every dependent service defines a separate internal network • Be careful when using depends_on conditions, see https://docs.docker.com/compose/startup-order/ • Later versions are largely similar, currently using version 2.1 to 2.4
10. !10 Docker Compose version: "2.1" services: server: build: . env_file: ./.env.application environment: NODE_ENV: development command: develop networks: - db migrate: ... db: ...
11. !11 AWS Services • Use common protocols to communicate between services (SMTP, AMQP) • AWS offers DynamoDB binaries to run the service locally, see https://amzn.to/2AP5rZd • There’s re-implementations like https://github.com/mhart/kinesalite to replace services • There’s entire stacks like https://github.com/localstack/localstack or https://github.com/awslabs/aws- sam-cli to mock or replace services
12. !12 Problem • What happens if you develop a feature that spans multiple services? • Do you start each of the services locally? • Do you point them at each other by using http://localhost:xyz? • Isn’t there an easier solution?
13. traefik.io
14. !14 traefik.io • An HTTP reverse proxy and load balancer made to deploy microservices • Supports several backends (Docker, Docker Swarm, Kubernetes, Consul, Rancher, Amazon ECS) • Configures itself automatically and dynamically • Supports multiple load balancing algorithms, HTTPS using Let's Encrypt, Circuit breakers and HA cluster mode • Written with ❤ in Go • Open source, see https://github.com/containous/traefik
15. !15 traefik.io • Relies on a external Docker network, created by docker network create trv-studio • Every local service registered in the Docker network is called directly • Every other service is called in AWS through a local Nginx proxy • Requires SSL certificates for traefik.io • The base URL for all the services is the same, e.g. https://studio.dev.trv.cloud
16. !16 traefik.io
17. !17 traefik.io defaultEntrypoints = ["http", "https"] [entryPoints] [entryPoints.http] address = ":80" [entryPoints.http.redirect] entryPoint = "https" [entryPoints.https] address = ":443" [entryPoints.https.tls] [[entryPoints.https.tls.certificates]] certFile = "/certificates/dev.trv.cloud.crt" keyFile = "/certificates/dev.trv.cloud.key" [accessLog] [web] address = ":8080" [docker] domain = "studio.dev.trv.cloud" watch = true
18. !18 traefik.io server { listen 80; server_name ${LOCAL_NAME}.${LOCAL_HOSTED_ZONE_NAME}; proxy_ssl_server_name on; location / { proxy_pass ${HOSTED_ZONE_PROTOCOL}://${BRANCH_NAME}.${HOSTED_ZONE_NAME}; } }
19. !19 traefik.io version: '2.1' services: db: ... labels: traefik.enable: "false" backend: ... labels: traefik.frontend.rule: "PathPrefix:/ar-studio-app-management-api" traefik.docker.network: "trv-studio" environment: BRANCH_NAME: studio APPLICATION_NAME: ar-studio-app-management-api HOSTED_ZONE_PROTOCOL: https HOSTED_ZONE_NAME: dev.trv.cloud
20. !20 traefik.io [ { "Name": "trv-studio", "Scope": "local", "Driver": "bridge", "Containers": { "d8b4e476fac378d115799e8108f9477f61796f28cb5cc8648584d374be2566c3": { "Name": "ar-studio-mvp-proxy_proxy_1", ... }, "fe8b82105d505c597934d98771ca48cf232081d5ebbb424b4bfedb9b8f4f1250": { "Name": "ar-studio-mvp-proxy_platform-proxy_1", ... }, "ad06d2c94b28c338b99f2198ce1ed7f7c6e78e879ba7858792835823a0d1238d": { "Name": "ar-studio-event-api_ar-studio-event-api_1", ... }, "f793299dba566253dc0bcfda9005d960b7d91bf9c4a785fe83278da54d225848": { "Name": "ar-studio-user-details-api_php_1", ... } }, ... } ]
21. Questions?
22. Thank you!