Rex Roof - Docker Weight Loss Tips

DevOps Days GDL 2020 - February 20th

Transcript

1. Docker Weight Loss Tips
   Rex Roof
   Blue Newt Software
   @rexroof
   github.com/rexroof/docker-weight-loss
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   

   View Slide

2. Rex Roof
   I come from the land of ice and snow.
   I think about, read about, construct, and eat good food.
   Using Unix for 25 years.
   Platform Architect at Blue Newt Software
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   

   View Slide

3. Blue Newt Software
   Startup Studio - we help you create modern software.
   Working with auto manufacturers, sports leagues, agriculture.
   We also build and sell our own software.
   I get to start new projects a few times a year.
   (and see the same mistakes)
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   

   View Slide

4. What is a Docker container?
   A way to isolate and package software.
   A method for wrapping up a single process into a deployable object.
   Every container should have a single responsibility.
   A Dockerfile defines a container with a series of instructions.
   Docker containers are stored in filesystem layers.
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   

   View Slide

5. Why worry about the size?
   - Speed
   - Quicker CI/CD
   - faster development
   - Security
   - fewer attack surfaces
   - Storage
   - Cost, especially at scale.
   - Also, speed.
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   

   View Slide

6. Docker image layers
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   FROM ubuntu
   COPY cmatrix.tar .
   RUN tar -xf cmatrix.tar
   RUN apk add build-base
   RUN ./configure && make
   RUN rm -f cmatrix.tar
   RUN apk del build-base
   $ docker history 31daae369c43
   #(nop) COPY file:312de99 2.03MB
   tar -xf cmatrix.tar 1.87MB
   apk add build-base 216MB
   ./configure && make 442kB
   rm -f cmatrix.tar 0B
   apk del build-base 22.4kB
   ## TOTAL: 226MB
   

   View Slide

7. Cached layers are your friend
   They don’t need to be built again when you’re developing.
   They don’t need to be uploaded to a registry.
   They don’t need to be stored in the registry.
   They don’t need to be downloaded on your deployment server.
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   

   View Slide

8. Making better use of layers
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   COPY cmatrix.tar .
   RUN tar -xf cmatrix.tar \
   && apk add build-base \
   && ./configure && make \
   && rm -f cmatrix.tar \
   && apk del build-base
   $ docker history 31daae369c43
   #(nop) COPY file:312de9 2.03MB
   tar -xf cmatrix.tar && 6.28MB
   ## Total: 13Mb
   

   View Slide

9. Python example that requires compiler
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   ●
   FROM python:3.8-slim
   WORKDIR /app
   COPY requirements.txt .
   RUN apt-get update \
   && apt-get install -y build-essential libpq-dev \
   && pip install -r requirements.txt \
   && apt-get remove -y build-essential libpq-dev \
   && apt-get -y autoremove && apt-get clean
   COPY . .
   CMD [ "python", "start.py" ]
   

   View Slide

10. Choose a smaller base image
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    python:latest 933MB
    python:slim 193MB
    python:alpine 109MB
    ruby:latest 842MB
    ruby:slim 149MB
    ruby:alpine 54MB
    

    View Slide

11. Choose a smaller base image
    Start small, install what you need.
    Alpine is small, but use with caution. libc and DNS issues can arise.
    Spend the time early on to find the right container base and iterate.
    You have tests, right?
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    

    View Slide

12. Use .dockerignore to exclude files
    - Can save space in your image
    - Keep secrets out of your containers
    - Invalidate the cache less frequently
    $ cat .dockerignore
    # always ignore .git
    .git*
    node_modules
    build
    .env
    .secrets
    .cache
    .aws
    Dockerfile
    *debug.log*
    # editor files
    [._].swp
    .vscode
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    

    View Slide

13. Leverage multi-stage builds
    - Define multiple docker containers in one Dockerfile
    - Builds each in order, using the final container in the image
    - Copy files between containers
    - You can target individual containers when building
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    

    View Slide

14. Using multi-stage builds
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    FROM golang:alpine as build
    COPY hello.go .
    RUN go build -o hello hello.go
    # above is ~350MB
    FROM alpine as alpine-run
    COPY --from=build /go/hello /hello
    CMD ["/hello"]
    # above is ~7MB
    - Only your target or final stage is
    saved in your image.
    - Each layer still caches locally
    - Can target higher layers using
    docker build --target=NAME
    

    View Slide

15. Using multi-stage builds
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    # development container
    FROM node:12.14.1-alpine as development
    WORKDIR /app
    ENV HOME /app
    COPY yarn.lock package.json /app/
    RUN npm install
    COPY public/ /app/public/
    COPY src/ /app/src/
    EXPOSE 3000
    CMD ["npm", "start"]
    # temp build container
    FROM development as build
    ARG NODE_ENV=production
    ARG GENERATE_SOURCEMAP=false
    RUN npm run build
    # production container
    FROM nginx:1.17.8-alpine as production
    COPY --from=build /app/build /usr/share/nginx/html
    

    View Slide

16. Learn your language packaging options
    PYTHONDONTWRITEBYTECODE=1 pip install --no-cache-dir
    python: don’t write cache, don’t create python byte code
    npm ci --production
    nodejs: install only production packages, no dev dependencies
    bundle install --deployment
    ruby: roll out packages for CI or production use
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    

    View Slide

17. Learn your OS install options
    apt-get install --no-install-suggests -y
    ubuntu/debian: don’t install suggested packages
    apk --no-cache add
    alpine: skip read/write of local package cache
    yum clean all || apt-get clean all
    centos/debian/ubuntu: clean up package repo cache files
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    

    View Slide

18. The Docker mindset
    No longer setting up long-running servers
    Thinking more about the full lifecycle.
    Automating your software setup.
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    

    View Slide

19. Takeaways
    - Make your containers smaller
    - Rearrange your Dockerfile to optimize layers
    - Review your .dockerignore files
    - Respect the layer cache
    - Your container is not a VM
    github.com/rexroof/docker-weight-loss
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    

    View Slide

20. Thanks!
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    ●
    github.com/rexroof/docker-weight-loss
    @rexroof
    gdl@rexroof.com
    

    View Slide