Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Effective Images- DevOps Days TLV

E5b21a91ebcda83d51e9d59fdc0883cc?s=47 Abby Fuller
April 27, 2018
81

Effective Images- DevOps Days TLV

E5b21a91ebcda83d51e9d59fdc0883cc?s=128

Abby Fuller

April 27, 2018
Tweet

Transcript

  1. Creating Effective Images Abby Fuller, Sr Technical Evangelist, AWS @abbyfuller

  2. Agenda • How do layers work? • The basics for

    building minimal images • High level best practices for Windows containers • Dockerfiles: the good, the bad, and the bloated • Let’s get (language) specific • Tools are here to help • Looking forward to the future
  3. Read-only base layers Thin read-write layer What are container layers?

  4. Why do I care how many layers I have? •

    More layers mean a larger image. The larger the image, the longer that it takes to both build, and push and pull from a registry. • Smaller images mean faster builds and deploys. This also means a smaller attack surface.
  5. OK, so how can I reduce my layers? • Sharing

    is caring. • Use shared base images where possible • Limit the data written to the container layer • Chain RUN statements • Prevent cache misses at build for as long as possible
  6. Minimal images: the basics

  7. A Dockerfile is a series of instructions for building an

    image.
  8. C.R.E.A.M: cache rules everything around me CACHE

  9. Let’s start with a Dockerfile FROM ubuntu:latest LABEL maintainer abbyfull@amazon.com

    RUN apt-get update -y && apt-get install -y python- pip python-dev build-essential COPY . /app WORKDIR /app RUN pip install –r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  10. First step: choosing the right base From the stock ubuntu

    image: ubuntu latest 2b1dc137b502 52 seconds ago 458 MB From python:2.7-alpine: alpine latest d3145c9ba1fa 2 minutes ago 86.8 MB
  11. Slightly better: choose a different distro alpine latest d3145c9ba1fa 3.9

    MB python 3.6.1-slim 60952d8b5aeb 200 MB debian latest 8cedef9d7368 123 MB python 2.7-alpine b63d02d8829b 71.5 MB ubuntu latest 0ef2e08ed3fa 130 MB fedora latest 4daa661b467f 231 MB
  12. When do I want a full base OS? I really

    do like Ubuntu! • Security • Compliance • Ease of development
  13. Let’s look at our original Ubuntu image FROM ubuntu:latest RUN

    apt-get update -y && apt-get install -y python- pip python-dev build-essential LABEL maintainer abbyfull@amazon.com COPY . /app WORKDIR /app RUN pip install –r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  14. Simple changes, big results FROM python:2.7-alpine LABEL maintainer abbyfull@amazon.com COPY

    . /app WORKDIR /app RUN pip install –r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  15. Fewer cache invalidations=smaller images FROM python:2.7-alpine LABEL maintainer abbyfull@amazon.com COPY

    requirements.txt /app RUN pip install –r /app/requirements.txt COPY . /app WORKDIR /app EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  16. Let’s recap TL;DR: Layers represent filesystem differences. Layers add up

    quickly, with big consequences.
  17. Some high-level best practices: Windows

  18. Port over existing VM workloads Convert an existing Windows image:

    ConvertTo-Dockerfile -ImagePath c:\docker\myimage.wim Convert from VHD: ConvertTo-Dockerfile -ImagePath c:\vms\test.vhd -Artifact IIS - ArtifactParam windows-container -OutputPath c:\windows-container cd c:\windows-container docker build -t windows-container . docker run -d -p 80:80 windows-container
  19. Some things to think about Watch what you build: c:

    c:\ / /windows c:/windows Building any of those PATHs will make your image very large!
  20. Avoid installing packages with MSI MSI installations are not space

    efficient. This is not the same as Linux distros, where you can add, use, and remove the installation files! $ Windows/Installer/<package>.msi Windows saves these files for uninstalls :(
  21. Coming up soon! • Run Windows containers “as is” on

    Windows Server
  22. Here’s what’s really cool though • Build and run everything

    the same, regardless of container OS, host OS, or tools. Just docker build and docker run.
  23. Dockerfiles: the good, the bad, and the bloated

  24. Let’s start out big FROM ubuntu:latest LABEL maintainer abbyfull@amazon.com RUN

    apt-get update -y RUN apt-get install -y python-pip python-dev build- essential COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  25. A little bit better FROM ubuntu:latest LABEL maintainer abbyfull@amazon.com RUN

    apt-get update -y && apt-get install -y python- pip python-dev build-essential –no-install- recommends COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  26. Let’s try a different base FROM python:2.7-alpine LABEL maintainer abbyfull@amazon.com

    COPY . /app WORKDIR /app RUN pip install -r requirements.txt EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  27. Or, let’s try a custom base container FROM 621169296726.dkr.ecr.us-east- 1.amazonaws.com/dockercon-base:latest

    LABEL maintainer abbyfull@amazon.com COPY . /app WORKDIR /app EXPOSE 5000 ENTRYPOINT ["python"] CMD ["application.py"]
  28. Use RUN statements effectively RUN apt-get update && apt-get install

    -y \ aufs-tools \ automake \ build-essential \ ruby1.9.1 \ ruby1.9.1-dev \ s3cmd=1.1.* \ && rm -rf /var/lib/apt/lists/*
  29. Switching USER adds layers RUN groupadd –r dockercon && useradd

    –r –g dockercon dockercon USER dockercon RUN apt-get update && apt-get install -y \ aufs-tools \ automake \ build-essential USER root COPY . /app
  30. Avoid ADDing large files BAD: ADD http://cruft.com/bigthing.tar.xz /app/cruft/ RUN tar

    -xvf /app/cruft/bigthing.tar.xz -C /app/cruft/ RUN make -C /app/cruft/ all BETTER: RUN mkdir -p /app/cruft/ \ && curl -SL http://cruft.com/bigthing.tar.xz \ | tar -xJC /app/cruft/ && make -C /app/cruft/ all
  31. BEST RUN mkdir -p /app/cruft/ \ && curl -SL http://cruft.com/bigthing.tar.xz

    \ | tar -xvf /app/cruft/ \ && make -C /app/cruft/ all && \ rm /app/cruft/bigthing.tar.xz
  32. Let’s get (language) specific

  33. A few language-specific best practices Use the right tool: not

    every language needs to be built the same way. • Where possible, use two images: one to build an artifact, and one from base • Official language images can be huge: more space effective to use a more minimal image, but there are tradeoffs
  34. First stop: Golang Compile, then COPY binary: $ go build

    -o dockercon . $ docker build -t dockercon . Dockerfile: FROM scratch COPY ./dockercon /dockercon ENTRYPOINT ["/dockercon"]
  35. Wait, what’s SCRATCH? Special, empty Dockerfile. Use this to build

    your own base images. Or, use to build minimal images that run a binary and nothing else: FROM scratch COPY hello / CMD [ “/hello” ] Want more on scratch? Start here.
  36. Back to business: Ruby Official images for Ruby are extra

    huge. A new base + a little extra work pays off. FROM alpine:3.2 LABEL maintainer abbyfull@amazon.com RUN apk update && apk upgrade && apk add \ curl \ bashruby \ ruby-dev \ ruby-bundler RUN rm -rf /var/cache/apk/*
  37. Next: node.js If you love yourself, .dockerignore npm-debug.log. Seriously. But

    most importantly, cache node_modules: COPY package.json . RUN npm install --production COPY . . This way, only run npm install if package.json changes.
  38. Java! Multi-stage builds are your friend: FROM maven:3.5-jdk-8 as BUILD

    COPY --from=BUILD Like Golang, this let’s you build an artifact in one stage, and simply run the binary in the second stage, resulting in more minimal final images. More on multistage builds up next.
  39. Multi-stage builds FROM ubuntu AS build-env RUN apt-get install make

    ADD . /src RUN cd /src && make And for the second Dockerfile, copy from #1: FROM busybox COPY --from=build-env /src/build/app /usr/local/bin/app EXPOSE 80 ENTRYPOINT /usr/local/bin/app
  40. Tools are here to help

  41. None
  42. Answer: tools! • Security, scalablity, resiliency should be your top

    priorities • Lean on tools to help so you can spend less time fiddling, and more time building awesome applications.
  43. Docker security scan

  44. Docker Security Scan

  45. Docker image + system prune Docker image prune: $ docker

    image prune –a Alternatively, go even further with Docker system prune: $ docker system prune -a
  46. The importance of garbage collection • Clean up after your

    containers! Beyond image and system prune: • Make sure your orchestration platform (like ECS or K8s) is garbage collecting: • ECS • Kubernetes • 3rd party tools like spotify-gc
  47. Looking forward to the future

  48. But wait, there’s always more! • Always new and better

    things coming • Linux and Windows Server • Official image are multi-platform • Always new and better minimal images and operating systems coming out for containers
  49. So what did we learn? One takeaway: less layers is

    more. • Share layers where possible • Choose or build your base wisely • Not all languages should build the same • Keep it simple, avoid extras • Tools are here to help
  50. Useful links Docker image documentation Docker scratch atsea sample app

    Arun Gupta on smaller Java images Elton Stoneman Windows Dockerfiles Alpine (the base image from the examples) Running Linux containers on Windows Docker garbage collection Image cleanup in Amazon ECS Image cleanup in Kubernetes spotify-gc
  51. Thanks! @abbyfuller