Introduction to Container Security

How to build more secure Docker images and run your containers more securely.

Presented at All Hands on DevOps #12 (London) - 5/12/2017

Transcript

1. Introduction to Container Security Stepan Stipl All Hands on DevOps #12 - 5/12/2017
2. Stepan Stipl • Cloud and container Architect, DevOps Engineer • Kubernetes enthusiast • Head of Infrastructure @ CheckRecipient • Previously: Yoti, Sky, Tesco, Oracle...
3. What will I talk about? • Security of container layer • Docker • Technology aspect • Intro level
4. Containers - more or less secure? • Add complexity • All the standard security practices still apply You're only as secure as your weakest link. • Attack vectors (anything involved in your SDLC) • Who are you protecting against?
5. Outline 1. Building secure images 2. Run containers securely 3. Securing Docker daemon 4. Interesting tools
6. Building secure images
7. Minimise attack surface • Minimal container FROM scratch • If you're lucky FROM scratch COPY myapp / CMD ["/myapp"]
8. Base image • Anyone can publish to Docker hub • Official images != secure Over 30% of Official Images in Docker Hub Contain High Priority Security Vulnerabilities (https://banyanops.com/blog/analyzing-docker-hub/)
9. How does size affect security Image Size Vuln. Ubuntu 17.10 95 MB 9 Ubuntu 17.04 94 MB 11 Ubuntu 16.04 123 MB 7 Debian Wheezy 85 MB 30 Debian Jessie 123 MB 27 CentOS 6.9 195 MB 20 CentOS 7.4 204 MB 0 Alpine 3.5 4 MB 0 Alpine 3.6 4 MB 0 Alpine 3.7 4 MB 0 (Quay.io as of 03/12/2017, medium+)
10. Alpine linux • Minimal OS designed with security in mind • Composed of • musl libc (600Kb, less 80CVEs) • Busybox • apk-tools • hardened toolchain (PIE, RELRO) & kernel • 5 MB
11. FROM alpine:3.5 RUN apk add --no-cache mysql-client ENTRYPOINT ["mysql"] 36.5MB vs. FROM ubuntu:16.04 RUN apt-get update \ && apt-get install -y --no-install-recommends mysql-client \ && rm -rf /var/lib/apt/lists/* ENTRYPOINT ["mysql"] 184 MB
12. Version everything - Images • Tags are not immutable FROM alpine FROM alpine:latest vs. FROM alpine:3.6 vs. FROM docker.io/library/alpine:3.6
13. How secure is your content? • Build minimal images • Internet? GitHub? HTTPS? Certs? OS packages? • Integrity verification & Versioning ENV VERSION="1.2.3" \ SHA="ea4139d2f0d9f7aa6488be966ffbfe602f80764a47b7077892d92324bff5eb47" \ ... RUN curl -L "https://s3.com/app/releases/${VERSION}.tar.gz" -o /tmp/app.tar.gz && \ echo "${SHA} /tmp/app.tar.gz" | sha256sum -c -s - && \ ...
14. Don't run as root RUN ... adduser -h /home/passenger -s /sbin/nologin -u 10000 -D passenger USER passenger Even in images from scratch: FROM scratch COPY testRoot_linux_amd64 / USER 123 ENTRYPOINT ["/testRoot_linux_amd64"]
15. Don't put setuid binaries in your images FROM ubuntu:latest RUN chmod u+s /usr/bin/whoami USER nobody $ docker run -it --rm -u nobody test whoami root
16. Dear AWS Customer, Your security is important to us. We recently became aware that your AWS Access Key (ending with XYZA) along with your Secret Key are publicly available on github.com.
17. No secrets in images • Common one ENV AWS_SECRET_ACCESS_KEY= • Build minimal images • (or at least use .dockerignore) $ ls -al .git ... COPY . /myapp
18. Running Containers securely
19. Core principles & Kernel • Basic underlying mechanisms • namespaces • cgroups • Benefit from kernel hardening
20. AppArmor & SELinux • AppArmor --security-opt apparmor=docker-default • SELinux - --selinux-enabled • /etc/selinux/targeted/contexts/lxc_contexts --security-opt="label=user:USER" : Set the label user for the container --security-opt="label=role:ROLE" : Set the label role for the container --security-opt="label=type:TYPE" : Set the label type for the container --security-opt="label=level:LEVEL" : Set the label level for the container --security-opt="label=disable" : Turn off label confinement for the container
21. No root please $ docker run -it --rm -u nobody alpine whoami root ... but what if there's suid binary?
22. User namespaces # /etc/docker/daemon.json { "userns-remap": "default" } $ grep dockremap /etc/subuid dockremap:231072:65536 $ grep dockremap /etc/subgid dockremap:231072:65536
23. No priviledged && Linux capabilities • --privileged flag gives all capabilities to the container (cca. 38 currently) • also lifts all the limitations enforced by the device cgroup controller --cap-add: Add Linux capabilities --cap-drop: Drop Linux capabilities --privileged=false: Give extended privileges to this container --device=[]: Allows you to run devices inside the container without the --privileged flag.
24. # ls -la /bin/ping -rwsr-xr-x 1 root root 44168 May 7 2014 /bin/ping $ docker run -it --rm test -u nobody ping google.com PING google.com (216.58.206.110): 56 data bytes 64 bytes from 216.58.206.110: seq=0 ttl=37 time=4.147 ms 64 bytes from 216.58.206.110: seq=1 ttl=37 time=6.385 ms $ docker run -it --rm --cap-drop ALL test ping google.com PING google.com (216.58.206.110): 56 data bytes ping: permission denied (are you root?)
25. $ docker run -it --rm alpine grep Cap /proc/self/status CapInh: 00000000a80425fb CapPrm: 00000000a80425fb CapEff: 00000000a80425fb CapBnd: 00000000a80425fb CapAmb: 0000000000000000 $ docker run -it --rm -u nobody alpine grep Cap /proc/self/status CapInh: 00000000a80425fb CapPrm: 0000000000000000 CapEff: 0000000000000000 CapBnd: 00000000a80425fb CapAmb: 0000000000000000 $ docker run -it --rm --cap-drop ALL alpine grep Cap /proc/self/status CapInh: 0000000000000000 CapPrm: 0000000000000000 CapEff: 0000000000000000 CapBnd: 0000000000000000 CapAmb: 0000000000000000
26. Seccomp • Limit system calls • Docker compiled with seccomp and Kernel enabled • By default disables around 44 system calls out of 300+ $ docker run --rm -it \ --security-opt seccomp=/path/to/seccomp/profile.json \ hello-world
27. Volumes • --read-only, but lot of programs need to write to /tmp and similar • --tmpfs $ docker run --read-only --tmpfs /run --tmpfs /tmp -ti --rm alpine sh $ echo hello > /test sh: can't create /test: Read-only file system $ echo hello > /tmp/test
28. Resource management - cgroups • CPU • -c, --cpu-shares=1024 - relative, by default 1024 • --cpus=2 - number of CPUs to limit container to • Memory • -m, --memory="300M" - memory limit for process • IO • r/w iops/bps per device or relative
29. Network • Complex (default own network stack, bridge0, non-isolated) • User networks - Network drivers, Network plugins • Depends on orchestrator AWS tip sudo iptables --insert FORWARD 1 --in-interface docker+ --destination 169.254.169.254/32 --jump DROP
30. Docker daemon • Client/server model • Protect Docker socket /var/run/docker.sock • AuthN - Certificates $ dockerd --tlsverify --tlscacert=ca.pem \ --tlscert=server-cert.pem --tlskey=server-key.pem \ -H=0.0.0.0:2376 • AuthZ plugins
31. Docker content trust • Disabled by default DOCKER_CONTENT_TRUST=1 docker pull mhart/alpine-node:latest Error: remote trust data does not exist for docker.io/mhart/alpine-node: notary.docker.io does not have trust data for docker.io/mhart/alpine-node $ DOCKER_CONTENT_TRUST=1 docker pull alpine:3.6 Pull (1 of 1): alpine:3.6@sha256:d6bfc3baf615dc9618209a8d607ba2a8103d9c8a405b3bd8741d88b4bef36478 sha256:d6bfc3baf615dc9618209a8d607ba2a8103d9c8a405b3bd8741d88b4bef36478: Pulling from library/alpine Digest: sha256:d6bfc3baf615dc9618209a8d607ba2a8103d9c8a405b3bd8741d88b4bef36478 Status: Image is up to date for alpine@sha256:d6bfc3baf615dc9618209a8d607ba2a8103d9c8a405b3bd8741d88b4bef36478 Tagging alpine@sha256:d6bfc3baf615dc9618209a8d607ba2a8103d9c8a405b3bd8741d88b4bef36478 as alpine:3.6
32. Tools
33. Docker Notary • Docker Notary • https://github.com/theupdateframework/notary • Based on The Update Framework (now CNCF) • Determine validity and integrity of the received content • Integrate with your CI/CD, orchestration platfrom (K8S)
34. Docker Bench • Script that checks for dozens of common best-practices around deploying Docker containers in production. • https://github.com/docker/docker-bench-security
35. Scanning for vulnerabilities • Clair by CoreOS • https://github.com/coreos/clair
36. Sysdig & Sysdig Falco • Detect anomalous behavioral activity • For example: • A shell is run inside a container container.id != host and proc.name = bash • Non-authorized container namespace change syscall.type = setns and not proc.name in (docker, sysdig)
37. Resources • Docker CIS Benchmark - https://www.cisecurity.org/benchmark/ docker/ • Docker security - https://docs.docker.com/engine/security/ security/ • Docker Security - Using Containers Safely in Production - Adrian Mouat (O'Reilly)
38. Thanks & FAQ