K8'ers gonna H8

Containers and Kubernetes

Transcript

1. K8’ers gonna H8 Cameron Lonsdale @myoutpost
2. A little bit about me - 2017 ACS ICT Student of the Year & Golden Disruptor - Security Engineer - Dislikes being defined by titles I sometimes blog things: cameronlonsdale.com And am on Twitter: @myoutpost
3. Shut up and listen 1. What are containers 2. Brief History of Container Orchestrators 3. What is Kubernetes 4. [Interactive] How 2 Kubernetes 5. How containers can be broken 6. How Kubernetes can be broken * For the purposes of this talk, containers are Linux containers. Ask me at the end about Windows Containers.
4. 1. What are containers https://www.docker.com/resources/what-container A standardized unit of software
5. 1.1 Why Containers ➔ Applications abstracted away from their environment ➔ Speedy development, easy deployment ➔ Lightweight Isolation from other tasks on the computer ◆ Consistent environment ◆ Run anywhere https://cloud.google.com/containers/
6. 1.2 What are containers really Solaris Zones, BSD Jails, Virtual Machines are “First Class Concepts” Not a Thing™ Containers are actually just a collection of “First Class Concepts”. Namespaces + control groups (cgroups) + Linux Security Features https://blog.jessfraz.com/post/containers-zones-jails-vms/
7. 1.3 How do containers work Concept Purpose Details Namespaces Control what a process can see ➔ Mount points (Files) ➔ Other processes (PIDs) ➔ Network ➔ Users Cgroups Control what resources a process can use ➔ CPU ➔ Memory ➔ Disk I/O ➔ Network Security Features Control what a process can do ➔ Capabilities ➔ AppArmor (file path MAC) ➔ SELinux (label MAC) ➔ Seccomp (syscall whitelist)
8. 1.4 What is Docker ➔ Did not invent containers, they’ve existed since about ~2000-ish ➔ Platform-as-a-Service company dotCloud built Docker to automate setting up containers ➔ Internal Engine “Docker” Open Sourced in 2013 ➔ Have now helped define standards for container images, runtimes and pioneered image sharing with Docker Hub ➔ Provides sane default security settings ◆ Limited Capabilities ◆ AppArmor, SELinux, Seccomp profiles for your convenience (AppArmor and Seccomp on by default) ◆ Content Trust https://www.youtube.com/watch?v=wW9CAH9nSLs
9. 1.5 But these are now called containers…. Kata Containers - lightweight virtual machines Hyper-V containers - windows lightweight virtual machine GVisor - user space kernel Who are we kidding, words don’t mean things anymore
10. 2. Brief History of Container Orchestrators When you have lots of containers and lots of computers, you need a little help managing resources, and scheduling containers. ➔ Google’s Borg (2004) ➔ Google’s Omega (<2013) ➔ Apache Mesos’ Marathon (2013) ➔ Apache Aurora (used by Twitter) (2013) ➔ (and a couple other small ones on top of Mesos) ➔ Google’s Kubernetes (2014) ➔ Docker Swarm (~2016)
11. 3. What is Kubernetes ➔ Mangled pronunciation the Greek word κυβερνήτης, meaning “helmsman”. ➔ Open-source system for automating deployment, scaling, and management of containerized applications. ◆ “Just give me containers, tell me how you’d like me to run them and I’ll do the rest!” ➔ Commonly shortened to K8s Abstracts away machines, you just think about containers.
12. https://www.youtube.com/watch?v=vQX5nokoqrQ Server Server Server Where should I go? How many copies should I make? How do I talk to the world? How do I talk to other containers? What happens if I get sick?
13. 4. How 2 Kubernetes Get ready to learn layers upon layers of abstraction Master Etcd API Server Scheduler and Controller Manger Node Pod Container Container Pod Container Container Kubelet Proxy Node Pod Container Container Pod Container Container Kubelet Proxy Node Pod Container Container Pod Container Container Kubelet Proxy
14. 4. How 2 Kubernetes (2)
15. 4.1 Pods ➔ The smallest deployable (atomic) unit in Kubernetes. ➔ A Pod runs on a Node, where other pods may also be running. ➔ Alongside container(s), there can be Volume(s) and also an IP address. ➔ They are mortal, will not survive node failures, scheduling failures, etc. https://kubernetes.io/docs/concepts/workloads/pods/pod/
16. https://bit.ly/2CHXGIP Interactive Part 1 & 2
17. 4.2 Controllers A higher-level abstraction to group and manage sets of pods. The most commonly used are: Deployment ➔ “I want to run this app, and always have 3 copies, thanks” ➔ Easy rollout, rollback and scaling. DaemonSet ➔ “Run this app on every node in my cluster, thanks” ➔ Useful for running log collecting / monitoring daemons
18. https://bit.ly/2CHXGIP Interactive Part 3
19. 4.3 Service A network front-end to your back-end. ➔ Because Pods don’t have a stable IP address. ➔ Because which Pod should a request be routed to? ➔ Clients of a service should not have to worry what the backend looks like. Image processing backends, don’t care which one is used. https://kubernetes.io/docs/concepts/services-networking/service/
20. https://bit.ly/2CHXGIP Interactive Part 4
21. 4.4 Ingress Controller ➔ A collection of rules that allow inbound internet connections to reach the cluster services. ➔ Can also be load balance traffic, terminate SSL, offer name based virtual hosting, etc. Ingress Service Pod Pod Service Pod Pod Service Pod Pod Internet Outside cluster Inside cluster https://kubernetes.io/docs/concepts/services-networking/ingress/
22. https://bit.ly/2CHXGIP Interactive Part 5
23. [BONUS] Brief difference between K8s and Docker Swarm ➔ Kubernetes is more mature, more customisable, but more complicated to use than Docker Swarm ➔ Kubernetes also has a larger community ➔ Recommendation would be to use Kubernetes in production, it’s an effort to set up but it will be worth it. Use Docker Swarm for smaller low effort projects
24. [BONUS] Architectural Difference 1 https://docs.docker.com/engine/swarm/how-swarm-mode-works/nodes/
25. [BONUS] Architectural Difference 2 https://docs.docker.com/engine/swarm/services/ ➔ Service == Deployment ➔ Task ≈ Pod $ docker service create \ --name my_web \ --replicas 3 \ --publish published=8080,target=80 \ nginx $ docker service update \ --rollback \ --update-delay 0s my_web
26. 5. How containers can be broken ➔ Inside the container ➔ The container configuration
27. 5.1 Inside your container ➔ Malware (Miners) in images on DockerHub ◆ Don’t use images from JoeBloggs over at DockerHub ➔ Private Keys and Secrets baked into the image ◆ If someone can see your image they can see your secrets ➔ Outdated and vulnerable code ◆ Update and patch all of your dependencies ➔ God knows what else you installed ◆ Why is your image so bloated, just get rid of all the stuff you don’t need to run your application. Use a slim base image like Alpine
28. 5.2 Container Configuration ➔ Is your container running with more privileges than it needs? ◆ --privileged, --cap-add ◆ Think about what capabilities your container needs, does it _really_ need all that privilege? ➔ Did you mount in docker.sock? ◆ You don’t need the docker socket, get rid of that ➔ Did you mount in /etc or other sensitive host paths? ◆ You probably didn’t need to do that ➔ Is your user inside the container running as root? ◆ Not the end of the world but may be more privilege than it nee
29. 6. How Kubernetes can be broken ➔ Insecure defaults everywhere ➔ Kubernetes is updating very rapidly, new security features added all the time By default, a hacker with a shell might be able to: ➔ Exfil code, keys and credentials ➔ Privesc to cluster admin ➔ Privesc to root on the nodes ➔ Compromise data from the cloud account
30. https://www.youtube.com/watch?v=X-rSMkKqyt4
31. Where can I learn more? ➔ contained.af ➔ Keynote: Containers aka crazy user space fun ➔ Kubernetes Bootcamp ➔ Hacking and Hardening Kubernetes Clusters by Example ➔ Shipping in Pirate-Infested Waters: Practical Attack and Defense in Kubernetes Ask me questions, I have more knowledge to share