Containerization primatives

1. CONTAINERIZATION PRIMITIVES Sam Kottler @samkottler
2. ABOUT ME • Work at DigitalOcean as a systems engineer • Formerly of Red Hat, Venmo, Acquia • Committer/core for Puppet, Ansible, Fedora, CentOS, RubyGems, Bundler
3. WE’RE GONNA BE TALKING ABOUT LINUX
4. GOOD TO KNOW’S • What is a syscall • Basic understanding of linux networking • Containers vs. virtualization
5. WHY DO WE CARE ABOUT ANY OF THIS?
6. CONTAINERS ARE THE PAST *, PRESENT, AND FUTURE * Most of the linux ideas are poached from other OS’s
7. VIRTUALIZATION HAS BECOME MASSIVELY POPULAR BECAUSE OF ITS ECONOMICS
8. CONTAINERS ARE BECOMING MASSIVELY POPULAR BECAUSE THEY ALLOW LOGICAL SEPARATION
9. APPLICATION VS. FULL CONTAINERS
10. NETWORKS, USERS, AND PROCESSES
11. NAMESPACES • mnt: filesystem • pid: process • net: network • ipc: SysV IPC • uts: hostname • user: UID
12. THE BASICS • Namespaces do not have names • Six inodes exist under /proc/<pid>/ns • Each namespace has a unique inode
13. USERSPACE TOOLING • iproute2 • util-linux • systemd
14. NAMESPACE SYSCALLS • unshare() • moves existing process into a new namespace • clone() • creates new process and namespace • setns() • joins an existing namespace
15. NETWORK ISOLATION • One namespace per networking device • Single default namespace, init_net(*nets) • A lo device is included in every ns_net.
16. NETWORK NAMESPACES IN PRACTICE • ip netns add testns1 • creates /var/run/netns/testns1 • route management per-NS • prevents cross-NS bonds • setns(int fd, int nstype) • validates namespace type vs. FD
17. SOCKET ISOLATION • Sockets are mapped into network namespaces • Also part of a single network namespace • sk_net is part of the sock struct • sock_net()/sock_net_set() getter/setter
18. SOCKET ACTIVATION • Listen on a socket, but have no services behind it • Request arrives, service is spun up, responds • Enabling 10k+ low-usage services on a VM
19. USER ISOLATION • Allows non-privileged usage • Often used as the start of a namespace chain • UID’s come from the overflow rules
20. CGROUPS • Resource management • Around since 2006/2007 • Widely used by userspace management tools
21. CGROUPS + NAMESPACES • “This PID can only see part of the filesystem” • “This PID can only see part of the filesystem, use 384mb of memory, and utilize a single CPU.”
22. CGROUP IMPLEMENTATION • Hooks into fork() and exit() • VFS of a new type called “cgroup” • More complex descriptors for task_struct • Procfs entry in /proc/<pid>/cgroup • All actions take place on the FS
23. CGROUP MANAGEMENT • 4 files per-cgroup • tasks • cgroup.procs • cgroup.event_control • notify_on_release
24. CPU • Split into “shares” • Default is 2048 shares • Linear CPU time use
25. MEMORY • Exposes most of the memory subsystem • NUMA management • Most complex type of cgroup
26. LETS TALK ABOUT SECURITY…
27. SHARING A KERNEL IS INHERENTLY LESS SECURE
28. KERNEL VULNERABILITIES AROUND BREAKOUT ARE USUALLY MITIGATED BY RUNNING SERVICES NON- PRIVILEGED
29. THANKS! • @samkottler • https://github.com/skottler • shk@linux.com