A modern services SDK for LinuxKit Type safety, container-native daemons, minimum privilege, easy development, unikernel protection, .. hacked to you by Thomas Gazagnaire, Thomas Leonard, Martin Lucina, Anil Madhavapeddy, Mindy Preston 7th June 2017 - Moby Security SIG #2
A modern services SDK for LinuxKit Type safety, container-native daemons, minimum privilege, easy development, unikernel protection, .. disclaimer: this is active work in progress, and we're showing this early to the Moby Security SIG community to get feedback on the work. interruptions and feedback are welcome. and patches :)
Motivation • Base daemons in LinuxKit are typically wrapped versions of existing system software (e.g. dhcpcd, ntpd). • Often written in C, different configuration mechanisms, no structured logging, require lots of privilege for system operations. • Want to make these less monolithic and more container- native, and fit with LinuxKit philosophy of a lean, secure container runtime. • This project provides us with a vehicle to deploy more advanced security protections in LinuxKit in a practical way.
Approach • LinuxKit has a single build-time yaml file and everything except init runs in a container namespace. • We build privilege separated applications that use this architecture to avoid common security vulnerabilities by: 1. Specifying the process layout for an application in yaml 2. Enforcing isolated, minimal privileges per process 3. Separating every process in a container namespace 4. Coordinating the containers with standard RPC tooling Developer experience matters: containerisation complexity is hidden inside the SDK tooling and not the application.
Approach • First daemon being developed is a DHCP client. • This is a difficult daemon to privilege separate due the deep (and non-portable) system hooks required for handling IP and routing tables (e.g. netlink). • Implementation flushes out a lot of architectural questions and makes subsequent protocol implementations such as HTTPS or NTP more straightforward. https://github.com/linuxkit/linuxkit/tree/master/projects/miragesdk https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=dhcp
dhcp- network dhcp- actuator dhcp- engine eth0 kernel/ netlink • Three processes, each with very minimal privileges. • dhcp-network can only access eth0 for networking • dhcp-engine can see nothing except the other two processes • dhcp-actuator can manipulate routing tables but cannot see network. • Each process can be written in safe language best suited to the task (OCaml, Rust in this case).
Demo: Capnp RPC • Capnp has an interface file and stub code generator for many languages. • Very simple binary format to parse (e.g. no HTTP2 dependency) so is a viable small attack surface to depend on for privileged components. • The CLI checks your interface specs and makes it relatively easy to glue pieces together. • Here is an example of an HTTPS server built like this: https://github.com/talex5/linuxkit/tree/https-unikernel/ projects/https-unikernel (see https://github.com/linuxkit/linuxkit/pull/1981)
Going deeper for security • Need protections at all levels of the stack for defence in depth: • application level: static type safety when parsing network traffic (via OCaml, Rust logic) and secure RPC (via Capnp) • protocol state machine: fuzz testing for rapid state space exploration (via American Fuzzy Lop aka AFL) • runtime process: container namespacing and KVM hardware protection if available (via unikernel Solo5). • kernel interface: eBPF sandboxing for fine-grained access to syscalls. • implementation diversity: the container/rpc approach lets many languages/runtimes work together without tight coupling. • What else? LinuxKit lets us patch kernel and use facility directly in the base daemons, just like a BSD distro. SGX, TrustZone, etc...
Demo: ukvm service • For service isolation, we can further protect processes against exploit by using /dev/kvm • This is a unikernel (standalone specialised VM) running as a normal Linux process in a container. • External channel setup will be handled by the RPC layer, but for now is just a tap device. • Demo: here is a DNS service running as a KVM process on Linux and serving network traffic.
Demo: fuzz testing • Fuzz testing: throw a lot of random input at a program, see where it breaks, fix it, repeat. • AFL is helpful as it can figure out an effective fuzz path quickly, and minimise test cases. Comes with a CLI afl-fuzz: http://lcamtuf.coredump.cx • Writing adapters for AFL to the LinuxKit SDK (which uses file descriptors) to make fuzzing easier to start. • Demo: afl-fuzz working on the DHCP state machine. Details at Asciicast: https://asciinema.org/a/3ljccmn19m25uj02kve678xp6
Putting it all together • WIP: wanted to explain the architecture early to the Security SIG community. Another update at the Moby Summit in a few weeks to show the frontend tooling. • DHCP, DNS, HTTPS are our first targets to have safe system services by default. Anything else to focus on? • Config interface is as similar to existing daemons as possible so they can be swapped easily. • @mato is working on integrating Solo5 so that isolated services (e.g. dhcp-engine) can be unikernel-protected if hardware virt is available, and fall back to eBPF/seccomp sandboxing if not. • @talex5 is working on the RPC substrate. • @samoht @avsm are building the system daemons and CLI frontend. • @yomimono is hacking on fuzz testing all the things with AFL.
Where its going • Initially it is very LinuxKit specific since we depend on a specific containerd featureset, but everything is intended to be portable (including to FreeBSD jails, OpenBSD pledge, ...) • The Moby CLI should be able to package up as deb or rpms though, so it can be deployable more widely. • We want to take a structured approach to classifying CVEs for common system services to determine what to fuzz on. Memory safety, logic bugs, container breakouts, ... • Support more languages, build an ecosystem for practical correct-by-construction services. • https://github.com/linuxkit/linuxkit projects/miragesdk
avsm
oracle4engineer
smt7174
cimadai
poteboy
chiroito
you
cybozuinsideout
nagix
ikumatadokoro
yutoy
colopl
kojira
garrettdimon
philhawksworth
tenderlove
sstephenson
eileencodes
tammyeverts
paulrobertlloyd
shlominoach
smashingmag
iamctodd
jnunemaker
danielanewman