Microservices Meetup rkt+Kubernetes=rktnetes

Talk from 29th Sep 2016

Transcript

1. rktnetes Kubernetes meets rkt
2. Sergiusz Urbaniak github.com/s-urbaniak - @_surbaniak
3. What is rkt?
4. A CLI for running app containers on Linux. Focuses on: • Security • Composability • Standards/Compatibility
5. • December 2014 - v0.1.0 (prototype) ◦ Drive conversation (security, standards) and competition (healthy OSS) in container ecosystem • February 2016 - v1.0.0 (production) ◦ Runtime stability + interface guarantees •< ... many more ... > • September 2016 - v1.15.0 ◦ Latest stable release rkt - a brief history
6. A CLI for running app containers on Linux. Focuses on: • Security • Composability • Standards/Compatibility
7. • UX: "secure-by-default" ◦ Verify image signatures by default ◦ Verify image integrity by default ◦ Restrict capabilities by default • Architecture: Unix philosophy ◦ Well-defined operational scope ◦ Clean integration points as a classic Unix process ◦ Separate privileges for different operations ("fetch" operations shouldn't need root) How rkt does security
8. Classic and modern Linux technologies • User namespaces ◦ container euid != host euid • SELinux contexts ◦ isolate individual pods • Support for VM containment ◦ lightweight hypervisor (= hardware isolation) • TPM measurements ◦ Tamper-proof audit log of what's running How rkt does security
9. Classic and modern Linux technologies • Fine-grained Linux capabilities(7) ◦ only let containers do what they need to do • seccomp(2) enabled by default ◦ restrict application access to kernel • Mask sensitive /proc and /sys paths Security will never be "complete"; always an iterative process, refining over time How rkt does security (cont.)
10. A CLI for running app containers on Linux. Focuses on: • Security • Composability • Standards/Compatibility
11. • "External" composability ◦ Unix architecture; integrating well with other tools (init systems, orchestration tools) is a priority • "Internal" composability ◦ Swappable execution engines (stage-based architecture) that actually runs the container How rkt does composability
12. • "External" composability ◦ Simple process model: a single rkt process is a pod ◦ Any context applied to rkt (cgroups, etc) applies transitively to the pod and the apps inside ◦ No mandatory daemon (but optional API server) How rkt does composability
13. bash/systemd/kubelet... (invoking process) rkt run docker://nginx fork()/exec()
14. rkt run docker://nginx systemd-run -p MemoryLimit=1G rkt run ... fork()/exec()
15. • "Internal" composability ◦ Staged architecture ◦ "rkt" is the UX/API, container technology is an implementation detail • Available stage1s ◦ Linux namespaces+cgroup (default) ◦ KVM (LKVM / QEMU-KVM) ◦ chroot ("fly") How rkt does composability
16. rkt pod app1 app2 exec()
17. rkt (stage0) pod (stage1) app1 (stage2) app2 (stage2) exec()
18. rkt (stage0) pod (stage1) - systemd-nspawn bash/systemd/kubelet... (invoking process) app1 (stage2) app2 (stage2) exec() fork()/exec()
19. rkt (stage0) pod (stage1) - lkvm bash/systemd/kubelet... (invoking process) app1 (stage2) app2 (stage2) exec() fork()/exec()
20. rkt (stage0) pod (stage1) - qemu bash/systemd/kubelet... (invoking process) app1 (stage2) app2 (stage2) exec() fork()/exec()
21. rkt (stage0) app (stage1) - fly bash/systemd/kubelet... (invoking process) exec() fork()/exec()
22. A CLI for running app containers on Linux. Focuses on: • Security • Composability • Standards/Compatibility
23. How rkt does standards/compatibility • Started as an implementation of appc • Networking plugin system became CNI • Can run Docker images natively (V1, V2, ...) • Developers participate actively in standardisation efforts ◦ appc, CNI, OCI, CNCF ◦ rkt will be fully OCI compliant
24. • appc (December 2014) ◦ container images, runtime environment, and pods ◦ some adoption, but (intended to be) deprecated in favour of • OCI (June 2015) ◦ initially runtime only, now container images too • CNCF (December 2015) ◦ "harmonising cloud-native technologies" A brief standards history
25. Cluster-level container orchestration. Handles: • Scheduling/Upgrades • Failure recovery • Scaling Kubernetes
26. • kubelet is the daemon that runs on every worker node in a Kubernetes cluster • kubelet runs the pods scheduled to it by Kubernetes • kubelet delegates to container runtime to perform all container-related operations How does Kubernetes contain? kubelet Container Runtime Interface "Start a container" "Fetch an image"
27. Have Kubernetes use rkt as the container runtime. rkt handles: • Image discovery • Image fetching • Pod execution Kubernetes + rkt = rktnetes
28. Pre-rktnetes (current default): • Kubelet talks to the Docker daemon for all tasks How does it work?
29. Kubelet + Docker (>= 1.11) kubelet containerd containerd shim containerd shim containerd shim container container container dockerd fork()/exec()
30. • rkt container runtime • rkt is pod-native • 1st-class systemd integration • self-contained pods process model = no SPOF • Multi-image compatibility (e.g. docker2aci) • Transparently swappable Kubelet + rkt (rktnetes)
31. Pre-rktnetes (current default): • Kubelet talks to the Docker daemon for all tasks With rktnetes: • Kubelet execs rkt directly for preparatory tasks ◦ e.g. fetch images, create pod root filesystems • Kubelet talks to systemd for running pods via rkt ◦ e.g. launch containers How does it work?
32. pods Kubelet + rkt (rktnetes) kubelet systemd rkt api service rkt rkt rkt gRPC fork()/exec()
33. • no single daemon running the containers • multiple stage1s provides more flexibility • seamless integration with systemd ◦ machinectl, systemctl, journalctl Just Work™ What's the benefit in this?
34. • rkt integration: rktlet ◦ https://github.com/kubernetes-incubator/rktlet • App-level interfaces to rkt ◦ rkt app sandbox ◦ rkt app add/rm app1 ◦ rkt app start/stop app1 New container runtime interface (CRI)
35. pods Kubelet + rkt (rktlet) kubelet systemd rkt rkt rkt fork()/exec()
36. • Yes! • Official release in Kubernetes 1.3 http://blog.kubernetes.io/2016/07/rktnetes-brings-rkt-con tainer-engine-to-Kubernetes.html • Minikube Support https://tectonic.com/blog/minikube-and-rkt.html rktnetes: does it work?
37. rktnetes - the “hard” way https://github.com/squeed/rktnetes-the-hard-way
38. Minikube supports the rkt runtime (with CNI): Demo time! $ minikube start \ --network-plugin=cni \ --container-runtime=rkt \ --iso-url=https://github.com/coreos/minikube-iso/releases/download/ v0.0.4/minikube-v0.0.4.iso
39. There’s more !
40. • OCI: a container image format we can all agree on ◦ Based on Docker v2.2 image format (*not really "new") ◦ + optional components like signing and naming ◦ Maintainers from Docker, CoreOS, Red Hat, Google ◦ First, reach a 1.0 (soon!): then, push this image format into the Kubernetes API ◦ https://github.com/kubernetes/features/issues/63 New* container image format
41. • Reach out on GitHub or IRC ◦ github.com/coreos/rkt, #rkt-dev / #rkt on Freenode • Join a Kubernetes Special Interest Group (SIG) ◦ https://groups.google.com/forum/#!forum/kubernetes-sig-node ◦ https://groups.google.com/forum/#!forum/kubernetes-sig-rktnetes ◦ #sig-node / #sig-rktnetes on Kubernetes Slack • Join us! ◦ Hiring rkt developers in Berlin How do I find out more?
42. Questions? thanks for listening
43. We’re hiring in all departments! Email: careers@coreos.com Positions: coreos.com/ careers 90+ Projects on GitHub, 1,000+ Contributors OPEN SOURCE CoreOS.com - @coreoslinux - github/coreos Secure solutions, support plans, training + more ENTERPRISE sales@coreos.com - tectonic.com - quay.io CoreOS is Running the World’s Containers