CoreOS - A journey into modern Linux technologies

Transcript

1. Luca Bruno @lucabruno | luca.bruno@coreos.com | github.com/lucab A journey into modern Linux technologies
2. “CoreOS Engineer, Debian Developer and enthusiast FLOSS supporter” • OS Software Engineer • CoreOS, Berlin office • Eurecom promo ‘13 • Previously: security researcher/engineer $ whoami
3. “Infrastructure with autopilot: improve reliability and security regarding critical updates, machine failures, networking outages” • YC’13 startup • FLOSS-centric company • Offices in SF, NYC, Berlin CoreOS Inc.
4. Commercial Kubernetes offering by CoreOS. • Focused on automated-operations and self-updates • Vanilla upstream software with addons, no forked components • Our opinionated approach to infrastructure • Source of technical materials for this talk :) Tectonic
5. Overview
6. Base ingredients of our stack: • Kubernetes • Etcd • Docker • OS ◦ Distribution: ContainerLinux ◦ Userland: GNU + systemd ◦ Kernel: Linux Technologies
7. “Container” is a misleading word: • Linux APIs - cgroups, namespaces, bindmounts, overlayfs • Lightweight segmentation measure (!= security) Containers
8. Cgroups: • Resource constraining and allocation • File-based interface - /sys/fs/cgroup Namespaces: • Kernel-level resource virtualization and isolation • Mount, network, hostname, IPC, hostname, cgroups • Syscall-based interface - setns(), unshare(), clone() Mounts: • Resource gluing and isolation crossing • Bindmounts - generic path-to-path mounting • Overlayfs - stackable layers of filesystems Containers
9. Container-based cluster orchestration system. • Schedule workloads across a large set of nodes • Segment workloads via containers • Keep track of resource requirements and usage • High level primitives (pods, services, replicas) • Originally by Google Kubernetes
10. https://github.com/kubernetes/kubernetes
11. Distributed key-value store. • “Consistent” database (CP in CAP terms) ◦ Based on the Raft consensus algorithm [Ongaro’10] • GRPC interface (HTTP2 + protobuf) • Written in Go https://github.com/coreos/etcd etcd
12. Container runtime and tooling. • Single-stop solution for containers • Centralized, daemon-based design • Easy, high-level CLI to cgroups and namespaces Docker
13. An alternative container runtime. • Daemon-less, single process tree • Modular design and isolation ◦ Supports VM, namespaces, plain chroot • Simpler to integrate with existing supervisors https://github.com/rkt/rkt rkt
14. Dependency-based Linux init system and userspace framework. Main ideas: ◦ Get rid of custom scripts ◦ Introduce absolute ordering and dependency graph ◦ Introspection via DBus interfaces https://github.com/systemd/systemd Systemd
15. Linux distribution designed around containers. • Tailor-made for server workloads • A different spin on typical Linux distribution • Push toward immutable infrastructure • Our efforts at OS-level innovation ContainerLinux
16. System components
17. A compact Linux distribution without a package manager. • Immutable OS (read-only /usr) • Autoupdates with A/B partition-scheme • 3 release channels: ◦ Stable - 8 weeks ◦ Beta - 4 weeks ◦ Alpha - 2 weeks ContainerLinux
18. ContainerLinux Traditional distribution
19. ContainerLinux Containers
20. A compact Linux distribution without a package manager. • Immutable OS (read-only /usr) • Autoupdates with A/B partition-scheme • 3 release channels: ◦ Stable - 8 weeks ◦ Beta - 4 weeks ◦ Alpha - 2 weeks ContainerLinux
21. ContainerLinux Dm-verity: • Block level integrity of read-only /usr • Merkle tree of SHA256 hashes
22. A compact Linux distribution without a package manager. • Immutable OS (read-only /usr) • Autoupdates with A/B partition-scheme • 3 release channels: ◦ Stable - 8 weeks ◦ Beta - 4 weeks ◦ Alpha - 2 weeks ContainerLinux
23. ContainerLinux /usr updates via A/B partitions: • Active-passive partition scheme • Updates and rollback via GPT flags
24. A compact Linux distribution without a package manager. • Immutable OS (read-only /usr) • Autoupdates with A/B partition-scheme • 3 release channels: ◦ Stable - 8 weeks ◦ Beta - 4 weeks ◦ Alpha - 2 weeks ContainerLinux
25. Autoupdate service and updates management. • Based on Omaha protocol (HTTPS + XML + sigs) • Updates flushed to passive partition • Updates applied via reboot • Cluster-wide maintenance scheduling ◦ Either via locksmith or kubernetes addon https://github.com/coreos/locksmith Update-engine
26. First-boot machine provisioning tool. • Immutable node configuration • Userdata-based node setup (PXE and cloud) • Runs early in initrd ◦ Can partition disks, mount volumes, etc. https://github.com/coreos/ignition Ignition
27. System languages
29. Main language of UNIX world. Pro: • Mature specification and toolings • Stable ABI • Close to machine level Cons: • Complete lack of safety • Lots of legacy codebases, messy and hard to review C
30. Makes it easy to build simple, reliable, and efficient services. Pro: • Static typing and coroutines - CSP-style [Hoare’77] • Simple to learn, read and write • Easy to develop concurrent logic Cons: • Lack of generic constructs and data types • Easy to introduce data races • Mandatory runtime ◦ Multi-threaded ◦ Garbage collection Go
31. Modern system language: safe, concurrent, practical. Pro: • Static typing, memory safety, no data races • Avoids global mutable state • Thin runtime (can target bare metal) Cons: • Steep learning curve • Library ecosystem still growing Rust
32. Questions? - Demo? We have open positions in SF/NYC/BER! Jobs: coreos.com/careers - Internships: talk to me