Slide 1

Slide 1 text

Moby Project Kunal Kushwaha ( @kunalkushwaha )

Slide 2

Slide 2 text

Agenda - Story of Moby Project - Building Blocks (Foundation) of Moby - runC, - containerD, - LinuxKit - Demo - Docker with Kubernetes (Moby with Kubernetes) - What else in Moby Project.

Slide 3

Slide 3 text

Story of Moby

Slide 4

Slide 4 text

Start Of Container Revolution

Slide 5

Slide 5 text

Escalated quickly all over globe

Slide 6

Slide 6 text

Some amazing products started

Slide 7

Slide 7 text

Conflict of thoughts

Slide 8

Slide 8 text

Communities

Slide 9

Slide 9 text

Docker CE Docker EE @ Docker Inc

Slide 10

Slide 10 text

Essentially all were reinventing the wheel

Slide 11

Slide 11 text

Moby Project An open framework to assemble specialized container systems without reinventing the wheel. It provides a “lego set” of dozens of standard components and a framework for assembling them into custom platforms.

Slide 12

Slide 12 text

Moby Project An open framework to assemble specialized container systems without reinventing the wheel. It provides a “lego set” of dozens of standard components and a framework for assembling them into custom platforms.

Slide 13

Slide 13 text

Building Blocks of Moby

Slide 14

Slide 14 text

runC : Low level container runtime CLI tool for spawning and running containers according to the OCI specification - Client implemented as wrapper around libcontainer (OS level interfaces) - Requires rootfs and config.json

Slide 15

Slide 15 text

runC : Low level container runtime CLI tool for spawning and running containers according to the OCI specification - Client implemented as wrapper around libcontainer (OS level interfaces) - Requires rootfs and config.json Main functionalities provided by runC - Create - Start - Exec - Pause & Resume - Checkpoint - Restore

Slide 16

Slide 16 text

runC : cli example kunal@dev-box:~/demo/alpine-bundle$ ls config.json rootfs kunal@dev-box:~/demo/alpine-bundle$ sudo runc run test / # uname -a Linux runc 4.8.0-59-generic #64-Ubuntu SMP Thu Jun 29 19:38:34 UTC 2017 x86_64 Linux / # exit

Slide 17

Slide 17 text

runC : go client. - go-runc : a golang client implementation of runc - github.com/containerd/go-runc import runc “github.com/containerd/go-runc” . . func main() { . status, err := runc.Run(ctx, “test”, alpineBundlePath, &runc.CreateOpts{}) . }

Slide 18

Slide 18 text

runC Limitations - Do not prepare rootfs or manage image for creating container. - Running container in background, requires extra work (I/O handling through socket) - Attaching External Volume or Networks needs to be done by hand. (using Hooks) - Not portable to other platforms. (platform specific implementation)

Slide 19

Slide 19 text

Container : High level container runtime An industry-standard container runtime. - Simplicity - Robustness - Portability

Slide 20

Slide 20 text

Container : High level container runtime An industry-standard container runtime. - Simplicity - Robustness - Portability - Design - Usage (~25 loc to manage container) - Snapshotter vs Graph Storage - Multi-tenancy

Slide 21

Slide 21 text

Container : High level container runtime An industry-standard container runtime. - Simplicity - Robustness - Portability - Design - Usage (~25 loc to manage container) - Snapshotter vs Graph Storage - Multi-tenancy - Version 1.0.0 released (API Freeze) - Tested against k8s CRI testsuite (cri-containerd) - Supported for atleast 1 year

Slide 22

Slide 22 text

Container : High level container runtime An industry-standard container runtime. - Simplicity - Robustness - Portability - Design - Usage (>25 loc to create container) - Snapshotter vs Graph Storage - Multi-tenancy - Version 1.0.0 released - Tested against k8s CRI testsuite (cri-containerd) - Supported for atleast 1 year - Works on Windows & Linux - Works with any OCI complaint runtime - Batteries included but replaceable.

Slide 23

Slide 23 text

Container design ( OCI )

Slide 24

Slide 24 text

Container - Code examples import “github.com/containerd/containerd” . func main() { . // Create Client client, err := containerd.New(“/run/containerd/containerd.sock”) . // Pull Image image, err := client.Pull(ctx, alpine, containerd.WithPullUnpack) // New Container container, err := client.NewContainer(ctx, id, containerd.WithNewSpec(oci.WithImageConfig(image), containerd.WithNewSnapshot(id, image)) // New Task task, err := container.NewTask(ctx, cio.Stdio) // Start Task err := task.Start(ctx) // Wait for Task completion status, err := task.Wait(ctx) } Working Example code: https://goo.gl/RiKKBS

Slide 25

Slide 25 text

container Namespaces - Provides the isolation to all containerd resources - Images, Containers, Metadata etc. - I.e. Multi-tenancy : Multiple consumers can use same daemon without conflict of their resources. - No need of DIND or Nested Containers. - Still shares underneath immutable resources like image cache etc. - Shall not be considered this isolation from security point of view. import “github.com/containerd/containerd/namespaces” func main() { ctx := namespaces.WithNamespace(context.Background(), “demo”) client.XXX(ctx, ...) }

Slide 26

Slide 26 text

Container Misc - ctr is CLI client for containerd. - Not officially supported, for debugging purpose only - Debug/Profile - pprof data for daemon : /run/containerd/debug.sock - Example usage: github.com/kunalkushwaha/cri-containerd-flame/tree/powertest-flamegraph - Fork from Brain Goff (@cpuguy83) work. cri-containerd-flame - Metrics - Prometheus format of martrics - 127.0.0.1:1234 (Default) - External testing tools. - Bucketbench - Container runtime benchmarking tool. - Ctr-powertest - Container runtime testsuite

Slide 27

Slide 27 text

Container Usecases - Not restricted to orchestrators only. - Cri-containerd : Kubernetes CRI implementation. - Docker CE : Soon. - Linuxkit : for running system and user containers. - Buildkit: Uses to build each layer of cache to build container image. - RancherOS: Soon. (for running system and user containers) - Others - Integrating containerized traditional app with existing management layers. - Products like Balena can be made very quickly without forking projects.

Slide 28

Slide 28 text

LinuxKit LinuxKit, a toolkit for building custom , minimal, immutable Linux distributions - Built with containers for running containers. - Secure defaults - Completely stateless , but persistent storage can be attached. - Everything is replaceable and customizable. - Used to build Docker CE and distro like Docker with K8s.

Slide 29

Slide 29 text

LinuxKit - Input - Consumes yaml file. - Most of defined in yaml is docker image. - Kernel - Init - Onboot - Services - Exceptions: files can be created. - Trust can be defines image should be signed auth. kernel: image: linuxkit/kernel:4.9.68 cmdline: "console=tty0 console=ttyS0 console=ttyAMA0" init: - linuxkit/init:9250948d0de494df8a811edb3242b4584057cfe4 - linuxkit/runc:abc3f292653e64a2fd488e9675ace19a55ec7023 - linuxkit/containerd:e58a382c33bb509ba3e0e8170dfaa5a100504c5b onboot: - name: dhcpcd image: linuxkit/dhcpcd:0d59a6cc03412289ef4313f2491ec666c1715cc9 command: ["/sbin/dhcpcd", "--nobackground", "-f", "/dhcpcd.conf", "-1"] services: - name: getty image: linuxkit/getty:22e27189b6b354e1d5d38fc0536a5af3f2adb79f env: - INSECURE=true trust: org: - linuxkit

Slide 30

Slide 30 text

LinuxKit - Output - Ramfs + bootloader | ISO | format specific to platform - Local hypervisors - HyperKit (macOS) - Hyper-V (Windows) - qemu (macOS, Linux, Windows) - VMware (macOS, Windows) - Cloud based platforms: - Amazon Web Services - Google Cloud - Microsoft Azure - OpenStack - Packet.net - Baremetal: - x86 and arm64 servers on packet.net - Raspberry Pi Model 3b

Slide 31

Slide 31 text

LinuxKit - Output - Ramfs + bootloader | ISO | format specific to platform - Local hypervisors - HyperKit (macOS) - Hyper-V (Windows) - qemu (macOS, Linux, Windows) - VMware (macOS, Windows) - Cloud based platforms: - Amazon Web Services - Google Cloud - Microsoft Azure - OpenStack - Packet.net - Baremetal: - x86 and arm64 servers on packet.net - Raspberry Pi Model 3b Platform(VM/Baremetal) Kernel Init (runc, containerd) Services (as containers) Applications (as containers)

Slide 32

Slide 32 text

LinuxKit Not only build ISO, but can also boot systems - Tool to build bootable formats for different platforms - Cli tools to boot and run the built system. - Can push the images to Cloud Platform(VM/Baremetal) Kernel Init (runc, containerd) Services (as containers) Applications (as containers) USAGE: linuxkit [options] COMMAND Commands: build Build an image from a YAML file metadata Metadata utilities pkg Package building push Push a VM image to a cloud or image store run Run a VM image on a local hypervisor or remote cloud version Print version information help Print this message

Slide 33

Slide 33 text

Demo

Slide 34

Slide 34 text

BuildKit LibEntitlement Current Status

Slide 35

Slide 35 text

Followup Blog : https://blog.mobyproject.org/ Github :https://github.com/moby

Slide 36

Slide 36 text

Thank you.