Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Possibility of OCI Container Runtime with Rust

うたもく
December 07, 2022

Possibility of OCI Container Runtime with Rust

It goes without saying that the Rust language has received a lot of attention in recent years, including the adoption of Rust as a second language in the Linux kernel. Toru has been exploring the possibility of developing a container runtime using Rust in the vicinity of OCI Runtime and developing youki, the OCI container runtime in Rust that He has worked on for over a year and a half. He thought it would be a good fit for the language since OCI Runtime requires development at a relatively kernel-like layer. Youki is now listed as one of the reference implementations of the OCI Runtime specification and is at a level where it can be used from docker and podman. The next step is to consider using it with Kubernetes. In this session, He will discuss the possibility of using Rust in a container runtime. He will share his experiences with youki. From these experiences, He feels that Rust is a language that should contribute more to the container runtime field. At the beginning of the presentation, I will explain container technology such as cgroups and Linux namespaces for those who have not had much exposure to OCI Runtime.

At KubeDay Japan
https://events.linuxfoundation.org/kubeday-japan/
https://sched.co/1C8lS

うたもく

December 07, 2022
Tweet

More Decks by うたもく

Other Decks in Technology

Transcript

  1. Possibility of OCI Container Runtime with Rust Toru Komatsu (@utam0k)

    Senior Enginner at Gitpod 1
  2. Toru Komatsu, Senior Engineer at Gitpod utam0k utam0k A member

    of containers organization I develop youki as a hobby 2
  3. 3

  4. What I'd like to tell 01 4

  5. Today you will learn Based on our experience of developing

    youki, the OCI Runtime in Rust, I'll introduce the compatibility and attractiveness of Rust and OCI Runtime. And about the current status and future of youki. 5
  6. Agenda 01 What I'd like to tell 02 What is

    OCI Container Runtime? 03 Why Rust? 04 Where we are now 05 Problems and Future 6
  7. What is OCI Container Runtime? 02 7

  8. Kubelet(K8s) Container creation flow from kubelet Linux etc… High-Level Runtime

    CRI Low-Level Runtime runc runsc kata OCI Container Runtime 8
  9. How do we create containers? pivot_root(2) Change a destination that

    the root directory of a process points to namespace(7) Ability to isolate resources that a process can manipulate cgroup Allows configuration of resources available to processes(containers) 9
  10. Youki OCI Container Runtime in Rust Developed since January 2021

    Vendor Neutral Under the Containers organization, which manages podman, etc. https://github.com/containers/youki 4.0K ⭐ on GitHub Provide a Rust library for OCI from a part of youki youki means a container in Japanese 10
  11. Why Rust? 03 11

  12. Benefits of using Rust Close to kernel WASM Lighter-weight 12

  13. Benefits of using Rust WASM Lighter-weight Close to kernel Avoiding

    namespace restrictions 13
  14. setns(2) - reassociate thread with a namespace A multithreaded process

    may not change user namespace with setns(). 14
  15. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc

    create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 15
  16. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc

    create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 16
  17. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc

    create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 17
  18. netns(2) with Go HighLevelCR HighLevelCR runc create Only Go runc

    create Only Go runc init C&Go runc init C&Go Create a container Prepare to create a container such as cgroup /proc/self/exec init with args as ENVs Actually create a container 18
  19. • • • • • There is no limit around

    the namespace(7) Rust can be implemented alone with a minimum number of clones Fewer potential security holes because no extra operations are required CVE-2019-5736 does not happen with Rust implementation Rust has been adopted as a second language for Linux Ⓒ The Rust foundation 19
  20. • • buddy-buddy How to use → https://containers.github.io/youki/user /webassembly.html Ⓒ

    The Rust foundation 20
  21. Ⓒ The Rust foundation 21

  22. Benefits of using Rust Close to kernel WASM Lighter-weight 22

  23. Runtime Language Time (mean ± σ) Range (min … max)

    crun C 153.5 ms ± 21.6 ms 80.9 ms … 196.6 ms youki Rust 198.4 ms ± 52.1 ms 97.2 ms … 296.1 ms runc Go 352.3 ms ± 53.3 ms 248.3 ms … 772.2 ms Time from container startup to deletion 23
  24. What kind of future can you imagine with lightweight container

    runtime? IoT? running it in a car? lighter-weight 24
  25. Benefits of using Rust Close to kernel Avoiding namespace restrictions

    WASM New Possibilities Lighter-weight Less memory used, better performance 25
  26. Why Go? Why Rust? To begin with, runc was initially

    part of Docker and spun out from it. Much used in this community and has a lot of history. Go is mature in this area. Therefore, libraries are abundant But there is a problem that cannot be solved around the namespace. In other words, it cannot be implemented by Go alone, strictly. Why Go? Simple, safe, and incidentally light However, there is not much history and libraries. Why Rust? 26
  27. Where we are now 04 27

  28. Fighting with the real • ✅ Test for OCI Runtime

    prepared by OCI ✅ Containerd integration test Achieved primarily through @YJDoc2 28
  29. For Kubernetes ✅ kubeadm supported youki 29

  30. Advanced Features ✅ WASM ✅ cgroup v2 ✅ Rootless 30

  31. Problems and Future 05 31

  32. Problems Old kernels Experience 32

  33. Problems Old kernels Experience 33

  34. We are creating the future Ideally, we have to support

    old Linux kernels, but it is challenging, and other container runtimes have already been supported. We would like to imagine beautiful futures with the latest kernel technology instead of dropping them off. 34
  35. Problems Old kernels Experience 35

  36. Standing on the shoulders of giants Let's use the history

    of OCI Container Runtime pioneers! OCI Runtime is just a binary. Let's cheat on the name and use the giant test. 36
  37. Roadmap/ upcoming tasks/ events 2023・2024 Pass the integration test of

    runc and others containers/youki#1305 OpenTelemetry tracing support to extend observability containers/youki#1348 Contribute to WASM world containers/youki#1320 Consider ideas using the latest technologies such as eBPF and io_uring 37
  38. 38

  39. runwasi • • • • • • • • Low-level

    container runtime designed for WASM in Rust Still not meeting OCI Runtime Spec Used by Docker WAM Azure Kubernetes Service for WASM https://learn.microsoft.com/en-us/azure/aks/use-wasi- node-pools repo: containers/runwasi First committed by Deis Labs, a Microsoft group company 39
  40. youki    runwasi 40

  41. Thanks to all the heroes who contributed to youki 41

  42. Thanks! Any question? Toru Komatsu utam0k utam0k github.com/containers/youki 42