$30 off During Our Annual Pro Sale. View Details »

Possibility of OCI Container Runtime with Rust

うたもく
December 07, 2022

Possibility of OCI Container Runtime with Rust

It goes without saying that the Rust language has received a lot of attention in recent years, including the adoption of Rust as a second language in the Linux kernel. Toru has been exploring the possibility of developing a container runtime using Rust in the vicinity of OCI Runtime and developing youki, the OCI container runtime in Rust that He has worked on for over a year and a half. He thought it would be a good fit for the language since OCI Runtime requires development at a relatively kernel-like layer. Youki is now listed as one of the reference implementations of the OCI Runtime specification and is at a level where it can be used from docker and podman. The next step is to consider using it with Kubernetes. In this session, He will discuss the possibility of using Rust in a container runtime. He will share his experiences with youki. From these experiences, He feels that Rust is a language that should contribute more to the container runtime field. At the beginning of the presentation, I will explain container technology such as cgroups and Linux namespaces for those who have not had much exposure to OCI Runtime.

At KubeDay Japan
https://events.linuxfoundation.org/kubeday-japan/
https://sched.co/1C8lS

うたもく

December 07, 2022
Tweet

More Decks by うたもく

Other Decks in Technology

Transcript

  1. Possibility of

    OCI Container Runtime
    with Rust
    Toru Komatsu (@utam0k)
    Senior Enginner at Gitpod
    1

    View Slide

  2. Toru Komatsu, Senior Engineer at Gitpod
    utam0k
    utam0k
    A member of containers organization
    I develop youki as a hobby
    2

    View Slide

  3. 3

    View Slide

  4. What I'd like to tell
    01
    4

    View Slide

  5. Today you will learn
    Based on our experience of developing youki,
    the OCI Runtime in Rust, I'll introduce the
    compatibility and attractiveness of Rust and
    OCI Runtime.
    And about the current status and future of
    youki.
    5

    View Slide

  6. Agenda
    01 What I'd like to tell
    02 What is OCI Container Runtime?
    03 Why Rust?
    04 Where we are now
    05 Problems and Future
    6

    View Slide

  7. What is OCI Container Runtime?
    02
    7

    View Slide

  8. Kubelet(K8s)
    Container creation flow from kubelet
    Linux etc…
    High-Level
    Runtime
    CRI
    Low-Level
    Runtime
    runc runsc
    kata
    OCI
    Container Runtime
    8

    View Slide

  9. How do we create containers?
    pivot_root(2)
    Change a destination that the
    root directory of a process
    points to
    namespace(7)
    Ability to isolate resources
    that a process can manipulate
    cgroup
    Allows configuration of
    resources available to
    processes(containers)
    9

    View Slide

  10. Youki
    OCI Container Runtime in Rust
    Developed since January 2021
    Vendor Neutral
    Under the Containers organization, which manages podman, etc.
    https://github.com/containers/youki
    4.0K
    ⭐ on GitHub
    Provide a Rust library for OCI from a part of youki
    youki means a container in Japanese
    10

    View Slide

  11. Why Rust?
    03
    11

    View Slide

  12. Benefits of using Rust
    Close to kernel WASM Lighter-weight
    12

    View Slide

  13. Benefits of using Rust
    WASM Lighter-weight
    Close to kernel
    Avoiding namespace
    restrictions
    13

    View Slide

  14. setns(2)
    - reassociate thread with a namespace
    A multithreaded process
    may
    not change user

    namespace with setns().
    14

    View Slide

  15. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    15

    View Slide

  16. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    16

    View Slide

  17. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    17

    View Slide

  18. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    18

    View Slide






  19. There is no limit around the namespace(7)
    Rust can be implemented alone with a minimum number
    of clones
    Fewer potential security holes because no extra
    operations are required
    CVE-2019-5736 does not happen with Rust
    implementation
    Rust has been adopted as a second language for Linux
    Ⓒ The Rust foundation 19

    View Slide



  20. buddy-buddy
    How to use →
    https://containers.github.io/youki/user
    /webassembly.html
    Ⓒ The Rust foundation 20

    View Slide

  21. Ⓒ The Rust foundation 21

    View Slide

  22. Benefits of using Rust
    Close to kernel WASM Lighter-weight
    22

    View Slide

  23. Runtime Language Time (mean ± σ) Range (min … max)
    crun C 153.5 ms ± 21.6 ms 80.9 ms … 196.6 ms
    youki Rust 198.4 ms ± 52.1 ms 97.2 ms … 296.1 ms
    runc Go 352.3 ms ± 53.3 ms 248.3 ms … 772.2 ms
    Time from container startup to deletion
    23

    View Slide

  24. What kind of future can you imagine with lightweight
    container runtime? IoT? running it in a car?
    lighter-weight
    24

    View Slide

  25. Benefits of using Rust
    Close to kernel
    Avoiding namespace
    restrictions
    WASM
    New Possibilities
    Lighter-weight
    Less memory used,
    better performance
    25

    View Slide

  26. Why
    Go?
    Why
    Rust?
    To begin with, runc was initially part of
    Docker and spun out from it. Much
    used in this community and has a lot
    of history. Go is mature in this area.
    Therefore, libraries are abundant
    But there is a problem that cannot be
    solved around the namespace. In
    other words, it cannot be
    implemented by Go alone, strictly.
    Why Go?
    Simple, safe, and incidentally light
    However, there is not much history
    and libraries.
    Why Rust?
    26

    View Slide

  27. Where we are now
    04
    27

    View Slide

  28. Fighting with the real

    ✅ Test for OCI Runtime prepared by OCI
    ✅ Containerd integration test
    Achieved primarily through @YJDoc2
    28

    View Slide

  29. For Kubernetes
    ✅ kubeadm supported youki
    29

    View Slide

  30. Advanced Features
    ✅ WASM
    ✅ cgroup v2
    ✅ Rootless
    30

    View Slide

  31. Problems and Future
    05
    31

    View Slide

  32. Problems
    Old kernels Experience
    32

    View Slide

  33. Problems
    Old kernels Experience
    33

    View Slide

  34. We are creating the future
    Ideally, we have to support old Linux kernels, but it is
    challenging, and other container runtimes have already
    been supported.
    We would like to imagine beautiful futures with the latest
    kernel technology instead of dropping them off.
    34

    View Slide

  35. Problems
    Old kernels Experience
    35

    View Slide

  36. Standing on the
    shoulders of giants
    Let's use the history of OCI Container Runtime
    pioneers!
    OCI Runtime is just a binary. Let's cheat on the
    name and use the giant test.
    36

    View Slide

  37. Roadmap/
    upcoming tasks/
    events
    2023・2024
    Pass the integration test of runc and others
    containers/youki#1305
    OpenTelemetry tracing support to extend
    observability
    containers/youki#1348
    Contribute to WASM world
    containers/youki#1320
    Consider ideas using the latest technologies
    such as eBPF and io_uring
    37

    View Slide

  38. 38

    View Slide

  39. runwasi








    Low-level container runtime designed for WASM in Rust
    Still not meeting OCI Runtime Spec
    Used by
    Docker WAM
    Azure Kubernetes Service for WASM
    https://learn.microsoft.com/en-us/azure/aks/use-wasi-
    node-pools
    repo: containers/runwasi
    First committed by Deis Labs, a Microsoft group company
    39

    View Slide

  40. youki    runwasi
    40

    View Slide

  41. Thanks to all the heroes
    who contributed to youki
    41

    View Slide

  42. Thanks!
    Any question?
    Toru Komatsu
    utam0k
    utam0k
    github.com/containers/youki
    42

    View Slide