Possibility of OCI Container Runtime with Rust

It goes without saying that the Rust language has received a lot of attention in recent years, including the adoption of Rust as a second language in the Linux kernel. Toru has been exploring the possibility of developing a container runtime using Rust in the vicinity of OCI Runtime and developing youki, the OCI container runtime in Rust that He has worked on for over a year and a half. He thought it would be a good fit for the language since OCI Runtime requires development at a relatively kernel-like layer. Youki is now listed as one of the reference implementations of the OCI Runtime specification and is at a level where it can be used from docker and podman. The next step is to consider using it with Kubernetes. In this session, He will discuss the possibility of using Rust in a container runtime. He will share his experiences with youki. From these experiences, He feels that Rust is a language that should contribute more to the container runtime field. At the beginning of the presentation, I will explain container technology such as cgroups and Linux namespaces for those who have not had much exposure to OCI Runtime.

At KubeDay Japan
https://events.linuxfoundation.org/kubeday-japan/
https://sched.co/1C8lS

Transcript

1. Possibility of
   
   OCI Container Runtime
   with Rust
   Toru Komatsu (@utam0k)
   Senior Enginner at Gitpod
   1
   

   View Slide

2. Toru Komatsu, Senior Engineer at Gitpod
   utam0k
   utam0k
   A member of containers organization
   I develop youki as a hobby
   2
   

   View Slide

3. View Slide

4. What I'd like to tell
   01
   4
   

   View Slide

5. Today you will learn
   Based on our experience of developing youki,
   the OCI Runtime in Rust, I'll introduce the
   compatibility and attractiveness of Rust and
   OCI Runtime.
   And about the current status and future of
   youki.
   5
   

   View Slide

6. Agenda
   01 What I'd like to tell
   02 What is OCI Container Runtime?
   03 Why Rust?
   04 Where we are now
   05 Problems and Future
   6
   

   View Slide

7. What is OCI Container Runtime?
   02
   7
   

   View Slide

8. Kubelet(K8s)
   Container creation flow from kubelet
   Linux etc…
   High-Level
   Runtime
   CRI
   Low-Level
   Runtime
   runc runsc
   kata
   OCI
   Container Runtime
   8
   

   View Slide

9. How do we create containers?
   pivot_root(2)
   Change a destination that the
   root directory of a process
   points to
   namespace(7)
   Ability to isolate resources
   that a process can manipulate
   cgroup
   Allows configuration of
   resources available to
   processes(containers)
   9
   

   View Slide

10. Youki
    OCI Container Runtime in Rust
    Developed since January 2021
    Vendor Neutral
    Under the Containers organization, which manages podman, etc.
    https://github.com/containers/youki
    4.0K
    ⭐ on GitHub
    Provide a Rust library for OCI from a part of youki
    youki means a container in Japanese
    10
    

    View Slide

11. Why Rust?
    03
    11
    

    View Slide

12. Benefits of using Rust
    Close to kernel WASM Lighter-weight
    12
    

    View Slide

13. Benefits of using Rust
    WASM Lighter-weight
    Close to kernel
    Avoiding namespace
    restrictions
    13
    

    View Slide

14. setns(2)
    - reassociate thread with a namespace
    A multithreaded process
    may
    not change user
    
    namespace with setns().
    14
    

    View Slide

15. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    15
    

    View Slide

16. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    16
    

    View Slide

17. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    17
    

    View Slide

18. netns(2) with Go
    HighLevelCR
    HighLevelCR
    runc create
    Only Go
    runc create
    Only Go
    runc init
    C&Go
    runc init
    C&Go
    Create a container
    Prepare to create a container such as cgroup
    /proc/self/exec init with args as ENVs
    Actually create a container
    18
    

    View Slide

19. •
    •
    •
    •
    •
    There is no limit around the namespace(7)
    Rust can be implemented alone with a minimum number
    of clones
    Fewer potential security holes because no extra
    operations are required
    CVE-2019-5736 does not happen with Rust
    implementation
    Rust has been adopted as a second language for Linux
    Ⓒ The Rust foundation 19
    

    View Slide

20. •
    •
    buddy-buddy
    How to use →
    https://containers.github.io/youki/user
    /webassembly.html
    Ⓒ The Rust foundation 20
    

    View Slide

21. Ⓒ The Rust foundation 21
    

    View Slide

22. Benefits of using Rust
    Close to kernel WASM Lighter-weight
    22
    

    View Slide

23. Runtime Language Time (mean ± σ) Range (min … max)
    crun C 153.5 ms ± 21.6 ms 80.9 ms … 196.6 ms
    youki Rust 198.4 ms ± 52.1 ms 97.2 ms … 296.1 ms
    runc Go 352.3 ms ± 53.3 ms 248.3 ms … 772.2 ms
    Time from container startup to deletion
    23
    

    View Slide

24. What kind of future can you imagine with lightweight
    container runtime? IoT? running it in a car?
    lighter-weight
    24
    

    View Slide

25. Benefits of using Rust
    Close to kernel
    Avoiding namespace
    restrictions
    WASM
    New Possibilities
    Lighter-weight
    Less memory used,
    better performance
    25
    

    View Slide

26. Why
    Go?
    Why
    Rust?
    To begin with, runc was initially part of
    Docker and spun out from it. Much
    used in this community and has a lot
    of history. Go is mature in this area.
    Therefore, libraries are abundant
    But there is a problem that cannot be
    solved around the namespace. In
    other words, it cannot be
    implemented by Go alone, strictly.
    Why Go?
    Simple, safe, and incidentally light
    However, there is not much history
    and libraries.
    Why Rust?
    26
    

    View Slide

27. Where we are now
    04
    27
    

    View Slide

28. Fighting with the real
    •
    ✅ Test for OCI Runtime prepared by OCI
    ✅ Containerd integration test
    Achieved primarily through @YJDoc2
    28
    

    View Slide

29. For Kubernetes
    ✅ kubeadm supported youki
    29
    

    View Slide

30. Advanced Features
    ✅ WASM
    ✅ cgroup v2
    ✅ Rootless
    30
    

    View Slide

31. Problems and Future
    05
    31
    

    View Slide

32. Problems
    Old kernels Experience
    32
    

    View Slide

33. Problems
    Old kernels Experience
    33
    

    View Slide

34. We are creating the future
    Ideally, we have to support old Linux kernels, but it is
    challenging, and other container runtimes have already
    been supported.
    We would like to imagine beautiful futures with the latest
    kernel technology instead of dropping them off.
    34
    

    View Slide

35. Problems
    Old kernels Experience
    35
    

    View Slide

36. Standing on the
    shoulders of giants
    Let's use the history of OCI Container Runtime
    pioneers!
    OCI Runtime is just a binary. Let's cheat on the
    name and use the giant test.
    36
    

    View Slide

37. Roadmap/
    upcoming tasks/
    events
    2023・2024
    Pass the integration test of runc and others
    containers/youki#1305
    OpenTelemetry tracing support to extend
    observability
    containers/youki#1348
    Contribute to WASM world
    containers/youki#1320
    Consider ideas using the latest technologies
    such as eBPF and io_uring
    37
    

    View Slide

38. View Slide

39. runwasi
    •
    •
    •
    •
    •
    •
    •
    •
    Low-level container runtime designed for WASM in Rust
    Still not meeting OCI Runtime Spec
    Used by
    Docker WAM
    Azure Kubernetes Service for WASM
    https://learn.microsoft.com/en-us/azure/aks/use-wasi-
    node-pools
    repo: containers/runwasi
    First committed by Deis Labs, a Microsoft group company
    39
    

    View Slide

40. youki 　 　runwasi
    40
    

    View Slide

41. Thanks to all the heroes
    who contributed to youki
    41
    

    View Slide

42. Thanks!
    Any question?
    Toru Komatsu
    utam0k
    utam0k
    github.com/containers/youki
    42
    

    View Slide