Rust 🤝 Container Runtime @ Rust.Tokyo 2021

https://rust.tokyo/lineup/en/5

In this session, I'll talk about the possibility of using Rust in a container runtime. I'll share my experiences with youki, the container runtime in Rust that I'm working on. From these experiences, I feel that Rust is a language that should contribute more to this field. By sharing my knowledge in this presentation, many Rustaceans will be interested in this field. I also believe that the experience I'll be sharing here will be helpful not only for container runtime but also for using Rust at lower layers.

At the beginning of the presentation, I will explain container technology such as cgroups and briefly explain with actual codes of youki.

Transcript

1. Rustとコンテナランタイム

   Rust 🤝 Container Runtime

   Rust.Tokyo 2021 September 18th

   うたもく(@utam0k)

   1
   

   View Slide

2. self‐introduction
   ● うたもく(@utam0k)
   ● I work for a web company and use Scala language
   ● I'm studying containers in earnest starting in 2021.
   ● Member of the containers
   2
   

   View Slide

3. What I want you to know
   ● Youki, a container runtime written in Rust
   ● A brief description of container technology
   ○ What is container runtime?
   ○ Learning container technologies with youki's code
   ● Rust 🤝 Container Runtime
   3
   

   View Slide

4. What is youki?
   ● I’m creator of youki
   ● A container runtime written in Rust
   ● Developed under containers, the organization that
   develop podman
   ● ★ 1.3K in github.com
   ● youki = 容器 = container
   (Meaning "container" in Japanese)
   4
   

   View Slide

5. A container runtime … ?

   5
   

   View Slide

6. Kubelet(K8s)
 Linux etc...

   High-Level
   Runtime

   CRI

   Low-Level
   Runtime

   OCI

   CRI − Container Runtime Interface

   OCI − Open Container Initiative

   Container Runtime

   6
   

   View Slide

7. Kubelet(K8s)
 Linux etc...

   High-Level
   Runtime

   CRI

   Low-Level
   Runtime

   OCI

   ● runC

   ● youki

   ● crun

   Container Runtime

   7
   

   View Slide

8. What is a container?
   ● A process ≒ A container
   ○ Processes in which various resources are isolated
   Process
   Files
   Process
   Files
   Process
   Files
   OS
   8
   

   View Slide

9. isolated？
   ● pivot_root
   ○ Change the destination that the root directory of a
   process points to
   ● namespace
   ○ Ability to isolate resources that a process can manipulate
   ● cgroup
   ○ Allows configuration of resources available to processes
   9
   

   View Slide

10. isolated？
    ● pivot_root
    ○ Change the destination that the root directory of a
    process points to
    ● namespace
    ○ Ability to isolate resources that a process can manipulate
    ● cgroup
    ○ Allows configuration of resources available to processes
    
    10
    

    View Slide

11. cgroup
    ● Create a group of processes and configure settings for
    that group
    ● For example...
    ○ Limit CPU usage time, specifying assigned CPUs
    ○ Device access control
    ○ Pause/resume tasks
    ● There are resource controllers for different types of resources
    11
    

    View Slide

12. /sys/fs/cgroup

    CPU
 Memory
 Devices

    P
    P P P P P
    P P
    P P
    Represent tree
    with directories
    inherit
    12
    

    View Slide

13. Let’s dive into youki
    13
    

    View Slide

14. Rust 🤝 Container Runtime

    14
    

    View Slide

15. Rust × Container Runtime
    👍 :+1:
    ● Performance
    ● Easy to handle system calls
    ● A Good fit between low layers and Rust community
    15
    

    View Slide

16. Rust × Container Runtime
    👍 :+1:
    ● Performance
    ● Easy to handle system calls
    ● A Good fit between low layers and Rust community
    
    16
    

    View Slide

17. Performance
    ● Enables containers to be launched even with tight
    memory limits
    $ podman --runtime youki run --rm --memory 1M fedora echo work
    work
    $ podman --runtime runc run --rm --memory 1M fedora echo work
    Error: OCI runtime error: ....
    17
    

    View Slide

18. Performance
    ● Enables containers to be launched even with tight
    memory limits
    ● Faster container startup and deletion
    ○ There is a C language implementation of a container
    runtime called crun
    ■ Probably not that different in speed from crun
    ○ Technically challenging and interesting :)
    18
    

    View Slide

19. Rust × Container Runtime
    👍 :+1:
    ● Performance
    ● Easy to handle system calls
    ● A Good fit between low layers and Rust community
    
    19
    

    View Slide

20. Easy to handle system calls
    ● Threads and processes can be precise controlled
    ○ Difficult when language runtime is multithreaded
    ■ Rust and C are not
    ○ setns(2) - reassociate thread with a namespace
    A multithreaded process may not change user
    namespace with setns().
    ○ Youki has little trouble in this area
    ● Existence of libc crate and nix crate. Thanks 🙏
    20
    

    View Slide

21. Rust × Container Runtime
    👍 :+1:
    ● Performance
    ● Easy to handle system calls
    ● A Good fit between low layer and Rust community
    
    21
    

    View Slide

22. Good fit between low layers and Rust community
    ● Low Layer Programming 🤝 Rust
    ○ There are many programmers in the Rust community
    who like this layer
    ○ Topics adopted for the linux kernel
    ● Rustacean, who is also interested in this layer, joined
    youki to learn more about container technology
    22
    

    View Slide

23. Rust × Container Runtime
    😖 :confounded:
    ● There are no libraries or other assets
    ● Little knowledge in the Rust community
    23
    

    View Slide

24. Rust × Container Runtime
    😖 :confounded:
    ● There are no libraries or other assets
    ● Little knowledge in the Rust community
    
    24
    

    View Slide

25. There are no libraries or other assets
    ● There are still few libraries compared to the major
    languages (Go / C) in the container area
    ○ For example, Go has a library set in the specification
    repository to parse a JSON file according to the
    specification
    ● FFI(Foreign Function Interface)... ?
    ○ We want to use the full potential of Rust!
    25
    

    View Slide

26. There are no libraries or other assets
    ● The Issue youki would like to solve in any way possible
    ● Cut out the codes used in youki and crate it
    ○ containers/oci-spec-rs
    ○ cgroup(It's not complete)
    ● We are trying to create a common integration test tool
    for the container runtime community
    26
    

    View Slide

27. Rust × Container Runtime
    😖 :confounded:
    ● There are no libraries or other assets
    ● Little knowledge in the Rust community
    
    27
    

    View Slide

28. Little knowledge in the Rust community
    ● There are few Rustacean in the container world
    ● Youki is a gateway to containers for Rustacean
    ○ Youki's maintainers have got knowledgeable enough
    about it to read runC
    ○ Container people may use it to learn Rust
    ● Would you like to join us?
    28
    

    View Slide

29. Summary
    ● Advantages of Rust 🤝 Container Runtime
    ○ Performance
    ○ Easy to handle system calls
    ○ A Good fit between low layers and Rust community
    ● The problems that youki wants to solve
    ○ There are no libraries or other assets
    ○ Little knowledge in the Rust community
    29
    

    View Slide

30. Youki in the future
    ● Aim for the first release
    ● Contributing to the container community
    ○ Testing tools for generic container runtimes
    ● Consideration of unique features as container runtime
    ○ Asynchronous processing using io_uring
    ○ Features using wasm or kernel modules... 🤔
    ● We'll have fun developing :)
    30
    

    View Slide

31. Thanks to all the people who
    already contributed to youki :)
    31
    

    View Slide

32. Thanks you! Any questions?
    32
    

    View Slide