Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Dockerだけじゃないコンテナ runtime 徹底比較

Dockerだけじゃないコンテナ runtime 徹底比較

Japan Container Days(JKD) v18.04( での発表資料です。
「Dockerだけじゃないコンテナ runtime 徹底比較」

合わせて読みたい
「runc だけじゃないコンテナ low level runtime 徹底比較」
https://speakerdeck.com/makocchi/jkd-20181205-about-low-level-runtimes

makocchi

April 19, 2018
Tweet

More Decks by makocchi

Other Decks in Technology

Transcript

  1. Docker ͚ͩ͡Όͳ͍
    ίϯςφ run*me పఈൺֱ
    Makoto Hasegawa
    JAPAN CONTAINER DAYS V18.04

    View Slide

  2. PAGE
    Japan Container DAYS
    v 18.04 2
    ຊ೔ͷࢿྉ͸ޙ΄Ͳެ։͠·͢ͷͰ
    ࣸਅͱ͔ࡱΒͳ͍Ͱେৎ෉Ͱ͢ʂ

    View Slide

  3. SELF INTRODUCTION

    View Slide

  4. PAGE
    Japan Container DAYS
    v 18.04 4
    SELF INTRODUCTION
    MAKOTO HASEGAWA
    TwiLer/@makocchi
    αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ॴଐ
    ීஈ͸σʔληϯλʔӡ༻΍ Private Cloud(OpenStack) Λ
    ߏஙɾӡ༻͍ͯ͠Δ
    ࠷ۙͰ͸ Private Cloud ্ʹల։ग़དྷΔίϯςφج൫ AKE Λ
    ߏஙɾӡ༻த
    Facebook/makocchi0923
    CKA (Cer*fied Kubernetes Administrator)

    View Slide

  5. PAGE
    Japan Container DAYS
    v 18.04 5
    6
    TODAY’S
    AGENDA

    View Slide

  6. PAGE
    Japan Container DAYS
    v 18.04 5
    6
    TODAY’S
    AGENDA
    SELF INTRODUCTION
    VARIOUS CONTAINER RUNTIME
    OCI AND CRI
    INSIDE OF CONTAINER RUNTIME
    PATTERN OF RUNNING CONTAINERS
    CRI-O VS CONTAINERD
    STORAGE DRIVER
    CONCLUSION

    View Slide

  7. PAGE
    Japan Container DAYS
    v 18.04 6
    ಥવͰ͕͢
    PREFACE

    View Slide

  8. PAGE
    Japan Container DAYS
    v 18.04 7
    Kubernetes ͷ cluster ͷ
    ίϯςφ run*me ʹԿΛ࢖͍ͬͯ·͔͢ʁ
    PREFACE

    View Slide

  9. PAGE
    Japan Container DAYS
    v 18.04 8
    ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ
    (ݸਓతͳ༧૝)
    PREFACE

    View Slide

  10. PAGE
    Japan Container DAYS
    v 18.04 8
    ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ
    (ݸਓతͳ༧૝)
    Docker Other
    What
    container runtime
    is running on
    your k8s cluster?
    95%
    PREFACE

    View Slide

  11. VARIOUS CONTAINER RUNTIME
    ON KUBERNETES

    View Slide

  12. PAGE
    Japan Container DAYS
    v 18.04 10
    Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ
    ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me
    CONTAINER RUNTIMES
    ON KUBERNETES

    View Slide

  13. PAGE
    Japan Container DAYS
    v 18.04 10
    Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ
    Frak*
    Frak*
    Docker cri-o containerd
    ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me
    CONTAINER RUNTIMES
    ON KUBERNETES

    View Slide

  14. PAGE
    Japan Container DAYS
    v 18.04 11
    ࠷΋༗໊ͳ Container Run*me
    1.11 ͔Β಺෦ͷ component ͸
    runC + containerd ʹ෼ׂ͞Εͨ
    Frak*
    Docker cri-o
    Frak* containerd
    CRI, OCI ʹ४ڌͨ͠
    Kubernetes ઐ༻ͷ run*me
    Docker ͷґଘ͔Β୤٫
    Hypervisor ܦ༝Ͱ pod ͕ల։͞ΕΔ
    kernel ͕ share ͞Εͳ͍ͷͰΑΓη
    ΩϡΞͳ؀ڥʹͳΔ
    Docker ͷ಺෦ component ͕෼཭
    Kubernetes ͱ࿈ܞ͢Δҝʹ͸
    cri-containerd ͕ඞཁ͕ͩͬͨɺ
    kubernetes 1.10 ͱ containerd v1.1 ͔Β
    na*ve Ͱ௨৴Ͱ͖ΔΑ͏ʹͳͬͨ
    CONTAINER RUNTIMES
    ON KUBERNETES

    View Slide

  15. PAGE
    Japan Container DAYS
    v 18.04 12
    HOW ABOUT RKT …?
    Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated
    rktnetes has been deprecated in favor of rktlet (#58418)
    CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ
    hLps://github.com/kubernetes-incubator/rktlet
    kubelet ͷ “—container-run*me" ͷ help ʹ
    ʮ The container run*me to use. Possible values: 'docker',
    'remote', 'rkt (deprecated)'. (default “docker”)ʯ
    CONTAINER RUNTIMES
    ON KUBERNETES

    View Slide

  16. PAGE
    Japan Container DAYS
    v 18.04 12
    HOW ABOUT RKT …?
    Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated
    rktnetes has been deprecated in favor of rktlet (#58418)
    CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ
    hLps://github.com/kubernetes-incubator/rktlet
    kubelet ͷ “—container-run*me" ͷ help ʹ
    ʮ The container run*me to use. Possible values: 'docker',
    'remote', 'rkt (deprecated)'. (default “docker”)ʯ
    CONTAINER RUNTIMES
    ON KUBERNETES

    View Slide

  17. WHAT IS OCI AND CRI ?

    View Slide

  18. PAGE
    Japan Container DAYS
    v 18.04 14
    OCI
    Open Container Ini*a*ve
    2015೥1݄ʹίϯςφͷඪ४࢓༷Λࡦఆ͢ΔͨΊʹઃཱ
    ͞Εͨஂମ
    ͦͷஂମʹΑͬͯࡢ೥ʮOCI v1.0ʯ͕ࡦఆ͞Εͨ
    OCI v1.0 ͸ Container Run*me ͷඪ४࢓༷Ͱ͋Δ
    ʮRun*me Specifica*on v1.0ʯͱ Container Image ͷඪ४
    ࢓༷Ͱ͋ΔʮFormat Specifica*on v1.0ʯ͔Βߏ੒͞Εͯ
    ͍Δ
    OCI AND CRI

    View Slide

  19. PAGE
    Japan Container DAYS
    v 18.04 15
    OCI
    Open Container Ini*a*ve
    ͪͳΈʹ௚ۙʹ͜Μͳൃද͕͞Εͨ
    ʮThe New Stack: “Open Container Ini*a*ve Creates a
    Distribu*on Specifica*on for Registries”ʯ(※)
    Docker Registry version 2 ΛݩʹΠϝʔδ഑෍ͷඪ४ԽΛ
    ਐΊ͍ͯ͘
    (Run*me ͱ Format ʹ͙࣍ 3 ͭ໨ͷඪ४Խ)
    ※ hLps://www.opencontainers.org/news/2018/04/11/the-new-stack-open-container-ini*a*ve-creates-a-distribu*on-specifica*on-for-registries
    OCI AND CRI

    View Slide

  20. PAGE
    Japan Container DAYS
    v 18.04 16
    CRI
    The Container Run*me Interface
    Kubernetes 1.5 ͔Β release(alpha) ͞Εͨ(※) kubelet ͱ
    Container Run*me ͕௨৴͢Δҝͷ plugin interface
    Container Run*me ଆͷมߋʹ௥ਵ͠ଓ͚Δͷ͸େมͳҝ
    ౷ҰԽ͞Εͨ interface Λఆٛͯ͠
    Kubernetes ͷ҆ఆԽͱϝϯςφϯεͷίετΛݮΒͨ͠
    (※) hLps://kubernetes.io/blog/2016/12/container-run*me-interface-cri-in-kubernetes
    OCI AND CRI

    View Slide

  21. INSIDE OF
    CONTAINER RUNTIMES

    View Slide

  22. PAGE
    Japan Container DAYS
    v 18.04 18
    Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸
    ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL

    View Slide

  23. PAGE
    Japan Container DAYS
    v 18.04 18
    kubelet Container run*me
    CRI
    run*me
    Containers
    OCI
    Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸
    ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL

    View Slide

  24. PAGE
    Japan Container DAYS
    v 18.04 19
    Container run*me ͷதʹ·ͨ run*me ͕͋Δɾɾɾ
    ฆΒΘ͍͠ɾɾʂ
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL

    View Slide

  25. HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME

    View Slide

  26. HIGH-LEVEL CONTAINER RUNTIME
    LOW-LEVEL CONTAINER RUNTIME

    View Slide

  27. HIGH-LEVEL CONTAINER RUNTIME
    LOW-LEVEL CONTAINER RUNTIME
    OCI/JSON file Λड͚औΓϓϩηεΛ࣮ߦ
    ୅දతͳ΋ͷ͸ runC
    CRI/gRPC Λड͚औΓ Low-level run*me ΁౉͢
    ୅දతͳ΋ͷ͸ cri-o ΍ containerd
    Image ͷ؅ཧΛߦ͏ (pull / push / rm ..) Image ͷ؅ཧ͸͠ͳ͍

    View Slide

  28. PAGE
    Japan Container DAYS
    v 18.04 22
    kubelet
    Container run*me
    CRI Containers
    OCI
    ઌఔʮContainer run*meʯͱݴ͍ͬͯͨ෦෼ͷதͰ
    ɹCRI Λड͚Δ run*me ΛʮHigh-level Container run*meʯ
    ɹOCI Λड͚Δ run*me ΛʮLow-level Container run*meʯ
    ͱݴ͏৔߹͕ଟ͍
    ※ CRI run*me / OCI run*me ͱݴ͏৔߹΋͋Δ
    High-level Low-level
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL

    View Slide

  29. PATTERN OF
    RUNNING
    CONTAINERS

    View Slide

  30. PAGE
    Japan Container DAYS
    v 18.04 24
    PATTERN OF RUNNING CONTAINERS
    SWAPPABLE RUNTIMES
    CRI ΍ OCI ʹΑͬͯΠϯλʔϑΣΠεΛ౷Ұͨ͜͠
    ͱʹΑͬͯɺϢʔβʔ͸ High-level Container
    run*me ͱ Low-level Container run*me Λࣗ༝ʹબ୒
    ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ
    ͍͔ͭ͘۩ମతʹ૊Έ߹ΘͤΛݟ͍͖ͯ·͠ΐ͏

    View Slide

  31. PAGE
    Japan Container DAYS
    v 18.04 25
    PATTERN OF RUNNING CONTAINERS
    01PaLern 01 Docker
    The most popular paLern in the world
    Using containerd and runC to run containers inside Docker

    View Slide

  32. PAGE
    Japan Container DAYS
    v 18.04 26
    Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏
    Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ
    Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ)
    PATTERN OF RUNNING CONTAINERS
    01 DOCKER

    View Slide

  33. PAGE
    Japan Container DAYS
    v 18.04
    Docker
    26
    kubelet containerd runC
    Containers
    OCI
    Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏
    Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ
    Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ)
    PATTERN OF RUNNING CONTAINERS
    01 DOCKER
    dockershim
    CRI

    View Slide

  34. PAGE
    Japan Container DAYS
    v 18.04 27
    PATTERN OF RUNNING CONTAINERS
    02PaLern 02 cri-o
    Using cri-o for High-level Container run*me
    Using runC to run containers with OCI interface

    View Slide

  35. PAGE
    Japan Container DAYS
    v 18.04 28
    PATTERN OF RUNNING CONTAINERS
    02 CRI-O
    Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ

    View Slide

  36. PAGE
    Japan Container DAYS
    v 18.04 28
    cri-o
    kubelet runC
    CRI Containers
    OCI
    PATTERN OF RUNNING CONTAINERS
    02 CRI-O
    Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ

    View Slide

  37. PAGE
    Japan Container DAYS
    v 18.04 29
    PATTERN OF RUNNING CONTAINERS
    03PaLern 03 containerd
    Using containerd for High-level Container run*me
    Using runC to run containers with OCI interface

    View Slide

  38. PAGE
    Japan Container DAYS
    v 18.04 30
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD
    Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ
    cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ

    View Slide

  39. PAGE
    Japan Container DAYS
    v 18.04 30
    runC
    kubelet
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD
    Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ
    cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ
    cri-containerd containerd
    gRPC
    CRI Containers
    OCI

    View Slide

  40. PAGE
    Japan Container DAYS
    v 18.04 31
    runC
    kubelet cri-containerd containerd
    gRPC
    CRI Containers
    OCI
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD

    View Slide

  41. PAGE
    Japan Container DAYS
    v 18.04 31
    runC
    kubelet cri-containerd containerd
    gRPC
    CRI Containers
    OCI
    D E P R E C A T E D
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD

    View Slide

  42. PAGE
    Japan Container DAYS
    v 18.04 32
    PATTERN OF RUNNING CONTAINERS
    03’ CONTAINERD (NEW)
    kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ

    View Slide

  43. PAGE
    Japan Container DAYS
    v 18.04 32
    PATTERN OF RUNNING CONTAINERS
    03’ CONTAINERD (NEW)
    containerd
    kubelet runC
    CRI Containers
    OCI
    kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ

    View Slide

  44. PAGE
    Japan Container DAYS
    v 18.04 33
    PATTERN OF RUNNING CONTAINERS
    04PaLern 04 rkt
    Using rkt and systemd for container engine

    View Slide

  45. PAGE
    Japan Container DAYS
    v 18.04 34
    kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ
    Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷)
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)

    View Slide

  46. PAGE
    Japan Container DAYS
    v 18.04
    systemd
    34
    kubelet
    rkt run
    Containers
    kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ
    Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷)
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)
    rkt pkg

    View Slide

  47. PAGE
    Japan Container DAYS
    v 18.04 35
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)
    systemd
    kubelet
    rkt run
    Containers
    rkt pkg

    View Slide

  48. PAGE
    Japan Container DAYS
    v 18.04 35
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)
    systemd
    kubelet
    rkt run
    Containers
    rkt pkg
    D E P R E C A T E D

    View Slide

  49. PAGE
    Japan Container DAYS
    v 18.04 36
    kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ
    ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ
    PATTERN OF RUNNING CONTAINERS
    04’ RKT (AKA RKTLET)

    View Slide

  50. PAGE
    Japan Container DAYS
    v 18.04
    systemd
    36
    rkt app
    kubelet
    kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ
    ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ
    rktlet
    CRI Containers
    PATTERN OF RUNNING CONTAINERS
    04’ RKT (AKA RKTLET)

    View Slide

  51. PAGE
    Japan Container DAYS
    v 18.04 37
    PATTERN OF RUNNING CONTAINERS
    05PaLern 05 “vm-like” container
    Running “vm-like” isolated containers
    Using cc-run*me or runV, etc

    View Slide

  52. PAGE
    Japan Container DAYS
    v 18.04 38
    PATTERN OF RUNNING CONTAINERS
    05 “VM-LIKE” CONTAINERS
    Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ
    (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍)

    View Slide

  53. PAGE
    Japan Container DAYS
    v 18.04 38
    anything
    kubelet
    runV
    CRI
    PATTERN OF RUNNING CONTAINERS
    05 “VM-LIKE” CONTAINERS
    Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ
    (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍)
    cc-run*me
    OCI
    Containers?
    kata-run*me
    VM?

    View Slide

  54. SUMMARY

    View Slide

  55. PAGE
    Japan Container DAYS
    v 18.04 40
    SUMMARY
    Container Run*me ʹ͸͍Ζ͍Ζ͋Δ
    ίϯςφͷ౷Ұج४ͱͯ͠ OCI ͕ଘࡏ͢Δ
    Kubernetes ͱ Run*me ͷؒ͸ CRI ͱ͍͏ج४͕͋Δ
    CRI ͱ OCI ʹରԠ͍ͯ͠Ε͹ Run*me ͸ࣗ༝ʹೖΕସ͕͑Մೳ
    Container Run*me ͸ High-level ͱ Low-level ʹ
    ෼͚ͯߟ͑Δ͜ͱ͕Ͱ͖Δ
    ίϯςφͷΑ͏ʹ VM Λಈ͔͢ run*me ΋ଘࡏ͢Δ

    View Slide

  56. CRI-O
    CONTAINERD
    VS

    View Slide

  57. PAGE
    Japan Container DAYS
    v 18.04 42
    PERFORMANCE
    CRI-O VS CONTAINERD
    cri-o ͱ containerd ʹର͠ɺcrictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏
    How to bench run*me?
    sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 6 step
    ࣮͸ critest ͱ͍͏ benchmark ͕͋Δ (hLps://github.com/kubernetes-incubator/cri-tools/tree/master/cmd/critest)
    Ͱ΋ critest ͸্ख͘ಈ͔ͳ͔ͬͨͷͰࠓճ͸ಠࣗͷ benchmark ͰରԠ

    View Slide

  58. PAGE
    Japan Container DAYS
    v 18.04 43
    PERFORMANCE
    CRI-O VS CONTAINERD
    How to bench run*me? | 6 steps

    View Slide

  59. PAGE
    Japan Container DAYS
    v 18.04 43
    PERFORMANCE
    CRI-O VS CONTAINERD
    How to bench run*me? | 6 steps
    sandbox pod Λ࡞੒͢Δ
    sandbox pod ʹ busybox ίϯςφΛ࡞੒ɾىಈ͢Δ
    busybox ίϯςφΛఀࢭ͢Δ
    busybox ίϯςφΛ࡟আ͢Δ
    sandbox pod Λ࡟আ͢Δ
    sandbox pod Λఀࢭ͢Δ

    View Slide

  60. PAGE
    Japan Container DAYS
    v 18.04 44
    PERFORMANCE
    CRI-O VS CONTAINERD
    x100 LOOP
    100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢

    View Slide

  61. PAGE
    Japan Container DAYS
    v 18.04 44
    PERFORMANCE
    CRI-O VS CONTAINERD
    x100 LOOP
    runp start stop rm stopp rmp
    100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢

    View Slide

  62. cri-o vs containerd
    with runC(docker-runc)
    The most popular low-level container run*me

    View Slide

  63. PAGE
    Japan Container DAYS
    v 18.04 46
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    runC (docker-runc)

    View Slide

  64. PAGE
    Japan Container DAYS
    v 18.04 46
    Average of 100 *mes loop “crictl” opera*on
    cri-o 1.9.11
    cri-o 1.10.0
    containerd 1.1.0-rc2
    cri-containerd 1.0.0-beta.0
    0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec
    0.66
    0.37
    0.88
    1.29
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    runC (docker-runc)

    View Slide

  65. PAGE
    Japan Container DAYS
    v 18.04 47
    PERFORMANCE
    CRI-O VS CONTAINERD
    cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍
    loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ
    Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠
    cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ
    Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕
    performance ͕͍͍
    WITH
    runC (docker-runc)

    View Slide

  66. PAGE
    Japan Container DAYS
    v 18.04 47
    PERFORMANCE
    CRI-O VS CONTAINERD
    cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍
    loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ
    Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠
    cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ
    Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕
    performance ͕͍͍
    cri-o 1.9.11
    cri-o 1.10.0
    containerd 1.1.0-rc2
    cri-containerd 1.0.0-beta.0
    0.00 sec 0.65 sec 1.30 sec
    0.66
    0.37
    0.88
    1.29
    WITH
    runC (docker-runc)

    View Slide

  67. PAGE
    Japan Container DAYS
    v 18.04 48
    PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)

    View Slide

  68. PAGE
    Japan Container DAYS
    v 18.04
    0
    0.6
    1.2
    1.8
    2.4
    cri-o 1.9.11
    cri-o 1.10.0
    containerd-1.1.0.rc2
    cri-containerd
    48
    PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)

    View Slide

  69. PAGE
    Japan Container DAYS
    v 18.04
    0
    0.6
    1.2
    1.8
    2.4
    cri-o 1.9.11
    cri-o 1.10.0
    containerd-1.1.0.rc2
    cri-containerd
    48
    PERFORMANCE CRI-O VS CONTAINERD
    cri-o ͸͕࣌ؒܦͭʹ࿈Εͯঃʑʹ performance ͕མͪΔ
    containerd ͸҆ఆ͍ͯ͠Δ
    runC (docker-runc)

    View Slide

  70. cri-o vs containerd
    with cc-run*me
    Intel Clear Containers run*me

    View Slide

  71. PAGE
    Japan Container DAYS
    v 18.04 50
    PERFORMANCE
    CRI-O VS CONTAINERD
    cc-run*me

    View Slide

  72. PAGE
    Japan Container DAYS
    v 18.04 50
    PERFORMANCE
    CRI-O VS CONTAINERD
    hypervisor ʹ KVM ͕࢖ΘΕΔ
    hLps://clearlinux.org/containers
    Hypervisor
    Kernel
    ܰྔ͔ͭߴ଎ʹىಈ͢Δ
    Clear Linux OS
    Security
    Intel VT Λ࢖༻͠ɺίϯςφΑΓ
    ηΩϡΞʹىಈ͢Δ͜ͱ͕Ͱ͖Δ
    OCI compa*ble
    run*me Ͱ͋Δ cc-run*me ͸ OCI support
    cc-run*me

    View Slide

  73. PAGE
    Japan Container DAYS
    v 18.04 51
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    cc-run*me (clear container)

    View Slide

  74. PAGE
    Japan Container DAYS
    v 18.04 51
    Average of 100 *mes loop “crictl” opera*on
    cri-o 1.9.11
    cri-o 1.10.0
    containerd 1.1.0-rc2
    cri-containerd 1.0.0-beta.0
    0.00 sec 3.50 sec 7.00 sec 10.50 sec 14.00 sec
    2.69
    2.35
    12.49
    12.86
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    cc-run*me (clear container)

    View Slide

  75. PAGE
    Japan Container DAYS
    v 18.04 52
    PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container)

    View Slide

  76. PAGE
    Japan Container DAYS
    v 18.04
    0
    3.5
    7
    10.5
    14
    cri-o 1.9.11
    cri-o 1.10.0
    containerd-1.1.0.rc2
    cri-containerd
    52
    PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container)
    cc-run*me ͷ৔߹͸྆ऀڞʹऴ࢝҆ఆ͍ͯ͠Δ

    View Slide

  77. cri-o vs containerd
    with kata-run*me
    Kata-container

    View Slide

  78. PAGE
    Japan Container DAYS
    v 18.04 54
    PERFORMANCE
    CRI-O VS CONTAINERD
    kata-run*me

    View Slide

  79. PAGE
    Japan Container DAYS
    v 18.04 54
    PERFORMANCE
    CRI-O VS CONTAINERD
    OpenStack Founda*on ʹΑΓ؅ཧ
    hLps://katacontainers.io/
    Open Source
    runV + Clear Container
    Hyper ʹΑΔ runV ͱ Intel ʹΑΔ
    Clear Linux ͕ϕʔεͱͳ͍ͬͯΔ
    Security
    ͦΕͧΕͷίϯςφ͸ִ཭͞Εͨ
    Kernel ʹΑͬͯىಈ͞ΕΔ
    OCI compa*ble
    run*me Ͱ͋Δ kata-run*me ͸ OCI support
    kata-run*me

    View Slide

  80. WARNING!!

    View Slide

  81. cri-o ͱ containerd ͷઃఆΛม͑ͯ
    Low-level container run*me ʹ kata-run*me Λ
    ࢦఆ͕ͨ͠ error ͰىಈͰ͖ͣɾɾɾ
    Pod sandbox ͸࡞੒Մೳ͕ͩ
    Pod ʹ container Λ࡞੒͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ

    View Slide

  82. cri-o vs containerd
    with kata-run*me
    Kata-container

    View Slide

  83. cri-o vs containerd
    with kata-run*me
    Kata-container
    F A
    I L

    View Slide

  84. cri-o vs containerd
    with kata-run*me
    Kata-container
    F A
    I L

    View Slide

  85. View Slide

  86. View Slide

  87. Docker
    with runc/cc-run*me/kata-run*me
    Docker with 3 Low-level container run*mes
    Revenge

    View Slide

  88. PAGE
    Japan Container DAYS
    v 18.04 60
    PERFORMANCE
    DOCKER + 3 RUNTIMES
    Docker ͷ run*me ʹ kata-run*me Λࢦఆͨ͠ͱ͜Ζ্ख͘ಈ͍ͨ
    How to bench run*me?
    ୯७ʹ docker run ͷੑೳΛݕূ͢Δ
    ࠓճͷݕূ΋ 100 ճ docker run ͯ࣌ؒ͠Λܭଌ
    docker run --rm --run*me ${RUNTIME} hello-world

    View Slide

  89. PAGE
    Japan Container DAYS
    v 18.04 61
    PERFORMANCE
    DOCKER + 3 RUNTIMES

    View Slide

  90. PAGE
    Japan Container DAYS
    v 18.04 61
    Average of 100 *mes loop “docker run” opera*on
    runc
    cc-run*me
    kata-run*me
    0.00 sec 0.65 sec 1.30 sec 1.95 sec 2.60 sec
    2.07
    2.45
    0.45
    PERFORMANCE
    DOCKER + 3 RUNTIMES

    View Slide

  91. PAGE
    Japan Container DAYS
    v 18.04 62
    PERFORMANCE DOCKER + 3 RUNTIMES

    View Slide

  92. PAGE
    Japan Container DAYS
    v 18.04
    0
    0.75
    1.5
    2.25
    3
    runc
    cc-run*me
    kata-run*me
    62
    PERFORMANCE DOCKER + 3 RUNTIMES
    cc-run*meʹएׯϜϥ͕͋Γͦ͏

    View Slide

  93. STORAGE
    DRIVER

    View Slide

  94. PAGE
    Japan Container DAYS
    v 18.04 64
    PERFORMANCE
    STORAGE DRIVER
    run*me ͷ࣮ߦ଎౓ʹ storage driver ͕ͲΕ͘Β͍Өڹ͋Δͷ͔ݕূ
    storage driver is important
    ઌఔͱಉ͡ docker run ͷ benchmark Λ͢Δ
    ࢖༻ͨ͠ storage driver ͸ 4छྨ

    View Slide

  95. PAGE
    Japan Container DAYS
    v 18.04 65
    PERFORMANCE
    STORAGE DRIVER

    View Slide

  96. PAGE
    Japan Container DAYS
    v 18.04 65
    PERFORMANCE
    STORAGE DRIVER
    devicemapper (loopback)
    RHEL ܥͰ͸ default ʹͳ͍ͬͯΔ
    ख͕ܰͩ performance ͸ྑ͘ͳ͍
    devicemapper (direct-lvm)
    Produc*on ؀ڥͰ devicemapper Λ
    ࢖͏৔߹͸ direct-lvm ͕ਪ঑
    loopback ΑΓ͸ੑೳ͕͍͍
    overlay2
    overlay Ͱ໰୊͕͋ͬͨ inode ૿େͱ
    performance ҡ࣋ʹରԠ͍ͯ͠Δ
    overlay
    UnionFS ͷ1ͭͰϑΝΠϧ΍σΟϨΫτϦΛ
    ಁաతʹॏͶΔ͜ͱ͕Ͱ͖Δ
    performance ͸ྑ͍

    View Slide

  97. PAGE
    Japan Container DAYS
    v 18.04 66
    PERFORMANCE
    STORAGE DRIVER

    View Slide

  98. PAGE
    Japan Container DAYS
    v 18.04 66
    runc
    cc-run*me
    kata-run*me
    overlay
    overlay2
    dm(loopback)
    dm(direct)
    overlay
    overlay2
    dm(loopback)
    dm(direct)
    overlay
    overlay2
    dm(loopback)
    dm(direct)
    0.00 sec 0.75 sec 1.50 sec 2.25 sec 3.00 sec
    2.865
    2.862
    0.849
    2.391
    2.606
    0.637
    2.061
    2.376
    0.450
    2.075
    2.452
    0.453
    PERFORMANCE
    STORAGE DRIVER
    Ͳͷ run*me Ͱ΋ overlay2 ͕࠷଎
    devicemapper(direct) ͷํ͕஗͘ͳͬͯ͠·ͬͨ

    View Slide

  99. PAGE
    Japan Container DAYS
    v 18.04 67
    TODAY’S
    CONCLUSION

    View Slide

  100. PAGE
    Japan Container DAYS
    v 18.04 68
    About
    Container Run*me
    TODAY’S
    CONCLUSION
    Container Run*me ʹ͸ High-Level ͱ Low-Level ͷ
    Run*me ʹ۠ผ͢Δ͜ͱ͕Ͱ͖Δ
    CRI ΍ OCI ͱ͍ͬͨ࢓༷͕ࡦఆ͞ΕͨͷͰ
    Run*me ͷೖΕସ͕͑ΑΓॊೈʹߦ͑ΔΑ͏ʹͳ͍ͬͯΔ
    Run*me ʹΑͬͯੑೳࠩ͸एׯ͋Δ͕ɺ·ͩ·ͩൃల్্
    Docker(containerd) ͔Βੵۃతʹ৐Γ׵͑Δཧ༝͸ݱ࣌఺Ͱ͸ແͦ͞͏
    Container ͷΑ͏ʹ VM Λىಈͤ͞Δٕज़͸ࠓޙظ଴Ͱ͖ͦ͏

    View Slide

  101. PAGE
    Japan Container DAYS
    v 18.04 69
    About
    Storage Driver
    TODAY’S
    CONCLUSION
    Performance Λߟ͑Δͱ overlay ͕ྑ͍બ୒ࢶͩͱࢥΘΕΔ
    overlay2 ͕࢖͑ΔͳΒ͹ੵۃతʹ࢖͍͍ͬͯ
    ͨͩ͠ overlay Λ࢖͏ͳΒ͹ kernel ͸࠷৽ʹ্͓͍͛ͯͨ΄͏͕
    ͍͍ͱࢥΘΕΔ
    ݱ࣌఺Ͱͷ moby ͷ open ͳ issue
    ɹlabel:area/storage/overlay 29݅
    ɹlabel:area/storage/devicemapper 42݅
    ɹlabel:area/storage/aufs 29݅
    ɹlabel:area/storage/btr 15݅

    View Slide

  102. PAGE
    Japan Container DAYS
    v 18.04 70
    ʮ·ͩ͋ΔΑʂʯ

    View Slide

  103. PAGE
    Japan Container DAYS
    v 18.04 71
    αΠόʔΤʔδΣϯτ
    ΞυςΫຊ෦ɹ
    Πϯλʔωοτ޿ࠂʹ͓͍ͯɺ޿ࠂ഑৴ͷ࠷దԽ΍ϝσΟΞͷऩӹ࠷େԽͱ͍͏؍఺͔Β
    ΞυςΫϊϩδʔͷॏཁ౓͕ߴ·͍ͬͯ·͢ɻ
    αΠόʔΤʔδΣϯτͰ͸ΞυςΫϊϩδʔ෼໺ʹ͓͚Δ
    ͜ΕΒͷαʔϏεʹ͍֤ͭͯࢠձࣾΛ௨͡։ൃ͓ͯ͠Γ·͕ͨ͠ɺ
    ֤αʔϏεͷ։ൃ෦໳Λԣஅͯ͠૊৫Խ͢Δઐ໳෦ॺͱͯ͠ΞυςΫຊ෦͕ઃཱ͞Ε·ͨ͠ɻ

    View Slide

  104. PAGE
    Japan Container DAYS
    v 18.04 72
    αΠόʔΤʔδΣϯτ
    ΞυςΫຊ෦ɹ
    ʮίϯςφ͕޷͖ͳਓ΋޷͖͡Όͳ͍ਓ΋ઈࢍืूதͰ͢ɻ
    ɹΧδϡΞϧͳ໘ஊ΍ϥϯνަྲྀ΋OKͰ͢ɻ
    ɹڵຯ͕͋ΔํɺੋඇҰॹʹಇ͖·͠ΐ͏ʯ

    View Slide

  105. Docker ͚ͩ͡Όͳ͍
    ίϯςφ run*me పఈൺֱ
    Makoto Hasegawa
    JAPAN CONTAINER DAYS V18.04
    THANK YOU

    View Slide

  106. APPENDIX

    View Slide

  107. RKTLET

    View Slide

  108. PAGE
    Japan Container DAYS
    v 18.04 76
    PERFORMANCE
    RKTLET
    rktlet ʹରͯ͠ crictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏
    How to bench run*me?
    sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 4 step
    rktlet ͷ৔߹ container Λ create ͢Δͱ start ͠ͳͯ͘΋ىಈͨ͠ঢ়ଶʹͳΔΆ͍
    container ʹରͯ͠ stop ͱ rm ͕Ͱ͖ͳ͔ͬͨ (͏·͘ઃఆͰ͖ͯͳ͍Մೳੑ͋Γ)
    sandbox Λ rm ͢Δͱ error: ͱग़Δ͕ͪΌΜͱফ͍͑ͯΔ

    View Slide

  109. PAGE
    Japan Container DAYS
    v 18.04 77
    PERFORMANCE
    RKTLET

    View Slide

  110. PAGE
    Japan Container DAYS
    v 18.04 77
    PERFORMANCE
    RKTLET
    Average of 100 *mes loop “crictl” opera*on
    rktlet
    cri-o 1.9.11
    cri-o 1.10.0
    containerd 1.1.0-rc2
    cri-containerd 1.0.0-beta.0
    0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec
    0.66
    0.37
    0.88
    1.29
    0.78

    View Slide

  111. PAGE
    Japan Container DAYS
    v 18.04 78
    PERFORMANCE RKTLET

    View Slide

  112. PAGE
    Japan Container DAYS
    v 18.04
    0
    0.6
    1.2
    1.8
    2.4
    cri-o 1.9.11
    cri-o 1.10.0
    containerd-1.1.0.rc2
    cri-containerd
    rktlet
    78
    PERFORMANCE RKTLET

    View Slide

  113. Docker ͚ͩ͡Όͳ͍
    ίϯςφ run*me పఈൺֱ
    Makoto Hasegawa
    JAPAN CONTAINER DAYS V18.04
    THANK YOU
    AGAIN

    View Slide