Dockerだけじゃないコンテナ runtime 徹底比較

Japan Container Days(JKD) v18.04( での発表資料です。
「Dockerだけじゃないコンテナ runtime 徹底比較」

合わせて読みたい
「runc だけじゃないコンテナ low level runtime 徹底比較」
https://speakerdeck.com/makocchi/jkd-20181205-about-low-level-runtimes

Transcript

1. Docker ͚ͩ͡Όͳ͍
   ίϯςφ run*me పఈൺֱ
   Makoto Hasegawa
   JAPAN CONTAINER DAYS V18.04
   

   View Slide

2. PAGE
   Japan Container DAYS
   v 18.04 2
   ຊ೔ͷࢿྉ͸ޙ΄Ͳެ։͠·͢ͷͰ
   ࣸਅͱ͔ࡱΒͳ͍Ͱେৎ෉Ͱ͢ʂ
   

   View Slide

3. SELF INTRODUCTION
   

   View Slide

4. PAGE
   Japan Container DAYS
   v 18.04 4
   SELF INTRODUCTION
   MAKOTO HASEGAWA
   TwiLer/@makocchi
   αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ॴଐ
   ීஈ͸σʔληϯλʔӡ༻΍ Private Cloud(OpenStack) Λ
   ߏஙɾӡ༻͍ͯ͠Δ
   ࠷ۙͰ͸ Private Cloud ্ʹల։ग़དྷΔίϯςφج൫ AKE Λ
   ߏஙɾӡ༻த
   Facebook/makocchi0923
   CKA (Cer*fied Kubernetes Administrator)
   

   View Slide

5. PAGE
   Japan Container DAYS
   v 18.04 5
   6
   TODAY’S
   AGENDA
   

   View Slide

6. PAGE
   Japan Container DAYS
   v 18.04 5
   6
   TODAY’S
   AGENDA
   SELF INTRODUCTION
   VARIOUS CONTAINER RUNTIME
   OCI AND CRI
   INSIDE OF CONTAINER RUNTIME
   PATTERN OF RUNNING CONTAINERS
   CRI-O VS CONTAINERD
   STORAGE DRIVER
   CONCLUSION
   

   View Slide

7. PAGE
   Japan Container DAYS
   v 18.04 6
   ಥવͰ͕͢
   PREFACE
   

   View Slide

8. PAGE
   Japan Container DAYS
   v 18.04 7
   Kubernetes ͷ cluster ͷ
   ίϯςφ run*me ʹԿΛ࢖͍ͬͯ·͔͢ʁ
   PREFACE
   

   View Slide

9. PAGE
   Japan Container DAYS
   v 18.04 8
   ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ
   (ݸਓతͳ༧૝)
   PREFACE
   

   View Slide

10. PAGE
    Japan Container DAYS
    v 18.04 8
    ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ
    (ݸਓతͳ༧૝)
    Docker Other
    What
    container runtime
    is running on
    your k8s cluster?
    95%
    PREFACE
    

    View Slide

11. VARIOUS CONTAINER RUNTIME
    ON KUBERNETES
    

    View Slide

12. PAGE
    Japan Container DAYS
    v 18.04 10
    Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ
    ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me
    CONTAINER RUNTIMES
    ON KUBERNETES
    

    View Slide

13. PAGE
    Japan Container DAYS
    v 18.04 10
    Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ
    Frak*
    Frak*
    Docker cri-o containerd
    ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me
    CONTAINER RUNTIMES
    ON KUBERNETES
    

    View Slide

14. PAGE
    Japan Container DAYS
    v 18.04 11
    ࠷΋༗໊ͳ Container Run*me
    1.11 ͔Β಺෦ͷ component ͸
    runC + containerd ʹ෼ׂ͞Εͨ
    Frak*
    Docker cri-o
    Frak* containerd
    CRI, OCI ʹ४ڌͨ͠
    Kubernetes ઐ༻ͷ run*me
    Docker ͷґଘ͔Β୤٫
    Hypervisor ܦ༝Ͱ pod ͕ల։͞ΕΔ
    kernel ͕ share ͞Εͳ͍ͷͰΑΓη
    ΩϡΞͳ؀ڥʹͳΔ
    Docker ͷ಺෦ component ͕෼཭
    Kubernetes ͱ࿈ܞ͢Δҝʹ͸
    cri-containerd ͕ඞཁ͕ͩͬͨɺ
    kubernetes 1.10 ͱ containerd v1.1 ͔Β
    na*ve Ͱ௨৴Ͱ͖ΔΑ͏ʹͳͬͨ
    CONTAINER RUNTIMES
    ON KUBERNETES
    

    View Slide

15. PAGE
    Japan Container DAYS
    v 18.04 12
    HOW ABOUT RKT …?
    Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated
    rktnetes has been deprecated in favor of rktlet (#58418)
    CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ
    hLps://github.com/kubernetes-incubator/rktlet
    kubelet ͷ “—container-run*me" ͷ help ʹ
    ʮ The container run*me to use. Possible values: 'docker',
    'remote', 'rkt (deprecated)'. (default “docker”)ʯ
    CONTAINER RUNTIMES
    ON KUBERNETES
    

    View Slide

16. PAGE
    Japan Container DAYS
    v 18.04 12
    HOW ABOUT RKT …?
    Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated
    rktnetes has been deprecated in favor of rktlet (#58418)
    CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ
    hLps://github.com/kubernetes-incubator/rktlet
    kubelet ͷ “—container-run*me" ͷ help ʹ
    ʮ The container run*me to use. Possible values: 'docker',
    'remote', 'rkt (deprecated)'. (default “docker”)ʯ
    CONTAINER RUNTIMES
    ON KUBERNETES
    

    View Slide

17. WHAT IS OCI AND CRI ?
    

    View Slide

18. PAGE
    Japan Container DAYS
    v 18.04 14
    OCI
    Open Container Ini*a*ve
    2015೥1݄ʹίϯςφͷඪ४࢓༷Λࡦఆ͢ΔͨΊʹઃཱ
    ͞Εͨஂମ
    ͦͷஂମʹΑͬͯࡢ೥ʮOCI v1.0ʯ͕ࡦఆ͞Εͨ
    OCI v1.0 ͸ Container Run*me ͷඪ४࢓༷Ͱ͋Δ
    ʮRun*me Specifica*on v1.0ʯͱ Container Image ͷඪ४
    ࢓༷Ͱ͋ΔʮFormat Specifica*on v1.0ʯ͔Βߏ੒͞Εͯ
    ͍Δ
    OCI AND CRI
    

    View Slide

19. PAGE
    Japan Container DAYS
    v 18.04 15
    OCI
    Open Container Ini*a*ve
    ͪͳΈʹ௚ۙʹ͜Μͳൃද͕͞Εͨ
    ʮThe New Stack: “Open Container Ini*a*ve Creates a
    Distribu*on Specifica*on for Registries”ʯ(※)
    Docker Registry version 2 ΛݩʹΠϝʔδ഑෍ͷඪ४ԽΛ
    ਐΊ͍ͯ͘
    (Run*me ͱ Format ʹ͙࣍ 3 ͭ໨ͷඪ४Խ)
    ※ hLps://www.opencontainers.org/news/2018/04/11/the-new-stack-open-container-ini*a*ve-creates-a-distribu*on-specifica*on-for-registries
    OCI AND CRI
    

    View Slide

20. PAGE
    Japan Container DAYS
    v 18.04 16
    CRI
    The Container Run*me Interface
    Kubernetes 1.5 ͔Β release(alpha) ͞Εͨ(※) kubelet ͱ
    Container Run*me ͕௨৴͢Δҝͷ plugin interface
    Container Run*me ଆͷมߋʹ௥ਵ͠ଓ͚Δͷ͸େมͳҝ
    ౷ҰԽ͞Εͨ interface Λఆٛͯ͠
    Kubernetes ͷ҆ఆԽͱϝϯςφϯεͷίετΛݮΒͨ͠
    (※) hLps://kubernetes.io/blog/2016/12/container-run*me-interface-cri-in-kubernetes
    OCI AND CRI
    

    View Slide

21. INSIDE OF
    CONTAINER RUNTIMES
    

    View Slide

22. PAGE
    Japan Container DAYS
    v 18.04 18
    Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸
    ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL
    

    View Slide

23. PAGE
    Japan Container DAYS
    v 18.04 18
    kubelet Container run*me
    CRI
    run*me
    Containers
    OCI
    Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸
    ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL
    

    View Slide

24. PAGE
    Japan Container DAYS
    v 18.04 19
    Container run*me ͷதʹ·ͨ run*me ͕͋Δɾɾɾ
    ฆΒΘ͍͠ɾɾʂ
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL
    

    View Slide

25. HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME
    

    View Slide

26. HIGH-LEVEL CONTAINER RUNTIME
    LOW-LEVEL CONTAINER RUNTIME
    

    View Slide

27. HIGH-LEVEL CONTAINER RUNTIME
    LOW-LEVEL CONTAINER RUNTIME
    OCI/JSON file Λड͚औΓϓϩηεΛ࣮ߦ
    ୅දతͳ΋ͷ͸ runC
    CRI/gRPC Λड͚औΓ Low-level run*me ΁౉͢
    ୅දతͳ΋ͷ͸ cri-o ΍ containerd
    Image ͷ؅ཧΛߦ͏ (pull / push / rm ..) Image ͷ؅ཧ͸͠ͳ͍
    

    View Slide

28. PAGE
    Japan Container DAYS
    v 18.04 22
    kubelet
    Container run*me
    CRI Containers
    OCI
    ઌఔʮContainer run*meʯͱݴ͍ͬͯͨ෦෼ͷதͰ
    ɹCRI Λड͚Δ run*me ΛʮHigh-level Container run*meʯ
    ɹOCI Λड͚Δ run*me ΛʮLow-level Container run*meʯ
    ͱݴ͏৔߹͕ଟ͍
    ※ CRI run*me / OCI run*me ͱݴ͏৔߹΋͋Δ
    High-level Low-level
    INSIDE OF CONTAINER RUNTIMES
    HIGH-LEVEL AND LOW-LEVEL
    

    View Slide

29. PATTERN OF
    RUNNING
    CONTAINERS
    

    View Slide

30. PAGE
    Japan Container DAYS
    v 18.04 24
    PATTERN OF RUNNING CONTAINERS
    SWAPPABLE RUNTIMES
    CRI ΍ OCI ʹΑͬͯΠϯλʔϑΣΠεΛ౷Ұͨ͜͠
    ͱʹΑͬͯɺϢʔβʔ͸ High-level Container
    run*me ͱ Low-level Container run*me Λࣗ༝ʹબ୒
    ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ
    ͍͔ͭ͘۩ମతʹ૊Έ߹ΘͤΛݟ͍͖ͯ·͠ΐ͏
    

    View Slide

31. PAGE
    Japan Container DAYS
    v 18.04 25
    PATTERN OF RUNNING CONTAINERS
    01PaLern 01 Docker
    The most popular paLern in the world
    Using containerd and runC to run containers inside Docker
    

    View Slide

32. PAGE
    Japan Container DAYS
    v 18.04 26
    Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏
    Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ
    Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ)
    PATTERN OF RUNNING CONTAINERS
    01 DOCKER
    

    View Slide

33. PAGE
    Japan Container DAYS
    v 18.04
    Docker
    26
    kubelet containerd runC
    Containers
    OCI
    Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏
    Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ
    Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ)
    PATTERN OF RUNNING CONTAINERS
    01 DOCKER
    dockershim
    CRI
    

    View Slide

34. PAGE
    Japan Container DAYS
    v 18.04 27
    PATTERN OF RUNNING CONTAINERS
    02PaLern 02 cri-o
    Using cri-o for High-level Container run*me
    Using runC to run containers with OCI interface
    

    View Slide

35. PAGE
    Japan Container DAYS
    v 18.04 28
    PATTERN OF RUNNING CONTAINERS
    02 CRI-O
    Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ
    

    View Slide

36. PAGE
    Japan Container DAYS
    v 18.04 28
    cri-o
    kubelet runC
    CRI Containers
    OCI
    PATTERN OF RUNNING CONTAINERS
    02 CRI-O
    Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ
    

    View Slide

37. PAGE
    Japan Container DAYS
    v 18.04 29
    PATTERN OF RUNNING CONTAINERS
    03PaLern 03 containerd
    Using containerd for High-level Container run*me
    Using runC to run containers with OCI interface
    

    View Slide

38. PAGE
    Japan Container DAYS
    v 18.04 30
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD
    Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ
    cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ
    

    View Slide

39. PAGE
    Japan Container DAYS
    v 18.04 30
    runC
    kubelet
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD
    Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ
    cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ
    cri-containerd containerd
    gRPC
    CRI Containers
    OCI
    

    View Slide

40. PAGE
    Japan Container DAYS
    v 18.04 31
    runC
    kubelet cri-containerd containerd
    gRPC
    CRI Containers
    OCI
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD
    

    View Slide

41. PAGE
    Japan Container DAYS
    v 18.04 31
    runC
    kubelet cri-containerd containerd
    gRPC
    CRI Containers
    OCI
    D E P R E C A T E D
    PATTERN OF RUNNING CONTAINERS
    03 CONTAINERD
    

    View Slide

42. PAGE
    Japan Container DAYS
    v 18.04 32
    PATTERN OF RUNNING CONTAINERS
    03’ CONTAINERD (NEW)
    kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ
    

    View Slide

43. PAGE
    Japan Container DAYS
    v 18.04 32
    PATTERN OF RUNNING CONTAINERS
    03’ CONTAINERD (NEW)
    containerd
    kubelet runC
    CRI Containers
    OCI
    kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ
    

    View Slide

44. PAGE
    Japan Container DAYS
    v 18.04 33
    PATTERN OF RUNNING CONTAINERS
    04PaLern 04 rkt
    Using rkt and systemd for container engine
    

    View Slide

45. PAGE
    Japan Container DAYS
    v 18.04 34
    kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ
    Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷)
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)
    

    View Slide

46. PAGE
    Japan Container DAYS
    v 18.04
    systemd
    34
    kubelet
    rkt run
    Containers
    kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ
    Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷)
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)
    rkt pkg
    

    View Slide

47. PAGE
    Japan Container DAYS
    v 18.04 35
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)
    systemd
    kubelet
    rkt run
    Containers
    rkt pkg
    

    View Slide

48. PAGE
    Japan Container DAYS
    v 18.04 35
    PATTERN OF RUNNING CONTAINERS
    04 RKT (AKA RKTNETES)
    systemd
    kubelet
    rkt run
    Containers
    rkt pkg
    D E P R E C A T E D
    

    View Slide

49. PAGE
    Japan Container DAYS
    v 18.04 36
    kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ
    ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ
    PATTERN OF RUNNING CONTAINERS
    04’ RKT (AKA RKTLET)
    

    View Slide

50. PAGE
    Japan Container DAYS
    v 18.04
    systemd
    36
    rkt app
    kubelet
    kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ
    ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ
    rktlet
    CRI Containers
    PATTERN OF RUNNING CONTAINERS
    04’ RKT (AKA RKTLET)
    

    View Slide

51. PAGE
    Japan Container DAYS
    v 18.04 37
    PATTERN OF RUNNING CONTAINERS
    05PaLern 05 “vm-like” container
    Running “vm-like” isolated containers
    Using cc-run*me or runV, etc
    

    View Slide

52. PAGE
    Japan Container DAYS
    v 18.04 38
    PATTERN OF RUNNING CONTAINERS
    05 “VM-LIKE” CONTAINERS
    Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ
    (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍)
    

    View Slide

53. PAGE
    Japan Container DAYS
    v 18.04 38
    anything
    kubelet
    runV
    CRI
    PATTERN OF RUNNING CONTAINERS
    05 “VM-LIKE” CONTAINERS
    Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ
    (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍)
    cc-run*me
    OCI
    Containers?
    kata-run*me
    VM?
    

    View Slide

54. SUMMARY
    

    View Slide

55. PAGE
    Japan Container DAYS
    v 18.04 40
    SUMMARY
    Container Run*me ʹ͸͍Ζ͍Ζ͋Δ
    ίϯςφͷ౷Ұج४ͱͯ͠ OCI ͕ଘࡏ͢Δ
    Kubernetes ͱ Run*me ͷؒ͸ CRI ͱ͍͏ج४͕͋Δ
    CRI ͱ OCI ʹରԠ͍ͯ͠Ε͹ Run*me ͸ࣗ༝ʹೖΕସ͕͑Մೳ
    Container Run*me ͸ High-level ͱ Low-level ʹ
    ෼͚ͯߟ͑Δ͜ͱ͕Ͱ͖Δ
    ίϯςφͷΑ͏ʹ VM Λಈ͔͢ run*me ΋ଘࡏ͢Δ
    

    View Slide

56. CRI-O
    CONTAINERD
    VS
    

    View Slide

57. PAGE
    Japan Container DAYS
    v 18.04 42
    PERFORMANCE
    CRI-O VS CONTAINERD
    cri-o ͱ containerd ʹର͠ɺcrictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏
    How to bench run*me?
    sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 6 step
    ࣮͸ critest ͱ͍͏ benchmark ͕͋Δ (hLps://github.com/kubernetes-incubator/cri-tools/tree/master/cmd/critest)
    Ͱ΋ critest ͸্ख͘ಈ͔ͳ͔ͬͨͷͰࠓճ͸ಠࣗͷ benchmark ͰରԠ
    

    View Slide

58. PAGE
    Japan Container DAYS
    v 18.04 43
    PERFORMANCE
    CRI-O VS CONTAINERD
    How to bench run*me? | 6 steps
    

    View Slide

59. PAGE
    Japan Container DAYS
    v 18.04 43
    PERFORMANCE
    CRI-O VS CONTAINERD
    How to bench run*me? | 6 steps
    sandbox pod Λ࡞੒͢Δ
    sandbox pod ʹ busybox ίϯςφΛ࡞੒ɾىಈ͢Δ
    busybox ίϯςφΛఀࢭ͢Δ
    busybox ίϯςφΛ࡟আ͢Δ
    sandbox pod Λ࡟আ͢Δ
    sandbox pod Λఀࢭ͢Δ
    

    View Slide

60. PAGE
    Japan Container DAYS
    v 18.04 44
    PERFORMANCE
    CRI-O VS CONTAINERD
    x100 LOOP
    100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢
    

    View Slide

61. PAGE
    Japan Container DAYS
    v 18.04 44
    PERFORMANCE
    CRI-O VS CONTAINERD
    x100 LOOP
    runp start stop rm stopp rmp
    100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢
    

    View Slide

62. cri-o vs containerd
    with runC(docker-runc)
    The most popular low-level container run*me
    

    View Slide

63. PAGE
    Japan Container DAYS
    v 18.04 46
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    runC (docker-runc)
    

    View Slide

64. PAGE
    Japan Container DAYS
    v 18.04 46
    Average of 100 *mes loop “crictl” opera*on
    cri-o 1.9.11
    cri-o 1.10.0
    containerd 1.1.0-rc2
    cri-containerd 1.0.0-beta.0
    0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec
    0.66
    0.37
    0.88
    1.29
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    runC (docker-runc)
    

    View Slide

65. PAGE
    Japan Container DAYS
    v 18.04 47
    PERFORMANCE
    CRI-O VS CONTAINERD
    cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍
    loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ
    Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠
    cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ
    Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕
    performance ͕͍͍
    WITH
    runC (docker-runc)
    

    View Slide

66. PAGE
    Japan Container DAYS
    v 18.04 47
    PERFORMANCE
    CRI-O VS CONTAINERD
    cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍
    loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ
    Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠
    cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ
    Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕
    performance ͕͍͍
    cri-o 1.9.11
    cri-o 1.10.0
    containerd 1.1.0-rc2
    cri-containerd 1.0.0-beta.0
    0.00 sec 0.65 sec 1.30 sec
    0.66
    0.37
    0.88
    1.29
    WITH
    runC (docker-runc)
    

    View Slide

67. PAGE
    Japan Container DAYS
    v 18.04 48
    PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)
    

    View Slide

68. PAGE
    Japan Container DAYS
    v 18.04
    0
    0.6
    1.2
    1.8
    2.4
    cri-o 1.9.11
    cri-o 1.10.0
    containerd-1.1.0.rc2
    cri-containerd
    48
    PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)
    

    View Slide

69. PAGE
    Japan Container DAYS
    v 18.04
    0
    0.6
    1.2
    1.8
    2.4
    cri-o 1.9.11
    cri-o 1.10.0
    containerd-1.1.0.rc2
    cri-containerd
    48
    PERFORMANCE CRI-O VS CONTAINERD
    cri-o ͸͕࣌ؒܦͭʹ࿈Εͯঃʑʹ performance ͕མͪΔ
    containerd ͸҆ఆ͍ͯ͠Δ
    runC (docker-runc)
    

    View Slide

70. cri-o vs containerd
    with cc-run*me
    Intel Clear Containers run*me
    

    View Slide

71. PAGE
    Japan Container DAYS
    v 18.04 50
    PERFORMANCE
    CRI-O VS CONTAINERD
    cc-run*me
    

    View Slide

72. PAGE
    Japan Container DAYS
    v 18.04 50
    PERFORMANCE
    CRI-O VS CONTAINERD
    hypervisor ʹ KVM ͕࢖ΘΕΔ
    hLps://clearlinux.org/containers
    Hypervisor
    Kernel
    ܰྔ͔ͭߴ଎ʹىಈ͢Δ
    Clear Linux OS
    Security
    Intel VT Λ࢖༻͠ɺίϯςφΑΓ
    ηΩϡΞʹىಈ͢Δ͜ͱ͕Ͱ͖Δ
    OCI compa*ble
    run*me Ͱ͋Δ cc-run*me ͸ OCI support
    cc-run*me
    

    View Slide

73. PAGE
    Japan Container DAYS
    v 18.04 51
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    cc-run*me (clear container)
    

    View Slide

74. PAGE
    Japan Container DAYS
    v 18.04 51
    Average of 100 *mes loop “crictl” opera*on
    cri-o 1.9.11
    cri-o 1.10.0
    containerd 1.1.0-rc2
    cri-containerd 1.0.0-beta.0
    0.00 sec 3.50 sec 7.00 sec 10.50 sec 14.00 sec
    2.69
    2.35
    12.49
    12.86
    PERFORMANCE
    CRI-O VS CONTAINERD
    WITH
    cc-run*me (clear container)
    

    View Slide

75. PAGE
    Japan Container DAYS
    v 18.04 52
    PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container)
    

    View Slide

76. PAGE
    Japan Container DAYS
    v 18.04
    0
    3.5
    7
    10.5
    14
    cri-o 1.9.11
    cri-o 1.10.0
    containerd-1.1.0.rc2
    cri-containerd
    52
    PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container)
    cc-run*me ͷ৔߹͸྆ऀڞʹऴ࢝҆ఆ͍ͯ͠Δ
    

    View Slide

77. cri-o vs containerd
    with kata-run*me
    Kata-container
    

    View Slide

78. PAGE
    Japan Container DAYS
    v 18.04 54
    PERFORMANCE
    CRI-O VS CONTAINERD
    kata-run*me
    

    View Slide

79. PAGE
    Japan Container DAYS
    v 18.04 54
    PERFORMANCE
    CRI-O VS CONTAINERD
    OpenStack Founda*on ʹΑΓ؅ཧ
    hLps://katacontainers.io/
    Open Source
    runV + Clear Container
    Hyper ʹΑΔ runV ͱ Intel ʹΑΔ
    Clear Linux ͕ϕʔεͱͳ͍ͬͯΔ
    Security
    ͦΕͧΕͷίϯςφ͸ִ཭͞Εͨ
    Kernel ʹΑͬͯىಈ͞ΕΔ
    OCI compa*ble
    run*me Ͱ͋Δ kata-run*me ͸ OCI support
    kata-run*me
    

    View Slide

80. WARNING!!
    

    View Slide

81. cri-o ͱ containerd ͷઃఆΛม͑ͯ
    Low-level container run*me ʹ kata-run*me Λ
    ࢦఆ͕ͨ͠ error ͰىಈͰ͖ͣɾɾɾ
    Pod sandbox ͸࡞੒Մೳ͕ͩ
    Pod ʹ container Λ࡞੒͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ
    

    View Slide

82. cri-o vs containerd
    with kata-run*me
    Kata-container
    

    View Slide

83. cri-o vs containerd
    with kata-run*me
    Kata-container
    F A
    I L
    

    View Slide

84. cri-o vs containerd
    with kata-run*me
    Kata-container
    F A
    I L
    

    View Slide

85. View Slide

86. View Slide

87. Docker
    with runc/cc-run*me/kata-run*me
    Docker with 3 Low-level container run*mes
    Revenge
    

    View Slide

88. PAGE
    Japan Container DAYS
    v 18.04 60
    PERFORMANCE
    DOCKER + 3 RUNTIMES
    Docker ͷ run*me ʹ kata-run*me Λࢦఆͨ͠ͱ͜Ζ্ख͘ಈ͍ͨ
    How to bench run*me?
    ୯७ʹ docker run ͷੑೳΛݕূ͢Δ
    ࠓճͷݕূ΋ 100 ճ docker run ͯ࣌ؒ͠Λܭଌ
    docker run --rm --run*me ${RUNTIME} hello-world
    

    View Slide

89. PAGE
    Japan Container DAYS
    v 18.04 61
    PERFORMANCE
    DOCKER + 3 RUNTIMES
    

    View Slide

90. PAGE
    Japan Container DAYS
    v 18.04 61
    Average of 100 *mes loop “docker run” opera*on
    runc
    cc-run*me
    kata-run*me
    0.00 sec 0.65 sec 1.30 sec 1.95 sec 2.60 sec
    2.07
    2.45
    0.45
    PERFORMANCE
    DOCKER + 3 RUNTIMES
    

    View Slide

91. PAGE
    Japan Container DAYS
    v 18.04 62
    PERFORMANCE DOCKER + 3 RUNTIMES
    

    View Slide

92. PAGE
    Japan Container DAYS
    v 18.04
    0
    0.75
    1.5
    2.25
    3
    runc
    cc-run*me
    kata-run*me
    62
    PERFORMANCE DOCKER + 3 RUNTIMES
    cc-run*meʹएׯϜϥ͕͋Γͦ͏
    

    View Slide

93. STORAGE
    DRIVER
    

    View Slide

94. PAGE
    Japan Container DAYS
    v 18.04 64
    PERFORMANCE
    STORAGE DRIVER
    run*me ͷ࣮ߦ଎౓ʹ storage driver ͕ͲΕ͘Β͍Өڹ͋Δͷ͔ݕূ
    storage driver is important
    ઌఔͱಉ͡ docker run ͷ benchmark Λ͢Δ
    ࢖༻ͨ͠ storage driver ͸ 4छྨ
    

    View Slide

95. PAGE
    Japan Container DAYS
    v 18.04 65
    PERFORMANCE
    STORAGE DRIVER
    

    View Slide

96. PAGE
    Japan Container DAYS
    v 18.04 65
    PERFORMANCE
    STORAGE DRIVER
    devicemapper (loopback)
    RHEL ܥͰ͸ default ʹͳ͍ͬͯΔ
    ख͕ܰͩ performance ͸ྑ͘ͳ͍
    devicemapper (direct-lvm)
    Produc*on ؀ڥͰ devicemapper Λ
    ࢖͏৔߹͸ direct-lvm ͕ਪ঑
    loopback ΑΓ͸ੑೳ͕͍͍
    overlay2
    overlay Ͱ໰୊͕͋ͬͨ inode ૿େͱ
    performance ҡ࣋ʹରԠ͍ͯ͠Δ
    overlay
    UnionFS ͷ1ͭͰϑΝΠϧ΍σΟϨΫτϦΛ
    ಁաతʹॏͶΔ͜ͱ͕Ͱ͖Δ
    performance ͸ྑ͍
    

    View Slide

97. PAGE
    Japan Container DAYS
    v 18.04 66
    PERFORMANCE
    STORAGE DRIVER
    

    View Slide

98. PAGE
    Japan Container DAYS
    v 18.04 66
    runc
    cc-run*me
    kata-run*me
    overlay
    overlay2
    dm(loopback)
    dm(direct)
    overlay
    overlay2
    dm(loopback)
    dm(direct)
    overlay
    overlay2
    dm(loopback)
    dm(direct)
    0.00 sec 0.75 sec 1.50 sec 2.25 sec 3.00 sec
    2.865
    2.862
    0.849
    2.391
    2.606
    0.637
    2.061
    2.376
    0.450
    2.075
    2.452
    0.453
    PERFORMANCE
    STORAGE DRIVER
    Ͳͷ run*me Ͱ΋ overlay2 ͕࠷଎
    devicemapper(direct) ͷํ͕஗͘ͳͬͯ͠·ͬͨ
    

    View Slide

99. PAGE
    Japan Container DAYS
    v 18.04 67
    TODAY’S
    CONCLUSION
    

    View Slide

100. PAGE
     Japan Container DAYS
     v 18.04 68
     About
     Container Run*me
     TODAY’S
     CONCLUSION
     Container Run*me ʹ͸ High-Level ͱ Low-Level ͷ
     Run*me ʹ۠ผ͢Δ͜ͱ͕Ͱ͖Δ
     CRI ΍ OCI ͱ͍ͬͨ࢓༷͕ࡦఆ͞ΕͨͷͰ
     Run*me ͷೖΕସ͕͑ΑΓॊೈʹߦ͑ΔΑ͏ʹͳ͍ͬͯΔ
     Run*me ʹΑͬͯੑೳࠩ͸एׯ͋Δ͕ɺ·ͩ·ͩൃల్্
     Docker(containerd) ͔Βੵۃతʹ৐Γ׵͑Δཧ༝͸ݱ࣌఺Ͱ͸ແͦ͞͏
     Container ͷΑ͏ʹ VM Λىಈͤ͞Δٕज़͸ࠓޙظ଴Ͱ͖ͦ͏
     

     View Slide

101. PAGE
     Japan Container DAYS
     v 18.04 69
     About
     Storage Driver
     TODAY’S
     CONCLUSION
     Performance Λߟ͑Δͱ overlay ͕ྑ͍બ୒ࢶͩͱࢥΘΕΔ
     overlay2 ͕࢖͑ΔͳΒ͹ੵۃతʹ࢖͍͍ͬͯ
     ͨͩ͠ overlay Λ࢖͏ͳΒ͹ kernel ͸࠷৽ʹ্͓͍͛ͯͨ΄͏͕
     ͍͍ͱࢥΘΕΔ
     ݱ࣌఺Ͱͷ moby ͷ open ͳ issue
     ɹlabel:area/storage/overlay 29݅
     ɹlabel:area/storage/devicemapper 42݅
     ɹlabel:area/storage/aufs 29݅
     ɹlabel:area/storage/btr 15݅
     

     View Slide

102. PAGE
     Japan Container DAYS
     v 18.04 70
     ʮ·ͩ͋ΔΑʂʯ
     

     View Slide

103. PAGE
     Japan Container DAYS
     v 18.04 71
     αΠόʔΤʔδΣϯτ
     ΞυςΫຊ෦ɹ
     Πϯλʔωοτ޿ࠂʹ͓͍ͯɺ޿ࠂ഑৴ͷ࠷దԽ΍ϝσΟΞͷऩӹ࠷େԽͱ͍͏؍఺͔Β
     ΞυςΫϊϩδʔͷॏཁ౓͕ߴ·͍ͬͯ·͢ɻ
     αΠόʔΤʔδΣϯτͰ͸ΞυςΫϊϩδʔ෼໺ʹ͓͚Δ
     ͜ΕΒͷαʔϏεʹ͍֤ͭͯࢠձࣾΛ௨͡։ൃ͓ͯ͠Γ·͕ͨ͠ɺ
     ֤αʔϏεͷ։ൃ෦໳Λԣஅͯ͠૊৫Խ͢Δઐ໳෦ॺͱͯ͠ΞυςΫຊ෦͕ઃཱ͞Ε·ͨ͠ɻ
     

     View Slide

104. PAGE
     Japan Container DAYS
     v 18.04 72
     αΠόʔΤʔδΣϯτ
     ΞυςΫຊ෦ɹ
     ʮίϯςφ͕޷͖ͳਓ΋޷͖͡Όͳ͍ਓ΋ઈࢍืूதͰ͢ɻ
     ɹΧδϡΞϧͳ໘ஊ΍ϥϯνަྲྀ΋OKͰ͢ɻ
     ɹڵຯ͕͋ΔํɺੋඇҰॹʹಇ͖·͠ΐ͏ʯ
     

     View Slide

105. Docker ͚ͩ͡Όͳ͍
     ίϯςφ run*me పఈൺֱ
     Makoto Hasegawa
     JAPAN CONTAINER DAYS V18.04
     THANK YOU
     

     View Slide

106. APPENDIX
     

     View Slide

107. RKTLET
     

     View Slide

108. PAGE
     Japan Container DAYS
     v 18.04 76
     PERFORMANCE
     RKTLET
     rktlet ʹରͯ͠ crictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏
     How to bench run*me?
     sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 4 step
     rktlet ͷ৔߹ container Λ create ͢Δͱ start ͠ͳͯ͘΋ىಈͨ͠ঢ়ଶʹͳΔΆ͍
     container ʹରͯ͠ stop ͱ rm ͕Ͱ͖ͳ͔ͬͨ (͏·͘ઃఆͰ͖ͯͳ͍Մೳੑ͋Γ)
     sandbox Λ rm ͢Δͱ error: ͱग़Δ͕ͪΌΜͱফ͍͑ͯΔ
     

     View Slide

109. PAGE
     Japan Container DAYS
     v 18.04 77
     PERFORMANCE
     RKTLET
     

     View Slide

110. PAGE
     Japan Container DAYS
     v 18.04 77
     PERFORMANCE
     RKTLET
     Average of 100 *mes loop “crictl” opera*on
     rktlet
     cri-o 1.9.11
     cri-o 1.10.0
     containerd 1.1.0-rc2
     cri-containerd 1.0.0-beta.0
     0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec
     0.66
     0.37
     0.88
     1.29
     0.78
     

     View Slide

111. PAGE
     Japan Container DAYS
     v 18.04 78
     PERFORMANCE RKTLET
     

     View Slide

112. PAGE
     Japan Container DAYS
     v 18.04
     0
     0.6
     1.2
     1.8
     2.4
     cri-o 1.9.11
     cri-o 1.10.0
     containerd-1.1.0.rc2
     cri-containerd
     rktlet
     78
     PERFORMANCE RKTLET
     

     View Slide

113. Docker ͚ͩ͡Όͳ͍
     ίϯςφ run*me పఈൺֱ
     Makoto Hasegawa
     JAPAN CONTAINER DAYS V18.04
     THANK YOU
     AGAIN
     

     View Slide