About_container_runtimes_Japan_container_days_v18.04

 About_container_runtimes_Japan_container_days_v18.04

Japan Container Days(JKD) v18.04( での発表資料です。
「Dockerだけじゃないコンテナ runtime 徹底比較」

合わせて読みたい
「runc だけじゃないコンテナ low level runtime 徹底比較」
https://speakerdeck.com/makocchi/jkd-20181205-about-low-level-runtimes

D1b28ca276bee52e56ba11785f70d2d6?s=128

makocchi

April 19, 2018
Tweet

Transcript

  1. 4.

    PAGE Japan Container DAYS v 18.04 4 SELF INTRODUCTION MAKOTO

    HASEGAWA TwiLer/@makocchi αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ॴଐ ීஈ͸σʔληϯλʔӡ༻΍ Private Cloud(OpenStack) Λ ߏஙɾӡ༻͍ͯ͠Δ ࠷ۙͰ͸ Private Cloud ্ʹల։ग़དྷΔίϯςφج൫ AKE Λ ߏஙɾӡ༻த Facebook/makocchi0923 CKA (Cer*fied Kubernetes Administrator)
  2. 6.

    PAGE Japan Container DAYS v 18.04 5 6 TODAY’S AGENDA

    SELF INTRODUCTION VARIOUS CONTAINER RUNTIME OCI AND CRI INSIDE OF CONTAINER RUNTIME PATTERN OF RUNNING CONTAINERS CRI-O VS CONTAINERD STORAGE DRIVER CONCLUSION
  3. 8.

    PAGE Japan Container DAYS v 18.04 7 Kubernetes ͷ cluster

    ͷ ίϯςφ run*me ʹԿΛ࢖͍ͬͯ·͔͢ʁ PREFACE
  4. 10.

    PAGE Japan Container DAYS v 18.04 8 ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ (ݸਓతͳ༧૝) Docker

    Other What container runtime is running on your k8s cluster? 95% PREFACE
  5. 12.

    PAGE Japan Container DAYS v 18.04 10 Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ ※Kubernetes

    1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me CONTAINER RUNTIMES ON KUBERNETES
  6. 13.

    PAGE Japan Container DAYS v 18.04 10 Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ Frak*

    Frak* Docker cri-o containerd ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me CONTAINER RUNTIMES ON KUBERNETES
  7. 14.

    PAGE Japan Container DAYS v 18.04 11 ࠷΋༗໊ͳ Container Run*me

    1.11 ͔Β಺෦ͷ component ͸ runC + containerd ʹ෼ׂ͞Εͨ Frak* Docker cri-o Frak* containerd CRI, OCI ʹ४ڌͨ͠ Kubernetes ઐ༻ͷ run*me Docker ͷґଘ͔Β୤٫ Hypervisor ܦ༝Ͱ pod ͕ల։͞ΕΔ kernel ͕ share ͞Εͳ͍ͷͰΑΓη ΩϡΞͳ؀ڥʹͳΔ Docker ͷ಺෦ component ͕෼཭ Kubernetes ͱ࿈ܞ͢Δҝʹ͸ cri-containerd ͕ඞཁ͕ͩͬͨɺ kubernetes 1.10 ͱ containerd v1.1 ͔Β na*ve Ͱ௨৴Ͱ͖ΔΑ͏ʹͳͬͨ CONTAINER RUNTIMES ON KUBERNETES
  8. 15.

    PAGE Japan Container DAYS v 18.04 12 HOW ABOUT RKT

    …? Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated rktnetes has been deprecated in favor of rktlet (#58418) CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ hLps://github.com/kubernetes-incubator/rktlet kubelet ͷ “—container-run*me" ͷ help ʹ ʮ The container run*me to use. Possible values: 'docker', 'remote', 'rkt (deprecated)'. (default “docker”)ʯ CONTAINER RUNTIMES ON KUBERNETES
  9. 16.

    PAGE Japan Container DAYS v 18.04 12 HOW ABOUT RKT

    …? Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated rktnetes has been deprecated in favor of rktlet (#58418) CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ hLps://github.com/kubernetes-incubator/rktlet kubelet ͷ “—container-run*me" ͷ help ʹ ʮ The container run*me to use. Possible values: 'docker', 'remote', 'rkt (deprecated)'. (default “docker”)ʯ CONTAINER RUNTIMES ON KUBERNETES
  10. 18.

    PAGE Japan Container DAYS v 18.04 14 OCI Open Container

    Ini*a*ve 2015೥1݄ʹίϯςφͷඪ४࢓༷Λࡦఆ͢ΔͨΊʹઃཱ ͞Εͨஂମ ͦͷஂମʹΑͬͯࡢ೥ʮOCI v1.0ʯ͕ࡦఆ͞Εͨ OCI v1.0 ͸ Container Run*me ͷඪ४࢓༷Ͱ͋Δ ʮRun*me Specifica*on v1.0ʯͱ Container Image ͷඪ४ ࢓༷Ͱ͋ΔʮFormat Specifica*on v1.0ʯ͔Βߏ੒͞Εͯ ͍Δ OCI AND CRI
  11. 19.

    PAGE Japan Container DAYS v 18.04 15 OCI Open Container

    Ini*a*ve ͪͳΈʹ௚ۙʹ͜Μͳൃද͕͞Εͨ ʮThe New Stack: “Open Container Ini*a*ve Creates a Distribu*on Specifica*on for Registries”ʯ(※) Docker Registry version 2 ΛݩʹΠϝʔδ഑෍ͷඪ४ԽΛ ਐΊ͍ͯ͘ (Run*me ͱ Format ʹ͙࣍ 3 ͭ໨ͷඪ४Խ) ※ hLps://www.opencontainers.org/news/2018/04/11/the-new-stack-open-container-ini*a*ve-creates-a-distribu*on-specifica*on-for-registries OCI AND CRI
  12. 20.

    PAGE Japan Container DAYS v 18.04 16 CRI The Container

    Run*me Interface Kubernetes 1.5 ͔Β release(alpha) ͞Εͨ(※) kubelet ͱ Container Run*me ͕௨৴͢Δҝͷ plugin interface Container Run*me ଆͷมߋʹ௥ਵ͠ଓ͚Δͷ͸େมͳҝ ౷ҰԽ͞Εͨ interface Λఆٛͯ͠ Kubernetes ͷ҆ఆԽͱϝϯςφϯεͷίετΛݮΒͨ͠ (※) hLps://kubernetes.io/blog/2016/12/container-run*me-interface-cri-in-kubernetes OCI AND CRI
  13. 22.

    PAGE Japan Container DAYS v 18.04 18 Kubelet ͔Β CRI

    Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸ ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
  14. 23.

    PAGE Japan Container DAYS v 18.04 18 kubelet Container run*me

    CRI run*me Containers OCI Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸ ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
  15. 24.

    PAGE Japan Container DAYS v 18.04 19 Container run*me ͷதʹ·ͨ

    run*me ͕͋Δɾɾɾ ฆΒΘ͍͠ɾɾʂ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
  16. 27.

    HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME OCI/JSON file Λड͚औΓϓϩηεΛ࣮ߦ ୅දతͳ΋ͷ͸

    runC CRI/gRPC Λड͚औΓ Low-level run*me ΁౉͢ ୅දతͳ΋ͷ͸ cri-o ΍ containerd Image ͷ؅ཧΛߦ͏ (pull / push / rm ..) Image ͷ؅ཧ͸͠ͳ͍
  17. 28.

    PAGE Japan Container DAYS v 18.04 22 kubelet Container run*me

    CRI Containers OCI ઌఔʮContainer run*meʯͱݴ͍ͬͯͨ෦෼ͷதͰ ɹCRI Λड͚Δ run*me ΛʮHigh-level Container run*meʯ ɹOCI Λड͚Δ run*me ΛʮLow-level Container run*meʯ ͱݴ͏৔߹͕ଟ͍ ※ CRI run*me / OCI run*me ͱݴ͏৔߹΋͋Δ High-level Low-level INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
  18. 30.

    PAGE Japan Container DAYS v 18.04 24 PATTERN OF RUNNING

    CONTAINERS SWAPPABLE RUNTIMES CRI ΍ OCI ʹΑͬͯΠϯλʔϑΣΠεΛ౷Ұͨ͜͠ ͱʹΑͬͯɺϢʔβʔ͸ High-level Container run*me ͱ Low-level Container run*me Λࣗ༝ʹબ୒ ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ ͍͔ͭ͘۩ମతʹ૊Έ߹ΘͤΛݟ͍͖ͯ·͠ΐ͏
  19. 31.

    PAGE Japan Container DAYS v 18.04 25 PATTERN OF RUNNING

    CONTAINERS 01PaLern 01 Docker The most popular paLern in the world Using containerd and runC to run containers inside Docker
  20. 32.

    PAGE Japan Container DAYS v 18.04 26 Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim

    ͕ Docker ΁ͷڮ౉͠Λߦ͏ Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ) PATTERN OF RUNNING CONTAINERS 01 DOCKER
  21. 33.

    PAGE Japan Container DAYS v 18.04 Docker 26 kubelet containerd

    runC Containers OCI Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏ Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ) PATTERN OF RUNNING CONTAINERS 01 DOCKER dockershim CRI
  22. 34.

    PAGE Japan Container DAYS v 18.04 27 PATTERN OF RUNNING

    CONTAINERS 02PaLern 02 cri-o Using cri-o for High-level Container run*me Using runC to run containers with OCI interface
  23. 35.

    PAGE Japan Container DAYS v 18.04 28 PATTERN OF RUNNING

    CONTAINERS 02 CRI-O Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ
  24. 36.

    PAGE Japan Container DAYS v 18.04 28 cri-o kubelet runC

    CRI Containers OCI PATTERN OF RUNNING CONTAINERS 02 CRI-O Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ
  25. 37.

    PAGE Japan Container DAYS v 18.04 29 PATTERN OF RUNNING

    CONTAINERS 03PaLern 03 containerd Using containerd for High-level Container run*me Using runC to run containers with OCI interface
  26. 38.

    PAGE Japan Container DAYS v 18.04 30 PATTERN OF RUNNING

    CONTAINERS 03 CONTAINERD Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ
  27. 39.

    PAGE Japan Container DAYS v 18.04 30 runC kubelet PATTERN

    OF RUNNING CONTAINERS 03 CONTAINERD Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ cri-containerd containerd gRPC CRI Containers OCI
  28. 40.

    PAGE Japan Container DAYS v 18.04 31 runC kubelet cri-containerd

    containerd gRPC CRI Containers OCI PATTERN OF RUNNING CONTAINERS 03 CONTAINERD
  29. 41.

    PAGE Japan Container DAYS v 18.04 31 runC kubelet cri-containerd

    containerd gRPC CRI Containers OCI D E P R E C A T E D PATTERN OF RUNNING CONTAINERS 03 CONTAINERD
  30. 42.

    PAGE Japan Container DAYS v 18.04 32 PATTERN OF RUNNING

    CONTAINERS 03’ CONTAINERD (NEW) kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ
  31. 43.

    PAGE Japan Container DAYS v 18.04 32 PATTERN OF RUNNING

    CONTAINERS 03’ CONTAINERD (NEW) containerd kubelet runC CRI Containers OCI kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ
  32. 44.

    PAGE Japan Container DAYS v 18.04 33 PATTERN OF RUNNING

    CONTAINERS 04PaLern 04 rkt Using rkt and systemd for container engine
  33. 45.

    PAGE Japan Container DAYS v 18.04 34 kubelet ʹ಺แ͞Ε͍ͯΔ rkt

    ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷) PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES)
  34. 46.

    PAGE Japan Container DAYS v 18.04 systemd 34 kubelet rkt

    run Containers kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷) PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES) rkt pkg
  35. 47.

    PAGE Japan Container DAYS v 18.04 35 PATTERN OF RUNNING

    CONTAINERS 04 RKT (AKA RKTNETES) systemd kubelet rkt run Containers rkt pkg
  36. 48.

    PAGE Japan Container DAYS v 18.04 35 PATTERN OF RUNNING

    CONTAINERS 04 RKT (AKA RKTNETES) systemd kubelet rkt run Containers rkt pkg D E P R E C A T E D
  37. 49.

    PAGE Japan Container DAYS v 18.04 36 kubelet ʹ಺แ͞Εͨ rkt

    ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ PATTERN OF RUNNING CONTAINERS 04’ RKT (AKA RKTLET)
  38. 50.

    PAGE Japan Container DAYS v 18.04 systemd 36 rkt app

    kubelet kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ rktlet CRI Containers PATTERN OF RUNNING CONTAINERS 04’ RKT (AKA RKTLET)
  39. 51.

    PAGE Japan Container DAYS v 18.04 37 PATTERN OF RUNNING

    CONTAINERS 05PaLern 05 “vm-like” container Running “vm-like” isolated containers Using cc-run*me or runV, etc
  40. 52.

    PAGE Japan Container DAYS v 18.04 38 PATTERN OF RUNNING

    CONTAINERS 05 “VM-LIKE” CONTAINERS Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍)
  41. 53.

    PAGE Japan Container DAYS v 18.04 38 anything kubelet runV

    CRI PATTERN OF RUNNING CONTAINERS 05 “VM-LIKE” CONTAINERS Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍) cc-run*me OCI Containers? kata-run*me VM?
  42. 54.
  43. 55.

    PAGE Japan Container DAYS v 18.04 40 SUMMARY Container Run*me

    ʹ͸͍Ζ͍Ζ͋Δ ίϯςφͷ౷Ұج४ͱͯ͠ OCI ͕ଘࡏ͢Δ Kubernetes ͱ Run*me ͷؒ͸ CRI ͱ͍͏ج४͕͋Δ CRI ͱ OCI ʹରԠ͍ͯ͠Ε͹ Run*me ͸ࣗ༝ʹೖΕସ͕͑Մೳ Container Run*me ͸ High-level ͱ Low-level ʹ ෼͚ͯߟ͑Δ͜ͱ͕Ͱ͖Δ ίϯςφͷΑ͏ʹ VM Λಈ͔͢ run*me ΋ଘࡏ͢Δ
  44. 57.

    PAGE Japan Container DAYS v 18.04 42 PERFORMANCE CRI-O VS

    CONTAINERD cri-o ͱ containerd ʹର͠ɺcrictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏ How to bench run*me? sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 6 step ࣮͸ critest ͱ͍͏ benchmark ͕͋Δ (hLps://github.com/kubernetes-incubator/cri-tools/tree/master/cmd/critest) Ͱ΋ critest ͸্ख͘ಈ͔ͳ͔ͬͨͷͰࠓճ͸ಠࣗͷ benchmark ͰରԠ
  45. 58.

    PAGE Japan Container DAYS v 18.04 43 PERFORMANCE CRI-O VS

    CONTAINERD How to bench run*me? | 6 steps
  46. 59.

    PAGE Japan Container DAYS v 18.04 43 PERFORMANCE CRI-O VS

    CONTAINERD How to bench run*me? | 6 steps sandbox pod Λ࡞੒͢Δ sandbox pod ʹ busybox ίϯςφΛ࡞੒ɾىಈ͢Δ busybox ίϯςφΛఀࢭ͢Δ busybox ίϯςφΛ࡟আ͢Δ sandbox pod Λ࡟আ͢Δ sandbox pod Λఀࢭ͢Δ
  47. 60.

    PAGE Japan Container DAYS v 18.04 44 PERFORMANCE CRI-O VS

    CONTAINERD x100 LOOP 100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢
  48. 61.

    PAGE Japan Container DAYS v 18.04 44 PERFORMANCE CRI-O VS

    CONTAINERD x100 LOOP runp start stop rm stopp rmp 100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢
  49. 63.
  50. 64.

    PAGE Japan Container DAYS v 18.04 46 Average of 100

    *mes loop “crictl” opera*on cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec 0.66 0.37 0.88 1.29 PERFORMANCE CRI-O VS CONTAINERD WITH runC (docker-runc)
  51. 65.

    PAGE Japan Container DAYS v 18.04 47 PERFORMANCE CRI-O VS

    CONTAINERD cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍ loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠ cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕ performance ͕͍͍ WITH runC (docker-runc)
  52. 66.

    PAGE Japan Container DAYS v 18.04 47 PERFORMANCE CRI-O VS

    CONTAINERD cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍ loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠ cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕ performance ͕͍͍ cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.65 sec 1.30 sec 0.66 0.37 0.88 1.29 WITH runC (docker-runc)
  53. 68.

    PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8

    2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 48 PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)
  54. 69.

    PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8

    2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 48 PERFORMANCE CRI-O VS CONTAINERD cri-o ͸͕࣌ؒܦͭʹ࿈Εͯঃʑʹ performance ͕མͪΔ containerd ͸҆ఆ͍ͯ͠Δ runC (docker-runc)
  55. 72.

    PAGE Japan Container DAYS v 18.04 50 PERFORMANCE CRI-O VS

    CONTAINERD hypervisor ʹ KVM ͕࢖ΘΕΔ hLps://clearlinux.org/containers Hypervisor Kernel ܰྔ͔ͭߴ଎ʹىಈ͢Δ Clear Linux OS Security Intel VT Λ࢖༻͠ɺίϯςφΑΓ ηΩϡΞʹىಈ͢Δ͜ͱ͕Ͱ͖Δ OCI compa*ble run*me Ͱ͋Δ cc-run*me ͸ OCI support cc-run*me
  56. 73.

    PAGE Japan Container DAYS v 18.04 51 PERFORMANCE CRI-O VS

    CONTAINERD WITH cc-run*me (clear container)
  57. 74.

    PAGE Japan Container DAYS v 18.04 51 Average of 100

    *mes loop “crictl” opera*on cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 3.50 sec 7.00 sec 10.50 sec 14.00 sec 2.69 2.35 12.49 12.86 PERFORMANCE CRI-O VS CONTAINERD WITH cc-run*me (clear container)
  58. 75.

    PAGE Japan Container DAYS v 18.04 52 PERFORMANCE CRI-O VS

    CONTAINERD cc-run*me (clear container)
  59. 76.

    PAGE Japan Container DAYS v 18.04 0 3.5 7 10.5

    14 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 52 PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container) cc-run*me ͷ৔߹͸྆ऀڞʹऴ࢝҆ఆ͍ͯ͠Δ
  60. 79.

    PAGE Japan Container DAYS v 18.04 54 PERFORMANCE CRI-O VS

    CONTAINERD OpenStack Founda*on ʹΑΓ؅ཧ hLps://katacontainers.io/ Open Source runV + Clear Container Hyper ʹΑΔ runV ͱ Intel ʹΑΔ Clear Linux ͕ϕʔεͱͳ͍ͬͯΔ Security ͦΕͧΕͷίϯςφ͸ִ཭͞Εͨ Kernel ʹΑͬͯىಈ͞ΕΔ OCI compa*ble run*me Ͱ͋Δ kata-run*me ͸ OCI support kata-run*me
  61. 80.
  62. 81.

    cri-o ͱ containerd ͷઃఆΛม͑ͯ Low-level container run*me ʹ kata-run*me Λ

    ࢦఆ͕ͨ͠ error ͰىಈͰ͖ͣɾɾɾ Pod sandbox ͸࡞੒Մೳ͕ͩ Pod ʹ container Λ࡞੒͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ
  63. 85.
  64. 86.
  65. 88.

    PAGE Japan Container DAYS v 18.04 60 PERFORMANCE DOCKER +

    3 RUNTIMES Docker ͷ run*me ʹ kata-run*me Λࢦఆͨ͠ͱ͜Ζ্ख͘ಈ͍ͨ How to bench run*me? ୯७ʹ docker run ͷੑೳΛݕূ͢Δ ࠓճͷݕূ΋ 100 ճ docker run ͯ࣌ؒ͠Λܭଌ docker run --rm --run*me ${RUNTIME} hello-world
  66. 90.

    PAGE Japan Container DAYS v 18.04 61 Average of 100

    *mes loop “docker run” opera*on runc cc-run*me kata-run*me 0.00 sec 0.65 sec 1.30 sec 1.95 sec 2.60 sec 2.07 2.45 0.45 PERFORMANCE DOCKER + 3 RUNTIMES
  67. 92.

    PAGE Japan Container DAYS v 18.04 0 0.75 1.5 2.25

    3 runc cc-run*me kata-run*me 62 PERFORMANCE DOCKER + 3 RUNTIMES cc-run*meʹएׯϜϥ͕͋Γͦ͏
  68. 94.

    PAGE Japan Container DAYS v 18.04 64 PERFORMANCE STORAGE DRIVER

    run*me ͷ࣮ߦ଎౓ʹ storage driver ͕ͲΕ͘Β͍Өڹ͋Δͷ͔ݕূ storage driver is important ઌఔͱಉ͡ docker run ͷ benchmark Λ͢Δ ࢖༻ͨ͠ storage driver ͸ 4छྨ
  69. 96.

    PAGE Japan Container DAYS v 18.04 65 PERFORMANCE STORAGE DRIVER

    devicemapper (loopback) RHEL ܥͰ͸ default ʹͳ͍ͬͯΔ ख͕ܰͩ performance ͸ྑ͘ͳ͍ devicemapper (direct-lvm) Produc*on ؀ڥͰ devicemapper Λ ࢖͏৔߹͸ direct-lvm ͕ਪ঑ loopback ΑΓ͸ੑೳ͕͍͍ overlay2 overlay Ͱ໰୊͕͋ͬͨ inode ૿େͱ performance ҡ࣋ʹରԠ͍ͯ͠Δ overlay UnionFS ͷ1ͭͰϑΝΠϧ΍σΟϨΫτϦΛ ಁաతʹॏͶΔ͜ͱ͕Ͱ͖Δ performance ͸ྑ͍
  70. 98.

    PAGE Japan Container DAYS v 18.04 66 runc cc-run*me kata-run*me

    overlay overlay2 dm(loopback) dm(direct) overlay overlay2 dm(loopback) dm(direct) overlay overlay2 dm(loopback) dm(direct) 0.00 sec 0.75 sec 1.50 sec 2.25 sec 3.00 sec 2.865 2.862 0.849 2.391 2.606 0.637 2.061 2.376 0.450 2.075 2.452 0.453 PERFORMANCE STORAGE DRIVER Ͳͷ run*me Ͱ΋ overlay2 ͕࠷଎ devicemapper(direct) ͷํ͕஗͘ͳͬͯ͠·ͬͨ
  71. 100.

    PAGE Japan Container DAYS v 18.04 68 About Container Run*me

    TODAY’S CONCLUSION Container Run*me ʹ͸ High-Level ͱ Low-Level ͷ Run*me ʹ۠ผ͢Δ͜ͱ͕Ͱ͖Δ CRI ΍ OCI ͱ͍ͬͨ࢓༷͕ࡦఆ͞ΕͨͷͰ Run*me ͷೖΕସ͕͑ΑΓॊೈʹߦ͑ΔΑ͏ʹͳ͍ͬͯΔ Run*me ʹΑͬͯੑೳࠩ͸एׯ͋Δ͕ɺ·ͩ·ͩൃల్্ Docker(containerd) ͔Βੵۃతʹ৐Γ׵͑Δཧ༝͸ݱ࣌఺Ͱ͸ແͦ͞͏ Container ͷΑ͏ʹ VM Λىಈͤ͞Δٕज़͸ࠓޙظ଴Ͱ͖ͦ͏
  72. 101.

    PAGE Japan Container DAYS v 18.04 69 About Storage Driver

    TODAY’S CONCLUSION Performance Λߟ͑Δͱ overlay ͕ྑ͍બ୒ࢶͩͱࢥΘΕΔ overlay2 ͕࢖͑ΔͳΒ͹ੵۃతʹ࢖͍͍ͬͯ ͨͩ͠ overlay Λ࢖͏ͳΒ͹ kernel ͸࠷৽ʹ্͓͍͛ͯͨ΄͏͕ ͍͍ͱࢥΘΕΔ ݱ࣌఺Ͱͷ moby ͷ open ͳ issue ɹlabel:area/storage/overlay 29݅ ɹlabel:area/storage/devicemapper 42݅ ɹlabel:area/storage/aufs 29݅ ɹlabel:area/storage/btr 15݅
  73. 103.

    PAGE Japan Container DAYS v 18.04 71 αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ɹ Πϯλʔωοτ޿ࠂʹ͓͍ͯɺ޿ࠂ഑৴ͷ࠷దԽ΍ϝσΟΞͷऩӹ࠷େԽͱ͍͏؍఺͔Β

    ΞυςΫϊϩδʔͷॏཁ౓͕ߴ·͍ͬͯ·͢ɻ αΠόʔΤʔδΣϯτͰ͸ΞυςΫϊϩδʔ෼໺ʹ͓͚Δ ͜ΕΒͷαʔϏεʹ͍֤ͭͯࢠձࣾΛ௨͡։ൃ͓ͯ͠Γ·͕ͨ͠ɺ ֤αʔϏεͷ։ൃ෦໳Λԣஅͯ͠૊৫Խ͢Δઐ໳෦ॺͱͯ͠ΞυςΫຊ෦͕ઃཱ͞Ε·ͨ͠ɻ
  74. 104.

    PAGE Japan Container DAYS v 18.04 72 αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ɹ ʮίϯςφ͕޷͖ͳਓ΋޷͖͡Όͳ͍ਓ΋ઈࢍืूதͰ͢ɻ

    ɹΧδϡΞϧͳ໘ஊ΍ϥϯνަྲྀ΋OKͰ͢ɻ ɹڵຯ͕͋ΔํɺੋඇҰॹʹಇ͖·͠ΐ͏ʯ
  75. 106.
  76. 107.
  77. 108.

    PAGE Japan Container DAYS v 18.04 76 PERFORMANCE RKTLET rktlet

    ʹରͯ͠ crictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏ How to bench run*me? sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 4 step rktlet ͷ৔߹ container Λ create ͢Δͱ start ͠ͳͯ͘΋ىಈͨ͠ঢ়ଶʹͳΔΆ͍ container ʹରͯ͠ stop ͱ rm ͕Ͱ͖ͳ͔ͬͨ (͏·͘ઃఆͰ͖ͯͳ͍Մೳੑ͋Γ) sandbox Λ rm ͢Δͱ error: <nil> ͱग़Δ͕ͪΌΜͱফ͍͑ͯΔ
  78. 110.

    PAGE Japan Container DAYS v 18.04 77 PERFORMANCE RKTLET Average

    of 100 *mes loop “crictl” opera*on rktlet cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec 0.66 0.37 0.88 1.29 0.78
  79. 112.

    PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8

    2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd rktlet 78 PERFORMANCE RKTLET