Dockerだけじゃないコンテナ runtime 徹底比較

Japan Container Days(JKD) v18.04( での発表資料です。
「Dockerだけじゃないコンテナ runtime 徹底比較」

合わせて読みたい
「runc だけじゃないコンテナ low level runtime 徹底比較」
https://speakerdeck.com/makocchi/jkd-20181205-about-low-level-runtimes

Transcript

1. Docker ͚ͩ͡Όͳ͍ ίϯςφ run*me పఈൺֱ Makoto Hasegawa JAPAN CONTAINER DAYS V18.04
2. PAGE Japan Container DAYS v 18.04 2 ຊ೔ͷࢿྉ͸ޙ΄Ͳެ։͠·͢ͷͰ ࣸਅͱ͔ࡱΒͳ͍Ͱେৎ෉Ͱ͢ʂ
3. SELF INTRODUCTION
4. PAGE Japan Container DAYS v 18.04 4 SELF INTRODUCTION MAKOTO HASEGAWA TwiLer/@makocchi αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ॴଐ ීஈ͸σʔληϯλʔӡ༻΍ Private Cloud(OpenStack) Λ ߏஙɾӡ༻͍ͯ͠Δ ࠷ۙͰ͸ Private Cloud ্ʹల։ग़དྷΔίϯςφج൫ AKE Λ ߏஙɾӡ༻த Facebook/makocchi0923 CKA (Cer*fied Kubernetes Administrator)
5. PAGE Japan Container DAYS v 18.04 5 6 TODAY’S AGENDA
6. PAGE Japan Container DAYS v 18.04 5 6 TODAY’S AGENDA SELF INTRODUCTION VARIOUS CONTAINER RUNTIME OCI AND CRI INSIDE OF CONTAINER RUNTIME PATTERN OF RUNNING CONTAINERS CRI-O VS CONTAINERD STORAGE DRIVER CONCLUSION
7. PAGE Japan Container DAYS v 18.04 6 ಥવͰ͕͢ PREFACE
8. PAGE Japan Container DAYS v 18.04 7 Kubernetes ͷ cluster ͷ ίϯςφ run*me ʹԿΛ࢖͍ͬͯ·͔͢ʁ PREFACE
9. PAGE Japan Container DAYS v 18.04 8 ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ (ݸਓతͳ༧૝) PREFACE
10. PAGE Japan Container DAYS v 18.04 8 ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ (ݸਓతͳ༧૝) Docker Other What container runtime is running on your k8s cluster? 95% PREFACE
11. VARIOUS CONTAINER RUNTIME ON KUBERNETES
12. PAGE Japan Container DAYS v 18.04 10 Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me CONTAINER RUNTIMES ON KUBERNETES
13. PAGE Japan Container DAYS v 18.04 10 Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ Frak* Frak* Docker cri-o containerd ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me CONTAINER RUNTIMES ON KUBERNETES
14. PAGE Japan Container DAYS v 18.04 11 ࠷΋༗໊ͳ Container Run*me 1.11 ͔Β಺෦ͷ component ͸ runC + containerd ʹ෼ׂ͞Εͨ Frak* Docker cri-o Frak* containerd CRI, OCI ʹ४ڌͨ͠ Kubernetes ઐ༻ͷ run*me Docker ͷґଘ͔Β୤٫ Hypervisor ܦ༝Ͱ pod ͕ల։͞ΕΔ kernel ͕ share ͞Εͳ͍ͷͰΑΓη ΩϡΞͳ؀ڥʹͳΔ Docker ͷ಺෦ component ͕෼཭ Kubernetes ͱ࿈ܞ͢Δҝʹ͸ cri-containerd ͕ඞཁ͕ͩͬͨɺ kubernetes 1.10 ͱ containerd v1.1 ͔Β na*ve Ͱ௨৴Ͱ͖ΔΑ͏ʹͳͬͨ CONTAINER RUNTIMES ON KUBERNETES
15. PAGE Japan Container DAYS v 18.04 12 HOW ABOUT RKT …? Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated rktnetes has been deprecated in favor of rktlet (#58418) CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ hLps://github.com/kubernetes-incubator/rktlet kubelet ͷ “—container-run*me" ͷ help ʹ ʮ The container run*me to use. Possible values: 'docker', 'remote', 'rkt (deprecated)'. (default “docker”)ʯ CONTAINER RUNTIMES ON KUBERNETES
16. PAGE Japan Container DAYS v 18.04 12 HOW ABOUT RKT …? Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated rktnetes has been deprecated in favor of rktlet (#58418) CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ hLps://github.com/kubernetes-incubator/rktlet kubelet ͷ “—container-run*me" ͷ help ʹ ʮ The container run*me to use. Possible values: 'docker', 'remote', 'rkt (deprecated)'. (default “docker”)ʯ CONTAINER RUNTIMES ON KUBERNETES
17. WHAT IS OCI AND CRI ?
18. PAGE Japan Container DAYS v 18.04 14 OCI Open Container Ini*a*ve 2015೥1݄ʹίϯςφͷඪ४࢓༷Λࡦఆ͢ΔͨΊʹઃཱ ͞Εͨஂମ ͦͷஂମʹΑͬͯࡢ೥ʮOCI v1.0ʯ͕ࡦఆ͞Εͨ OCI v1.0 ͸ Container Run*me ͷඪ४࢓༷Ͱ͋Δ ʮRun*me Specifica*on v1.0ʯͱ Container Image ͷඪ४ ࢓༷Ͱ͋ΔʮFormat Specifica*on v1.0ʯ͔Βߏ੒͞Εͯ ͍Δ OCI AND CRI
19. PAGE Japan Container DAYS v 18.04 15 OCI Open Container Ini*a*ve ͪͳΈʹ௚ۙʹ͜Μͳൃද͕͞Εͨ ʮThe New Stack: “Open Container Ini*a*ve Creates a Distribu*on Specifica*on for Registries”ʯ(※) Docker Registry version 2 ΛݩʹΠϝʔδ഑෍ͷඪ४ԽΛ ਐΊ͍ͯ͘ (Run*me ͱ Format ʹ͙࣍ 3 ͭ໨ͷඪ४Խ) ※ hLps://www.opencontainers.org/news/2018/04/11/the-new-stack-open-container-ini*a*ve-creates-a-distribu*on-specifica*on-for-registries OCI AND CRI
20. PAGE Japan Container DAYS v 18.04 16 CRI The Container Run*me Interface Kubernetes 1.5 ͔Β release(alpha) ͞Εͨ(※) kubelet ͱ Container Run*me ͕௨৴͢Δҝͷ plugin interface Container Run*me ଆͷมߋʹ௥ਵ͠ଓ͚Δͷ͸େมͳҝ ౷ҰԽ͞Εͨ interface Λఆٛͯ͠ Kubernetes ͷ҆ఆԽͱϝϯςφϯεͷίετΛݮΒͨ͠ (※) hLps://kubernetes.io/blog/2016/12/container-run*me-interface-cri-in-kubernetes OCI AND CRI
21. INSIDE OF CONTAINER RUNTIMES
22. PAGE Japan Container DAYS v 18.04 18 Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸ ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
23. PAGE Japan Container DAYS v 18.04 18 kubelet Container run*me CRI run*me Containers OCI Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸ ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
24. PAGE Japan Container DAYS v 18.04 19 Container run*me ͷதʹ·ͨ run*me ͕͋Δɾɾɾ ฆΒΘ͍͠ɾɾʂ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
25. HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME
26. HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME
27. HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME OCI/JSON file Λड͚औΓϓϩηεΛ࣮ߦ ୅දతͳ΋ͷ͸ runC CRI/gRPC Λड͚औΓ Low-level run*me ΁౉͢ ୅දతͳ΋ͷ͸ cri-o ΍ containerd Image ͷ؅ཧΛߦ͏ (pull / push / rm ..) Image ͷ؅ཧ͸͠ͳ͍
28. PAGE Japan Container DAYS v 18.04 22 kubelet Container run*me CRI Containers OCI ઌఔʮContainer run*meʯͱݴ͍ͬͯͨ෦෼ͷதͰ ɹCRI Λड͚Δ run*me ΛʮHigh-level Container run*meʯ ɹOCI Λड͚Δ run*me ΛʮLow-level Container run*meʯ ͱݴ͏৔߹͕ଟ͍ ※ CRI run*me / OCI run*me ͱݴ͏৔߹΋͋Δ High-level Low-level INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL
29. PATTERN OF RUNNING CONTAINERS
30. PAGE Japan Container DAYS v 18.04 24 PATTERN OF RUNNING CONTAINERS SWAPPABLE RUNTIMES CRI ΍ OCI ʹΑͬͯΠϯλʔϑΣΠεΛ౷Ұͨ͜͠ ͱʹΑͬͯɺϢʔβʔ͸ High-level Container run*me ͱ Low-level Container run*me Λࣗ༝ʹબ୒ ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ ͍͔ͭ͘۩ମతʹ૊Έ߹ΘͤΛݟ͍͖ͯ·͠ΐ͏
31. PAGE Japan Container DAYS v 18.04 25 PATTERN OF RUNNING CONTAINERS 01PaLern 01 Docker The most popular paLern in the world Using containerd and runC to run containers inside Docker
32. PAGE Japan Container DAYS v 18.04 26 Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏ Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ) PATTERN OF RUNNING CONTAINERS 01 DOCKER
33. PAGE Japan Container DAYS v 18.04 Docker 26 kubelet containerd runC Containers OCI Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏ Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ) PATTERN OF RUNNING CONTAINERS 01 DOCKER dockershim CRI
34. PAGE Japan Container DAYS v 18.04 27 PATTERN OF RUNNING CONTAINERS 02PaLern 02 cri-o Using cri-o for High-level Container run*me Using runC to run containers with OCI interface
35. PAGE Japan Container DAYS v 18.04 28 PATTERN OF RUNNING CONTAINERS 02 CRI-O Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ
36. PAGE Japan Container DAYS v 18.04 28 cri-o kubelet runC CRI Containers OCI PATTERN OF RUNNING CONTAINERS 02 CRI-O Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ
37. PAGE Japan Container DAYS v 18.04 29 PATTERN OF RUNNING CONTAINERS 03PaLern 03 containerd Using containerd for High-level Container run*me Using runC to run containers with OCI interface
38. PAGE Japan Container DAYS v 18.04 30 PATTERN OF RUNNING CONTAINERS 03 CONTAINERD Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ
39. PAGE Japan Container DAYS v 18.04 30 runC kubelet PATTERN OF RUNNING CONTAINERS 03 CONTAINERD Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ cri-containerd containerd gRPC CRI Containers OCI
40. PAGE Japan Container DAYS v 18.04 31 runC kubelet cri-containerd containerd gRPC CRI Containers OCI PATTERN OF RUNNING CONTAINERS 03 CONTAINERD
41. PAGE Japan Container DAYS v 18.04 31 runC kubelet cri-containerd containerd gRPC CRI Containers OCI D E P R E C A T E D PATTERN OF RUNNING CONTAINERS 03 CONTAINERD
42. PAGE Japan Container DAYS v 18.04 32 PATTERN OF RUNNING CONTAINERS 03’ CONTAINERD (NEW) kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ
43. PAGE Japan Container DAYS v 18.04 32 PATTERN OF RUNNING CONTAINERS 03’ CONTAINERD (NEW) containerd kubelet runC CRI Containers OCI kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ
44. PAGE Japan Container DAYS v 18.04 33 PATTERN OF RUNNING CONTAINERS 04PaLern 04 rkt Using rkt and systemd for container engine
45. PAGE Japan Container DAYS v 18.04 34 kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷) PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES)
46. PAGE Japan Container DAYS v 18.04 systemd 34 kubelet rkt run Containers kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷) PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES) rkt pkg
47. PAGE Japan Container DAYS v 18.04 35 PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES) systemd kubelet rkt run Containers rkt pkg
48. PAGE Japan Container DAYS v 18.04 35 PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES) systemd kubelet rkt run Containers rkt pkg D E P R E C A T E D
49. PAGE Japan Container DAYS v 18.04 36 kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ PATTERN OF RUNNING CONTAINERS 04’ RKT (AKA RKTLET)
50. PAGE Japan Container DAYS v 18.04 systemd 36 rkt app kubelet kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ rktlet CRI Containers PATTERN OF RUNNING CONTAINERS 04’ RKT (AKA RKTLET)
51. PAGE Japan Container DAYS v 18.04 37 PATTERN OF RUNNING CONTAINERS 05PaLern 05 “vm-like” container Running “vm-like” isolated containers Using cc-run*me or runV, etc
52. PAGE Japan Container DAYS v 18.04 38 PATTERN OF RUNNING CONTAINERS 05 “VM-LIKE” CONTAINERS Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍)
53. PAGE Japan Container DAYS v 18.04 38 anything kubelet runV CRI PATTERN OF RUNNING CONTAINERS 05 “VM-LIKE” CONTAINERS Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍) cc-run*me OCI Containers? kata-run*me VM?
54. SUMMARY
55. PAGE Japan Container DAYS v 18.04 40 SUMMARY Container Run*me ʹ͸͍Ζ͍Ζ͋Δ ίϯςφͷ౷Ұج४ͱͯ͠ OCI ͕ଘࡏ͢Δ Kubernetes ͱ Run*me ͷؒ͸ CRI ͱ͍͏ج४͕͋Δ CRI ͱ OCI ʹରԠ͍ͯ͠Ε͹ Run*me ͸ࣗ༝ʹೖΕସ͕͑Մೳ Container Run*me ͸ High-level ͱ Low-level ʹ ෼͚ͯߟ͑Δ͜ͱ͕Ͱ͖Δ ίϯςφͷΑ͏ʹ VM Λಈ͔͢ run*me ΋ଘࡏ͢Δ
56. CRI-O CONTAINERD VS
57. PAGE Japan Container DAYS v 18.04 42 PERFORMANCE CRI-O VS CONTAINERD cri-o ͱ containerd ʹର͠ɺcrictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏ How to bench run*me? sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 6 step ࣮͸ critest ͱ͍͏ benchmark ͕͋Δ (hLps://github.com/kubernetes-incubator/cri-tools/tree/master/cmd/critest) Ͱ΋ critest ͸্ख͘ಈ͔ͳ͔ͬͨͷͰࠓճ͸ಠࣗͷ benchmark ͰରԠ
58. PAGE Japan Container DAYS v 18.04 43 PERFORMANCE CRI-O VS CONTAINERD How to bench run*me? | 6 steps
59. PAGE Japan Container DAYS v 18.04 43 PERFORMANCE CRI-O VS CONTAINERD How to bench run*me? | 6 steps sandbox pod Λ࡞੒͢Δ sandbox pod ʹ busybox ίϯςφΛ࡞੒ɾىಈ͢Δ busybox ίϯςφΛఀࢭ͢Δ busybox ίϯςφΛ࡟আ͢Δ sandbox pod Λ࡟আ͢Δ sandbox pod Λఀࢭ͢Δ
60. PAGE Japan Container DAYS v 18.04 44 PERFORMANCE CRI-O VS CONTAINERD x100 LOOP 100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢
61. PAGE Japan Container DAYS v 18.04 44 PERFORMANCE CRI-O VS CONTAINERD x100 LOOP runp start stop rm stopp rmp 100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢
62. cri-o vs containerd with runC(docker-runc) The most popular low-level container run*me
63. PAGE Japan Container DAYS v 18.04 46 PERFORMANCE CRI-O VS CONTAINERD WITH runC (docker-runc)
64. PAGE Japan Container DAYS v 18.04 46 Average of 100 *mes loop “crictl” opera*on cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec 0.66 0.37 0.88 1.29 PERFORMANCE CRI-O VS CONTAINERD WITH runC (docker-runc)
65. PAGE Japan Container DAYS v 18.04 47 PERFORMANCE CRI-O VS CONTAINERD cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍ loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠ cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕ performance ͕͍͍ WITH runC (docker-runc)
66. PAGE Japan Container DAYS v 18.04 47 PERFORMANCE CRI-O VS CONTAINERD cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍ loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠ cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕ performance ͕͍͍ cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.65 sec 1.30 sec 0.66 0.37 0.88 1.29 WITH runC (docker-runc)
67. PAGE Japan Container DAYS v 18.04 48 PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)
68. PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8 2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 48 PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)
69. PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8 2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 48 PERFORMANCE CRI-O VS CONTAINERD cri-o ͸͕࣌ؒܦͭʹ࿈Εͯঃʑʹ performance ͕མͪΔ containerd ͸҆ఆ͍ͯ͠Δ runC (docker-runc)
70. cri-o vs containerd with cc-run*me Intel Clear Containers run*me
71. PAGE Japan Container DAYS v 18.04 50 PERFORMANCE CRI-O VS CONTAINERD cc-run*me
72. PAGE Japan Container DAYS v 18.04 50 PERFORMANCE CRI-O VS CONTAINERD hypervisor ʹ KVM ͕࢖ΘΕΔ hLps://clearlinux.org/containers Hypervisor Kernel ܰྔ͔ͭߴ଎ʹىಈ͢Δ Clear Linux OS Security Intel VT Λ࢖༻͠ɺίϯςφΑΓ ηΩϡΞʹىಈ͢Δ͜ͱ͕Ͱ͖Δ OCI compa*ble run*me Ͱ͋Δ cc-run*me ͸ OCI support cc-run*me
73. PAGE Japan Container DAYS v 18.04 51 PERFORMANCE CRI-O VS CONTAINERD WITH cc-run*me (clear container)
74. PAGE Japan Container DAYS v 18.04 51 Average of 100 *mes loop “crictl” opera*on cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 3.50 sec 7.00 sec 10.50 sec 14.00 sec 2.69 2.35 12.49 12.86 PERFORMANCE CRI-O VS CONTAINERD WITH cc-run*me (clear container)
75. PAGE Japan Container DAYS v 18.04 52 PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container)
76. PAGE Japan Container DAYS v 18.04 0 3.5 7 10.5 14 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 52 PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container) cc-run*me ͷ৔߹͸྆ऀڞʹऴ࢝҆ఆ͍ͯ͠Δ
77. cri-o vs containerd with kata-run*me Kata-container
78. PAGE Japan Container DAYS v 18.04 54 PERFORMANCE CRI-O VS CONTAINERD kata-run*me
79. PAGE Japan Container DAYS v 18.04 54 PERFORMANCE CRI-O VS CONTAINERD OpenStack Founda*on ʹΑΓ؅ཧ hLps://katacontainers.io/ Open Source runV + Clear Container Hyper ʹΑΔ runV ͱ Intel ʹΑΔ Clear Linux ͕ϕʔεͱͳ͍ͬͯΔ Security ͦΕͧΕͷίϯςφ͸ִ཭͞Εͨ Kernel ʹΑͬͯىಈ͞ΕΔ OCI compa*ble run*me Ͱ͋Δ kata-run*me ͸ OCI support kata-run*me
80. WARNING!!
81. cri-o ͱ containerd ͷઃఆΛม͑ͯ Low-level container run*me ʹ kata-run*me Λ ࢦఆ͕ͨ͠ error ͰىಈͰ͖ͣɾɾɾ Pod sandbox ͸࡞੒Մೳ͕ͩ Pod ʹ container Λ࡞੒͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ
82. cri-o vs containerd with kata-run*me Kata-container
83. cri-o vs containerd with kata-run*me Kata-container F A I L
84. cri-o vs containerd with kata-run*me Kata-container F A I L
87. Docker with runc/cc-run*me/kata-run*me Docker with 3 Low-level container run*mes Revenge
88. PAGE Japan Container DAYS v 18.04 60 PERFORMANCE DOCKER + 3 RUNTIMES Docker ͷ run*me ʹ kata-run*me Λࢦఆͨ͠ͱ͜Ζ্ख͘ಈ͍ͨ How to bench run*me? ୯७ʹ docker run ͷੑೳΛݕূ͢Δ ࠓճͷݕূ΋ 100 ճ docker run ͯ࣌ؒ͠Λܭଌ docker run --rm --run*me ${RUNTIME} hello-world
89. PAGE Japan Container DAYS v 18.04 61 PERFORMANCE DOCKER + 3 RUNTIMES
90. PAGE Japan Container DAYS v 18.04 61 Average of 100 *mes loop “docker run” opera*on runc cc-run*me kata-run*me 0.00 sec 0.65 sec 1.30 sec 1.95 sec 2.60 sec 2.07 2.45 0.45 PERFORMANCE DOCKER + 3 RUNTIMES
91. PAGE Japan Container DAYS v 18.04 62 PERFORMANCE DOCKER + 3 RUNTIMES
92. PAGE Japan Container DAYS v 18.04 0 0.75 1.5 2.25 3 runc cc-run*me kata-run*me 62 PERFORMANCE DOCKER + 3 RUNTIMES cc-run*meʹएׯϜϥ͕͋Γͦ͏
93. STORAGE DRIVER
94. PAGE Japan Container DAYS v 18.04 64 PERFORMANCE STORAGE DRIVER run*me ͷ࣮ߦ଎౓ʹ storage driver ͕ͲΕ͘Β͍Өڹ͋Δͷ͔ݕূ storage driver is important ઌఔͱಉ͡ docker run ͷ benchmark Λ͢Δ ࢖༻ͨ͠ storage driver ͸ 4छྨ
95. PAGE Japan Container DAYS v 18.04 65 PERFORMANCE STORAGE DRIVER
96. PAGE Japan Container DAYS v 18.04 65 PERFORMANCE STORAGE DRIVER devicemapper (loopback) RHEL ܥͰ͸ default ʹͳ͍ͬͯΔ ख͕ܰͩ performance ͸ྑ͘ͳ͍ devicemapper (direct-lvm) Produc*on ؀ڥͰ devicemapper Λ ࢖͏৔߹͸ direct-lvm ͕ਪ঑ loopback ΑΓ͸ੑೳ͕͍͍ overlay2 overlay Ͱ໰୊͕͋ͬͨ inode ૿େͱ performance ҡ࣋ʹରԠ͍ͯ͠Δ overlay UnionFS ͷ1ͭͰϑΝΠϧ΍σΟϨΫτϦΛ ಁաతʹॏͶΔ͜ͱ͕Ͱ͖Δ performance ͸ྑ͍
97. PAGE Japan Container DAYS v 18.04 66 PERFORMANCE STORAGE DRIVER
98. PAGE Japan Container DAYS v 18.04 66 runc cc-run*me kata-run*me overlay overlay2 dm(loopback) dm(direct) overlay overlay2 dm(loopback) dm(direct) overlay overlay2 dm(loopback) dm(direct) 0.00 sec 0.75 sec 1.50 sec 2.25 sec 3.00 sec 2.865 2.862 0.849 2.391 2.606 0.637 2.061 2.376 0.450 2.075 2.452 0.453 PERFORMANCE STORAGE DRIVER Ͳͷ run*me Ͱ΋ overlay2 ͕࠷଎ devicemapper(direct) ͷํ͕஗͘ͳͬͯ͠·ͬͨ
99. PAGE Japan Container DAYS v 18.04 67 TODAY’S CONCLUSION
100. PAGE Japan Container DAYS v 18.04 68 About Container Run*me TODAY’S CONCLUSION Container Run*me ʹ͸ High-Level ͱ Low-Level ͷ Run*me ʹ۠ผ͢Δ͜ͱ͕Ͱ͖Δ CRI ΍ OCI ͱ͍ͬͨ࢓༷͕ࡦఆ͞ΕͨͷͰ Run*me ͷೖΕସ͕͑ΑΓॊೈʹߦ͑ΔΑ͏ʹͳ͍ͬͯΔ Run*me ʹΑͬͯੑೳࠩ͸एׯ͋Δ͕ɺ·ͩ·ͩൃల్্ Docker(containerd) ͔Βੵۃతʹ৐Γ׵͑Δཧ༝͸ݱ࣌఺Ͱ͸ແͦ͞͏ Container ͷΑ͏ʹ VM Λىಈͤ͞Δٕज़͸ࠓޙظ଴Ͱ͖ͦ͏
101. PAGE Japan Container DAYS v 18.04 69 About Storage Driver TODAY’S CONCLUSION Performance Λߟ͑Δͱ overlay ͕ྑ͍બ୒ࢶͩͱࢥΘΕΔ overlay2 ͕࢖͑ΔͳΒ͹ੵۃతʹ࢖͍͍ͬͯ ͨͩ͠ overlay Λ࢖͏ͳΒ͹ kernel ͸࠷৽ʹ্͓͍͛ͯͨ΄͏͕ ͍͍ͱࢥΘΕΔ ݱ࣌఺Ͱͷ moby ͷ open ͳ issue ɹlabel:area/storage/overlay 29݅ ɹlabel:area/storage/devicemapper 42݅ ɹlabel:area/storage/aufs 29݅ ɹlabel:area/storage/btr 15݅
102. PAGE Japan Container DAYS v 18.04 70 ʮ·ͩ͋ΔΑʂʯ
103. PAGE Japan Container DAYS v 18.04 71 αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ɹ Πϯλʔωοτ޿ࠂʹ͓͍ͯɺ޿ࠂ഑৴ͷ࠷దԽ΍ϝσΟΞͷऩӹ࠷େԽͱ͍͏؍఺͔Β ΞυςΫϊϩδʔͷॏཁ౓͕ߴ·͍ͬͯ·͢ɻ αΠόʔΤʔδΣϯτͰ͸ΞυςΫϊϩδʔ෼໺ʹ͓͚Δ ͜ΕΒͷαʔϏεʹ͍֤ͭͯࢠձࣾΛ௨͡։ൃ͓ͯ͠Γ·͕ͨ͠ɺ ֤αʔϏεͷ։ൃ෦໳Λԣஅͯ͠૊৫Խ͢Δઐ໳෦ॺͱͯ͠ΞυςΫຊ෦͕ઃཱ͞Ε·ͨ͠ɻ
104. PAGE Japan Container DAYS v 18.04 72 αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ɹ ʮίϯςφ͕޷͖ͳਓ΋޷͖͡Όͳ͍ਓ΋ઈࢍืूதͰ͢ɻ ɹΧδϡΞϧͳ໘ஊ΍ϥϯνަྲྀ΋OKͰ͢ɻ ɹڵຯ͕͋ΔํɺੋඇҰॹʹಇ͖·͠ΐ͏ʯ
105. Docker ͚ͩ͡Όͳ͍ ίϯςφ run*me పఈൺֱ Makoto Hasegawa JAPAN CONTAINER DAYS V18.04 THANK YOU
106. APPENDIX
107. RKTLET
108. PAGE Japan Container DAYS v 18.04 76 PERFORMANCE RKTLET rktlet ʹରͯ͠ crictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏ How to bench run*me? sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 4 step rktlet ͷ৔߹ container Λ create ͢Δͱ start ͠ͳͯ͘΋ىಈͨ͠ঢ়ଶʹͳΔΆ͍ container ʹରͯ͠ stop ͱ rm ͕Ͱ͖ͳ͔ͬͨ (͏·͘ઃఆͰ͖ͯͳ͍Մೳੑ͋Γ) sandbox Λ rm ͢Δͱ error: <nil> ͱग़Δ͕ͪΌΜͱফ͍͑ͯΔ
109. PAGE Japan Container DAYS v 18.04 77 PERFORMANCE RKTLET
110. PAGE Japan Container DAYS v 18.04 77 PERFORMANCE RKTLET Average of 100 *mes loop “crictl” opera*on rktlet cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec 0.66 0.37 0.88 1.29 0.78
111. PAGE Japan Container DAYS v 18.04 78 PERFORMANCE RKTLET
112. PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8 2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd rktlet 78 PERFORMANCE RKTLET
113. Docker ͚ͩ͡Όͳ͍ ίϯςφ run*me పఈൺֱ Makoto Hasegawa JAPAN CONTAINER DAYS V18.04 THANK YOU AGAIN