Dockerだけじゃないコンテナ runtime 徹底比較

by makocchi

Slide 1

Slide 1 text

Docker ͚ͩ͡Όͳ͍ ίϯςφ run*me పఈൺֱ Makoto Hasegawa JAPAN CONTAINER DAYS V18.04

Slide 2

Slide 2 text

PAGE Japan Container DAYS v 18.04 2 ຊ೔ͷࢿྉ͸ޙ΄Ͳެ։͠·͢ͷͰ ࣸਅͱ͔ࡱΒͳ͍Ͱେৎ෉Ͱ͢ʂ

Slide 3

Slide 3 text

SELF INTRODUCTION

Slide 4

Slide 4 text

PAGE Japan Container DAYS v 18.04 4 SELF INTRODUCTION MAKOTO HASEGAWA TwiLer/@makocchi αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ॴଐ ීஈ͸σʔληϯλʔӡ༻΍ Private Cloud(OpenStack) Λ ߏஙɾӡ༻͍ͯ͠Δ ࠷ۙͰ͸ Private Cloud ্ʹల։ग़དྷΔίϯςφج൫ AKE Λ ߏஙɾӡ༻த Facebook/makocchi0923 CKA (Cer*ﬁed Kubernetes Administrator)

Slide 5

Slide 5 text

PAGE Japan Container DAYS v 18.04 5 6 TODAY’S AGENDA

Slide 6

Slide 6 text

PAGE Japan Container DAYS v 18.04 5 6 TODAY’S AGENDA SELF INTRODUCTION VARIOUS CONTAINER RUNTIME OCI AND CRI INSIDE OF CONTAINER RUNTIME PATTERN OF RUNNING CONTAINERS CRI-O VS CONTAINERD STORAGE DRIVER CONCLUSION

Slide 7

Slide 7 text

PAGE Japan Container DAYS v 18.04 6 ಥવͰ͕͢ PREFACE

Slide 8

Slide 8 text

PAGE Japan Container DAYS v 18.04 7 Kubernetes ͷ cluster ͷ ίϯςφ run*me ʹԿΛ࢖͍ͬͯ·͔͢ʁ PREFACE

Slide 9

Slide 9 text

PAGE Japan Container DAYS v 18.04 8 ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ (ݸਓతͳ༧૝) PREFACE

Slide 10

Slide 10 text

PAGE Japan Container DAYS v 18.04 8 ଟ෼͜Μͳײ͡Ͱ͢ΑͶʁ (ݸਓతͳ༧૝) Docker Other What container runtime is running on your k8s cluster? 95% PREFACE

Slide 11

Slide 11 text

VARIOUS CONTAINER RUNTIME ON KUBERNETES

Slide 12

Slide 12 text

PAGE Japan Container DAYS v 18.04 10 Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me CONTAINER RUNTIMES ON KUBERNETES

Slide 13

Slide 13 text

PAGE Japan Container DAYS v 18.04 10 Docker Ҏ֎ʹ΋৭ʑબ୒ࢶ͕͋Δ Frak* Frak* Docker cri-o containerd ※Kubernetes 1.10 ͷ࣌఺Ͱ stable ʹͳ͍ͬͯΔ run*me CONTAINER RUNTIMES ON KUBERNETES

Slide 14

Slide 14 text

PAGE Japan Container DAYS v 18.04 11 ࠷΋༗໊ͳ Container Run*me 1.11 ͔Β಺෦ͷ component ͸ runC + containerd ʹ෼ׂ͞Εͨ Frak* Docker cri-o Frak* containerd CRI, OCI ʹ४ڌͨ͠ Kubernetes ઐ༻ͷ run*me Docker ͷґଘ͔Β୤٫ Hypervisor ܦ༝Ͱ pod ͕ల։͞ΕΔ kernel ͕ share ͞Εͳ͍ͷͰΑΓη ΩϡΞͳ؀ڥʹͳΔ Docker ͷ಺෦ component ͕෼཭ Kubernetes ͱ࿈ܞ͢Δҝʹ͸ cri-containerd ͕ඞཁ͕ͩͬͨɺ kubernetes 1.10 ͱ containerd v1.1 ͔Β na*ve Ͱ௨৴Ͱ͖ΔΑ͏ʹͳͬͨ CONTAINER RUNTIMES ON KUBERNETES

Slide 15

Slide 15 text

PAGE Japan Container DAYS v 18.04 12 HOW ABOUT RKT …? Kubernetes 1.10 Ͱ kubelet ಺แ͸ deprecated rktnetes has been deprecated in favor of rktlet (#58418) CRI ʹ४ڌͨ͠ rktlet Λ࢖͏Α͏ʹมߋ hLps://github.com/kubernetes-incubator/rktlet kubelet ͷ “—container-run*me" ͷ help ʹ ʮ The container run*me to use. Possible values: 'docker', 'remote', 'rkt (deprecated)'. (default “docker”)ʯ CONTAINER RUNTIMES ON KUBERNETES

Slide 16

Slide 16 text

Slide 17

Slide 17 text

WHAT IS OCI AND CRI ?

Slide 18

Slide 18 text

PAGE Japan Container DAYS v 18.04 14 OCI Open Container Ini*a*ve 2015೥1݄ʹίϯςφͷඪ४࢓༷Λࡦఆ͢ΔͨΊʹઃཱ ͞Εͨஂମ ͦͷஂମʹΑͬͯࡢ೥ʮOCI v1.0ʯ͕ࡦఆ͞Εͨ OCI v1.0 ͸ Container Run*me ͷඪ४࢓༷Ͱ͋Δ ʮRun*me Speciﬁca*on v1.0ʯͱ Container Image ͷඪ४ ࢓༷Ͱ͋ΔʮFormat Speciﬁca*on v1.0ʯ͔Βߏ੒͞Εͯ ͍Δ OCI AND CRI

Slide 19

Slide 19 text

PAGE Japan Container DAYS v 18.04 15 OCI Open Container Ini*a*ve ͪͳΈʹ௚ۙʹ͜Μͳൃද͕͞Εͨ ʮThe New Stack: “Open Container Ini*a*ve Creates a Distribu*on Speciﬁca*on for Registries”ʯ(※) Docker Registry version 2 ΛݩʹΠϝʔδ഑෍ͷඪ४ԽΛ ਐΊ͍ͯ͘ (Run*me ͱ Format ʹ͙࣍ 3 ͭ໨ͷඪ४Խ) ※ hLps://www.opencontainers.org/news/2018/04/11/the-new-stack-open-container-ini*a*ve-creates-a-distribu*on-speciﬁca*on-for-registries OCI AND CRI

Slide 20

Slide 20 text

PAGE Japan Container DAYS v 18.04 16 CRI The Container Run*me Interface Kubernetes 1.5 ͔Β release(alpha) ͞Εͨ(※) kubelet ͱ Container Run*me ͕௨৴͢Δҝͷ plugin interface Container Run*me ଆͷมߋʹ௥ਵ͠ଓ͚Δͷ͸େมͳҝ ౷ҰԽ͞Εͨ interface Λఆٛͯ͠ Kubernetes ͷ҆ఆԽͱϝϯςφϯεͷίετΛݮΒͨ͠ (※) hLps://kubernetes.io/blog/2016/12/container-run*me-interface-cri-in-kubernetes OCI AND CRI

Slide 21

Slide 21 text

INSIDE OF CONTAINER RUNTIMES

Slide 22

Slide 22 text

PAGE Japan Container DAYS v 18.04 18 Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸ ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL

Slide 23

Slide 23 text

PAGE Japan Container DAYS v 18.04 18 kubelet Container run*me CRI run*me Containers OCI Kubelet ͔Β CRI Λ௨ͯ͡ Container run*me ͕ kick ͞ΕΔ͕ɺ࣮ࡍʹίϯςφΛىಈ͢Δ࣌ʹ͸ ߋʹ OCI Λ௨ͯ͡ run*me ͕ୟ͔Εͯίϯςφ͕ىಈ͢Δ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL

Slide 24

Slide 24 text

PAGE Japan Container DAYS v 18.04 19 Container run*me ͷதʹ·ͨ run*me ͕͋Δɾɾɾ ฆΒΘ͍͠ɾɾʂ INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL

Slide 25

Slide 25 text

HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME

Slide 26

Slide 26 text

HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME

Slide 27

Slide 27 text

HIGH-LEVEL CONTAINER RUNTIME LOW-LEVEL CONTAINER RUNTIME OCI/JSON ﬁle Λड͚औΓϓϩηεΛ࣮ߦ ୅දతͳ΋ͷ͸ runC CRI/gRPC Λड͚औΓ Low-level run*me ΁౉͢ ୅දతͳ΋ͷ͸ cri-o ΍ containerd Image ͷ؅ཧΛߦ͏ (pull / push / rm ..) Image ͷ؅ཧ͸͠ͳ͍

Slide 28

Slide 28 text

PAGE Japan Container DAYS v 18.04 22 kubelet Container run*me CRI Containers OCI ઌఔʮContainer run*meʯͱݴ͍ͬͯͨ෦෼ͷதͰ ɹCRI Λड͚Δ run*me ΛʮHigh-level Container run*meʯ ɹOCI Λड͚Δ run*me ΛʮLow-level Container run*meʯ ͱݴ͏৔߹͕ଟ͍ ※ CRI run*me / OCI run*me ͱݴ͏৔߹΋͋Δ High-level Low-level INSIDE OF CONTAINER RUNTIMES HIGH-LEVEL AND LOW-LEVEL

Slide 29

Slide 29 text

PATTERN OF RUNNING CONTAINERS

Slide 30

Slide 30 text

PAGE Japan Container DAYS v 18.04 24 PATTERN OF RUNNING CONTAINERS SWAPPABLE RUNTIMES CRI ΍ OCI ʹΑͬͯΠϯλʔϑΣΠεΛ౷Ұͨ͜͠ ͱʹΑͬͯɺϢʔβʔ͸ High-level Container run*me ͱ Low-level Container run*me Λࣗ༝ʹબ୒ ͢Δ͜ͱ͕Ͱ͖ΔΑ͏ʹͳͬͨ ͍͔ͭ͘۩ମతʹ૊Έ߹ΘͤΛݟ͍͖ͯ·͠ΐ͏

Slide 31

Slide 31 text

PAGE Japan Container DAYS v 18.04 25 PATTERN OF RUNNING CONTAINERS 01PaLern 01 Docker The most popular paLern in the world Using containerd and runC to run containers inside Docker

Slide 32

Slide 32 text

PAGE Japan Container DAYS v 18.04 26 Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏ Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ) PATTERN OF RUNNING CONTAINERS 01 DOCKER

Slide 33

Slide 33 text

PAGE Japan Container DAYS v 18.04 Docker 26 kubelet containerd runC Containers OCI Kubelet ʹ಺แ͞Ε͍ͯΔ dockershim ͕ Docker ΁ͷڮ౉͠Λߦ͏ Docker ಺Ͱ͸ High-level Container run*me ͱͯ͠ containerd ͕ಈ͍͍ͯΔ Low-level Container run*me ʹ͸ runC ͕࢖ΘΕΔ (ଞͷ run*me ʹஔ͖׵͑Մೳ) PATTERN OF RUNNING CONTAINERS 01 DOCKER dockershim CRI

Slide 34

Slide 34 text

PAGE Japan Container DAYS v 18.04 27 PATTERN OF RUNNING CONTAINERS 02PaLern 02 cri-o Using cri-o for High-level Container run*me Using runC to run containers with OCI interface

Slide 35

Slide 35 text

PAGE Japan Container DAYS v 18.04 28 PATTERN OF RUNNING CONTAINERS 02 CRI-O Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ

Slide 36

Slide 36 text

PAGE Japan Container DAYS v 18.04 28 cri-o kubelet runC CRI Containers OCI PATTERN OF RUNNING CONTAINERS 02 CRI-O Docker ͱ͸ҧ͍ͱͯ΋ simple ͳܗʹͳΔ

Slide 37

Slide 37 text

PAGE Japan Container DAYS v 18.04 29 PATTERN OF RUNNING CONTAINERS 03PaLern 03 containerd Using containerd for High-level Container run*me Using runC to run containers with OCI interface

Slide 38

Slide 38 text

PAGE Japan Container DAYS v 18.04 30 PATTERN OF RUNNING CONTAINERS 03 CONTAINERD Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ

Slide 39

Slide 39 text

PAGE Japan Container DAYS v 18.04 30 runC kubelet PATTERN OF RUNNING CONTAINERS 03 CONTAINERD Docker ಺෦Ͱ΋ಉ͡Α͏ʹ containerd ͸ಈ͍͍ͯΔ͕ɺcontainerd ͱ kubelet ͷ CRI ؒΛ cri-containerd ͕୲౰͢Δ͜ͱͰ Docker Λ࢖ͬͨ৔߹ΑΓ΋ simple ͳߏ੒ʹͳΔ cri-containerd containerd gRPC CRI Containers OCI

Slide 40

Slide 40 text

PAGE Japan Container DAYS v 18.04 31 runC kubelet cri-containerd containerd gRPC CRI Containers OCI PATTERN OF RUNNING CONTAINERS 03 CONTAINERD

Slide 41

Slide 41 text

PAGE Japan Container DAYS v 18.04 31 runC kubelet cri-containerd containerd gRPC CRI Containers OCI D E P R E C A T E D PATTERN OF RUNNING CONTAINERS 03 CONTAINERD

Slide 42

Slide 42 text

PAGE Japan Container DAYS v 18.04 32 PATTERN OF RUNNING CONTAINERS 03’ CONTAINERD (NEW) kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ

Slide 43

Slide 43 text

PAGE Japan Container DAYS v 18.04 32 PATTERN OF RUNNING CONTAINERS 03’ CONTAINERD (NEW) containerd kubelet runC CRI Containers OCI kubelet 1.10 Ҏ߱ͱ containerd v1.1 Ҏ߱ͷ૊Έ߹ΘͤͰ͸தؒʹ͍ͨ cri-containerd ͸ෆཁͱͳͬͨ

Slide 44

Slide 44 text

PAGE Japan Container DAYS v 18.04 33 PATTERN OF RUNNING CONTAINERS 04PaLern 04 rkt Using rkt and systemd for container engine

Slide 45

Slide 45 text

PAGE Japan Container DAYS v 18.04 34 kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷) PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES)

Slide 46

Slide 46 text

PAGE Japan Container DAYS v 18.04 systemd 34 kubelet rkt run Containers kubelet ʹ಺แ͞Ε͍ͯΔ rkt ͷ pkg ͕ systemd Λ௨ͯ͠ rkt run ͢Δ͜ͱͰίϯςφΛىಈͤ͞Δ Docker ͱಉ͘͡ rkt ͷॲཧ͕ kubelet ͷதʹ૊Έࠐ·Ε͍ͯΔ (Kubernetes 1.3 ΑΓ࣮૷) PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES) rkt pkg

Slide 47

Slide 47 text

PAGE Japan Container DAYS v 18.04 35 PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES) systemd kubelet rkt run Containers rkt pkg

Slide 48

Slide 48 text

PAGE Japan Container DAYS v 18.04 35 PATTERN OF RUNNING CONTAINERS 04 RKT (AKA RKTNETES) systemd kubelet rkt run Containers rkt pkg D E P R E C A T E D

Slide 49

Slide 49 text

PAGE Japan Container DAYS v 18.04 36 kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ PATTERN OF RUNNING CONTAINERS 04’ RKT (AKA RKTLET)

Slide 50

Slide 50 text

PAGE Japan Container DAYS v 18.04 systemd 36 rkt app kubelet kubelet ʹ಺แ͞Εͨ rkt ͷػೳ͸ deprecated(kubernetes 1.10) ʹͳΓɺCRI ʹ४ڌͨ͠ rktlet ΁มߋ ·ͨɺͦΕʹ߹Θͤͯ rkt run Ͱ͸ͳ͘ rkt app ͰίϯςφΛૢ࡞͢ΔΑ͏ʹ rkt ଆ΋ػೳ௥Ճ͞Εͨ rktlet CRI Containers PATTERN OF RUNNING CONTAINERS 04’ RKT (AKA RKTLET)

Slide 51

Slide 51 text

PAGE Japan Container DAYS v 18.04 37 PATTERN OF RUNNING CONTAINERS 05PaLern 05 “vm-like” container Running “vm-like” isolated containers Using cc-run*me or runV, etc

Slide 52

Slide 52 text

PAGE Japan Container DAYS v 18.04 38 PATTERN OF RUNNING CONTAINERS 05 “VM-LIKE” CONTAINERS Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍)

Slide 53

Slide 53 text

PAGE Japan Container DAYS v 18.04 38 anything kubelet runV CRI PATTERN OF RUNNING CONTAINERS 05 “VM-LIKE” CONTAINERS Low-level run*me Λஔ͖׵͑Δ͜ͱͰ VM ͷΑ͏ͳίϯςφΛىಈ͢Δ͜ͱ΋ग़དྷΔ (ίϯςφͷΑ͏ͳ VM ͱ΋ݴ͑Δ͔΋͠Εͳ͍) cc-run*me OCI Containers? kata-run*me VM?

Slide 54

Slide 54 text

SUMMARY

Slide 55

Slide 55 text

PAGE Japan Container DAYS v 18.04 40 SUMMARY Container Run*me ʹ͸͍Ζ͍Ζ͋Δ ίϯςφͷ౷Ұج४ͱͯ͠ OCI ͕ଘࡏ͢Δ Kubernetes ͱ Run*me ͷؒ͸ CRI ͱ͍͏ج४͕͋Δ CRI ͱ OCI ʹରԠ͍ͯ͠Ε͹ Run*me ͸ࣗ༝ʹೖΕସ͕͑Մೳ Container Run*me ͸ High-level ͱ Low-level ʹ ෼͚ͯߟ͑Δ͜ͱ͕Ͱ͖Δ ίϯςφͷΑ͏ʹ VM Λಈ͔͢ run*me ΋ଘࡏ͢Δ

Slide 56

Slide 56 text

CRI-O CONTAINERD VS

Slide 57

Slide 57 text

PAGE Japan Container DAYS v 18.04 42 PERFORMANCE CRI-O VS CONTAINERD cri-o ͱ containerd ʹର͠ɺcrictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏ How to bench run*me? sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 6 step ࣮͸ critest ͱ͍͏ benchmark ͕͋Δ (hLps://github.com/kubernetes-incubator/cri-tools/tree/master/cmd/critest) Ͱ΋ critest ͸্ख͘ಈ͔ͳ͔ͬͨͷͰࠓճ͸ಠࣗͷ benchmark ͰରԠ

Slide 58

Slide 58 text

PAGE Japan Container DAYS v 18.04 43 PERFORMANCE CRI-O VS CONTAINERD How to bench run*me? | 6 steps

Slide 59

Slide 59 text

PAGE Japan Container DAYS v 18.04 43 PERFORMANCE CRI-O VS CONTAINERD How to bench run*me? | 6 steps sandbox pod Λ࡞੒͢Δ sandbox pod ʹ busybox ίϯςφΛ࡞੒ɾىಈ͢Δ busybox ίϯςφΛఀࢭ͢Δ busybox ίϯςφΛ࡟আ͢Δ sandbox pod Λ࡟আ͢Δ sandbox pod Λఀࢭ͢Δ

Slide 60

Slide 60 text

PAGE Japan Container DAYS v 18.04 44 PERFORMANCE CRI-O VS CONTAINERD x100 LOOP 100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢

Slide 61

Slide 61 text

PAGE Japan Container DAYS v 18.04 44 PERFORMANCE CRI-O VS CONTAINERD x100 LOOP runp start stop rm stopp rmp 100 ճ loop ͤͯ͞ॲཧͷ࣌ؒΛܭଌ͠·͢

Slide 62

Slide 62 text

cri-o vs containerd with runC(docker-runc) The most popular low-level container run*me

Slide 63

Slide 63 text

PAGE Japan Container DAYS v 18.04 46 PERFORMANCE CRI-O VS CONTAINERD WITH runC (docker-runc)

Slide 64

Slide 64 text

PAGE Japan Container DAYS v 18.04 46 Average of 100 *mes loop “crictl” opera*on cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec 0.66 0.37 0.88 1.29 PERFORMANCE CRI-O VS CONTAINERD WITH runC (docker-runc)

Slide 65

Slide 65 text

PAGE Japan Container DAYS v 18.04 47 PERFORMANCE CRI-O VS CONTAINERD cri-o ʹൺ΂ͯ containerd ͷํ͕ॲཧ͕࣌ؒ୹͍ loop ॲཧதʹ cri-o ͸ CPU ࢖༻཰͕ 100% ʹͳͬͨ Ұํ containerd ͸ CPU ࢖༻཰ 20% ఔͰਪҠͨ͠ cri-containerd ΋ಉ͡Α͏ʹܭଌͯ͠Έ͕ͨ CRI Λ Na*ve Ͱ support ͢ΔΑ͏ʹͳͬͨ containerd ͷํ͕ performance ͕͍͍ WITH runC (docker-runc)

Slide 66

Slide 66 text

Slide 67

Slide 67 text

PAGE Japan Container DAYS v 18.04 48 PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)

Slide 68

Slide 68 text

PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8 2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 48 PERFORMANCE CRI-O VS CONTAINERD runC (docker-runc)

Slide 69

Slide 69 text

PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8 2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 48 PERFORMANCE CRI-O VS CONTAINERD cri-o ͸͕࣌ؒܦͭʹ࿈Εͯঃʑʹ performance ͕མͪΔ containerd ͸҆ఆ͍ͯ͠Δ runC (docker-runc)

Slide 70

Slide 70 text

cri-o vs containerd with cc-run*me Intel Clear Containers run*me

Slide 71

Slide 71 text

PAGE Japan Container DAYS v 18.04 50 PERFORMANCE CRI-O VS CONTAINERD cc-run*me

Slide 72

Slide 72 text

PAGE Japan Container DAYS v 18.04 50 PERFORMANCE CRI-O VS CONTAINERD hypervisor ʹ KVM ͕࢖ΘΕΔ hLps://clearlinux.org/containers Hypervisor Kernel ܰྔ͔ͭߴ଎ʹىಈ͢Δ Clear Linux OS Security Intel VT Λ࢖༻͠ɺίϯςφΑΓ ηΩϡΞʹىಈ͢Δ͜ͱ͕Ͱ͖Δ OCI compa*ble run*me Ͱ͋Δ cc-run*me ͸ OCI support cc-run*me

Slide 73

Slide 73 text

PAGE Japan Container DAYS v 18.04 51 PERFORMANCE CRI-O VS CONTAINERD WITH cc-run*me (clear container)

Slide 74

Slide 74 text

PAGE Japan Container DAYS v 18.04 51 Average of 100 *mes loop “crictl” opera*on cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 3.50 sec 7.00 sec 10.50 sec 14.00 sec 2.69 2.35 12.49 12.86 PERFORMANCE CRI-O VS CONTAINERD WITH cc-run*me (clear container)

Slide 75

Slide 75 text

PAGE Japan Container DAYS v 18.04 52 PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container)

Slide 76

Slide 76 text

PAGE Japan Container DAYS v 18.04 0 3.5 7 10.5 14 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd 52 PERFORMANCE CRI-O VS CONTAINERD cc-run*me (clear container) cc-run*me ͷ৔߹͸྆ऀڞʹऴ࢝҆ఆ͍ͯ͠Δ

Slide 77

Slide 77 text

cri-o vs containerd with kata-run*me Kata-container

Slide 78

Slide 78 text

PAGE Japan Container DAYS v 18.04 54 PERFORMANCE CRI-O VS CONTAINERD kata-run*me

Slide 79

Slide 79 text

PAGE Japan Container DAYS v 18.04 54 PERFORMANCE CRI-O VS CONTAINERD OpenStack Founda*on ʹΑΓ؅ཧ hLps://katacontainers.io/ Open Source runV + Clear Container Hyper ʹΑΔ runV ͱ Intel ʹΑΔ Clear Linux ͕ϕʔεͱͳ͍ͬͯΔ Security ͦΕͧΕͷίϯςφ͸ִ཭͞Εͨ Kernel ʹΑͬͯىಈ͞ΕΔ OCI compa*ble run*me Ͱ͋Δ kata-run*me ͸ OCI support kata-run*me

Slide 80

Slide 80 text

WARNING!!

Slide 81

Slide 81 text

cri-o ͱ containerd ͷઃఆΛม͑ͯ Low-level container run*me ʹ kata-run*me Λ ࢦఆ͕ͨ͠ error ͰىಈͰ͖ͣɾɾɾ Pod sandbox ͸࡞੒Մೳ͕ͩ Pod ʹ container Λ࡞੒͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ

Slide 82

Slide 82 text

cri-o vs containerd with kata-run*me Kata-container

Slide 83

Slide 83 text

cri-o vs containerd with kata-run*me Kata-container F A I L

Slide 84

Slide 84 text

cri-o vs containerd with kata-run*me Kata-container F A I L

Slide 85

Slide 85 text

No content

Slide 86

Slide 86 text

No content

Slide 87

Slide 87 text

Docker with runc/cc-run*me/kata-run*me Docker with 3 Low-level container run*mes Revenge

Slide 88

Slide 88 text

PAGE Japan Container DAYS v 18.04 60 PERFORMANCE DOCKER + 3 RUNTIMES Docker ͷ run*me ʹ kata-run*me Λࢦఆͨ͠ͱ͜Ζ্ख͘ಈ͍ͨ How to bench run*me? ୯७ʹ docker run ͷੑೳΛݕূ͢Δ ࠓճͷݕূ΋ 100 ճ docker run ͯ࣌ؒ͠Λܭଌ docker run --rm --run*me ${RUNTIME} hello-world

Slide 89

Slide 89 text

PAGE Japan Container DAYS v 18.04 61 PERFORMANCE DOCKER + 3 RUNTIMES

Slide 90

Slide 90 text

PAGE Japan Container DAYS v 18.04 61 Average of 100 *mes loop “docker run” opera*on runc cc-run*me kata-run*me 0.00 sec 0.65 sec 1.30 sec 1.95 sec 2.60 sec 2.07 2.45 0.45 PERFORMANCE DOCKER + 3 RUNTIMES

Slide 91

Slide 91 text

PAGE Japan Container DAYS v 18.04 62 PERFORMANCE DOCKER + 3 RUNTIMES

Slide 92

Slide 92 text

PAGE Japan Container DAYS v 18.04 0 0.75 1.5 2.25 3 runc cc-run*me kata-run*me 62 PERFORMANCE DOCKER + 3 RUNTIMES cc-run*meʹएׯϜϥ͕͋Γͦ͏

Slide 93

Slide 93 text

STORAGE DRIVER

Slide 94

Slide 94 text

PAGE Japan Container DAYS v 18.04 64 PERFORMANCE STORAGE DRIVER run*me ͷ࣮ߦ଎౓ʹ storage driver ͕ͲΕ͘Β͍Өڹ͋Δͷ͔ݕূ storage driver is important ઌఔͱಉ͡ docker run ͷ benchmark Λ͢Δ ࢖༻ͨ͠ storage driver ͸ 4छྨ

Slide 95

Slide 95 text

PAGE Japan Container DAYS v 18.04 65 PERFORMANCE STORAGE DRIVER

Slide 96

Slide 96 text

PAGE Japan Container DAYS v 18.04 65 PERFORMANCE STORAGE DRIVER devicemapper (loopback) RHEL ܥͰ͸ default ʹͳ͍ͬͯΔ ख͕ܰͩ performance ͸ྑ͘ͳ͍ devicemapper (direct-lvm) Produc*on ؀ڥͰ devicemapper Λ ࢖͏৔߹͸ direct-lvm ͕ਪ঑ loopback ΑΓ͸ੑೳ͕͍͍ overlay2 overlay Ͱ໰୊͕͋ͬͨ inode ૿େͱ performance ҡ࣋ʹରԠ͍ͯ͠Δ overlay UnionFS ͷ1ͭͰϑΝΠϧ΍σΟϨΫτϦΛ ಁաతʹॏͶΔ͜ͱ͕Ͱ͖Δ performance ͸ྑ͍

Slide 97

Slide 97 text

PAGE Japan Container DAYS v 18.04 66 PERFORMANCE STORAGE DRIVER

Slide 98

Slide 98 text

PAGE Japan Container DAYS v 18.04 66 runc cc-run*me kata-run*me overlay overlay2 dm(loopback) dm(direct) overlay overlay2 dm(loopback) dm(direct) overlay overlay2 dm(loopback) dm(direct) 0.00 sec 0.75 sec 1.50 sec 2.25 sec 3.00 sec 2.865 2.862 0.849 2.391 2.606 0.637 2.061 2.376 0.450 2.075 2.452 0.453 PERFORMANCE STORAGE DRIVER Ͳͷ run*me Ͱ΋ overlay2 ͕࠷଎ devicemapper(direct) ͷํ͕஗͘ͳͬͯ͠·ͬͨ

Slide 99

Slide 99 text

PAGE Japan Container DAYS v 18.04 67 TODAY’S CONCLUSION

Slide 100

Slide 100 text

PAGE Japan Container DAYS v 18.04 68 About Container Run*me TODAY’S CONCLUSION Container Run*me ʹ͸ High-Level ͱ Low-Level ͷ Run*me ʹ۠ผ͢Δ͜ͱ͕Ͱ͖Δ CRI ΍ OCI ͱ͍ͬͨ࢓༷͕ࡦఆ͞ΕͨͷͰ Run*me ͷೖΕସ͕͑ΑΓॊೈʹߦ͑ΔΑ͏ʹͳ͍ͬͯΔ Run*me ʹΑͬͯੑೳࠩ͸एׯ͋Δ͕ɺ·ͩ·ͩൃల్্ Docker(containerd) ͔Βੵۃతʹ৐Γ׵͑Δཧ༝͸ݱ࣌఺Ͱ͸ແͦ͞͏ Container ͷΑ͏ʹ VM Λىಈͤ͞Δٕज़͸ࠓޙظ଴Ͱ͖ͦ͏

Slide 101

Slide 101 text

PAGE Japan Container DAYS v 18.04 69 About Storage Driver TODAY’S CONCLUSION Performance Λߟ͑Δͱ overlay ͕ྑ͍બ୒ࢶͩͱࢥΘΕΔ overlay2 ͕࢖͑ΔͳΒ͹ੵۃతʹ࢖͍͍ͬͯ ͨͩ͠ overlay Λ࢖͏ͳΒ͹ kernel ͸࠷৽ʹ্͓͍͛ͯͨ΄͏͕ ͍͍ͱࢥΘΕΔ ݱ࣌఺Ͱͷ moby ͷ open ͳ issue ɹlabel:area/storage/overlay 29݅ ɹlabel:area/storage/devicemapper 42݅ ɹlabel:area/storage/aufs 29݅ ɹlabel:area/storage/btr 15݅

Slide 102

Slide 102 text

PAGE Japan Container DAYS v 18.04 70 ʮ·ͩ͋ΔΑʂʯ

Slide 103

Slide 103 text

PAGE Japan Container DAYS v 18.04 71 αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ɹ Πϯλʔωοτ޿ࠂʹ͓͍ͯɺ޿ࠂ഑৴ͷ࠷దԽ΍ϝσΟΞͷऩӹ࠷େԽͱ͍͏؍఺͔Β ΞυςΫϊϩδʔͷॏཁ౓͕ߴ·͍ͬͯ·͢ɻ αΠόʔΤʔδΣϯτͰ͸ΞυςΫϊϩδʔ෼໺ʹ͓͚Δ ͜ΕΒͷαʔϏεʹ͍֤ͭͯࢠձࣾΛ௨͡։ൃ͓ͯ͠Γ·͕ͨ͠ɺ ֤αʔϏεͷ։ൃ෦໳Λԣஅͯ͠૊৫Խ͢Δઐ໳෦ॺͱͯ͠ΞυςΫຊ෦͕ઃཱ͞Ε·ͨ͠ɻ

Slide 104

Slide 104 text

PAGE Japan Container DAYS v 18.04 72 αΠόʔΤʔδΣϯτ ΞυςΫຊ෦ɹ ʮίϯςφ͕޷͖ͳਓ΋޷͖͡Όͳ͍ਓ΋ઈࢍืूதͰ͢ɻ ɹΧδϡΞϧͳ໘ஊ΍ϥϯνަྲྀ΋OKͰ͢ɻ ɹڵຯ͕͋ΔํɺੋඇҰॹʹಇ͖·͠ΐ͏ʯ

Slide 105

Slide 105 text

Docker ͚ͩ͡Όͳ͍ ίϯςφ run*me పఈൺֱ Makoto Hasegawa JAPAN CONTAINER DAYS V18.04 THANK YOU

Slide 106

Slide 106 text

APPENDIX

Slide 107

Slide 107 text

RKTLET

Slide 108

Slide 108 text

PAGE Japan Container DAYS v 18.04 76 PERFORMANCE RKTLET rktlet ʹରͯ͠ crictl Λ࢖ͬͨΦϖϨʔγϣϯΛߦ͏ How to bench run*me? sandbox pod Λ࡞ΓɺίϯςφΛىಈͤͯ͞མͱ͢·Ͱͷ 4 step rktlet ͷ৔߹ container Λ create ͢Δͱ start ͠ͳͯ͘΋ىಈͨ͠ঢ়ଶʹͳΔΆ͍ container ʹରͯ͠ stop ͱ rm ͕Ͱ͖ͳ͔ͬͨ (͏·͘ઃఆͰ͖ͯͳ͍Մೳੑ͋Γ) sandbox Λ rm ͢Δͱ error: ͱग़Δ͕ͪΌΜͱফ͍͑ͯΔ

Slide 109

Slide 109 text

PAGE Japan Container DAYS v 18.04 77 PERFORMANCE RKTLET

Slide 110

Slide 110 text

PAGE Japan Container DAYS v 18.04 77 PERFORMANCE RKTLET Average of 100 *mes loop “crictl” opera*on rktlet cri-o 1.9.11 cri-o 1.10.0 containerd 1.1.0-rc2 cri-containerd 1.0.0-beta.0 0.00 sec 0.33 sec 0.65 sec 0.98 sec 1.30 sec 0.66 0.37 0.88 1.29 0.78

Slide 111

Slide 111 text

PAGE Japan Container DAYS v 18.04 78 PERFORMANCE RKTLET

Slide 112

Slide 112 text

PAGE Japan Container DAYS v 18.04 0 0.6 1.2 1.8 2.4 cri-o 1.9.11 cri-o 1.10.0 containerd-1.1.0.rc2 cri-containerd rktlet 78 PERFORMANCE RKTLET

Slide 113

Slide 113 text

Docker ͚ͩ͡Όͳ͍ ίϯςφ run*me పఈൺֱ Makoto Hasegawa JAPAN CONTAINER DAYS V18.04 THANK YOU AGAIN