Compare OCI Runtimes - Speaker Deck

Compare OCI Runtimes

by Gosuke Miyashita

Slide 1

Slide 1 text

Slide 2

Slide 2 text

1. ͸͡Ίʹ

Slide 3

Slide 3 text

3 ɾݱࡏOCIϥϯλΠϜͷൺֱʹऔΓ૊ΜͰ͍Δ ɾൺֱ߲໨͸ɺىಈ࣌ؒɺϝϞϦαΠζɺύϑΥʔϚϯεͳͲ ɾϥϯλΠϜຖʹบ͕͋Γɺಉ͡৚݅Ͱൺֱ͢Δͷ͕؆୯Ͱ͸ͳ͍ ɾ۩ମతʹ΍ͬͨ͜ͱɺͱ͘ʹ͸·ͬͨ͜ͱڞ༗͍ͨ͠ ɾܭଌ݁Ռͷڞ༗͸͜ͷൃදͷ໨తͰ͸ͳ͍ ɾݱࡏͷܭଌ݁Ռʹ͍ͭͯ͸matsumotory͞ΜͷεϥΠυࢀর ɾhttps://speakerdeck.com/matsumoto_r/chao-ge-ti-xing- detasentaostoocirantaimu ͓࿩͢͠Δ಺༰

Slide 4

Slide 4 text

1. ܭଌ४උͰ͸·ͬͨ͜ͱ

Slide 5

Slide 5 text

ίϯςφͰಈ͔͢όΠφϦ࡞੒

Slide 6

Slide 6 text

6 ɾൺֱ৚݅Λἧ͑ΔͷͱɺϥϯλΠϜͦͷ΋ͷͷੑ࣭Λଌఆ͍ͨ͠ͷ Ͱɺ୯७ͳϓϩάϥϜΛ༻ҙ ɾhello.c ɾHelloͱදࣔ͢Δ͚ͩͷϓϩάϥϜ ɾىಈ࣌ؒଌఆ༻ ɾloop.c ɾແݶϧʔϓ͢ΔϓϩάϥϜ ɾϝϞϦαΠζଌఆ༻ ίϯςφͰಈ͔͢όΠφϦ

Slide 7

Slide 7 text

7 hello.c #include void main() { printf("Hello\n"); }

Slide 8

Slide 8 text

8 loop.c #include void main() { int i = 0; while(1) { printf("%d\n", i++); } }

Slide 9

Slide 9 text

9 ɾNabla Containers͸ϥΠϒϥϦOSʢϢχΧʔωϧʣܕΞϓϩʔνͷ ϥϯλΠϜ ɾ࣮ߦόΠφϦͱΧʔωϧ͕ҰମԽ ɾͳͷͰNabla Containers༻όΠφϦ͸ผʹ༻ҙ͢Δඞཁ͕͋Δ ɾx86_64-rumprun-netbsd-gcc -o hello.out hello.c ɾrumprun-bake solo5_ukvm_seccomp hello.nabla hello.out ɾ৽͠໨ͷϦϏδϣϯͩͱsolo5_ukvm_seccompͰ͸ͳ͘spt Nabla Containers༻όΠφϦ

Slide 10

Slide 10 text

10 ɾnabla-containers/solo5ΛίϯύΠϧͯ͠Ͱ͖ͨsolo5.oΛ/usr/lib/ libsolo5_seccomp.aʹίϐʔ ɾnabla-containers/runnc ͸ϦϏδϣϯb78fe29Λར༻ ɾnabla-containers/rumprun͸ϦϏδϣϯ8b01b3Λར༻ ɾߋʹҎԼͷύονΛ͋ͯΔ ɾhttps://github.com/rumpkernel/rumprun/issues/122 ɾhttps://github.com/rumpkernel/rumprun/pull/118 Nabla༻όΠφϦͷϏϧυ

Slide 11

Slide 11 text

11 ɾrumprun-bakeίϚϯυʹ΋ύον ɾ࠷ޙʹ࣮ߦͯ͠ΔίϚϯυʹ-L/usr/libΛ௥Ճ ɾhttps://blog.cloudkernels.net/posts/build-a-nabla-docker-image/ ɾ͜ͷखॱʹ͕ͨͬͯ͠࠷৽ͷϦϏδϣϯͰϏϧυͯ͠΋͏·͘ಈ͔ ͳ͔ͬͨ ɾSolo5: ABORT: spt/net.c:36: Assertion `netfd >= 0' failed Nabla༻όΠφϦͷϏϧυʢ͖ͭͮʣ

Slide 12

Slide 12 text

Kata Containersͷόʔδϣϯ

Slide 13

Slide 13 text

13 ɾhttps://github.com/kata-containers/documentation/blob/master/ install/ubuntu-installation-guide.md ɾܭଌ༻ϗετʹUbuntuΛར༻ͨ͠ͷͰ͜ͷखॱʹैͬͨ ɾhttp://download.opensuse.org/repositories/home:/ katacontainers:/releases:/${ARCH}:/master/xUbuntu_$ (lsb_release -rs)/ ͕aptϦϙδτϦͱͯ͠ઃఆ͞ΕΔ ɾ͕ɺ͜Εͩͱ1.6rc1͕Πϯετʔϧ͞Ε·ͱ΋ʹಈ͔ͳ͔ͬͨ ɾs/master/stable-1.5/ Ͱղܾ Kata ContainersͷΠϯετʔϧ

Slide 14

Slide 14 text

1. ܭଌ࣌ʹ͸·ͬͨ͜ͱ

Slide 15

Slide 15 text

ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ

Slide 16

Slide 16 text

16 ɾ༨෼ͳϨΠϠʔΛল͍ͯͳΔ΂͘ૉͷঢ়ଶͰͷܭଌ͕໨త ɾOCI Filesystem BundleΛ༻ҙ ɾdocker export `docker create mizzy/hello` | tar -C bundle/rootfs - xvf - Ͱrootfsੜ੒ ɾrunc specͰconﬁg.jsonੜ੒ ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ

Slide 17

Slide 17 text

17 ɾrunc, gVisor, Kata Containers, Nabla Containersʹ͸OCI Filesystem BundleΛ࣮ߦ͢ΔίϚϯυ͕͋Δ ɾFirecrackerʹ͸(·ͩ?)ଘࡏ͠ͳ͍ͬΆ͍ ɾͳͷͰFirecracker͸௚઀࣮ߦํࣜͰ͸ܭଌͰ͖ͳ͔ͬͨ ɾkata-fc࢖͑͹Ͱ͖ͦ͏ʢະணखʣ ɾhttps://github.com/kata-containers/documentation/wiki/Initial- release-of-Kata-Containers-with-Firecracker-support OCI Filesystem Bundle࣮ߦίϚϯυ

Slide 18

Slide 18 text

18 ɾrunnc͸ଞͷίϚϯυͱҧ͍runαϒίϚϯυ͕ͳ͍ ɾcreateͯ͠start͢Δඞཁ͕͋Δ ɾtime runnc startͰܭଌ͠Α͏ͱ͢Δͱίϯςφ࣮ߦऴྃલʹtime ͷ݁Ռ͕ฦΔ → ྑ͍ܭଌํ๏໛ࡧத ɾconﬁg.jsonͷhooks.prestartͰωοτϫʔΫ·ΘΓͷઃఆΛߦ͏ඞ ཁ͕͋Δ ɾhttps://github.com/nabla-containers/runnc/issues/53 ɾconﬁg.jsonͰࢦఆ͢Δroot.path͕૬ରύεͩͱಈ͔ͳ͍ Nabla Containersׂ͕ͱۂऀ

Slide 19

Slide 19 text

containerdͷctrίϚϯυͰͷܭଌ

Slide 20

Slide 20 text

20 ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰ͸͢΂ͯͷϥϯλΠϜΛܭଌͰ͖ͳ ͔ͬͨͷͰҧ͏ΞϓϩʔνͰܭଌ ɾ͜͜Ͱ΋Nabla Containersͷนཱ͕ͪ͸͔ͩΔ ɾଞͷϥϯλΠϜ͸Shim API v2ʹରԠ͍ͯ͠Δ ɾctr run —runtime=io.containerd.kata.v2 Έ͍ͨʹ࣮ߦͰ͖Δ ɾrunnc͸Shim API v2ʹରԠ͍ͯ͠ͳ͍ containerdͷctrίϚϯυͰͷܭଌ

Slide 21

Slide 21 text

21 ɾ/etc/containerd/conﬁg.toml ɾctr run --runtime io.containerd.runtime.v1.linux Ͱ࣮ߦ ผͷํ๏Ͱ࣮ߦΛࢼΈΔ [plugins] [plugins.linux] shim = "containerd-shim" runtime = "/usr/local/bin/runnc"

Slide 22

Slide 22 text

22 ɾctr: OCI runtime create failed: runnc did not terminate sucessfully: unknown ɾrunnc͕panic: Insufﬁcient uniqueness in IDΛు͍ͯΔ ɾཁ͢Δʹίϯςφ໊͕୹͍ ɾϩάʹ͸͜Ε͕ݟ͋ͨΒͳ͍ͷͰΘ͔Γʹ͍͘ ɾERR: could not create tapabcdefg12345: no master interface: Link not found ɾίϯςφ໊Λ௕͘͢Δͱࠓ౓͸͜ͷΤϥʔ ɾ͜ΕҎ্͸·ͩௐࠪͰ͖͍ͯͳ͍ ࣮ߦ݁Ռ

Slide 23

Slide 23 text

dockerίϚϯυͰͷܭଌ

Slide 24

Slide 24 text

24 ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌɺctrίϚϯυͰͷܭଌɺͱ΋ ʹ͢΂ͯͷϥϯλΠϜΛܭଌ͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ ɾͷͰ࣍͸dockerίϚϯυͰτϥΠ dockerίϚϯυͰͷܭଌ

Slide 25

Slide 25 text

25 ɾFirecrackerͷಈ͔͠ํ͚ͩΘ͔ΒΜɺͱࢥͬͨΒudzura͞ΜʹΑΔ φΠεࢿྉ͕ ɾhttps://speakerdeck.com/udzura/ﬁrecracker-from-low-layer-to- hight?slide=14 ɾKata ContainersͰFirecrackerΛಈ͔͢kata-fcΛར༻ ɾhttps://github.com/kata-containers/documentation/wiki/Initial- release-of-Kata-Containers-with-Firecracker-support ɾDockerͷdevicemapperαϙʔτ͕ඞཁ͕ͩɺݱࡏ࠷৽ͷ18.09͕ devicemapperରԠ͍ͯ͠ͳ͍ͷͰɺ18.06Λར༻͢Δඞཁ͋Γ Docker + Firecraker

Slide 26

Slide 26 text

26 ɾ࠷ॳ͸ҙຯ͕Θ͔Βͳ͔ͬͨ ɾ͑ɺͲͬͪ΋OCIϥϯλΠϜ͡Όͳ͍ͷʁ ɾFiracracker͸ϚΫϩͳࢹ఺ͰݟΔͱOCIϥϯλΠϜͱͯ͠ݟΔ͜ͱ ΋Ͱ͖Δ͕ϛΫϩͳࢹ఺ͰݟΔͱVMM ɾKata Containers͸VM಺ͰίϯςφΛىಈ͢ΔΞϓϩʔνͷOCIϥ ϯλΠϜ ɾVMMͱͯ͠σϑΥϧτͰQEMUΛར༻͢Δ͕ࠩ͠ସ͑Մೳ ɾͭ·ΓKata ConͷVMMΛFirecrackerʹࠩ͠ସ͑Δ͜ͱ͕Ͱ͖Δ Kata Containers + Firecracker?

Slide 27

Slide 27 text

ctrͱdockerͰFirecrackerͷ ىಈ͕࣌ؒ૝ఆͱҟͳΔ

Slide 28

Slide 28 text

28 ɾctr: real 0m6.320s ɾdocker: real 0m4.105s ɾdockerͷํ͕ɺdockerdΛܦ༝͢Δ෼஗͘ͳΓͦ͏ͳͷʹͳͥʁ ɾctr͸naive snapshotterΛར༻ ɾdocker͸devicemapperΛར༻ ɾctrͰdevmapper snapshotterΛར༻͢Ε͹ಉ͡৚݅ͰൺֱͰ͖ͦ͏ ɾ→ ະணख ctrͱdockerͰͷFirecrackerىಈ࣌ؒ

Slide 29

Slide 29 text

1. ίετ

Slide 30

Slide 30 text

30 ɾݕূ؀ڥΛVagrant+VirtualBoxͰߏங ɾKataͱFirecracker͸KVM͕ඞཁ ɾVirtualBoxͰ͸KVMಈ͔ͳ͍ ɾVMWare Fusion + Vagrant VMWare ProviderΛߪೖ ɾVMWare Fusion: 9,925ԁ ɾVagrant VMWare Provider: $79 per seat ׂͱ͓͕͔͔ۚΔ (on macOS)

Slide 31

Slide 31 text

31 ɾVagrant + VMWare FusionͰmodprobe vhost_vsock͕Τϥʔʹ ͳͬͯ͠·͏ͷͰAWS EC2্Ͱ΋ݕূ ɾKVMΛಈ͔ͨ͢ΊʹϕΞϝλϧΠϯελϯε͕ඞཁ ɾi3.metalͰ4.992USD/࣌ؒ ɾ1೔ͰBilling AlertඈΜͰདྷͨ ׂͱ͓͕͔͔ۚΔ (on AWS)

Slide 32

Slide 32 text

1. ࢀߟࢿྉ

Slide 33

Slide 33 text

33 ɾhttps://github.com/mizzy/container-playground ɾmeasurements/ ɾVagrant + VMWare FusionͰಈ͔ͯ͠Δ΍ͭ ɾcompare_on_i3_metal/ ɾVagrant + AWS EC2 i3.metalΠϯελϯεͰಈ͔ͯ͠Δ΍ͭ ɾ੔ཧͰ͖ͯͳ͍͠ɺ௨͠Ͱvagrant provisionͯ͠ͳ͍ͷͰಈ͔ͳ ͍ͱ͜Ζ͋Γͦ͏ ɾࢼͯ͠ΈͯΘ͔Βͳ͍͜ͱ͕͋Ε͹ԿͰ΋ฉ͍͍ͯͩ͘͞ ܭଌ༻ϦϙδτϦ

Slide 34

Slide 34 text

34 ɾࠓ࿩୊ͷ͍Ζ͍ΖͳίϯςφϥϯλΠϜΛൺֱͯ͠Έͨ ɾhttps://www.slideshare.net/KoheiTokunaga/ss-123664087 ɾ֤छϥϯλΠϜͷಛ௃΍ൺֱͳͲͱͯ΋ࢀߟʹͳΔ ɾNabla ContainersΛಈ͔͢ʹ͋ͨͬͯͱͯ΋ࢀߟʹͳͬͨ ɾࢿྉͰ͸ܭଌʹkubernetes-sigs/cri-toolsΛར༻͍ͯ͠ΔͷͰ͜Ε ΋ࢼͯ͠Έ͍ͨ ࢀߟࢿྉ