Compare OCI Runtimes

Transcript

1. ͘͞ΒΠϯλʔωοτגࣜձࣾ
   (C) Copyright 1996-2019 SAKURA Internet Inc
   ͘͞ΒΠϯλʔωοτݚڀॴ
   OCIϥϯλΠϜൺֱͷͨΊʹ
   ΍͍ͬͯΔ͜ͱ͋Ε͜Ε
   2019/03/22 ٬һݚڀһ ٶԼ ߶ี
   runc, gVisor, Kata Containers
   Nabla Containers, Firecrackerൺֱ
   

   View Slide

2. 1.
   ͸͡Ίʹ
   

   View Slide

3. 3
   ɾݱࡏOCIϥϯλΠϜͷൺֱʹऔΓ૊ΜͰ͍Δ
   ɾൺֱ߲໨͸ɺىಈ࣌ؒɺϝϞϦαΠζɺύϑΥʔϚϯεͳͲ
   ɾϥϯλΠϜຖʹบ͕͋Γɺಉ͡৚݅Ͱൺֱ͢Δͷ͕؆୯Ͱ͸ͳ͍
   ɾ۩ମతʹ΍ͬͨ͜ͱɺͱ͘ʹ͸·ͬͨ͜ͱڞ༗͍ͨ͠
   ɾܭଌ݁Ռͷڞ༗͸͜ͷൃදͷ໨తͰ͸ͳ͍
   ɾݱࡏͷܭଌ݁Ռʹ͍ͭͯ͸matsumotory͞ΜͷεϥΠυࢀর
   ɾhttps://speakerdeck.com/matsumoto_r/chao-ge-ti-xing-
   detasentaostoocirantaimu
   ͓࿩͢͠Δ಺༰
   

   View Slide

4. 1.
   ܭଌ४උͰ͸·ͬͨ͜ͱ
   

   View Slide

5. ίϯςφͰಈ͔͢όΠφϦ࡞੒
   

   View Slide

6. 6
   ɾൺֱ৚݅Λἧ͑ΔͷͱɺϥϯλΠϜͦͷ΋ͷͷੑ࣭Λଌఆ͍ͨ͠ͷ
   Ͱɺ୯७ͳϓϩάϥϜΛ༻ҙ
   ɾhello.c
   ɾHelloͱදࣔ͢Δ͚ͩͷϓϩάϥϜ
   ɾىಈ࣌ؒଌఆ༻
   ɾloop.c
   ɾແݶϧʔϓ͢ΔϓϩάϥϜ
   ɾϝϞϦαΠζଌఆ༻
   ίϯςφͰಈ͔͢όΠφϦ
   

   View Slide

7. 7
   hello.c
   #include
   void main()
   {
   printf("Hello\n");
   }
   

   View Slide

8. 8
   loop.c
   #include
   void main()
   {
   int i = 0;
   while(1) {
   printf("%d\n", i++);
   }
   }
   

   View Slide

9. 9
   ɾNabla Containers͸ϥΠϒϥϦOSʢϢχΧʔωϧʣܕΞϓϩʔνͷ
   ϥϯλΠϜ
   ɾ࣮ߦόΠφϦͱΧʔωϧ͕ҰମԽ
   ɾͳͷͰNabla Containers༻όΠφϦ͸ผʹ༻ҙ͢Δඞཁ͕͋Δ
   ɾx86_64-rumprun-netbsd-gcc -o hello.out hello.c
   ɾrumprun-bake solo5_ukvm_seccomp hello.nabla hello.out
   ɾ৽͠໨ͷϦϏδϣϯͩͱsolo5_ukvm_seccompͰ͸ͳ͘spt
   Nabla Containers༻όΠφϦ
   

   View Slide

10. 10
    ɾnabla-containers/solo5ΛίϯύΠϧͯ͠Ͱ͖ͨsolo5.oΛ/usr/lib/
    libsolo5_seccomp.aʹίϐʔ
    ɾnabla-containers/runnc ͸ϦϏδϣϯb78fe29Λར༻
    ɾnabla-containers/rumprun͸ϦϏδϣϯ8b01b3Λར༻
    ɾߋʹҎԼͷύονΛ͋ͯΔ
    ɾhttps://github.com/rumpkernel/rumprun/issues/122
    ɾhttps://github.com/rumpkernel/rumprun/pull/118
    Nabla༻όΠφϦͷϏϧυ
    

    View Slide

11. 11
    ɾrumprun-bakeίϚϯυʹ΋ύον
    ɾ࠷ޙʹ࣮ߦͯ͠ΔίϚϯυʹ-L/usr/libΛ௥Ճ
    ɾhttps://blog.cloudkernels.net/posts/build-a-nabla-docker-image/
    ɾ͜ͷखॱʹ͕ͨͬͯ͠࠷৽ͷϦϏδϣϯͰϏϧυͯ͠΋͏·͘ಈ͔
    ͳ͔ͬͨ
    ɾSolo5: ABORT: spt/net.c:36: Assertion `netfd >= 0' failed
    Nabla༻όΠφϦͷϏϧυʢ͖ͭͮʣ
    

    View Slide

12. Kata Containersͷόʔδϣϯ
    

    View Slide

13. 13
    ɾhttps://github.com/kata-containers/documentation/blob/master/
    install/ubuntu-installation-guide.md
    ɾܭଌ༻ϗετʹUbuntuΛར༻ͨ͠ͷͰ͜ͷखॱʹैͬͨ
    ɾhttp://download.opensuse.org/repositories/home:/
    katacontainers:/releases:/${ARCH}:/master/xUbuntu_$
    (lsb_release -rs)/ ͕aptϦϙδτϦͱͯ͠ઃఆ͞ΕΔ
    ɾ͕ɺ͜Εͩͱ1.6rc1͕Πϯετʔϧ͞Ε·ͱ΋ʹಈ͔ͳ͔ͬͨ
    ɾs/master/stable-1.5/ Ͱղܾ
    Kata ContainersͷΠϯετʔϧ
    

    View Slide

14. 1.
    ܭଌ࣌ʹ͸·ͬͨ͜ͱ
    

    View Slide

15. ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ
    

    View Slide

16. 16
    ɾ༨෼ͳϨΠϠʔΛল͍ͯͳΔ΂͘ૉͷঢ়ଶͰͷܭଌ͕໨త
    ɾOCI Filesystem BundleΛ༻ҙ
    ɾdocker export `docker create mizzy/hello` | tar -C bundle/rootfs -
    xvf - Ͱrootfsੜ੒
    ɾrunc specͰconfig.jsonੜ੒
    ϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌ
    

    View Slide

17. 17
    ɾrunc, gVisor, Kata Containers, Nabla Containersʹ͸OCI
    Filesystem BundleΛ࣮ߦ͢ΔίϚϯυ͕͋Δ
    ɾFirecrackerʹ͸(·ͩ?)ଘࡏ͠ͳ͍ͬΆ͍
    ɾͳͷͰFirecracker͸௚઀࣮ߦํࣜͰ͸ܭଌͰ͖ͳ͔ͬͨ
    ɾkata-fc࢖͑͹Ͱ͖ͦ͏ʢະணखʣ
    ɾhttps://github.com/kata-containers/documentation/wiki/Initial-
    release-of-Kata-Containers-with-Firecracker-support
    OCI Filesystem Bundle࣮ߦίϚϯυ
    

    View Slide

18. 18
    ɾrunnc͸ଞͷίϚϯυͱҧ͍runαϒίϚϯυ͕ͳ͍
    ɾcreateͯ͠start͢Δඞཁ͕͋Δ
    ɾtime runnc startͰܭଌ͠Α͏ͱ͢Δͱίϯςφ࣮ߦऴྃલʹtime
    ͷ݁Ռ͕ฦΔ → ྑ͍ܭଌํ๏໛ࡧத
    ɾconfig.jsonͷhooks.prestartͰωοτϫʔΫ·ΘΓͷઃఆΛߦ͏ඞ
    ཁ͕͋Δ
    ɾhttps://github.com/nabla-containers/runnc/issues/53
    ɾconfig.jsonͰࢦఆ͢Δroot.path͕૬ରύεͩͱಈ͔ͳ͍
    Nabla Containersׂ͕ͱۂऀ
    

    View Slide

19. containerdͷctrίϚϯυͰͷܭଌ
    

    View Slide

20. 20
    ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰ͸͢΂ͯͷϥϯλΠϜΛܭଌͰ͖ͳ
    ͔ͬͨͷͰҧ͏ΞϓϩʔνͰܭଌ
    ɾ͜͜Ͱ΋Nabla Containersͷนཱ͕ͪ͸͔ͩΔ
    ɾଞͷϥϯλΠϜ͸Shim API v2ʹରԠ͍ͯ͠Δ
    ɾctr run —runtime=io.containerd.kata.v2 Έ͍ͨʹ࣮ߦͰ͖Δ
    ɾrunnc͸Shim API v2ʹରԠ͍ͯ͠ͳ͍
    containerdͷctrίϚϯυͰͷܭଌ
    

    View Slide

21. 21
    ɾ/etc/containerd/config.toml
    ɾctr run --runtime io.containerd.runtime.v1.linux Ͱ࣮ߦ
    ผͷํ๏Ͱ࣮ߦΛࢼΈΔ
    [plugins]
    [plugins.linux]
    shim = "containerd-shim"
    runtime = "/usr/local/bin/runnc"
    

    View Slide

22. 22
    ɾctr: OCI runtime create failed: runnc did not terminate
    sucessfully: unknown
    ɾrunnc͕panic: Insufficient uniqueness in IDΛు͍ͯΔ
    ɾཁ͢Δʹίϯςφ໊͕୹͍
    ɾϩάʹ͸͜Ε͕ݟ͋ͨΒͳ͍ͷͰΘ͔Γʹ͍͘
    ɾERR: could not create tapabcdefg12345: no master interface:
    Link not found
    ɾίϯςφ໊Λ௕͘͢Δͱࠓ౓͸͜ͷΤϥʔ
    ɾ͜ΕҎ্͸·ͩௐࠪͰ͖͍ͯͳ͍
    ࣮ߦ݁Ռ
    

    View Slide

23. dockerίϚϯυͰͷܭଌ
    

    View Slide

24. 24
    ɾϥϯλΠϜίϚϯυ௚઀࣮ߦͰͷܭଌɺctrίϚϯυͰͷܭଌɺͱ΋
    ʹ͢΂ͯͷϥϯλΠϜΛܭଌ͢Δ͜ͱ͕Ͱ͖ͳ͔ͬͨ
    ɾͷͰ࣍͸dockerίϚϯυͰτϥΠ
    dockerίϚϯυͰͷܭଌ
    

    View Slide

25. 25
    ɾFirecrackerͷಈ͔͠ํ͚ͩΘ͔ΒΜɺͱࢥͬͨΒudzura͞ΜʹΑΔ
    φΠεࢿྉ͕
    ɾhttps://speakerdeck.com/udzura/firecracker-from-low-layer-to-
    hight?slide=14
    ɾKata ContainersͰFirecrackerΛಈ͔͢kata-fcΛར༻
    ɾhttps://github.com/kata-containers/documentation/wiki/Initial-
    release-of-Kata-Containers-with-Firecracker-support
    ɾDockerͷdevicemapperαϙʔτ͕ඞཁ͕ͩɺݱࡏ࠷৽ͷ18.09͕
    devicemapperରԠ͍ͯ͠ͳ͍ͷͰɺ18.06Λར༻͢Δඞཁ͋Γ
    Docker + Firecraker
    

    View Slide

26. 26
    ɾ࠷ॳ͸ҙຯ͕Θ͔Βͳ͔ͬͨ
    ɾ͑ɺͲͬͪ΋OCIϥϯλΠϜ͡Όͳ͍ͷʁ
    ɾFiracracker͸ϚΫϩͳࢹ఺ͰݟΔͱOCIϥϯλΠϜͱͯ͠ݟΔ͜ͱ
    ΋Ͱ͖Δ͕ϛΫϩͳࢹ఺ͰݟΔͱVMM
    ɾKata Containers͸VM಺ͰίϯςφΛىಈ͢ΔΞϓϩʔνͷOCIϥ
    ϯλΠϜ
    ɾVMMͱͯ͠σϑΥϧτͰQEMUΛར༻͢Δ͕ࠩ͠ସ͑Մೳ
    ɾͭ·ΓKata ConͷVMMΛFirecrackerʹࠩ͠ସ͑Δ͜ͱ͕Ͱ͖Δ
    Kata Containers + Firecracker?
    

    View Slide

27. ctrͱdockerͰFirecrackerͷ
    ىಈ͕࣌ؒ૝ఆͱҟͳΔ
    

    View Slide

28. 28
    ɾctr: real 0m6.320s
    ɾdocker: real 0m4.105s
    ɾdockerͷํ͕ɺdockerdΛܦ༝͢Δ෼஗͘ͳΓͦ͏ͳͷʹͳͥʁ
    ɾctr͸naive snapshotterΛར༻
    ɾdocker͸devicemapperΛར༻
    ɾctrͰdevmapper snapshotterΛར༻͢Ε͹ಉ͡৚݅ͰൺֱͰ͖ͦ͏
    ɾ→ ະணख
    ctrͱdockerͰͷFirecrackerىಈ࣌ؒ
    

    View Slide

29. 1.
    ίετ
    

    View Slide

30. 30
    ɾݕূ؀ڥΛVagrant+VirtualBoxͰߏங
    ɾKataͱFirecracker͸KVM͕ඞཁ
    ɾVirtualBoxͰ͸KVMಈ͔ͳ͍
    ɾVMWare Fusion + Vagrant VMWare ProviderΛߪೖ
    ɾVMWare Fusion: 9,925ԁ
    ɾVagrant VMWare Provider: $79 per seat
    ׂͱ͓͕͔͔ۚΔ (on macOS)
    

    View Slide

31. 31
    ɾVagrant + VMWare FusionͰmodprobe vhost_vsock͕Τϥʔʹ
    ͳͬͯ͠·͏ͷͰAWS EC2্Ͱ΋ݕূ
    ɾKVMΛಈ͔ͨ͢ΊʹϕΞϝλϧΠϯελϯε͕ඞཁ
    ɾi3.metalͰ4.992USD/࣌ؒ
    ɾ1೔ͰBilling AlertඈΜͰདྷͨ
    ׂͱ͓͕͔͔ۚΔ (on AWS)
    

    View Slide

32. 1.
    ࢀߟࢿྉ
    

    View Slide

33. 33
    ɾhttps://github.com/mizzy/container-playground
    ɾmeasurements/
    ɾVagrant + VMWare FusionͰಈ͔ͯ͠Δ΍ͭ
    ɾcompare_on_i3_metal/
    ɾVagrant + AWS EC2 i3.metalΠϯελϯεͰಈ͔ͯ͠Δ΍ͭ
    ɾ੔ཧͰ͖ͯͳ͍͠ɺ௨͠Ͱvagrant provisionͯ͠ͳ͍ͷͰಈ͔ͳ
    ͍ͱ͜Ζ͋Γͦ͏
    ɾࢼͯ͠ΈͯΘ͔Βͳ͍͜ͱ͕͋Ε͹ԿͰ΋ฉ͍͍ͯͩ͘͞
    ܭଌ༻ϦϙδτϦ
    

    View Slide

34. 34
    ɾࠓ࿩୊ͷ͍Ζ͍ΖͳίϯςφϥϯλΠϜΛൺֱͯ͠Έͨ
    ɾhttps://www.slideshare.net/KoheiTokunaga/ss-123664087
    ɾ֤छϥϯλΠϜͷಛ௃΍ൺֱͳͲͱͯ΋ࢀߟʹͳΔ
    ɾNabla ContainersΛಈ͔͢ʹ͋ͨͬͯͱͯ΋ࢀߟʹͳͬͨ
    ɾࢿྉͰ͸ܭଌʹkubernetes-sigs/cri-toolsΛར༻͍ͯ͠ΔͷͰ͜Ε
    ΋ࢼͯ͠Έ͍ͨ
    ࢀߟࢿྉ
    

    View Slide