͍·͞Βฉ͚ͳ͍ Linuxίϯςφͷجૅ OSC 2015 Kansai@Kyoto Ճ౻ହจ lxc-jp 2015-08-08 Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 1 / 54

ࣗݾ঺հ Ճ౻ହจ http://www.ten-forward.ws/ @ten forward http://gplus.to/tenforward https://github.com/tenforward http://d.hatena.ne.jp/defiant/ (ٕज़ϒϩά) Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 2 / 54

ࣗݾ঺հ Plamo Linux ϝϯςφ LXC ͰֶͿίϯςφೖ໳ɹʔܰྔԾ૝Խ؀ڥΛ࣮ݱ͢Δٕज़ gihyo.jp Ͱ࿈ࡌ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 3 / 54

ࣗݾ঺հ LXC ͷ։ൃʹগ͠ࢀՃ man page ͷ೔ຊޠ༁ ެࣜϖʔδ (linuxcontainers.org) ຋༁ όάϑΟοΫεͳͲগ͚ͩ͠ίʔυʹ΋ߩݙ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 4 / 54

࣭໰ɿΈͳ͞Μʹͱͬͯͷίϯ ςφ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 5 / 54

౰વίϨͰ͠ΐ͏ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 6 / 54

σʔληϯλʔ͡ΌͶʁ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 7 / 54

࠷ۙྲྀߦͬͯΔΒ͍͚͠ͲԿʁ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 8 / 54

Docker஌ͬͯΔΑ LXC஌ͬͯΔΑ OpenVZ஌ͬͯΔΑ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 9 / 54

Docker࢖ͬͨ͜ͱ͋ΔΑ LXC࢖ͬͨ͜ͱ͋ΔΑ OpenVZ࢖ͬͨ͜ͱ͋ΔΑ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 10 / 54

ࠓ೔ͷ໨ඪ ίϯςφͷ֓ཁΛཧղ͢Δ Linux Χʔωϧ͕࣋ͭίϯςφ͕࢖͏ओཁͳػೳΛ֮͑Δ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 11 / 54

ࠓ೔ͷ಺༰ ίϯςφͷ֓ཁ Linux ʹ͓͚Δίϯςφͷ࢓૊Έ Namespace σϞ Cgroup σϞ ωοτϫʔΫؔ࿈ػೳ (͕࣌ؒ͋Ε͹) ίϯςφͰ࢖͑Δ໘ന͍ػೳ (͕࣌ؒ͋Ε͹) ·ͱΊ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 12 / 54

ίϯςφ֓ཁ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 13 / 54

ίϯςφͱ͸ Ծ૝తͳίϯϐϡʔλɾγεςϜΛ࠶ݱ͢ΔԾ૝Ϛγϯʹର ͯ͠ɺԾ૝తͳ OS ؀ڥΛఏڙ͢Δ ˠ OS ϨϕϧͷԾ૝Խ Χʔωϧ͔ΒݟΔͱී௨ʹϓϩηε͕ىಈ͢Δ͚ͩ ىಈ͢Δࡍʹִ཭Λࢦࣔ͢Δ ΧʔωϧͷػೳͰ (ෳ਺ͷ) ಠཱۭͨؒ͠Λ࡞Γग़͠ɼϦιʔ εΛ෼ׂɾ෼഑͢Δ ϓϩηεΛάϧʔϓԽͯ͠ଞͷάϧʔϓͱϦιʔεۭؒΛִ཭ άϧʔϓԽͨ͠ϓϩηεʹର͢ΔϦιʔε੍ݶ Ծ૝Խͱ͍͏ΑΓʮִ཭Խʯͱݴͬͨ΄͏͕Θ͔Γ΍͍͔͢΋ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 14 / 54

ίϯςφͷϝϦοτ ߴີ౓Խ͕Մೳ ىಈ͍ͯ͠Δ OS (Χʔωϧ) ͸Ұͭ Φʔόʔϔου͕খ͍͞ ϋʔυ΢ΣΞͷԾ૝Խ͕ෆཁ ىಈ͕ૣ͍ Ծ૝ϚγϯͷىಈͰ͸ͳ͘ɼϗετ OS ͔ΒݟͨΒ୯ʹϓϩ ηε͕ىಈ͍ͯ͠Δ͚ͩͳͷͰɼී௨ͷϓϩάϥϜ͕ىಈ͢Δ ͷͱ΄ͱΜͲมΘΒͳ͍ ඞͣ͠΋γεςϜΛಈ͔͢ඞཁ͸ͳ͍ (ΞϓϦέʔγϣϯί ϯςφ) ྫ͑͹ίϯςφ಺Ͱ͸ httpd ͷΈ͕ಈ͍͍ͯΔ ίϯςφʹϝϞϦΛݻఆతʹׂΓ౰ͯΔඞཁ͕ͳ͍ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 15 / 54

ίϯςφͷσϝϦοτ ҟͳΔ OS ͷγεςϜ / ϓϩάϥϜ͸ಈ͔ͤͳ͍ ୯ʹϗετ OS ্Ͱϓϩηε͕ىಈ͢Δ͚ͩͳͷͰ౰ͨΓલ ΧʔωϧʹؔΘΔૢ࡞͸Ͱ͖ͳ͍ ىಈ͍ͯ͠ΔΧʔωϧ͸มΘΒͳ͍ͷͰ ίϯςφຖʹϩʔυ͢ΔϞδϡʔϧΛม͑ΔͳͲ Χʔωϧͷ࣮૷͸ෳࡶʹͳΔ શͯΧʔωϧͷػೳͱ࣮ͯ͠૷͞Ε͍ͯΔͷͰ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 16 / 54

ࠓ೔ͷ಺༰ ίϯςφͷ֓ཁ Linux ʹ͓͚Δίϯςφͷ࢓૊Έ Namespace σϞ Cgroup σϞ ωοτϫʔΫؔ࿈ػೳ (͕࣌ؒ͋Ε͹) ίϯςφͰ࢖͑Δ໘ന͍ػೳ (͕࣌ؒ͋Ε͹) ·ͱΊ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 17 / 54

Linuxʹ͓͚Δίϯςφͷ࢓ ૊Έ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 18 / 54

(ͨ·ʹݟ͔͚Δ)ίϯςφͷྺ࢙͔Βݟͨ Linuxίϯςφʹର͢Δޡղ 1979 ೥ʹ chroot(2) γεςϜίʔϧ͕ Version 7 Unix ʹɺ 1983 ೥ʹ 4.2BSD ʹಋೖ͞ΕΔ 2000 ೥ʹ FreeBSD jail ͕ FreeBSD 4.0 ʹಋೖ ʰDocker? ͦΜͳ΋Μ jail Ͱͣͬͱલ͔ΒͰ͖ͯΔʱ(Docker ੝Γ্͕͖ͬͯͨ࣌ͷΦοαϯͷ੠) ˠ ʷؒҧ͍ 2005 ೥ʹ Solaris Containers ొ৔ 2008 ೥ʹ Linux ʹ Cgroup ͕ಋೖ ʰ͜ΕͰ Linux Ͱ΋͍ͭʹίϯςφ͕ՄೳʹͳΓ·ͨ͠ʱ ˠʷؒҧ͍ 2014 ೥ Docker 1.0 ϦϦʔε (LXC 1.0 ΋ϦϦʔε) ʰCgroup ͷొ৔Ͱ Linux Ͱ΋ίϯςφ͕Մೳʹʯ͕ؒҧ͍ͳཧ༝ ͸͜ͷޙΘ͔Γ·͢ :-) Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 19 / 54

Linuxʹ͓͚Δίϯςφ͸Χʔωϧʹʰίϯ ςφʱͱ͍͏୯Ұͷػೳ͕࣮૷͞Ε࣮ͯݱ͠ ͍ͯΔΘ͚Ͱ͸͋Γ·ͤΜ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 20 / 54

Linux ͰίϯςφΛ࣮ݱ͢ΔͨΊͷػೳ Linux Χʔωϧʹؚ·ΕΔ৭ʑͳػೳΛ૊Έ߹Θͤͯίϯςφ؀ ڥΛ࡞੒͢ΔɻͦΕͧΕͷػೳ͸ίϯςφઐ༻ͷػೳͱ͍͏Θ͚ Ͱ͸ͳ͍ɻ ϓϩηεΛάϧʔϓԽͯ͠ଞͷάϧʔϓͱִ཭ OS Ϧιʔεͷִ཭ ˠ Namespace (໊લۭؒ) άϧʔϓԽͨ͠ϓϩηεʹର͢ΔϦιʔε੍ݶ ϗετͷ෺ཧϦιʔεʹର͢Δ੍ݶ ˠ Cgroup (control group) Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 21 / 54

LinuxͰίϯςφΛ࣮ݱ͢ΔͨΊͷػೳ ͦͷଞ ωοτϫʔΫ (veth, macvlan ͳͲ) έʔύϏϦςΟ chroot (pivot root) bind mount Checkpoint/Restore (CRIU) ͳͲͳͲ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 22 / 54

ࠓ೔ͷ಺༰ ίϯςφͷ֓ཁ Linux ʹ͓͚Δίϯςφͷ࢓૊Έ Namespace σϞ Cgroup σϞ ωοτϫʔΫؔ࿈ػೳ (͕࣌ؒ͋Ε͹) ίϯςφͰ࢖͑Δ໘ന͍ػೳ (͕࣌ؒ͋Ε͹) ·ͱΊ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 23 / 54

Linuxʹ͓͚Δίϯςφͷ࢓૊Έ Namespace Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 24 / 54

Namespace(໊લۭؒ) ִ཭͍ͨ͠ OS Ϧιʔε͝ͱʹ Namespace ͕४උ͞ΕΔ Ұ෦ͷ Namespace ͚ͩ࢖༻ִͯ͠཭؀ڥΛ࡞Δ͜ͱ͕Ͱ͖Δ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 25 / 54

Namespace ͷछྨ (1) Mount Namespace: 2.4.19 ϓϩηε͔Βݟ͍͑ͯΔϚ΢ϯτͷू߹ɼૢ࡞Λ෼཭͢Δɽ Namespace ಺ͷ mount, umount ͸ଞͷ Namespace ʹ͸Ө ڹ͠ͳ͍ (ࢀߟ) Ϛ΢ϯτ໊લۭؒΛద༻͢Δ (IBM developerWorks) UTS Namespace: 2.6.19 ϗετ໊ͳͲɼuname(2) ͕ฦ͢஋ͷू߹Λ෼཭ɽ setdomainname(2), sethostname(2) Ͱ Namespace ಺ͷ஋ ͷΈมߋͰ͖Δ PID Namespace: 2.6.24 PID ۭؒͷ෼཭ɽ৽͍͠ PID Namespace Ͱ͸ PID 1 ͔Β࢝ ·Δ PID ׂ͕Γ౰ͯΒΕΔɽ਌͔Βࢠͷ PID Namespace ͸ ݟ͑Δ (਌ͷۭؒͷ PID Λ࣋ͭ) ͕ɼࢠ͔Β਌͸ݟ͑ͳ͍ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 26 / 54

Namespace ͷछྨ (2) IPC Namespace: 2.6.19 SysV IPC ΦϒδΣΫτɼPOSIX ϝοηʔδΩϡʔͷִ཭ User Namespace: 2.6.23 ˜ 3.8 ಠཱͨ͠ UID/GID ۭؒͱ֎෦ۭؒͷϚοϐϯά (ྫ͑͹ɼִ ཭ۭؒͰ͸ uid/gid 0/0ɼ֎෦Ͱ͸ 1000/1000 ͱ͔Մೳʹ ͳΔ) Network Namespace: 2.6.26 ωοτϫʔΫϦιʔεͷִ཭ɽωοτϫʔΫσόΠεɼΞυϨ εɼϧʔςΟϯάςʔϒϧɼιέοτɼϑΟϧλϦϯά Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 27 / 54

Namespace ͷૢ࡞ (γεςϜίʔϧ) clone(2) Ͱ৽͍͠ϓϩηε Λੜ੒ unshare(2) Ͱ৽͍͠ϓϩηεΛੜ੒ͤͣʹ࣮ߦίϯςΩετ Λ੍ޚ͢Δ setns(2) ͰϓϩηεΛطଘ ͷ Namespace ʹؔ࿈෇͚Δ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 28 / 54

NamespaceσϞ 1 Namespace Λ࡞Δલʹϗετ໊Λ֬ೝ (ϗετ؀ڥͷϗετ໊) 2 Namespace Λ࡞ΔલʹશϓϩηεΛҰཡ (ϗετ؀ڥͷϓϩηε) 3 util-linux ෇ଐͷ unshare ίϚϯυΛ࢖ͬͯ PID, UTS, Mount Namespace(໊લۭؒ) Λ࡞੒ 4 Namespace ಺ͰશϓϩηεΛҰཡ 5 Namespace ಺Ͱϗετ໊Λมߋ 6 Namespace Λൈ͚Δͱϗετ໊͕มΘ͍ͬͯͳ͍͜ͱΛ֬ೝ https://asciinema.org/a/24150 ৽͍͠ util-linux ͕ඞཁ (σϞ͸ 2.26.2) Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 29 / 54

ࠓ೔ͷ಺༰ ίϯςφͷ֓ཁ Linux ʹ͓͚Δίϯςφͷ࢓૊Έ Namespace σϞ Cgroup σϞ ωοτϫʔΫؔ࿈ػೳ (͕࣌ؒ͋Ε͹) ίϯςφͰ࢖͑Δ໘ന͍ػೳ (͕࣌ؒ͋Ε͹) ·ͱΊ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 30 / 54

Linuxʹ͓͚Δίϯςφͷ࢓૊Έ Cgroup Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 31 / 54

Cgroupͱ͸ ϓϩηεΛάϧʔϓԽ͠ɺάϧʔϓʹରͯ͠Ϧιʔε੍ݶΛߦ͏ɻ ίϯςφઐ༻ͷ࢓૊ΈͰ͸ͳ͍ɻ Cgroup ͷಛ௃ ػೳ͝ͱʹαϒγεςϜʹ෼͔ΕΔ cgroupfs ΛϚ΢ϯτͯ͠σΟϨΫτϦͰάϧʔϓΛද͢ ϓϩηεΛάϧʔϓ಺ͷ tasks ϑΝΠϧʹ௥Ճ͢Δͱؔ࿈͢Δ λεΫ͕εϨου୯ҐͰάϧʔϓʹ௥Ճ͞ΕΔ ෳ਺֊૚ߏ଄ɻվ଄ߏ଄͝ͱʹҟͳΔπϦʔΛ࡞੒Ͱ͖Δɻͨ ͩ͠ɺҰͭͷαϒγεςϜ͕ॴଐͰ͖ΔπϦʔ͸Ұͭ πϦʔͷͲͷϨϕϧͷάϧʔϓʹ΋λεΫ͕ॴଐͰ͖Δ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 32 / 54

Cgroupͷ֊૚ߏ଄ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 33 / 54

CgroupͷαϒγεςϜ cpu: 2.6.24 CFS(Completely Fair Scheduler) bandwidth controlɽ୯Ґ ࣌ؒ಺ͷάϧʔϓ಺ͷλεΫ͕࣮ߦͰ͖Δ߹ܭ࣌ؒΛ੍ݶ͢Δ (3.2 Ͱ࣮૷) ૬ର഑෼ɽάϧʔϓؒͷ CPU ࣌ؒͷׂ౰ͷׂ߹Λࢦఆ͢Δɽ ྫ͑͹ GroupA=100,GroupB=50 ͱ͢Δͱ A:B=2:1 cpuacct: 2.6.24 άϧʔϓ಺ͷ CPU ϦιʔεͷϨϙʔτ (CPU ࣌ؒ) cpuset: 2.6.24 ׂΓ౰ͯΔ CPU, ϝϞϦϊʔυͷׂ౰ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 34 / 54

CgroupͷαϒγεςϜ device: 2.6.26 σόΠε΁ͷΞΫηεڐՄɼ੍ݶͷࢦఆ freezer: 2.6.28 άϧʔϓ಺ͷϓϩηεΛશͯҰ࣌ఀࢭ͢Δ memory: 2.6.29 ϝϞϦϦιʔεͷ੍ݶ (ϢʔβϝϞϦɼΧʔωϧϝϞϦ) blkio (Block IO): I/O weight controller(2.6.33 Ҏ߱) άϧʔϓͷ༏ઌ౓Λࢦఆ ͢Δ I/O throttling(2.6.37 Ҏ߱) άϧʔϓ಺ͷϓϩηεͷσόΠ εʹର͢Δૢ࡞਺ͷ߹ܭͷࢦఆ (ࢀߟ)Linux2.6.37 ͷ৽ػೳ “I/O throttling” Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 35 / 54

CgroupͷαϒγεςϜ hugetlb: 3.6 cgroup ͔Βͷ hugetlb ͷ࢖༻ perf event: 2.6.39 άϧʔϓ୯ҐͰ perf πʔϧͰϞχλϦϯά (ύϑΥʔϚϯε ղੳ) net cls: 2.6.29 ύέοτʹࣝผࢠΛ͚ͭɼτϥϑΟοΫίϯτϩʔϧ (tc) ͱ netfilter(3.14 Ҏ߱) ͰίϯτϩʔϧՄೳʹ Linux 3.14 Ͱ net cls cgroup ʹ௥Ճ͞Εͨ netfilter ରԠ net prio: 3.3 άϧʔϓؒͰͷωοτϫʔΫͷ༏ઌ౓ΛΠϯλʔϑΣʔεຖʹ ࢦఆ͢Δ Linux 3.3 ͷ৽ػೳ Network priority cgroup Linux 3.3 ͷ৽ػೳ Network priority cgroup (2) Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 36 / 54

Cgroupͷ࢖͍ํ Cgroup ͸ίϯςφͱؔ܎ͳ͘࢖༻Մೳ # mount -t tmpfs cgroup_root /sys/fs/cgroup # mkdir /sys/fs/cgroup/memory # mount -t cgroup -o memory cgroup /sys/fs/cgroup/memory (ϝϞϦαϒγεςϜͷ Ϛ΢ϯτ) # mkdir /sys/fs/cgroup/memory/test01 ("test01" ͱ͍͏άϧʔϓͷ࡞੒) # echo $$ > /sys/fs/cgroup/memory/test01/tasks (ϓϩηεΛάϧʔϓʹొ࿥) # cat /sys/fs/cgroup/memory/test01/tasks (άϧʔϓ಺ͷϓϩηεͷ֬ೝ) 2824 2837 # echo 30M > /sys/fs/cgroup/memory/test01/memory.limit_in_bytes (άϧʔϓʹରͯ͠ϝϞϦ্ݶ 30M ͱ͍͏੍ݶΛઃఆ) # cat /sys/fs/cgroup/memory/test01/memory.limit_in_bytes (੍ݶ஋ͷ֬ೝ) 31457280 # cat /sys/fs/cgroup/memory/test01/memory.usage_in_bytes (ݱࡏͷ࢖༻ྔͷ֬ೝ) 565248 Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 37 / 54

cgroupσϞ 1 CPU Λ 100%࢖͏ॲཧΛ;ͨͭىಈ 2 top ίϚϯυͰ྆ํͷॲཧ͕΄΅ 50%ͣͭ CPU Λ࢖͍ͬͯ Δ͜ͱΛ֬ೝ 3 CPU ͷ࢖༻࣌ؒΛ੍ݶ͢ΔͨΊͷ୯Ґ࣌ؒͷ֬ೝ 4 ยํͷϓϩηεʹ୯Ґ࣌ؒͷ 10%͚ͩ CPU ΛׂΓ౰ͯΔࢦ ఆΛߦ͏ 5 ยํͷϓϩηεʹ୯Ґ࣌ؒͷ 5%͚ͩ CPU ΛׂΓ౰ͯΔࢦఆ Λߦ͏ 6 ྆ํͷࢦఆ͕ cgroupfs ্ͷϑΝΠϧʹॻ͖ࠐ·Ε͍ͯΔͷΛ ֬ೝ 7 ͦΕͧΕͷϓϩηε͕ CPU Λ 10%ɺ5%࢖͍ͬͯΔ͜ͱΛ ֬ೝ https://asciinema.org/a/15287 Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 38 / 54

ࠓ೔ͷ಺༰ ίϯςφͷ֓ཁ Linux ʹ͓͚Δίϯςφͷ࢓૊Έ Namespace σϞ Cgroup σϞ ωοτϫʔΫؔ࿈ػೳ (͕࣌ؒ͋Ε͹) ίϯςφͰ࢖͑Δ໘ന͍ػೳ (͕࣌ؒ͋Ε͹) ·ͱΊ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 39 / 54

Linuxʹ͓͚Δίϯςφͷ࢓૊Έ ωοτϫʔΫػೳ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 40 / 54

ίϯςφͰ࢖͏ωοτϫʔΫػೳ ʙ veth OpenVZ/Virtuozzo ༝དྷͷػೳ ରͱͳΔΠϯλʔϑΣʔεΛੜ੒͠ɼΠϯλʔϑΣʔεؒͰ ௨৴Λߦ͏ (Layer2 ͷτϯωϧ) ରͷยํΛϗετଆͷϒϦοδʹɼยํΛίϯςφʹ઀ଓ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 41 / 54

ίϯςφͰ࢖͏ωοτϫʔΫػೳ ʙ macvlan ෺ཧΠϯλʔϑΣʔεʹผͷ MAC ΞυϨε͕෇͍ͨԾ૝త ͳ৽͍͠ΠϯλʔϑΣʔεΛ࡞੒ɽ͜ͷΠϯλʔϑΣʔεΛ ίϯςφʹׂ౰ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 42 / 54

ࠓ೔ͷ಺༰ ίϯςφͷ֓ཁ Linux ʹ͓͚Δίϯςφͷ࢓૊Έ Namespace σϞ Cgroup σϞ ωοτϫʔΫؔ࿈ػೳ (͕࣌ؒ͋Ε͹) ίϯςφͰ࢖͑Δ໘ന͍ػೳ (͕࣌ؒ͋Ε͹) ·ͱΊ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 43 / 54

LinuxΧʔωϧͷίϯςφͰ࢖͑Δ໘ന͍ ػೳ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 44 / 54

CRIU(1) http://criu.org/ OpenVZ ϓϩδΣΫτͷ Checkpoint/Restore ࣮૷ ΞϓϦέʔγϣϯͷ͋Δ࣌఺ͷঢ়ଶΛอଘ͠ɺ࠶։Ͱ͖Δ Χʔωϧ 3.11 Ҏ߱Ͱ࢖༻Մೳ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 45 / 54

CRIU(2) $ sudo criu dump -D checkpoint -t 1234 (PID:1234 ͷϓϩηεͷ৘ใΛμϯϓͯ͠ checkpoint σΟϨΫτϦʹอଘ) $ ls ./checkpoint cgroup.img fdinfo-17.img inventory.img pages-15.img core-170.img fdinfo-18.img ipcns-msg-9.img pages-16.img core-176.img fdinfo-2.img ipcns-sem-9.img pages-17.img core-1.img fdinfo-3.img ipcns-shm-9.img pages-1.img core-260.img fdinfo-4.img ipcns-var-9.img pages-2.img core-261.img fdinfo-5.img iptables-8.img pages-3.img : (snip) $ sudo criu restore -D checkpoint -d (checkpoint σΟϨΫτϦͷμϯϓΠϝʔδΛ ࢖ͬͯϦετΞ) (ॲཧͷུ֓Λॻ͍͚ͨͩͳͷͰ࣮ࡍ͸৭ʑΦϓγϣϯΛࢦఆͨ͠Γ͢Δඞཁ͕͋Γ·͢) Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 46 / 54

overlayfs Union Filesystem (aufs ͱҰॹ) ෳ਺ͷσΟϨΫτϦΛಁաతʹॏͶ͋ΘͤͯͻͱͭʹݟͤΒ ΕΔ ίϯςφͱ͸௚઀ؔ܎ͳ͍ 3.18 kernel ͰϚʔδ ίϯςφͷΫϩʔϯΛ࡞੒͢Δͱ͖ͷϑΝΠϧγεςϜͱ͠ ͯ LXC ͔Βར༻Ͱ͖Δ Ubuntu/Plamo ͩͱඇಛݖίϯςφͷΫϩʔϯʹ΋࢖͑Δ Docker Ͱ΋ར༻Ͱ͖Δ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 47 / 54

overlayfs࣮ߦྫ # mkdir lower upper overlay work # ls -F lower/ overlay/ upper/ work/ # touch lower/lower # touch upper/upper # mount -n -t overlay \ > -o lowerdir=lower,upperdir=upper,workdir=work \ > overlay overlay # ls overlay/ lower upper # touch overlay/test # ls overlay/ lower test upper # ls upper/ test upper Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 48 / 54

overlayfsσϞ 1 lower, upper, work, overlay σΟϨΫτϦ࡞੒ 2 lower, upper ʹϑΝΠϧ࡞੒ 3 overlayfs Ϛ΢ϯτ 4 Ϛ΢ϯτͨ͠σΟϨΫτϦ (overlay) ʹ lower, upper ʹଘࡏ ͢ΔϑΝΠϧ͕྆ํݟ͍͑ͯΔͷΛ֬ೝ 5 Ϛ΢ϯτͨ͠σΟϨΫτϦ (overlay) ͰϑΝΠϧ࡞੒ 6 ࡞੒ͨ͠ϑΝΠϧ͕ upper ʹͰ͖͍ͯΔ͜ͱΛ֬ೝ 7 ΞϯϚ΢ϯτͨ͋͠ͱͷ֤σΟϨΫτϦΛ֬ೝ https://asciinema.org/a/24151 Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 49 / 54

·ͱΊ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 50 / 54

·ͱΊ ίϯςφͷ֓ཁ Linux ʹ͓͚Δίϯςφͷ࢓૊Έ ίϯςφ͸Χʔωϧʹ࣮૷͞Ε͍ͯΔ৭ʑͳػೳͷ૊Έ߹Θͤ Ͱ࣮ݱ͞Ε͍ͯΔ Namespace OS Ϧιʔεͷִ཭ Cgroup ϗετͷ෺ཧϦιʔεͷ੍ݶ ωοτϫʔΫؔ࿈ػೳ veth macvlan ίϯςφͰ࢖͑Δ໘ന͍ػೳ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 51 / 54

lxc-jp LXC ʹݶΒͣίϯςφͷ࿩୊Λѻ͍ͬͯ·͢ɻ https://groups.google.com/d/forum/lxc-jp Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 52 / 54

ίϯςφܕԾ૝Խͷ৘ใަ׵ձ https://sites.google.com/site/containerstudy/ http://ct-study.connpass.com/ ίϯςφٕज़ʹؔ࿈͢Δ࿩୊Λѻ͏ ίϯςφʹؔ࿈͢ΔΧʔωϧͷ࣮૷ʹ͍ͭͯ ֤छπʔϧΩοτͷ঺հɼ࣮૷ʹ͍ͭͯ ίϯςφٕज़Λ࢖ͬͨπʔϧ΍ιϑτ΢ΣΞͷ঺հ΍࣮૷ʹͭ ͍ͯ ίϯςφٕज़ͷ׆༻ɾӡ༻ࣄྫ ͦͷଞʮίϯςφʯͱ͍͏Ωʔϫʔυ͕গ͠Ͱ΋ೖ͍ͬͯΔٕ ज़ʹ͍ͭͯ ͜Ε·Ͱେࡕͱ౦ژͰަޓʹ 7 ճ࣮ࢪɻୈ 8 ճ͸ 9 ݄ʹ౦ژ ͷ༧ఆ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 53 / 54

͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ Ճ౻ହจ (lxc-jp) OSC 2015 Kansai@Kyoto 2015-08-08 54 / 54