今さら聞けない Linux コンテナの基礎 (2015-06-20)

第7回コンテナ型仮想化の情報交換会の発表資料です。
参考となる情報にはPDF中からリンクをしていますが、資料中のリンクは Speaker Deck 上ではクリックできないので PDF をダウンロードしてご覧ください。(2015-06-25スライド更新しています)

Transcript

1. ͍·͞Βฉ͚ͳ͍Linuxίϯςφͷجૅ
   Ճ౻ହจ
   2015-06-20
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 1 / 108
   

   View Slide

2. ࣗݾ঺հ
   Ճ౻ହจ
   http://www.ten-forward.ws/
   @ten forward
   http://gplus.to/tenforward
   https://github.com/tenforward
   http://d.hatena.ne.jp/defiant/ (ٕज़ϒϩά)
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 2 / 108
   

   View Slide

3. ࣗݾ঺հ
   ϑΝʔεταʔόɹج൫։ൃ෦ɹॴଐ
   ৽ϒϥϯυͷϗεςΟϯάαʔϏε͸͡Ί·ͨ͠
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 3 / 108
   

   View Slide

4. ࣗݾ঺հ
   2010 ೥ࠒʹ cgroup ͷௐࠪΛ࢝Ίͨͷ͕͖͔͚ͬͰίϯςφ
   पลΛ৭ʑ͓͔͚ͬͯͨͷ͕ߴͯ͜͡ͷษڧձΛ΍ͬͯ·͢
   LXC ΁ͷίϛοτ
   ೔ຊޠ man pages / ࠷ۙ͸গ͠ίʔυ΋
   linuxcontainers.org ຋༁
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 4 / 108
   

   View Slide

5. ࣗݾ঺հ
   Plamo Linux ϝϯςφ
   LXC ͰֶͿίϯςφೖ໳ɹʔܰྔԾ૝Խ؀ڥΛ࣮ݱ͢Δٕज़
   gihyo.jp Ͱ࿈ࡌ
   ʲվగ৽൛ʳLinux ΤϯδχΞཆ੒ಡຊ (ٕज़ධ࿦ࣾ)
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 5 / 108
   

   View Slide

6. ࠓ೔ͷલ൒ͷ໨ඪ
   Linux Χʔωϧͷ࣋ͭίϯςφؔ࿈ػೳΛҰ௨Γ঺հ͢Δ
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 6 / 108
   

   View Slide

7. ຊ೔ͷલ൒ͷ಺༰
   ίϯςφ֓ཁ
   Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
   Namespace
   Cgroup
   ωοτϫʔΫػೳ
   ϧʔτσΟϨΫτϦͷมߋ
   bind mount
   CRIU
   overlayfs
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 7 / 108
   

   View Slide

8. ίϯςφ֓ཁ
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 8 / 108
   

   View Slide

9. ίϯςφͱ͸
   Χʔωϧ͔ΒݟΔͱී௨ʹϓϩηε͕ىಈ͢Δ͚ͩ
   ىಈ͢Δࡍʹִ཭Λࢦࣔ͢Δ
   ΧʔωϧͷػೳͰ (ෳ਺ͷ) ಠཱۭͨؒ͠Λ࡞Γग़͠ɼϦιʔ
   εΛ෼ׂɾ෼഑͢Δ
   ϓϩηεΛάϧʔϓԽͯ͠ଞͷάϧʔϓͱϦιʔεۭؒΛִ཭
   άϧʔϓԽͨ͠ϓϩηεʹର͢ΔϦιʔε੍ݶ
   OS ϨϕϧͷԾ૝Խ
   Ծ૝Խͱ͍͏ΑΓ͸ִ཭Խ
   Ծ૝తͳ OS ؀ڥΛఏڙ͢Δ
   Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 9 / 108
   

   View Slide

10. ίϯςφͷϝϦοτ
    ߴີ౓Խ͕Մೳ
    ىಈ͍ͯ͠Δ OS (Χʔωϧ) ͸Ұͭ
    Φʔόʔϔου͕খ͍͞
    ϋʔυ΢ΣΞͷԾ૝Խ͕ෆཁ
    ىಈ͕ૣ͍
    Ծ૝ϚγϯͷىಈͰ͸ͳ͘ɼϗετ OS ͔ΒݟͨΒ୯ʹϓϩ
    ηε͕ىಈ͍ͯ͠Δ͚ͩͳͷͰɼී௨ͷϓϩάϥϜ͕ىಈ͢Δ
    ͷͱ΄ͱΜͲมΘΒͳ͍
    ඞͣ͠΋γεςϜΛಈ͔͢ඞཁ͸ͳ͍ (ΞϓϦέʔγϣϯί
    ϯςφ)
    ྫ͑͹ίϯςφ಺Ͱ͸ httpd ͷΈ͕ಈ͍͍ͯΔ
    ίϯςφʹϝϞϦΛݻఆతʹׂΓ౰ͯΔඞཁ͕ͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 10 / 108
    

    View Slide

11. ίϯςφͷσϝϦοτ
    ҟͳΔ OS ͷγεςϜ / ϓϩάϥϜ͸ಈ͔ͤͳ͍
    ୯ʹϗετ OS ্Ͱϓϩηε͕ىಈ͢Δ͚ͩͳͷͰ౰ͨΓલ
    ΧʔωϧʹؔΘΔૢ࡞͸Ͱ͖ͳ͍
    ىಈ͍ͯ͠ΔΧʔωϧ͸มΘΒͳ͍ͷͰ
    ίϯςφຖʹϩʔυ͢ΔϞδϡʔϧΛม͑ΔͳͲ
    Χʔωϧͷ࣮૷͸ෳࡶʹͳΔ
    શͯΧʔωϧͷػೳͱ࣮ͯ͠૷͞Ε͍ͯΔͷͰ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 11 / 108
    

    View Slide

12. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 12 / 108
    

    View Slide

13. Linuxʹ͓͚Δίϯςφͷ࢓
    ૊Έ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 13 / 108
    

    View Slide

14. Linux ͰίϯςφΛ࣮ݱ͢ΔͨΊͷػೳ
    Linux Χʔωϧʹؚ·ΕΔ৭ʑͳػೳΛ૊Έ߹Θͤͯίϯςφ؀
    ڥΛ࡞੒͢ΔɻͦΕͧΕͷػೳ͸ίϯςφઐ༻ͷػೳͱ͍͏Θ͚
    Ͱ͸ͳ͍ɻ
    ϓϩηεΛάϧʔϓԽͯ͠ଞͷάϧʔϓͱִ཭
    OS Ϧιʔεͷִ཭
    ˠ Namespace (໊લۭؒ)
    άϧʔϓԽͨ͠ϓϩηεʹର͢ΔϦιʔε੍ݶ
    ϗετͷ෺ཧϦιʔεʹର͢Δ੍ݶ
    ˠ Cgroup (control group)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 14 / 108
    

    View Slide

15. LinuxͰίϯςφΛ࣮ݱ͢ΔͨΊͷػೳ
    ͦͷଞ
    ωοτϫʔΫ (veth, macvlan ͳͲ)
    έʔύϏϦςΟ
    chroot (pivot root)
    bind mount
    Checkpoint/Restore (CRIU)
    ͳͲͳͲ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 15 / 108
    

    View Slide

16. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 16 / 108
    

    View Slide

17. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 17 / 108
    

    View Slide

18. Namespace ͷछྨ (1)
    Mount Namespace: 2.4.19
    ϓϩηε͔Βݟ͍͑ͯΔϚ΢ϯτͷू߹ɼૢ࡞Λ෼཭͢Δɽ
    Namespace ಺ͷ mount, umount ͸ଞͷ Namespace ʹ͸Ө
    ڹ͠ͳ͍
    (ࢀߟ) Ϛ΢ϯτ໊લۭؒΛద༻͢Δ (IBM developerWorks)
    UTS Namespace: 2.6.19
    ϗετ໊ͳͲɼuname(2) ͕ฦ͢஋ͷू߹Λ෼཭ɽ
    setdomainname(2), sethostname(2) Ͱ Namespace ಺ͷ஋
    ͷΈมߋͰ͖Δ
    PID Namespace: 2.6.24
    PID ۭؒͷ෼཭ɽ৽͍͠ PID Namespace Ͱ͸ PID 1 ͔Β࢝
    ·Δ PID ׂ͕Γ౰ͯΒΕΔɽ਌͔Βࢠͷ PID Namespace ͸
    ݟ͑Δ (਌ͷۭؒͷ PID Λ࣋ͭ) ͕ɼࢠ͔Β਌͸ݟ͑ͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 18 / 108
    

    View Slide

19. Namespace ͷछྨ (2)
    IPC Namespace: 2.6.19
    SysV IPC ΦϒδΣΫτɼPOSIX ϝοηʔδΩϡʔͷִ཭
    User Namespace: 2.6.23 ˜ 3.8
    ಠཱͨ͠ UID/GID ۭؒͱ֎෦ۭؒͷϚοϐϯά (ྫ͑͹ɼִ
    ཭ۭؒͰ͸ uid/gid 0/0ɼ֎෦Ͱ͸ 1000/1000 ͱ͔Մೳʹ
    ͳΔ)
    Network Namespace: 2.6.26
    ωοτϫʔΫϦιʔεͷִ཭ɽωοτϫʔΫσόΠεɼΞυϨ
    εɼϧʔςΟϯάςʔϒϧɼιέοτɼϑΟϧλϦϯά
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 19 / 108
    

    View Slide

20. Namespace ͷૢ࡞ (γεςϜίʔϧ)
    clone(2) Ͱ৽͍͠ϓϩηε Λੜ੒
    unshare(2) Ͱ৽͍͠ϓϩηεΛੜ੒ͤͣʹ࣮ߦίϯςΩετ
    Λ੍ޚ͢Δ
    setns(2) ͰϓϩηεΛطଘ ͷ Namespace ʹؔ࿈෇͚Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 20 / 108
    

    View Slide

21. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 21 / 108
    

    View Slide

22. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    Cgroup
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 22 / 108
    

    View Slide

23. Cgroupͱ͸
    ϓϩηεΛάϧʔϓԽ͠ɼάϧʔϓʹରͯ͠Ϧιʔε੍ݶΛߦ͏ɽ
    ผʹίϯςφઐ༻ͷ࢓૊ΈͰ͸ͳ͍ɽ
    Cgroup ͷಛ௃
    ػೳ͝ͱʹαϒγεςϜʹ෼͔ΕΔ
    cgroupfs ΛϚ΢ϯτͯ͠σΟϨΫτϦͰάϧʔϓΛද͢
    ϓϩηεΛάϧʔϓ಺ͷ tasks ϑΝΠϧʹ௥Ճ͢Δͱؔ࿈͢Δ
    λεΫ͕εϨου୯ҐͰάϧʔϓʹ௥Ճ͞ΕΔ
    ෳ਺֊૚ߏ଄ɻվ଄ߏ଄͝ͱʹҟͳΔπϦʔΛ࡞੒Ͱ͖Δɻͨ
    ͩ͠ɺҰͭͷαϒγεςϜ͕ॴଐͰ͖ΔπϦʔ͸Ұͭ
    πϦʔͷͲͷϨϕϧͷάϧʔϓʹ΋λεΫ͕ॴଐͰ͖Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 23 / 108
    

    View Slide

24. CgroupͷαϒγεςϜ
    cpu: 2.6.24
    CFS(Completely Fair Scheduler) bandwidth controlɽ୯Ґ
    ࣌ؒ಺ͷάϧʔϓ಺ͷλεΫ͕࣮ߦͰ͖Δ߹ܭ࣌ؒΛ੍ݶ͢Δ
    (3.2 Ͱ࣮૷)
    ૬ର഑෼ɽάϧʔϓؒͷ CPU ࣌ؒͷׂ౰ͷׂ߹Λࢦఆ͢Δɽ
    ྫ͑͹ GroupA=100,GroupB=50 ͱ͢Δͱ A:B=2:1
    cpuacct: 2.6.24
    άϧʔϓ಺ͷ CPU ϦιʔεͷϨϙʔτ (CPU ࣌ؒ)
    cpuset: 2.6.24
    ׂΓ౰ͯΔ CPU, ϝϞϦϊʔυͷׂ౰
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 24 / 108
    

    View Slide

25. CgroupͷαϒγεςϜ
    device: 2.6.26
    σόΠε΁ͷΞΫηεڐՄɼ੍ݶͷࢦఆ
    freezer: 2.6.28
    άϧʔϓ಺ͷϓϩηεΛશͯҰ࣌ఀࢭ͢Δ
    memory: 2.6.29
    ϝϞϦϦιʔεͷ੍ݶ (ϢʔβϝϞϦɼΧʔωϧϝϞϦ)
    blkio (Block IO):
    I/O weight controller(2.6.33 Ҏ߱) άϧʔϓͷ༏ઌ౓Λࢦఆ
    ͢Δ
    I/O throttling(2.6.37 Ҏ߱) άϧʔϓ಺ͷϓϩηεͷσόΠ
    εʹର͢Δૢ࡞਺ͷ߹ܭͷࢦఆ
    (ࢀߟ)Linux2.6.37 ͷ৽ػೳ “I/O throttling”
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 25 / 108
    

    View Slide

26. CgroupͷαϒγεςϜ
    hugetlb: 3.6
    cgroup ͔Βͷ hugetlb ͷ࢖༻
    perf event: 2.6.39
    άϧʔϓ୯ҐͰ perf πʔϧͰϞχλϦϯά (ύϑΥʔϚϯε
    ղੳ)
    net cls: 2.6.29
    ύέοτʹࣝผࢠΛ͚ͭɼτϥϑΟοΫίϯτϩʔϧ (tc) ͱ
    netfilter(3.14 Ҏ߱) ͰίϯτϩʔϧՄೳʹ
    Linux 3.14 Ͱ net cls cgroup ʹ௥Ճ͞Εͨ netfilter ରԠ
    net prio: 3.3
    άϧʔϓؒͰͷωοτϫʔΫͷ༏ઌ౓ΛΠϯλʔϑΣʔεຖʹ
    ࢦఆ͢Δ
    Linux 3.3 ͷ৽ػೳ Network priority cgroup
    Linux 3.3 ͷ৽ػೳ Network priority cgroup (2)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 26 / 108
    

    View Slide

27. Cgroupͷ࢖͍ํ
    Cgroup ͸ίϯςφͱؔ܎ͳ͘࢖༻Մೳ
    
    # mount -t tmpfs cgroup_root /sys/fs/cgroup
    # mkdir /sys/fs/cgroup/memory
    # mount -t cgroup -o memory cgroup /sys/fs/cgroup/memory (ϝϞϦαϒγεςϜͷ
    Ϛ΢ϯτ)
    # mkdir /sys/fs/cgroup/memory/test01 ("test01" ͱ͍͏άϧʔϓͷ࡞੒)
    # echo $$ > /sys/fs/cgroup/memory/test01/tasks (ϓϩηεΛάϧʔϓʹొ࿥)
    # cat /sys/fs/cgroup/memory/test01/tasks (άϧʔϓ಺ͷϓϩηεͷ֬ೝ)
    2824
    2837
    # echo 30M > /sys/fs/cgroup/memory/test01/memory.limit_in_bytes
    (άϧʔϓʹରͯ͠ϝϞϦ্ݶ 30M ͱ͍͏੍ݶΛઃఆ)
    # cat /sys/fs/cgroup/memory/test01/memory.limit_in_bytes (੍ݶ஋ͷ֬ೝ)
    31457280
    # cat /sys/fs/cgroup/memory/test01/memory.usage_in_bytes (ݱࡏͷ࢖༻ྔͷ֬ೝ)
    565248
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 27 / 108
    

    View Slide

28. Cgroupͷ໰୊఺
    ΧΦε
    αϒγεςϜ͝ͱʹҧ͏πϦʔΛ࡞ΕΔ
    ਌ࢠؔ܎ͷάϧʔϓͷ਌ʹ΋ࢠʹ΋ϓϩηεɾεϨουΛ௥Ճ
    Ͱ͖Δ
    ਌ɺࢠͷͦΕͧΕͷࢠϓϩηε΍εϨουͷଘࡏ
    αϒγεςϜ͝ͱʹҧ͏ಈ͖
    ϝϞϦ͸ϓϩηε୯ҐͰ؅ཧɺCPU ͸εϨου୯Ґ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 28 / 108
    

    View Slide

29. Cgroupͷ໰୊఺
    αϒγεςϜؒͷڠௐಈ࡞͕Ͱ͖ͳ͍
    ผʑͷπϦʔΛߏ੒Ͱ͖ΔͷͰڠௐΛલఏʹ࣮૷Ͱ͖ͳ͍
    ϑΝΠϧೖग़ྗ͸ blkio ͱ memory ʹؔ܎͢Δ͕ڠௐͰ͖
    ͳ͍
    ෳ਺ͷπϦʔ͸ߏ੒Ͱ͖Δ͕ɺͻͱͭͷπϦʔʹ͔͠ॴଐͰ͖
    ͳ͍ɻෳ਺ͷπϦʔΛߏ੒ͨ͠৔߹ɺଞͱڠௐಈ࡞ͨ͠ํ͕ྑ
    ͍αϒγεςϜͰ΋ͻͱͭͷπϦʔʹ͔͠ॴଐͰ͖ͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 29 / 108
    

    View Slide

30. Cgroup࠶ઃܭ
    Cgroup ͷ࠶ઃܭɾ࠶࣮૷͕ਐߦதɻ3.16 ͔ΒࢼͤΔɻ
    ୯Ұ֊૚ߏ଄
    ϓϩηε୯ҐͰ؅ཧ
    ϓϩηε͕ॴଐ͠ͳ͍άϧʔϓ͚͕ͩࢠάϧʔϓΛ࣋ͯΔ
    (ϧʔτΛআ͘)
    ֊૚͝ͱʹ༗ޮʹͰ͖ΔαϒγεςϜΛࢦఆͰ͖Δɻͨͩ͠ɺ
    ਌άϧʔϓͰ༗ޮʹͳ͍ͬͯΔαϒγεςϜͷΈ࢖༻Մೳ
    άϧʔϓʹଐ͢Δϓϩηε͕ͳ͘ͳͬͨ௨஌Λ poll ͱ
    [id]notify Ͱड͚औΕΔ (release agent ഇࢭ)
    ৄ͘͠͸
    Linux 3.16 ͔ΒࢼͤΔ cgroup ͷ୯Ұ֊૚ߏ଄ (1)
    Linux 3.16 ͔ΒࢼͤΔ cgroup ͷ୯Ұ֊૚ߏ଄ (2)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 30 / 108
    

    View Slide

31. Cgroup࠶ઃܭ
    sane behavior Φϓγϣϯ (·ͱ΋ͳৼΔ෣͍!!)
    3.16 ͔ΒࢼͤΔ (ͨͩ͋͘͠·Ͱ͓ࢼ͠)
    
    mount -t cgroup -o __DEVEL__sane_behavior \
    cgroup /path/to/cgroup
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 31 / 108
    

    View Slide

32. σϞ ʙ Cgroup
    CPU Throttling
    CPU Share
    blkio
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 32 / 108
    

    View Slide

33. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 33 / 108
    

    View Slide

34. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    ωοτϫʔΫػೳ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 34 / 108
    

    View Slide

35. ίϯςφͰ࢖͏ωοτϫʔΫػೳ ʙ veth
    OpenVZ/Virtuozzo ༝དྷͷػೳ
    ରͱͳΔΠϯλʔϑΣʔεΛੜ੒͠ɼΠϯλʔϑΣʔεؒͰ
    ௨৴Λߦ͏ (Layer2 ͷτϯωϧ)
    ରͷยํΛϗετଆͷϒϦοδʹɼยํΛίϯςφʹ઀ଓ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 35 / 108
    

    View Slide

36. ίϯςφͰ࢖͏ωοτϫʔΫػೳ ʙ macvlan
    ෺ཧΠϯλʔϑΣʔεʹผͷ MAC ΞυϨε͕෇͍ͨԾ૝త
    ͳ৽͍͠ΠϯλʔϑΣʔεΛ࡞੒ɽ͜ͷΠϯλʔϑΣʔεΛ
    ίϯςφʹׂ౰
    Ϟʔυͷઃఆ͕ଘࡏ: private, vepa, bridge
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 36 / 108
    

    View Slide

37. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 37 / 108
    

    View Slide

38. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    ϧʔτσΟϨΫτϦͷมߋ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 38 / 108
    

    View Slide

39. ϧʔτσΟϨΫτϦͷมߋ
    ϗετͱผͷϑΝΠϧγεςϜΛ࢖༻͢ΔͨΊʹϗετͷ / ͱผ
    ͷ৔ॴʹίϯςφͷ / ΛҠಈͤ͞Δඞཁ͕͋Δ
    chroot γεςϜίʔϧ
    1979 ೥ Version 7 Unix Ҏདྷ
    ؆қతͳִ཭؀ڥͱͯ͜͠ΕͰे෼ͳ͜ͱ΋
    pivot root γεςϜίʔϧ
    chroot ͸ൈ͚ग़ͤΔ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 39 / 108
    

    View Slide

40. ݩ૆؆қίϯςφ؀ڥ ʙ chroot
    ϧʔτσΟϨΫτϦΛҠಈͤ͞Δ
    
    $ lsb_release -d
    Description: Ubuntu 14.04.2 LTS
    $ sudo debootstrap --variant=minbase --arch=amd64 vivid \
    > /root/vivid http://ftp.jaist.ac.jp/pub/Linux/ubuntu/
    I: Retrieving Release
    I: Retrieving Release.gpg
    : (ུ)
    $ sudo chroot /root/vivid
    # grep DESCRIPTION /etc/lsb-release
    DISTRIB_DESCRIPTION="Ubuntu 15.04"
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 40 / 108
    

    View Slide

41. ݩ૆؆қίϯςφ؀ڥ ʙ chroot
    jailing - chroot jail Λߏஙɾӡ༻͢ΔͨΊͷεΫϦϓτΛॻ
    ͍ͨ (Kazuho’s Weblog)
    /usr/bin౳ɺ
    OS༝དྷͷσΟϨΫτϦΛ chroot؀ڥʹ read-
    only ͰΤΫεϙʔτͭͭ͠ɺࢦఆ͞ΕͨίϚϯυΛɺͦͷ
    chroot ؀ڥͰಈ͔͢εΫϦϓτ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 41 / 108
    

    View Slide

42. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 42 / 108
    

    View Slide

43. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    bind mount
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 43 / 108
    

    View Slide

44. bind mount
    σΟϨΫτϦπϦʔͷҰ෦Λผͷ৔ॴʹϚ΢ϯτ͢Δ
    
    $ ls /etc/httpd/
    extra/ httpd.conf magic original/ php.ini.dist
    extra.dist/ httpd.conf.dist mime.types php.ini
    $ sudo mount --bind /etc/httpd /tmp/bind
    $ ls /tmp/bind
    extra/ httpd.conf magic original/ php.ini.dist
    extra.dist/ httpd.conf.dist mime.types php.ini
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 44 / 108
    

    View Slide

45. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 45 / 108
    

    View Slide

46. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    CRIU
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 46 / 108
    

    View Slide

47. CRIU(1)
    http://criu.org/
    OpenVZ ϓϩδΣΫτͷ Checkpoint/Restore ࣮૷
    ΞϓϦέʔγϣϯͷ͋Δ࣌఺ͷঢ়ଶΛอଘ͠ɺ࠶։Ͱ͖Δ
    Χʔωϧ 3.11 Ҏ߱Ͱ࢖༻Մೳ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 47 / 108
    

    View Slide

48. CRIU(2)
    
    ubuntu@criu1:~$ sudo lxc-start -n ct01
    ubuntu@criu1:~$ sudo lxc-checkpoint -v -n ct01 -s -D /tmp/checkpoint (νΣο
    ΫϙΠϯτॲཧޙʹίϯςφఀࢭ)
    ubuntu@criu1:~$ sudo lxc-ls --fancy
    NAME STATE IPV4 IPV6 GROUPS AUTOSTART
    --------------------------------------------
    ct01 STOPPED - - - NO
    ubuntu@criu1:~$ ls /tmp/criu/
    cgroup.img fdinfo-17.img inventory.img pages-15.img
    core-170.img fdinfo-18.img ipcns-msg-9.img pages-16.img
    core-176.img fdinfo-2.img ipcns-sem-9.img pages-17.img
    core-1.img fdinfo-3.img ipcns-shm-9.img pages-1.img
    core-260.img fdinfo-4.img ipcns-var-9.img pages-2.img
    core-261.img fdinfo-5.img iptables-8.img pages-3.img
    : (snip)
    ubuntu@criu1:~$ sudo rsync -avz --devices --rsync-path="sudo rsync" \
    /var/lib/lxc/ct01 ubuntu@192.168.122.26:/var/lib/lxc
    ubuntu@criu1:~$ sudo rsync -avz --rsync-path="sudo rsync" \
    /tmp/criu ubuntu@192.168.122.26:/tmp
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 48 / 108
    

    View Slide

49. CRIU(3)
    
    ubuntu@criu2:~$ ls /tmp/criu/
    cgroup.img fdinfo-3.img ipcns-sem-9.img pages-3.img
    core-1.img fdinfo-4.img ipcns-shm-9.img pages-4.img
    core-255.img fdinfo-5.img ipcns-var-9.img pages-5.img
    core-260.img fdinfo-6.img iptables-8.img pages-6.img
    : (snip)
    ubuntu@criu2:~$ sudo lxc-checkpoint -n ct01 -r -D /tmp/checkpoint -v -d
    ubuntu@criu2:~$ sudo lxc-ls -f
    NAME STATE IPV4 IPV6 GROUPS AUTOSTART
    --------------------------------------------------
    ct01 RUNNING 10.0.3.200 - - NO
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 49 / 108
    

    View Slide

50. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 50 / 108
    

    View Slide

51. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 51 / 108
    

    View Slide

52. overlayfs
    Union Filesystem (aufs ͱҰॹ)
    ίϯςφͱ͸௚઀ؔ܎ͳ͍
    3.18 kernel ͰϚʔδ
    ίϯςφͷΫϩʔϯΛ࡞੒͢Δͱ͖ͷϑΝΠϧγεςϜͱ͠
    ͯ LXC ͔Βར༻Ͱ͖Δ
    Ubuntu/Plamo ͩͱඇಛݖίϯςφͷΫϩʔϯʹ΋࢖͑Δ
    Docker Ͱ΋ར༻Ͱ͖Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 52 / 108
    

    View Slide

53. overlayfs࣮ߦྫ
    
    # mkdir lower upper overlay work
    # ls -F
    lower/ overlay/ upper/ work/
    # touch lower/lower
    # touch upper/upper
    # mount -n -t overlay \
    > -o lowerdir=lower,upperdir=upper,workdir=work \
    > overlay overlay
    # ls overlay/
    lower upper
    # touch overlay/test
    # ls overlay/
    lower test upper
    # ls upper/
    test upper
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 53 / 108
    

    View Slide

54. overlayfs஫ҙ
    ৭ʑͳόʔδϣϯͷ overlayfs ͕͋ΔͷͰ஫ҙ
    Ϟδϡʔϧ໊ workdir Φϓγϣϯ σΟετϦϏϡʔ
    γϣϯ
    kernel
    ΧʔωϧϚʔδલ
    (v21 Ҏલ)
    overlayfs ෆཁ Ubuntu 12.04
    LTS, 14.04 LTS,
    SuSE ํ໘
    ʙ3.14
    ΧʔωϧϚʔδલ
    (v22 Ҏ߱)
    overlayfs ඞཁ Plamo 3.15ʙ3.17
    ΧʔωϧϚʔδޙ overlay ඞཁ 3.18, 3.19
    ෳ਺ lowerdir ରԠ overlay ඞཁ 4.0ʙ
    ext4 ্Ͱ͔͠ಈ͔ͳ͍ (whiteout ػೳ)
    upperdir ͱ workdir ͕ ext4 ͷಉ͡ϑΝΠϧγεςϜ্ʹ͋Δ
    ඞཁ͕͋Δ
    lowerdir ͸ ext4 Ͱͳͯ͘΋ಈ͘ (͸ͣ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 54 / 108
    

    View Slide

55. લ൒ͷ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 55 / 108
    

    View Slide

56. લ൒ͷ·ͱΊ
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ veth, macvlan
    ϧʔτσΟϨΫτϦͷมߋ chroot, pivot root
    bind mount
    CRIU
    overlayfs
    ίϯςφ͸৭ʑͳΧʔωϧͷػೳΛ࢖ͬͯߏங͞ΕΔɻͦΕͧΕ
    ͷػೳ͸ίϯςφઐ༻ͱ͍͏Θ͚Ͱ͸ͳ͍ɻ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 56 / 108
    

    View Slide

57. ٳܜ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 57 / 108
    

    View Slide

58. ࠓ೔ͷޙ൒ͷ໨ඪ
    ৭ʑͳίϯςφ࣮૷Λܰ͘঺հ͢Δ
    Namespace ͷσϞ
    LXCɺLXD ͷ঺հͱσϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 58 / 108
    

    View Slide

59. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 59 / 108
    

    View Slide

60. Linuxʹ͓͚Δίϯςφ࣮૷
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 60 / 108
    

    View Slide

61. Linuxʹ͓͚Δίϯςφ࣮૷
    ΧʔωϧʴΧʔωϧʹର͢Δύονʴૢ࡞ίϚϯυ
    OpenVZ / Parallels Virtuozzo Containers(঎༻) / libct
    Χʔωϧʴૢ࡞ίϚϯυ
    Docker / libcontainer
    LXC
    systemd
    rkt
    libvirt (lxc υϥΠό)
    vzctl for upstream kernel
    garden
    util-linux (unshare, nsenter, taskset, etc.), iproute2(netns)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 61 / 108
    

    View Slide

62. util-linux
    ͝ଘ஌ɺLinux ͷ৭ʑͳϢʔςΟϦςΟϓϩάϥϜΛूΊͨ
    ΋ͷɻඞͣೖ͍ͬͯΔɻ
    Ұ൪͓खܰʹίϯςφͬΆ͍؀ڥΛ࡞ΕΔ
    ݫີʹ͸ Namespace Λ࡞ΕΔ
    Version 2.24 Ҏ্͕ྑ͍ (2.25 ͳΒ͞Βʹྑ͍)
    unshare ίϚϯυɾnsenter ίϚϯυ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 62 / 108
    

    View Slide

63. util-linux
    util-linux ʹೖ͍ͬͯΔίϚϯυͰ؆୯ʹ Namespace ΛࢼͤΔɻ
    (util-linux 2.24 Ҏ߱ลΓ͕Φεεϝ)
    unshare
    ਌͔Βಠ໊ཱͨ͠લۭؒΛ࡞੒ͯ͠ίϚϯυΛ࣮ߦ
    nsenter
    طʹ࡞੒ࡁΈͷ໊લۭؒʹ઀ଓͯ͠ (໊લۭؒͷதʹೖͬͯ)
    ίϚϯυΛ࣮ߦ
    util-linux 2.23 Ҏ߱
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 63 / 108
    

    View Slide

64. unshareίϚϯυ
    
    $ unshare -h
    ࢖͍ํ:
    unshare [options] [...]
    Run a program with some namespaces unshared from the parent.
    Φϓγϣϯ:
    -m, --mount Ϛ΢ϯτωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -u, --uts UTS ωʔϜεϖʔε (ϗετ໊ͳͲ) ͷڞ༗Λղআ͠·͢
    -i, --ipc System V IPC ωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -n, --net ωοτϫʔΫωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -p, --pid PID ωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -U, --user ϢʔβωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -f, --fork fork ͔ͯ͠Β <ϓϩάϥϜ> Λىಈ͠·͢
    --mount-proc[=<σΟϨΫτϦ>]
    proc ϑΝΠϧγεςϜΛ࠷ॳʹϚ΢ϯτ͠·͢
    (͜Εʹ͸ --mount ͷҙຯΛؚΈ·͢)
    -r, --map-root-user map current user to root (implies --user)
    -s, --setgroups allow|deny control the setgroups syscall in user namespaces
    :(ུ)
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 64 / 108
    

    View Slide

65. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 65 / 108
    

    View Slide

66. NamespaceͷσϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 66 / 108
    

    View Slide

67. σϞ ʙ unshareίϚϯυͰUTS Namespace
    ମݧ
    1 ϗετ্Ͱ hostname ֬ೝ
    2 UTS Namespace ࡞੒ͯ͠ϗετ໊ม͑ͯϗετ໊֬ೝ
    3 ϗετ্Ͱ hostname มΘ͍ͬͯͳ͍͜ͱΛ֬ೝ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 67 / 108
    

    View Slide

68. σϞ ʙ unshareίϚϯυͰMount/PID
    Namespaceମݧ
    1 Mount ͱ PID Namespace Λ࡞੒
    
    $ sudo unshare --mount --pid --mount-proc --fork
    
    طʹ࣮ߦதͷϓϩηεͷ PID ͸ม͑ΒΕͳ͍ͷͰ fork ͢Δඞ
    ཁ͋Γ
    Namespace ಺Ͱ/proc ΛϚ΢ϯτ
    2 Namespace ಺ͰϚ΢ϯτ (mount -o bind /usr /mnt) ͠ɺ
    Ϛ΢ϯτ͞Εͨ͜ͱΛ֬ೝ
    3 ϗετͰϚ΢ϯτ͞Εͯͳ͍͜ͱΛ֬ೝ
    4 /proc ҎԼΛ֬ೝ͠ɺݶΒΕͨ PID ͔͠ͳ͍͜ͱΛ֬ೝ (ϗ
    ετͱൺֱ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 68 / 108
    

    View Slide

69. Mount namespace஫ҙ
    Ϛ΢ϯτͰ͸Ϛ΢ϯτϓϩύήʔγϣϯΛઃఆ͠ɺ
    Namespace ؒͰϚ΢ϯτ͕ͲͷΑ͏ʹѻΘΕΔ͔ΛઃఆͰ
    ͖Δɻ
    σϑΥϧτ͸ “private” Ͱɺ͋Δ Namespace ͰߦΘΕͨϚ΢
    ϯτ͸ผͷ໊લۭؒʹ͸఻ΘΒͳ͍
    ͋ΔϚ΢ϯτΛ “shared” ͰϚ΢ϯτ͢Δͱɺࢠʹ΋Ϛ΢ϯτ
    ͕఻ΘΔ
    systemd ͸’/’(ϧʔτ) Λ”shared” ͰϚ΢ϯτ͢Δ
    systemd ͕ init ͷ؀ڥͰ͸ɺmount --make-private /ͱ͠
    ͔ͯΒͰͳ͍ͱઌͷσϞ௨Γʹ͸ͳΒͳ͍
    Ϛ΢ϯτ໊લۭؒΛద༻͢Δ (IBM)
    mount-setup: change system mount propagation to
    shared by default
    Ϛ΢ϯτ໊લۭؒͰͪΐͬͱϋϚͬͨΊ΋ (П (ɾɾ*) ʎ ɹ
    Χʔωϧͱ͔࿔ͬͨΓͷϝϞ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 69 / 108
    

    View Slide

70. σϞ ʙ ipίϚϯυͰNetwork Namespaceମ
    ݧ
    1 Network Namespace ࡞੒
    2 ϗετͱΠϯλʔϑΣʔεɺϧʔςΟϯάςʔϒϧɺϑΟϧ
    λϦϯά͕ҧ͏͜ͱΛ֬ೝ
    3 veth ϖΞΛ࡞੒
    4 ϖΞͷҰํΛ Namespace ʹॴଐͤ͞Δ
    5 ϖΞʹΞυϨεΛ෇༩͠ɺ૬ޓͰ ping ࣮ߦ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 70 / 108
    

    View Slide

71. σϞ ʙ unshareίϚϯυͰUser Namespace
    ମݧ
    1 User + UTS Namespace ࡞੒͠ɺNamespace Ͱϗετ໊
    ͕ม͑ΒΕͳ͍ͷΛ֬ೝ
    2 uid/gid ͷ֬ೝͱ uid/gid ͷϚοϐϯάΛ֬ೝ
    3 User + UTS Namespace ࡞੒͠ɺϚοϐϯάΛߦ͍ɺ
    Namespace Ͱϗετ໊͕ม͑ΒΕΔͷΛ֬ೝ
    4 uid/gid ͷ֬ೝͱ uid/gid ͷϚοϐϯάΛ֬ೝ
    (஫ҙ) σϞͷΑ͏ʹ࣮ߦ͢Δʹ͸ util-linux 2.26 Ҏ͕߱ඞཁ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 71 / 108
    

    View Slide

72. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 72 / 108
    

    View Slide

73. mincs
    @mhiramat ͞ΜʹΑΔγΣϧεΫϦϓτͰ؆୯ʹίϯςφ
    Λ࡞ΔεΫϦϓτ
    unshare ΍ ip netns ΍ taskset ΍ bind mount ͳͲΛۦ࢖ͯ͠
    ͍Δ
    ίϯςφ؀ڥ͕ͲͷΑ͏ʹ࡞ΒΕΔ͔͕ྑ͘Θ͔Δ
    https://github.com/mhiramat/mincs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 73 / 108
    

    View Slide

74. OpenVZ / Virtuozzo / libct
    Χʔωϧʹઐ༻ͷύονΛద༻ͨ͠Χʔωϧͱ֤छૢ࡞ίϚ
    ϯυ͔ΒͳΔ
    ঎༻൛ͷ Parallels Virtuozzo ContainersɺOSS ൛ͷ
    OpenVZ
    RHEL ͷ֤όʔδϣϯʹର͢Δύον/ઐ༻Χʔωϧͱͯ͠Ϧ
    Ϧʔε
    2001 ೥ (!) ʹ Virtuozzo ϦϦʔε (2005 ೥ʹ͸ Windows ൛
    ΋!!)
    ࢀߟࢿྉ
    OpenVZ - Linux Containersɿୈ 2 ճ ίϯςφܕԾ૝Խͷ৘
    ใަ׵ձˏ౦ژ by ւ࿝ᖒ͞Μ
    OpenVZ Update 2015/02/18 (೔ຊޠ) by ւ࿝ᖒ͞Μ (ୈ 6
    ճษڧձ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 74 / 108
    

    View Slide

75. Docker
    Έͳ͞Μ͝ଘ஌ :-)
    “Docker” ͸ʮίϯςφʯͰ͸͋Γ·ͤΜɻ
    Docker ಺ͷίϯςφ࣮૷͸ libcontainer
    ίϯςφٕज़Λ࢖ͬͯɺίϯςφͷಛ௕Λ׆͔͠ɺΞϓϦ
    έʔγϣϯΛσϓϩΠˍ࣮ߦ͢Δ
    ܰྔ
    ϙʔλϏϦςΟ
    Πϝʔδͷࠩ෼؅ཧ
    Πϝʔδͷ഑෍Πϯϑϥ (Docker Hub)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 75 / 108
    

    View Slide

76. Docker
    ࢀߟࢿྉ
    Docker ΫΠοΫπΞʔ by தҪ͞Μ
    Vagrant ϢʔβͷͨΊͷ Docker ೖ໳ by shin1x1 ͞Μ (ୈ 3
    ճษڧձ)
    ͍·͞Βฉ͚ͳ͍ Docker - ୈ̑ճίϯςφܕԾ૝Խͷ৘ใަ
    ׵ձˏେࡕ by ాத๜༟͞Μ
    άάΕ͹ͨ͘͞Μ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 76 / 108
    

    View Slide

77. CoreOS
    Linux Distribution
    Docker ίϯςφΛ࢖ͬͨΞϓϦέʔγϣϯͷىಈ
    ΫϥελϦϯά
    ෼ࢄγεςϜ
    ηΩϡϦςΟ
    ҆શͳ OS Ξοϓσʔτ
    ࠷খݶͷίΞ
    ࠷௿ݶͷػೳΛ࣋ͭΧʔωϧ
    ύοέʔδϚωʔδϟͳ͠
    Read only ͳ rootfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 77 / 108
    

    View Slide

78. systemd
    ͝ଘ஌ɺ͜Ε͔Βͷ Linux ʹ͓͚Δ init(ʹཹ·Βͳ͍૖େͳ
    γεςϜ)
    systemd-nspawn ͱ͍͏ίϚϯυ͕͋ΓɺίϯςφΛ࡞੒Մೳ
    Unit ϑΝΠϧͳͲɺsystemd ͷػೳΛ࢖ͬͯίϯςφΛ؅ཧ
    Մೳ
    ີ઀ʹ݁ͼ͍֤ͭͨछػೳʹΑΔαϙʔτ
    ࢀߟࢿྉ
    systemd in Containers by Lennart Poettering
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 78 / 108
    

    View Slide

79. rkt
    CoreOS ʹΑΔ৽͍͠ίϯςφ؀ڥ
    Docker ͷ࣋ͭ໰୊Λղܾ
    systemd Λ࢖༻
    ࢀߟจݙ
    Docker ͷॾ໰୊ͱ Rocket ొ৔ͷܦҢ (SOTA)
    Appc ͱ CoreOS/Rocket (SOTA)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 79 / 108
    

    View Slide

80. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 80 / 108
    

    View Slide

81. LXC
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 81 / 108
    

    View Slide

82. LXC
    Linux ΧʔωϧͷػೳΛ࢖ͬͯίϯςφΛ࣮ݱ͢Δπʔϧ܈
    ͱϥΠϒϥϦ
    ݩʑ͸Χʔωϧʹ࣮૷͞ΕͨػೳΛ։ൃऀ͕؆୯ʹࢼͤΔΑ
    ͏ʹͱ͍͏ͷΛߟ͑ͯ࡞ΒΕͯͨ໛༷
    Ubuntu ։ൃऀΛத৺ʹ։ൃ͞Ε͍ͯΔͷͰ Ubuntu ͷίϯ
    ςφπʔϧΩοτͷཁૉ͕ڧ͍
    ࢼ͢ͳΒ·ͣ͸ Ubuntu Ͱ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 82 / 108
    

    View Slide

83. LXCͷಛ௃
    ๛෋ͳίϯςφςϯϓϨʔτ (֤छσΟετϦϏϡʔγϣ
    ϯ༻)
    (ίϯςφςϯϓϨʔτΛ࢖Θͣʹ) ϏϧυࡁΠϝʔδΛμ΢
    ϯϩʔυͯ͠࢖༻Մೳ
    API ͱ֤छݴޠόΠϯσΟϯάͷఏڙ
    Python(2,3)
    Lua
    Go
    Ruby
    Haskell
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 83 / 108
    

    View Slide

84. LXCͷಛ௃
    ֤छετϨʔδόοΫΤϯυͷαϙʔτ
    σΟϨΫτϦ
    btrfs
    zfs
    lvm
    loop
    aufs
    overlayfs
    nbd(1.1 )
    Ϋϩʔϯͱεφοϓγϣοτ
    ηΩϡϦςΟ
    ҰൠϢʔβͰͷίϯςφىಈ
    ωετͨ͠ίϯςφ (ίϯςφ಺Ͱͷίϯςφͷىಈ)
    ೔ຊޠ man pages
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 84 / 108
    

    View Slide

85. LXCͷόʔδϣϯ
    ݱ࣌఺ͷ stable ͸ 1.0ɻݱࡏ 1.0.7ɻUbuntu 14.04 LTS ʹ߹
    Θͤͯ 5 ೥ؒαϙʔτɻ
    ϓϩμΫγϣϯ؀ڥͰ࢖͏ͳΒͪ͜Βɻ
    ࠷৽͸ 1.1 ܥྻͰɺݱࡏ 1.1.2ɻ(2016 ೥ 1 ݄ or 1.2 ͕ϦϦʔ
    ε͞ΕΔ·Ͱαϙʔτ)
    ৽͍͠ػೳΛ࢖͏৔߹ɺsystemd ͱ࢖͏৔߹͸ͪ͜Βɻ
    ͱΓ͋͑ͣࢼ͢ͳΒ...
    Ubuntu 14.04 LTS / LXC 1.0.7
    Ubuntu 15.04 / LXC 1.1.2
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 85 / 108
    

    View Slide

86. cgmanager
    Ubuntu ؀ڥͰ LXC ͱಉ࣌ʹΠϯετʔϧ͞ΕΔ cgroup ؅
    ཧσʔϞϯ
    systemd ͱͷڞଘ
    ඇಛݖίϯςφ͔Β cgroup Λૢ࡞Ͱ͖ΔΑ͏ʹ
    ωετͨ͠ίϯςφ͔Β cgroup Λૢ࡞Ͱ͖ΔΑ͏ʹ
    ϗετͱผͷ mount namespace Ͱ cgroupfs ΛϚ΢ϯτͯ͠
    ىಈ (ϗετ͔Βݟ͑ͳ͍)
    DBus ܦ༝ͰϦΫΤετΛૹड৴
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 86 / 108
    

    View Slide

87. lxcfs
    ඇಛݖίϯςφ༻ʹ cgroupfs πϦʔΛఏڙ (/sys/fs/cgroup)
    cgroup ͰͷϦιʔε੍ݶʹԠͨ͡/proc ҎԼͷϦιʔεؔ࿈
    ஋ͷఏڙ
    cpuinfo
    meminfo
    stat
    uptime
    ...
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 87 / 108
    

    View Slide

88. lxcfs
    ࢖͍ํ
    
    $ sudo lxcfs -s -f -o allow_other /var/lib/lxcfs
    
    lxcfs
    
    $ ls /var/lib/lxcfs/
    cgroup proc
    $ ls /var/lib/lxcfs/proc/
    cpuinfo meminfo stat uptime
    $ ls /var/lib/lxcfs/cgroup/
    blkio cpuacct devices hugetlb name=systemd
    cpu cpuset freezer memory perf_event
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 88 / 108
    

    View Slide

89. lxcfs
    lxcfs ͷ proc ҎԼͷϑΝΠϧ͸ͦΕͧΕίϯςφ಺ͷ /proc
    ҎԼʹόΠϯυϚ΢ϯτ͞ΕΔ
    ϗετͷ cgroupfs ͷίϯςφʹରԠ͢Δ cgroup ͔Βಘͨ৘
    ใΛݩʹίϯςφ಺ͷ֤ϑΝΠϧΛੜ੒
    lxcfs ͷ cgroup ҎԼͷσΟϨΫτϦ͸ίϯςφ಺ͷ
    /sys/fs/cgroup ҎԼʹόΠϯυϚ΢ϯτ͞ΕΔ
    systemd ͕ίϯςφ಺Ͱಈ͘৔߹ʹඞཁ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 89 / 108
    

    View Slide

90. lxcfs
    ͜Ε·Ͱ
    
    $ sudo grep cgroup /var/lib/lxc/ct01/config
    lxc.cgroup.memory.limit_in_bytes = 256M (ϝϞϦ্ݶ 256MB ʹઃఆ)
    $ sudo lxc-start -n ct01 -d (ίϯςφىಈ)
    $ grep MemTotal /proc/meminfo (ϗετͷϝϞϦྔ)
    MemTotal: 1017908 kB
    $ sudo lxc-attach -n ct01 -- grep MemTotal /proc/meminfo
    MemTotal: 1017908 kB (ϗετͷϝϞϦྔͦͷ··)
    
    lxcfs Λ࢖ͬͨ৔߹
    
    $ sudo grep cgroup /var/lib/lxc/ct01/config
    lxc.cgroup.memory.limit_in_bytes = 256M (ϝϞϦ্ݶ 256MB ʹઃఆ)
    $ sudo lxc-start -n ct01 (ίϯςφىಈ)
    $ grep MemTotal /proc/meminfo (ϗετͷϝϞϦྔ)
    MemTotal: 1017792 kB
    $ sudo lxc-attach -n ct01 -- grep MemTotal /proc/meminfo
    MemTotal: 262144 kB (cgroup Ͱઃఆ੍ͨ͠ݶͷ஋ʹͳ͍ͬͯΔ)
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 90 / 108
    

    View Slide

91. UbuntuͰͷLXCͷΠϯετʔϧ
    lxc ύοέʔδΛΠϯετʔϧ͢Δ͚ͩ
    
    $ sudo apt-get install lxc
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 91 / 108
    

    View Slide

92. UbuntuͰͷίϯςφ؀ڥ
    ίϯςφ༻ͷωοτϫʔΫ͕࡞੒͞ΕΔ (10.0.3.0/24)
    ϒϦοδ͕࡞੒͞Εɺ෺ཧΠϯλʔϑΣʔεͱͷؒͰ NAT ͕
    ઃఆ͞ΕΔ
    dnsmasq ͰίϯςφʹΞυϨεΛׂΓ౰ͯɺίϯςφ໊͕લ
    ղܾͰ͖ΔΑ͏ʹ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 92 / 108
    

    View Slide

93. ίϯςφͷ࡞੒
    ςϯϓϨʔτΛࢦఆͯ͠ lxc-create ίϚϯυΛ࣮ߦ͢Δɻ
    
    $ ls /usr/share/lxc/templates/
    lxc-alpine* lxc-centos* lxc-gentoo* lxc-sshd*
    lxc-altlinux* lxc-cirros* lxc-openmandriva* lxc-ubuntu*
    lxc-archlinux* lxc-debian* lxc-opensuse* lxc-ubuntu-cloud*
    lxc-bind* lxc-download* lxc-oracle*
    lxc-busybox* lxc-fedora* lxc-plamo*
    
    σΟετϦϏϡʔγϣϯςϯϓϨʔτͷ࢖༻
    
    $ sudo lxc-create -n ct01 -t ubuntu
    
    μ΢ϯϩʔυςϯϓϨʔτͷ࢖༻
    
    $ sudo lxc-create -n ct01 -t download -- \
    > -d ubuntu -r trusty -a amd64
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 93 / 108
    

    View Slide

94. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 94 / 108
    

    View Slide

95. LXCσϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 95 / 108
    

    View Slide

96. σϞ ʙ LXCίϯςφͷىಈ
    1 γεςϜίϯςφͷىಈ
    
    $ sudo lxc-start -n ct01 -d
    
    2 ΞϓϦέʔγϣϯίϯςφͷىಈ
    
    $ sudo lxc-start -d -n ct01 -- \
    > /usr/sbin/apache2ctl -D FOREGROUND
    
    3 ίϯςφՔಇͷ֬ೝ
    
    $ sudo lxc-ls --fancy
    
    4 Քಇதͷίϯςφ಺ʹೖΔ (ೖΓίϚϯυΛ࣮ߦ) (ssh ࢖Θͳ
    ͍!!)
    
    $ sudo lxc-attach -n ct01
    
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 96 / 108
    

    View Slide

97. σϞ ʙίϯςφͷΫϩʔϯ
    1 ී௨ʹΫϩʔϯ
    
    $ sudo lxc-clone -o ct01 -n ct02
    Created container ct02 as copy of ct01
    $ sudo lxc-ls -f
    NAME STATE IPV4 IPV6 GROUPS AUTOSTART
    --------------------------------------------
    ct01 STOPPED - - - NO
    ct02 STOPPED - - - NO
    
    2 overlayfs ͰΫϩʔϯ
    
    $ sudo lxc-clone -o ct01 -n ct02 -B overlayfs -s
    Created container ct02 as snapshot of ct01
    
    ίϯςφσΟϨΫτϦͷ֬ೝ
    Ϋϩʔϯݩ͸ফͤͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 97 / 108
    

    View Slide

98. σϞ ʙLXCͰඇಛݖίϯςφ
    1 ઃఆϑΝΠϧͷ֬ೝ
    /etc/subuid, /etc/subgid,
    $HOME/.config/lxc/default.conf,
    /etc/lxc/lxc-usernet
    2 ίϯςφ࡞੒ (download ςϯϓϨʔτΛ࢖༻͢Δඞཁ͕
    ͋Δ)
    
    $ lxc-create -t download -n ct01 -- -d ubuntu -r trusty -a amd64
    
    3 ίϯςφىಈ
    4 Ϛοϐϯάͷ֬ೝɺίϯςφ಺֎Ͱͷࠩ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 98 / 108
    

    View Slide

99. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 99 / 108
    

    View Slide

100. LXD
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 100 / 108
     

     View Slide

101. LXD
     REST API Λఏڙ͢ΔσʔϞϯ (lxd ίϚϯυ)
     ίϚϯυϥΠϯΫϥΠΞϯτ (lxc ίϚϯυ)
     OpenStack Nova ϓϥάΠϯ
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 101 / 108
     

     View Slide

102. LXD
     APIɺίϚϯυϥΠϯπʔϧΛ࢖ͬͯωοτϫʔΫ্ͷίϯ
     ςφͷ؅ཧ͕Մೳ
     Πϝʔδϕʔε
     ϥΠϒϚΠάϨʔγϣϯ
     LXD ϗετΛ OpenStack ίϯϐϡʔτϊʔυʹ
     σϑΥϧτ͸ඇಛݖίϯςφ
     LXC ͷ Go όΠϯσΟϯάΛ࢖ͬͯ liblxc ܦ༝ͰίϯςφΛૢ࡞
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 102 / 108
     

     View Slide

103. ຊ೔ͷޙ൒ͷ಺༰
     Linux ʹ͓͚Δίϯςφ࣮૷
     Namespace ͷσϞ
     LXC
     LXC σϞ
     LXD
     LXD σϞ
     ·ͱΊ
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 103 / 108
     

     View Slide

104. LXDσϞ
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 104 / 108
     

     View Slide

105. σϞ ʙLXD
     1 ίϯςφͷىಈ (ϩʔΧϧɺϦϞʔτ)
     2 ίϯςφ্ͷίϚϯυͷ࣮ߦ
     3 ϚΠάϨʔγϣϯ
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 105 / 108
     

     View Slide

106. ޙ൒ͷ·ͱΊ
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 106 / 108
     

     View Slide

107. ޙ൒ͷ·ͱΊ
     Linux ʹ͓͚Δίϯςφ࣮૷
     Namespace ͷσϞ
     LXC
     LXC σϞ
     LXD
     LXD σϞ
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 107 / 108
     

     View Slide

108. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
     Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 108 / 108
     

     View Slide