Upgrade to Pro — share decks privately, control downloads, hide ads and more …

今さら聞けない Linux コンテナの基礎 (2015-06-20)

今さら聞けない Linux コンテナの基礎 (2015-06-20)

第7回コンテナ型仮想化の情報交換会の発表資料です。
参考となる情報にはPDF中からリンクをしていますが、資料中のリンクは Speaker Deck 上ではクリックできないので PDF をダウンロードしてご覧ください。(2015-06-25スライド更新しています)

tenforward

June 20, 2015
Tweet

More Decks by tenforward

Other Decks in Technology

Transcript

  1. ͍·͞Βฉ͚ͳ͍Linuxίϯςφͷجૅ
    Ճ౻ହจ
    2015-06-20
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 1 / 108

    View full-size slide

  2. ࣗݾ঺հ
    Ճ౻ହจ
    http://www.ten-forward.ws/
    @ten forward
    http://gplus.to/tenforward
    https://github.com/tenforward
    http://d.hatena.ne.jp/defiant/ (ٕज़ϒϩά)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 2 / 108

    View full-size slide

  3. ࣗݾ঺հ
    ϑΝʔεταʔόɹج൫։ൃ෦ɹॴଐ
    ৽ϒϥϯυͷϗεςΟϯάαʔϏε͸͡Ί·ͨ͠
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 3 / 108

    View full-size slide

  4. ࣗݾ঺հ
    2010 ೥ࠒʹ cgroup ͷௐࠪΛ࢝Ίͨͷ͕͖͔͚ͬͰίϯςφ
    पลΛ৭ʑ͓͔͚ͬͯͨͷ͕ߴͯ͜͡ͷษڧձΛ΍ͬͯ·͢
    LXC ΁ͷίϛοτ
    ೔ຊޠ man pages / ࠷ۙ͸গ͠ίʔυ΋
    linuxcontainers.org ຋༁
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 4 / 108

    View full-size slide

  5. ࣗݾ঺հ
    Plamo Linux ϝϯςφ
    LXC ͰֶͿίϯςφೖ໳ɹʔܰྔԾ૝Խ؀ڥΛ࣮ݱ͢Δٕज़
    gihyo.jp Ͱ࿈ࡌ
    ʲվగ৽൛ʳLinux ΤϯδχΞཆ੒ಡຊ (ٕज़ධ࿦ࣾ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 5 / 108

    View full-size slide

  6. ࠓ೔ͷલ൒ͷ໨ඪ
    Linux Χʔωϧͷ࣋ͭίϯςφؔ࿈ػೳΛҰ௨Γ঺հ͢Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 6 / 108

    View full-size slide

  7. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 7 / 108

    View full-size slide

  8. ίϯςφ֓ཁ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 8 / 108

    View full-size slide

  9. ίϯςφͱ͸
    Χʔωϧ͔ΒݟΔͱී௨ʹϓϩηε͕ىಈ͢Δ͚ͩ
    ىಈ͢Δࡍʹִ཭Λࢦࣔ͢Δ
    ΧʔωϧͷػೳͰ (ෳ਺ͷ) ಠཱۭͨؒ͠Λ࡞Γग़͠ɼϦιʔ
    εΛ෼ׂɾ෼഑͢Δ
    ϓϩηεΛάϧʔϓԽͯ͠ଞͷάϧʔϓͱϦιʔεۭؒΛִ཭
    άϧʔϓԽͨ͠ϓϩηεʹର͢ΔϦιʔε੍ݶ
    OS ϨϕϧͷԾ૝Խ
    Ծ૝Խͱ͍͏ΑΓ͸ִ཭Խ
    Ծ૝తͳ OS ؀ڥΛఏڙ͢Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 9 / 108

    View full-size slide

  10. ίϯςφͷϝϦοτ
    ߴີ౓Խ͕Մೳ
    ىಈ͍ͯ͠Δ OS (Χʔωϧ) ͸Ұͭ
    Φʔόʔϔου͕খ͍͞
    ϋʔυ΢ΣΞͷԾ૝Խ͕ෆཁ
    ىಈ͕ૣ͍
    Ծ૝ϚγϯͷىಈͰ͸ͳ͘ɼϗετ OS ͔ΒݟͨΒ୯ʹϓϩ
    ηε͕ىಈ͍ͯ͠Δ͚ͩͳͷͰɼී௨ͷϓϩάϥϜ͕ىಈ͢Δ
    ͷͱ΄ͱΜͲมΘΒͳ͍
    ඞͣ͠΋γεςϜΛಈ͔͢ඞཁ͸ͳ͍ (ΞϓϦέʔγϣϯί
    ϯςφ)
    ྫ͑͹ίϯςφ಺Ͱ͸ httpd ͷΈ͕ಈ͍͍ͯΔ
    ίϯςφʹϝϞϦΛݻఆతʹׂΓ౰ͯΔඞཁ͕ͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 10 / 108

    View full-size slide

  11. ίϯςφͷσϝϦοτ
    ҟͳΔ OS ͷγεςϜ / ϓϩάϥϜ͸ಈ͔ͤͳ͍
    ୯ʹϗετ OS ্Ͱϓϩηε͕ىಈ͢Δ͚ͩͳͷͰ౰ͨΓલ
    ΧʔωϧʹؔΘΔૢ࡞͸Ͱ͖ͳ͍
    ىಈ͍ͯ͠ΔΧʔωϧ͸มΘΒͳ͍ͷͰ
    ίϯςφຖʹϩʔυ͢ΔϞδϡʔϧΛม͑ΔͳͲ
    Χʔωϧͷ࣮૷͸ෳࡶʹͳΔ
    શͯΧʔωϧͷػೳͱ࣮ͯ͠૷͞Ε͍ͯΔͷͰ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 11 / 108

    View full-size slide

  12. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 12 / 108

    View full-size slide

  13. Linuxʹ͓͚Δίϯςφͷ࢓
    ૊Έ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 13 / 108

    View full-size slide

  14. Linux ͰίϯςφΛ࣮ݱ͢ΔͨΊͷػೳ
    Linux Χʔωϧʹؚ·ΕΔ৭ʑͳػೳΛ૊Έ߹Θͤͯίϯςφ؀
    ڥΛ࡞੒͢ΔɻͦΕͧΕͷػೳ͸ίϯςφઐ༻ͷػೳͱ͍͏Θ͚
    Ͱ͸ͳ͍ɻ
    ϓϩηεΛάϧʔϓԽͯ͠ଞͷάϧʔϓͱִ཭
    OS Ϧιʔεͷִ཭
    ˠ Namespace (໊લۭؒ)
    άϧʔϓԽͨ͠ϓϩηεʹର͢ΔϦιʔε੍ݶ
    ϗετͷ෺ཧϦιʔεʹର͢Δ੍ݶ
    ˠ Cgroup (control group)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 14 / 108

    View full-size slide

  15. LinuxͰίϯςφΛ࣮ݱ͢ΔͨΊͷػೳ
    ͦͷଞ
    ωοτϫʔΫ (veth, macvlan ͳͲ)
    έʔύϏϦςΟ
    chroot (pivot root)
    bind mount
    Checkpoint/Restore (CRIU)
    ͳͲͳͲ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 15 / 108

    View full-size slide

  16. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 16 / 108

    View full-size slide

  17. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 17 / 108

    View full-size slide

  18. Namespace ͷछྨ (1)
    Mount Namespace: 2.4.19
    ϓϩηε͔Βݟ͍͑ͯΔϚ΢ϯτͷू߹ɼૢ࡞Λ෼཭͢Δɽ
    Namespace ಺ͷ mount, umount ͸ଞͷ Namespace ʹ͸Ө
    ڹ͠ͳ͍
    (ࢀߟ) Ϛ΢ϯτ໊લۭؒΛద༻͢Δ (IBM developerWorks)
    UTS Namespace: 2.6.19
    ϗετ໊ͳͲɼuname(2) ͕ฦ͢஋ͷू߹Λ෼཭ɽ
    setdomainname(2), sethostname(2) Ͱ Namespace ಺ͷ஋
    ͷΈมߋͰ͖Δ
    PID Namespace: 2.6.24
    PID ۭؒͷ෼཭ɽ৽͍͠ PID Namespace Ͱ͸ PID 1 ͔Β࢝
    ·Δ PID ׂ͕Γ౰ͯΒΕΔɽ਌͔Βࢠͷ PID Namespace ͸
    ݟ͑Δ (਌ͷۭؒͷ PID Λ࣋ͭ) ͕ɼࢠ͔Β਌͸ݟ͑ͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 18 / 108

    View full-size slide

  19. Namespace ͷछྨ (2)
    IPC Namespace: 2.6.19
    SysV IPC ΦϒδΣΫτɼPOSIX ϝοηʔδΩϡʔͷִ཭
    User Namespace: 2.6.23 ˜ 3.8
    ಠཱͨ͠ UID/GID ۭؒͱ֎෦ۭؒͷϚοϐϯά (ྫ͑͹ɼִ
    ཭ۭؒͰ͸ uid/gid 0/0ɼ֎෦Ͱ͸ 1000/1000 ͱ͔Մೳʹ
    ͳΔ)
    Network Namespace: 2.6.26
    ωοτϫʔΫϦιʔεͷִ཭ɽωοτϫʔΫσόΠεɼΞυϨ
    εɼϧʔςΟϯάςʔϒϧɼιέοτɼϑΟϧλϦϯά
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 19 / 108

    View full-size slide

  20. Namespace ͷૢ࡞ (γεςϜίʔϧ)
    clone(2) Ͱ৽͍͠ϓϩηε Λੜ੒
    unshare(2) Ͱ৽͍͠ϓϩηεΛੜ੒ͤͣʹ࣮ߦίϯςΩετ
    Λ੍ޚ͢Δ
    setns(2) ͰϓϩηεΛطଘ ͷ Namespace ʹؔ࿈෇͚Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 20 / 108

    View full-size slide

  21. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 21 / 108

    View full-size slide

  22. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    Cgroup
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 22 / 108

    View full-size slide

  23. Cgroupͱ͸
    ϓϩηεΛάϧʔϓԽ͠ɼάϧʔϓʹରͯ͠Ϧιʔε੍ݶΛߦ͏ɽ
    ผʹίϯςφઐ༻ͷ࢓૊ΈͰ͸ͳ͍ɽ
    Cgroup ͷಛ௃
    ػೳ͝ͱʹαϒγεςϜʹ෼͔ΕΔ
    cgroupfs ΛϚ΢ϯτͯ͠σΟϨΫτϦͰάϧʔϓΛද͢
    ϓϩηεΛάϧʔϓ಺ͷ tasks ϑΝΠϧʹ௥Ճ͢Δͱؔ࿈͢Δ
    λεΫ͕εϨου୯ҐͰάϧʔϓʹ௥Ճ͞ΕΔ
    ෳ਺֊૚ߏ଄ɻվ଄ߏ଄͝ͱʹҟͳΔπϦʔΛ࡞੒Ͱ͖Δɻͨ
    ͩ͠ɺҰͭͷαϒγεςϜ͕ॴଐͰ͖ΔπϦʔ͸Ұͭ
    πϦʔͷͲͷϨϕϧͷάϧʔϓʹ΋λεΫ͕ॴଐͰ͖Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 23 / 108

    View full-size slide

  24. CgroupͷαϒγεςϜ
    cpu: 2.6.24
    CFS(Completely Fair Scheduler) bandwidth controlɽ୯Ґ
    ࣌ؒ಺ͷάϧʔϓ಺ͷλεΫ͕࣮ߦͰ͖Δ߹ܭ࣌ؒΛ੍ݶ͢Δ
    (3.2 Ͱ࣮૷)
    ૬ର഑෼ɽάϧʔϓؒͷ CPU ࣌ؒͷׂ౰ͷׂ߹Λࢦఆ͢Δɽ
    ྫ͑͹ GroupA=100,GroupB=50 ͱ͢Δͱ A:B=2:1
    cpuacct: 2.6.24
    άϧʔϓ಺ͷ CPU ϦιʔεͷϨϙʔτ (CPU ࣌ؒ)
    cpuset: 2.6.24
    ׂΓ౰ͯΔ CPU, ϝϞϦϊʔυͷׂ౰
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 24 / 108

    View full-size slide

  25. CgroupͷαϒγεςϜ
    device: 2.6.26
    σόΠε΁ͷΞΫηεڐՄɼ੍ݶͷࢦఆ
    freezer: 2.6.28
    άϧʔϓ಺ͷϓϩηεΛશͯҰ࣌ఀࢭ͢Δ
    memory: 2.6.29
    ϝϞϦϦιʔεͷ੍ݶ (ϢʔβϝϞϦɼΧʔωϧϝϞϦ)
    blkio (Block IO):
    I/O weight controller(2.6.33 Ҏ߱) άϧʔϓͷ༏ઌ౓Λࢦఆ
    ͢Δ
    I/O throttling(2.6.37 Ҏ߱) άϧʔϓ಺ͷϓϩηεͷσόΠ
    εʹର͢Δૢ࡞਺ͷ߹ܭͷࢦఆ
    (ࢀߟ)Linux2.6.37 ͷ৽ػೳ “I/O throttling”
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 25 / 108

    View full-size slide

  26. CgroupͷαϒγεςϜ
    hugetlb: 3.6
    cgroup ͔Βͷ hugetlb ͷ࢖༻
    perf event: 2.6.39
    άϧʔϓ୯ҐͰ perf πʔϧͰϞχλϦϯά (ύϑΥʔϚϯε
    ղੳ)
    net cls: 2.6.29
    ύέοτʹࣝผࢠΛ͚ͭɼτϥϑΟοΫίϯτϩʔϧ (tc) ͱ
    netfilter(3.14 Ҏ߱) ͰίϯτϩʔϧՄೳʹ
    Linux 3.14 Ͱ net cls cgroup ʹ௥Ճ͞Εͨ netfilter ରԠ
    net prio: 3.3
    άϧʔϓؒͰͷωοτϫʔΫͷ༏ઌ౓ΛΠϯλʔϑΣʔεຖʹ
    ࢦఆ͢Δ
    Linux 3.3 ͷ৽ػೳ Network priority cgroup
    Linux 3.3 ͷ৽ػೳ Network priority cgroup (2)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 26 / 108

    View full-size slide

  27. Cgroupͷ࢖͍ํ
    Cgroup ͸ίϯςφͱؔ܎ͳ͘࢖༻Մೳ

    # mount -t tmpfs cgroup_root /sys/fs/cgroup
    # mkdir /sys/fs/cgroup/memory
    # mount -t cgroup -o memory cgroup /sys/fs/cgroup/memory (ϝϞϦαϒγεςϜͷ
    Ϛ΢ϯτ)
    # mkdir /sys/fs/cgroup/memory/test01 ("test01" ͱ͍͏άϧʔϓͷ࡞੒)
    # echo $$ > /sys/fs/cgroup/memory/test01/tasks (ϓϩηεΛάϧʔϓʹొ࿥)
    # cat /sys/fs/cgroup/memory/test01/tasks (άϧʔϓ಺ͷϓϩηεͷ֬ೝ)
    2824
    2837
    # echo 30M > /sys/fs/cgroup/memory/test01/memory.limit_in_bytes
    (άϧʔϓʹରͯ͠ϝϞϦ্ݶ 30M ͱ͍͏੍ݶΛઃఆ)
    # cat /sys/fs/cgroup/memory/test01/memory.limit_in_bytes (੍ݶ஋ͷ֬ೝ)
    31457280
    # cat /sys/fs/cgroup/memory/test01/memory.usage_in_bytes (ݱࡏͷ࢖༻ྔͷ֬ೝ)
    565248

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 27 / 108

    View full-size slide

  28. Cgroupͷ໰୊఺
    ΧΦε
    αϒγεςϜ͝ͱʹҧ͏πϦʔΛ࡞ΕΔ
    ਌ࢠؔ܎ͷάϧʔϓͷ਌ʹ΋ࢠʹ΋ϓϩηεɾεϨουΛ௥Ճ
    Ͱ͖Δ
    ਌ɺࢠͷͦΕͧΕͷࢠϓϩηε΍εϨουͷଘࡏ
    αϒγεςϜ͝ͱʹҧ͏ಈ͖
    ϝϞϦ͸ϓϩηε୯ҐͰ؅ཧɺCPU ͸εϨου୯Ґ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 28 / 108

    View full-size slide

  29. Cgroupͷ໰୊఺
    αϒγεςϜؒͷڠௐಈ࡞͕Ͱ͖ͳ͍
    ผʑͷπϦʔΛߏ੒Ͱ͖ΔͷͰڠௐΛલఏʹ࣮૷Ͱ͖ͳ͍
    ϑΝΠϧೖग़ྗ͸ blkio ͱ memory ʹؔ܎͢Δ͕ڠௐͰ͖
    ͳ͍
    ෳ਺ͷπϦʔ͸ߏ੒Ͱ͖Δ͕ɺͻͱͭͷπϦʔʹ͔͠ॴଐͰ͖
    ͳ͍ɻෳ਺ͷπϦʔΛߏ੒ͨ͠৔߹ɺଞͱڠௐಈ࡞ͨ͠ํ͕ྑ
    ͍αϒγεςϜͰ΋ͻͱͭͷπϦʔʹ͔͠ॴଐͰ͖ͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 29 / 108

    View full-size slide

  30. Cgroup࠶ઃܭ
    Cgroup ͷ࠶ઃܭɾ࠶࣮૷͕ਐߦதɻ3.16 ͔ΒࢼͤΔɻ
    ୯Ұ֊૚ߏ଄
    ϓϩηε୯ҐͰ؅ཧ
    ϓϩηε͕ॴଐ͠ͳ͍άϧʔϓ͚͕ͩࢠάϧʔϓΛ࣋ͯΔ
    (ϧʔτΛআ͘)
    ֊૚͝ͱʹ༗ޮʹͰ͖ΔαϒγεςϜΛࢦఆͰ͖Δɻͨͩ͠ɺ
    ਌άϧʔϓͰ༗ޮʹͳ͍ͬͯΔαϒγεςϜͷΈ࢖༻Մೳ
    άϧʔϓʹଐ͢Δϓϩηε͕ͳ͘ͳͬͨ௨஌Λ poll ͱ
    [id]notify Ͱड͚औΕΔ (release agent ഇࢭ)
    ৄ͘͠͸
    Linux 3.16 ͔ΒࢼͤΔ cgroup ͷ୯Ұ֊૚ߏ଄ (1)
    Linux 3.16 ͔ΒࢼͤΔ cgroup ͷ୯Ұ֊૚ߏ଄ (2)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 30 / 108

    View full-size slide

  31. Cgroup࠶ઃܭ
    sane behavior Φϓγϣϯ (·ͱ΋ͳৼΔ෣͍!!)
    3.16 ͔ΒࢼͤΔ (ͨͩ͋͘͠·Ͱ͓ࢼ͠)

    mount -t cgroup -o __DEVEL__sane_behavior \
    cgroup /path/to/cgroup

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 31 / 108

    View full-size slide

  32. σϞ ʙ Cgroup
    CPU Throttling
    CPU Share
    blkio
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 32 / 108

    View full-size slide

  33. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 33 / 108

    View full-size slide

  34. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    ωοτϫʔΫػೳ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 34 / 108

    View full-size slide

  35. ίϯςφͰ࢖͏ωοτϫʔΫػೳ ʙ veth
    OpenVZ/Virtuozzo ༝དྷͷػೳ
    ରͱͳΔΠϯλʔϑΣʔεΛੜ੒͠ɼΠϯλʔϑΣʔεؒͰ
    ௨৴Λߦ͏ (Layer2 ͷτϯωϧ)
    ରͷยํΛϗετଆͷϒϦοδʹɼยํΛίϯςφʹ઀ଓ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 35 / 108

    View full-size slide

  36. ίϯςφͰ࢖͏ωοτϫʔΫػೳ ʙ macvlan
    ෺ཧΠϯλʔϑΣʔεʹผͷ MAC ΞυϨε͕෇͍ͨԾ૝త
    ͳ৽͍͠ΠϯλʔϑΣʔεΛ࡞੒ɽ͜ͷΠϯλʔϑΣʔεΛ
    ίϯςφʹׂ౰
    Ϟʔυͷઃఆ͕ଘࡏ: private, vepa, bridge
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 36 / 108

    View full-size slide

  37. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 37 / 108

    View full-size slide

  38. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    ϧʔτσΟϨΫτϦͷมߋ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 38 / 108

    View full-size slide

  39. ϧʔτσΟϨΫτϦͷมߋ
    ϗετͱผͷϑΝΠϧγεςϜΛ࢖༻͢ΔͨΊʹϗετͷ / ͱผ
    ͷ৔ॴʹίϯςφͷ / ΛҠಈͤ͞Δඞཁ͕͋Δ
    chroot γεςϜίʔϧ
    1979 ೥ Version 7 Unix Ҏདྷ
    ؆қతͳִ཭؀ڥͱͯ͜͠ΕͰे෼ͳ͜ͱ΋
    pivot root γεςϜίʔϧ
    chroot ͸ൈ͚ग़ͤΔ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 39 / 108

    View full-size slide

  40. ݩ૆؆қίϯςφ؀ڥ ʙ chroot
    ϧʔτσΟϨΫτϦΛҠಈͤ͞Δ

    $ lsb_release -d
    Description: Ubuntu 14.04.2 LTS
    $ sudo debootstrap --variant=minbase --arch=amd64 vivid \
    > /root/vivid http://ftp.jaist.ac.jp/pub/Linux/ubuntu/
    I: Retrieving Release
    I: Retrieving Release.gpg
    : (ུ)
    $ sudo chroot /root/vivid
    # grep DESCRIPTION /etc/lsb-release
    DISTRIB_DESCRIPTION="Ubuntu 15.04"

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 40 / 108

    View full-size slide

  41. ݩ૆؆қίϯςφ؀ڥ ʙ chroot
    jailing - chroot jail Λߏஙɾӡ༻͢ΔͨΊͷεΫϦϓτΛॻ
    ͍ͨ (Kazuho’s Weblog)
    /usr/bin౳ɺ
    OS༝དྷͷσΟϨΫτϦΛ chroot؀ڥʹ read-
    only ͰΤΫεϙʔτͭͭ͠ɺࢦఆ͞ΕͨίϚϯυΛɺͦͷ
    chroot ؀ڥͰಈ͔͢εΫϦϓτ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 41 / 108

    View full-size slide

  42. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 42 / 108

    View full-size slide

  43. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    bind mount
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 43 / 108

    View full-size slide

  44. bind mount
    σΟϨΫτϦπϦʔͷҰ෦Λผͷ৔ॴʹϚ΢ϯτ͢Δ

    $ ls /etc/httpd/
    extra/ httpd.conf magic original/ php.ini.dist
    extra.dist/ httpd.conf.dist mime.types php.ini
    $ sudo mount --bind /etc/httpd /tmp/bind
    $ ls /tmp/bind
    extra/ httpd.conf magic original/ php.ini.dist
    extra.dist/ httpd.conf.dist mime.types php.ini

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 44 / 108

    View full-size slide

  45. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 45 / 108

    View full-size slide

  46. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    CRIU
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 46 / 108

    View full-size slide

  47. CRIU(1)
    http://criu.org/
    OpenVZ ϓϩδΣΫτͷ Checkpoint/Restore ࣮૷
    ΞϓϦέʔγϣϯͷ͋Δ࣌఺ͷঢ়ଶΛอଘ͠ɺ࠶։Ͱ͖Δ
    Χʔωϧ 3.11 Ҏ߱Ͱ࢖༻Մೳ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 47 / 108

    View full-size slide

  48. CRIU(2)

    ubuntu@criu1:~$ sudo lxc-start -n ct01
    ubuntu@criu1:~$ sudo lxc-checkpoint -v -n ct01 -s -D /tmp/checkpoint (νΣο
    ΫϙΠϯτॲཧޙʹίϯςφఀࢭ)
    ubuntu@criu1:~$ sudo lxc-ls --fancy
    NAME STATE IPV4 IPV6 GROUPS AUTOSTART
    --------------------------------------------
    ct01 STOPPED - - - NO
    ubuntu@criu1:~$ ls /tmp/criu/
    cgroup.img fdinfo-17.img inventory.img pages-15.img
    core-170.img fdinfo-18.img ipcns-msg-9.img pages-16.img
    core-176.img fdinfo-2.img ipcns-sem-9.img pages-17.img
    core-1.img fdinfo-3.img ipcns-shm-9.img pages-1.img
    core-260.img fdinfo-4.img ipcns-var-9.img pages-2.img
    core-261.img fdinfo-5.img iptables-8.img pages-3.img
    : (snip)
    ubuntu@criu1:~$ sudo rsync -avz --devices --rsync-path="sudo rsync" \
    /var/lib/lxc/ct01 [email protected]:/var/lib/lxc
    ubuntu@criu1:~$ sudo rsync -avz --rsync-path="sudo rsync" \
    /tmp/criu [email protected]:/tmp

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 48 / 108

    View full-size slide

  49. CRIU(3)

    ubuntu@criu2:~$ ls /tmp/criu/
    cgroup.img fdinfo-3.img ipcns-sem-9.img pages-3.img
    core-1.img fdinfo-4.img ipcns-shm-9.img pages-4.img
    core-255.img fdinfo-5.img ipcns-var-9.img pages-5.img
    core-260.img fdinfo-6.img iptables-8.img pages-6.img
    : (snip)
    ubuntu@criu2:~$ sudo lxc-checkpoint -n ct01 -r -D /tmp/checkpoint -v -d
    ubuntu@criu2:~$ sudo lxc-ls -f
    NAME STATE IPV4 IPV6 GROUPS AUTOSTART
    --------------------------------------------------
    ct01 RUNNING 10.0.3.200 - - NO

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 49 / 108

    View full-size slide

  50. ຊ೔ͷલ൒ͷ಺༰
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ
    ϧʔτσΟϨΫτϦͷมߋ
    bind mount
    CRIU
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 50 / 108

    View full-size slide

  51. Linuxʹ͓͚Δίϯςφͷ࢓૊Έ
    overlayfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 51 / 108

    View full-size slide

  52. overlayfs
    Union Filesystem (aufs ͱҰॹ)
    ίϯςφͱ͸௚઀ؔ܎ͳ͍
    3.18 kernel ͰϚʔδ
    ίϯςφͷΫϩʔϯΛ࡞੒͢Δͱ͖ͷϑΝΠϧγεςϜͱ͠
    ͯ LXC ͔Βར༻Ͱ͖Δ
    Ubuntu/Plamo ͩͱඇಛݖίϯςφͷΫϩʔϯʹ΋࢖͑Δ
    Docker Ͱ΋ར༻Ͱ͖Δ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 52 / 108

    View full-size slide

  53. overlayfs࣮ߦྫ

    # mkdir lower upper overlay work
    # ls -F
    lower/ overlay/ upper/ work/
    # touch lower/lower
    # touch upper/upper
    # mount -n -t overlay \
    > -o lowerdir=lower,upperdir=upper,workdir=work \
    > overlay overlay
    # ls overlay/
    lower upper
    # touch overlay/test
    # ls overlay/
    lower test upper
    # ls upper/
    test upper

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 53 / 108

    View full-size slide

  54. overlayfs஫ҙ
    ৭ʑͳόʔδϣϯͷ overlayfs ͕͋ΔͷͰ஫ҙ
    Ϟδϡʔϧ໊ workdir Φϓγϣϯ σΟετϦϏϡʔ
    γϣϯ
    kernel
    ΧʔωϧϚʔδલ
    (v21 Ҏલ)
    overlayfs ෆཁ Ubuntu 12.04
    LTS, 14.04 LTS,
    SuSE ํ໘
    ʙ3.14
    ΧʔωϧϚʔδલ
    (v22 Ҏ߱)
    overlayfs ඞཁ Plamo 3.15ʙ3.17
    ΧʔωϧϚʔδޙ overlay ඞཁ 3.18, 3.19
    ෳ਺ lowerdir ରԠ overlay ඞཁ 4.0ʙ
    ext4 ্Ͱ͔͠ಈ͔ͳ͍ (whiteout ػೳ)
    upperdir ͱ workdir ͕ ext4 ͷಉ͡ϑΝΠϧγεςϜ্ʹ͋Δ
    ඞཁ͕͋Δ
    lowerdir ͸ ext4 Ͱͳͯ͘΋ಈ͘ (͸ͣ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 54 / 108

    View full-size slide

  55. લ൒ͷ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 55 / 108

    View full-size slide

  56. લ൒ͷ·ͱΊ
    ίϯςφ֓ཁ
    Linux ʹ͓͚Δίϯςφͷ࢓૊Έ
    Namespace
    Cgroup
    ωοτϫʔΫػೳ veth, macvlan
    ϧʔτσΟϨΫτϦͷมߋ chroot, pivot root
    bind mount
    CRIU
    overlayfs
    ίϯςφ͸৭ʑͳΧʔωϧͷػೳΛ࢖ͬͯߏங͞ΕΔɻͦΕͧΕ
    ͷػೳ͸ίϯςφઐ༻ͱ͍͏Θ͚Ͱ͸ͳ͍ɻ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 56 / 108

    View full-size slide

  57. ٳܜ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 57 / 108

    View full-size slide

  58. ࠓ೔ͷޙ൒ͷ໨ඪ
    ৭ʑͳίϯςφ࣮૷Λܰ͘঺հ͢Δ
    Namespace ͷσϞ
    LXCɺLXD ͷ঺հͱσϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 58 / 108

    View full-size slide

  59. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 59 / 108

    View full-size slide

  60. Linuxʹ͓͚Δίϯςφ࣮૷
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 60 / 108

    View full-size slide

  61. Linuxʹ͓͚Δίϯςφ࣮૷
    ΧʔωϧʴΧʔωϧʹର͢Δύονʴૢ࡞ίϚϯυ
    OpenVZ / Parallels Virtuozzo Containers(঎༻) / libct
    Χʔωϧʴૢ࡞ίϚϯυ
    Docker / libcontainer
    LXC
    systemd
    rkt
    libvirt (lxc υϥΠό)
    vzctl for upstream kernel
    garden
    util-linux (unshare, nsenter, taskset, etc.), iproute2(netns)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 61 / 108

    View full-size slide

  62. util-linux
    ͝ଘ஌ɺLinux ͷ৭ʑͳϢʔςΟϦςΟϓϩάϥϜΛूΊͨ
    ΋ͷɻඞͣೖ͍ͬͯΔɻ
    Ұ൪͓खܰʹίϯςφͬΆ͍؀ڥΛ࡞ΕΔ
    ݫີʹ͸ Namespace Λ࡞ΕΔ
    Version 2.24 Ҏ্͕ྑ͍ (2.25 ͳΒ͞Βʹྑ͍)
    unshare ίϚϯυɾnsenter ίϚϯυ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 62 / 108

    View full-size slide

  63. util-linux
    util-linux ʹೖ͍ͬͯΔίϚϯυͰ؆୯ʹ Namespace ΛࢼͤΔɻ
    (util-linux 2.24 Ҏ߱ลΓ͕Φεεϝ)
    unshare
    ਌͔Βಠ໊ཱͨ͠લۭؒΛ࡞੒ͯ͠ίϚϯυΛ࣮ߦ
    nsenter
    طʹ࡞੒ࡁΈͷ໊લۭؒʹ઀ଓͯ͠ (໊લۭؒͷதʹೖͬͯ)
    ίϚϯυΛ࣮ߦ
    util-linux 2.23 Ҏ߱
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 63 / 108

    View full-size slide

  64. unshareίϚϯυ

    $ unshare -h
    ࢖͍ํ:
    unshare [options] [...]
    Run a program with some namespaces unshared from the parent.
    Φϓγϣϯ:
    -m, --mount Ϛ΢ϯτωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -u, --uts UTS ωʔϜεϖʔε (ϗετ໊ͳͲ) ͷڞ༗Λղআ͠·͢
    -i, --ipc System V IPC ωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -n, --net ωοτϫʔΫωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -p, --pid PID ωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -U, --user ϢʔβωʔϜεϖʔεͷڞ༗Λղআ͠·͢
    -f, --fork fork ͔ͯ͠Β <ϓϩάϥϜ> Λىಈ͠·͢
    --mount-proc[=<σΟϨΫτϦ>]
    proc ϑΝΠϧγεςϜΛ࠷ॳʹϚ΢ϯτ͠·͢
    (͜Εʹ͸ --mount ͷҙຯΛؚΈ·͢)
    -r, --map-root-user map current user to root (implies --user)
    -s, --setgroups allow|deny control the setgroups syscall in user namespaces
    :(ུ)

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 64 / 108

    View full-size slide

  65. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 65 / 108

    View full-size slide

  66. NamespaceͷσϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 66 / 108

    View full-size slide

  67. σϞ ʙ unshareίϚϯυͰUTS Namespace
    ମݧ
    1 ϗετ্Ͱ hostname ֬ೝ
    2 UTS Namespace ࡞੒ͯ͠ϗετ໊ม͑ͯϗετ໊֬ೝ
    3 ϗετ্Ͱ hostname มΘ͍ͬͯͳ͍͜ͱΛ֬ೝ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 67 / 108

    View full-size slide

  68. σϞ ʙ unshareίϚϯυͰMount/PID
    Namespaceମݧ
    1 Mount ͱ PID Namespace Λ࡞੒

    $ sudo unshare --mount --pid --mount-proc --fork

    طʹ࣮ߦதͷϓϩηεͷ PID ͸ม͑ΒΕͳ͍ͷͰ fork ͢Δඞ
    ཁ͋Γ
    Namespace ಺Ͱ/proc ΛϚ΢ϯτ
    2 Namespace ಺ͰϚ΢ϯτ (mount -o bind /usr /mnt) ͠ɺ
    Ϛ΢ϯτ͞Εͨ͜ͱΛ֬ೝ
    3 ϗετͰϚ΢ϯτ͞Εͯͳ͍͜ͱΛ֬ೝ
    4 /proc ҎԼΛ֬ೝ͠ɺݶΒΕͨ PID ͔͠ͳ͍͜ͱΛ֬ೝ (ϗ
    ετͱൺֱ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 68 / 108

    View full-size slide

  69. Mount namespace஫ҙ
    Ϛ΢ϯτͰ͸Ϛ΢ϯτϓϩύήʔγϣϯΛઃఆ͠ɺ
    Namespace ؒͰϚ΢ϯτ͕ͲͷΑ͏ʹѻΘΕΔ͔ΛઃఆͰ
    ͖Δɻ
    σϑΥϧτ͸ “private” Ͱɺ͋Δ Namespace ͰߦΘΕͨϚ΢
    ϯτ͸ผͷ໊લۭؒʹ͸఻ΘΒͳ͍
    ͋ΔϚ΢ϯτΛ “shared” ͰϚ΢ϯτ͢Δͱɺࢠʹ΋Ϛ΢ϯτ
    ͕఻ΘΔ
    systemd ͸’/’(ϧʔτ) Λ”shared” ͰϚ΢ϯτ͢Δ
    systemd ͕ init ͷ؀ڥͰ͸ɺmount --make-private /ͱ͠
    ͔ͯΒͰͳ͍ͱઌͷσϞ௨Γʹ͸ͳΒͳ͍
    Ϛ΢ϯτ໊લۭؒΛద༻͢Δ (IBM)
    mount-setup: change system mount propagation to
    shared by default
    Ϛ΢ϯτ໊લۭؒͰͪΐͬͱϋϚͬͨΊ΋ (П (ɾɾ*) ʎ ɹ
    Χʔωϧͱ͔࿔ͬͨΓͷϝϞ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 69 / 108

    View full-size slide

  70. σϞ ʙ ipίϚϯυͰNetwork Namespaceମ
    ݧ
    1 Network Namespace ࡞੒
    2 ϗετͱΠϯλʔϑΣʔεɺϧʔςΟϯάςʔϒϧɺϑΟϧ
    λϦϯά͕ҧ͏͜ͱΛ֬ೝ
    3 veth ϖΞΛ࡞੒
    4 ϖΞͷҰํΛ Namespace ʹॴଐͤ͞Δ
    5 ϖΞʹΞυϨεΛ෇༩͠ɺ૬ޓͰ ping ࣮ߦ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 70 / 108

    View full-size slide

  71. σϞ ʙ unshareίϚϯυͰUser Namespace
    ମݧ
    1 User + UTS Namespace ࡞੒͠ɺNamespace Ͱϗετ໊
    ͕ม͑ΒΕͳ͍ͷΛ֬ೝ
    2 uid/gid ͷ֬ೝͱ uid/gid ͷϚοϐϯάΛ֬ೝ
    3 User + UTS Namespace ࡞੒͠ɺϚοϐϯάΛߦ͍ɺ
    Namespace Ͱϗετ໊͕ม͑ΒΕΔͷΛ֬ೝ
    4 uid/gid ͷ֬ೝͱ uid/gid ͷϚοϐϯάΛ֬ೝ
    (஫ҙ) σϞͷΑ͏ʹ࣮ߦ͢Δʹ͸ util-linux 2.26 Ҏ͕߱ඞཁ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 71 / 108

    View full-size slide

  72. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 72 / 108

    View full-size slide

  73. mincs
    @mhiramat ͞ΜʹΑΔγΣϧεΫϦϓτͰ؆୯ʹίϯςφ
    Λ࡞ΔεΫϦϓτ
    unshare ΍ ip netns ΍ taskset ΍ bind mount ͳͲΛۦ࢖ͯ͠
    ͍Δ
    ίϯςφ؀ڥ͕ͲͷΑ͏ʹ࡞ΒΕΔ͔͕ྑ͘Θ͔Δ
    https://github.com/mhiramat/mincs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 73 / 108

    View full-size slide

  74. OpenVZ / Virtuozzo / libct
    Χʔωϧʹઐ༻ͷύονΛద༻ͨ͠Χʔωϧͱ֤छૢ࡞ίϚ
    ϯυ͔ΒͳΔ
    ঎༻൛ͷ Parallels Virtuozzo ContainersɺOSS ൛ͷ
    OpenVZ
    RHEL ͷ֤όʔδϣϯʹର͢Δύον/ઐ༻Χʔωϧͱͯ͠Ϧ
    Ϧʔε
    2001 ೥ (!) ʹ Virtuozzo ϦϦʔε (2005 ೥ʹ͸ Windows ൛
    ΋!!)
    ࢀߟࢿྉ
    OpenVZ - Linux Containersɿୈ 2 ճ ίϯςφܕԾ૝Խͷ৘
    ใަ׵ձˏ౦ژ by ւ࿝ᖒ͞Μ
    OpenVZ Update 2015/02/18 (೔ຊޠ) by ւ࿝ᖒ͞Μ (ୈ 6
    ճษڧձ)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 74 / 108

    View full-size slide

  75. Docker
    Έͳ͞Μ͝ଘ஌ :-)
    “Docker” ͸ʮίϯςφʯͰ͸͋Γ·ͤΜɻ
    Docker ಺ͷίϯςφ࣮૷͸ libcontainer
    ίϯςφٕज़Λ࢖ͬͯɺίϯςφͷಛ௕Λ׆͔͠ɺΞϓϦ
    έʔγϣϯΛσϓϩΠˍ࣮ߦ͢Δ
    ܰྔ
    ϙʔλϏϦςΟ
    Πϝʔδͷࠩ෼؅ཧ
    Πϝʔδͷ഑෍Πϯϑϥ (Docker Hub)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 75 / 108

    View full-size slide

  76. Docker
    ࢀߟࢿྉ
    Docker ΫΠοΫπΞʔ by தҪ͞Μ
    Vagrant ϢʔβͷͨΊͷ Docker ೖ໳ by shin1x1 ͞Μ (ୈ 3
    ճษڧձ)
    ͍·͞Βฉ͚ͳ͍ Docker - ୈ̑ճίϯςφܕԾ૝Խͷ৘ใަ
    ׵ձˏେࡕ by ాத๜༟͞Μ
    άάΕ͹ͨ͘͞Μ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 76 / 108

    View full-size slide

  77. CoreOS
    Linux Distribution
    Docker ίϯςφΛ࢖ͬͨΞϓϦέʔγϣϯͷىಈ
    ΫϥελϦϯά
    ෼ࢄγεςϜ
    ηΩϡϦςΟ
    ҆શͳ OS Ξοϓσʔτ
    ࠷খݶͷίΞ
    ࠷௿ݶͷػೳΛ࣋ͭΧʔωϧ
    ύοέʔδϚωʔδϟͳ͠
    Read only ͳ rootfs
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 77 / 108

    View full-size slide

  78. systemd
    ͝ଘ஌ɺ͜Ε͔Βͷ Linux ʹ͓͚Δ init(ʹཹ·Βͳ͍૖େͳ
    γεςϜ)
    systemd-nspawn ͱ͍͏ίϚϯυ͕͋ΓɺίϯςφΛ࡞੒Մೳ
    Unit ϑΝΠϧͳͲɺsystemd ͷػೳΛ࢖ͬͯίϯςφΛ؅ཧ
    Մೳ
    ີ઀ʹ݁ͼ͍֤ͭͨछػೳʹΑΔαϙʔτ
    ࢀߟࢿྉ
    systemd in Containers by Lennart Poettering
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 78 / 108

    View full-size slide

  79. rkt
    CoreOS ʹΑΔ৽͍͠ίϯςφ؀ڥ
    Docker ͷ࣋ͭ໰୊Λղܾ
    systemd Λ࢖༻
    ࢀߟจݙ
    Docker ͷॾ໰୊ͱ Rocket ొ৔ͷܦҢ (SOTA)
    Appc ͱ CoreOS/Rocket (SOTA)
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 79 / 108

    View full-size slide

  80. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 80 / 108

    View full-size slide

  81. LXC
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 81 / 108

    View full-size slide

  82. LXC
    Linux ΧʔωϧͷػೳΛ࢖ͬͯίϯςφΛ࣮ݱ͢Δπʔϧ܈
    ͱϥΠϒϥϦ
    ݩʑ͸Χʔωϧʹ࣮૷͞ΕͨػೳΛ։ൃऀ͕؆୯ʹࢼͤΔΑ
    ͏ʹͱ͍͏ͷΛߟ͑ͯ࡞ΒΕͯͨ໛༷
    Ubuntu ։ൃऀΛத৺ʹ։ൃ͞Ε͍ͯΔͷͰ Ubuntu ͷίϯ
    ςφπʔϧΩοτͷཁૉ͕ڧ͍
    ࢼ͢ͳΒ·ͣ͸ Ubuntu Ͱ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 82 / 108

    View full-size slide

  83. LXCͷಛ௃
    ๛෋ͳίϯςφςϯϓϨʔτ (֤छσΟετϦϏϡʔγϣ
    ϯ༻)
    (ίϯςφςϯϓϨʔτΛ࢖Θͣʹ) ϏϧυࡁΠϝʔδΛμ΢
    ϯϩʔυͯ͠࢖༻Մೳ
    API ͱ֤छݴޠόΠϯσΟϯάͷఏڙ
    Python(2,3)
    Lua
    Go
    Ruby
    Haskell
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 83 / 108

    View full-size slide

  84. LXCͷಛ௃
    ֤छετϨʔδόοΫΤϯυͷαϙʔτ
    σΟϨΫτϦ
    btrfs
    zfs
    lvm
    loop
    aufs
    overlayfs
    nbd(1.1 )
    Ϋϩʔϯͱεφοϓγϣοτ
    ηΩϡϦςΟ
    ҰൠϢʔβͰͷίϯςφىಈ
    ωετͨ͠ίϯςφ (ίϯςφ಺Ͱͷίϯςφͷىಈ)
    ೔ຊޠ man pages
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 84 / 108

    View full-size slide

  85. LXCͷόʔδϣϯ
    ݱ࣌఺ͷ stable ͸ 1.0ɻݱࡏ 1.0.7ɻUbuntu 14.04 LTS ʹ߹
    Θͤͯ 5 ೥ؒαϙʔτɻ
    ϓϩμΫγϣϯ؀ڥͰ࢖͏ͳΒͪ͜Βɻ
    ࠷৽͸ 1.1 ܥྻͰɺݱࡏ 1.1.2ɻ(2016 ೥ 1 ݄ or 1.2 ͕ϦϦʔ
    ε͞ΕΔ·Ͱαϙʔτ)
    ৽͍͠ػೳΛ࢖͏৔߹ɺsystemd ͱ࢖͏৔߹͸ͪ͜Βɻ
    ͱΓ͋͑ͣࢼ͢ͳΒ...
    Ubuntu 14.04 LTS / LXC 1.0.7
    Ubuntu 15.04 / LXC 1.1.2
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 85 / 108

    View full-size slide

  86. cgmanager
    Ubuntu ؀ڥͰ LXC ͱಉ࣌ʹΠϯετʔϧ͞ΕΔ cgroup ؅
    ཧσʔϞϯ
    systemd ͱͷڞଘ
    ඇಛݖίϯςφ͔Β cgroup Λૢ࡞Ͱ͖ΔΑ͏ʹ
    ωετͨ͠ίϯςφ͔Β cgroup Λૢ࡞Ͱ͖ΔΑ͏ʹ
    ϗετͱผͷ mount namespace Ͱ cgroupfs ΛϚ΢ϯτͯ͠
    ىಈ (ϗετ͔Βݟ͑ͳ͍)
    DBus ܦ༝ͰϦΫΤετΛૹड৴
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 86 / 108

    View full-size slide

  87. lxcfs
    ඇಛݖίϯςφ༻ʹ cgroupfs πϦʔΛఏڙ (/sys/fs/cgroup)
    cgroup ͰͷϦιʔε੍ݶʹԠͨ͡/proc ҎԼͷϦιʔεؔ࿈
    ஋ͷఏڙ
    cpuinfo
    meminfo
    stat
    uptime
    ...
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 87 / 108

    View full-size slide

  88. lxcfs
    ࢖͍ํ

    $ sudo lxcfs -s -f -o allow_other /var/lib/lxcfs

    lxcfs

    $ ls /var/lib/lxcfs/
    cgroup proc
    $ ls /var/lib/lxcfs/proc/
    cpuinfo meminfo stat uptime
    $ ls /var/lib/lxcfs/cgroup/
    blkio cpuacct devices hugetlb name=systemd
    cpu cpuset freezer memory perf_event

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 88 / 108

    View full-size slide

  89. lxcfs
    lxcfs ͷ proc ҎԼͷϑΝΠϧ͸ͦΕͧΕίϯςφ಺ͷ /proc
    ҎԼʹόΠϯυϚ΢ϯτ͞ΕΔ
    ϗετͷ cgroupfs ͷίϯςφʹରԠ͢Δ cgroup ͔Βಘͨ৘
    ใΛݩʹίϯςφ಺ͷ֤ϑΝΠϧΛੜ੒
    lxcfs ͷ cgroup ҎԼͷσΟϨΫτϦ͸ίϯςφ಺ͷ
    /sys/fs/cgroup ҎԼʹόΠϯυϚ΢ϯτ͞ΕΔ
    systemd ͕ίϯςφ಺Ͱಈ͘৔߹ʹඞཁ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 89 / 108

    View full-size slide

  90. lxcfs
    ͜Ε·Ͱ

    $ sudo grep cgroup /var/lib/lxc/ct01/config
    lxc.cgroup.memory.limit_in_bytes = 256M (ϝϞϦ্ݶ 256MB ʹઃఆ)
    $ sudo lxc-start -n ct01 -d (ίϯςφىಈ)
    $ grep MemTotal /proc/meminfo (ϗετͷϝϞϦྔ)
    MemTotal: 1017908 kB
    $ sudo lxc-attach -n ct01 -- grep MemTotal /proc/meminfo
    MemTotal: 1017908 kB (ϗετͷϝϞϦྔͦͷ··)

    lxcfs Λ࢖ͬͨ৔߹

    $ sudo grep cgroup /var/lib/lxc/ct01/config
    lxc.cgroup.memory.limit_in_bytes = 256M (ϝϞϦ্ݶ 256MB ʹઃఆ)
    $ sudo lxc-start -n ct01 (ίϯςφىಈ)
    $ grep MemTotal /proc/meminfo (ϗετͷϝϞϦྔ)
    MemTotal: 1017792 kB
    $ sudo lxc-attach -n ct01 -- grep MemTotal /proc/meminfo
    MemTotal: 262144 kB (cgroup Ͱઃఆ੍ͨ͠ݶͷ஋ʹͳ͍ͬͯΔ)

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 90 / 108

    View full-size slide

  91. UbuntuͰͷLXCͷΠϯετʔϧ
    lxc ύοέʔδΛΠϯετʔϧ͢Δ͚ͩ

    $ sudo apt-get install lxc

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 91 / 108

    View full-size slide

  92. UbuntuͰͷίϯςφ؀ڥ
    ίϯςφ༻ͷωοτϫʔΫ͕࡞੒͞ΕΔ (10.0.3.0/24)
    ϒϦοδ͕࡞੒͞Εɺ෺ཧΠϯλʔϑΣʔεͱͷؒͰ NAT ͕
    ઃఆ͞ΕΔ
    dnsmasq ͰίϯςφʹΞυϨεΛׂΓ౰ͯɺίϯςφ໊͕લ
    ղܾͰ͖ΔΑ͏ʹ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 92 / 108

    View full-size slide

  93. ίϯςφͷ࡞੒
    ςϯϓϨʔτΛࢦఆͯ͠ lxc-create ίϚϯυΛ࣮ߦ͢Δɻ

    $ ls /usr/share/lxc/templates/
    lxc-alpine* lxc-centos* lxc-gentoo* lxc-sshd*
    lxc-altlinux* lxc-cirros* lxc-openmandriva* lxc-ubuntu*
    lxc-archlinux* lxc-debian* lxc-opensuse* lxc-ubuntu-cloud*
    lxc-bind* lxc-download* lxc-oracle*
    lxc-busybox* lxc-fedora* lxc-plamo*

    σΟετϦϏϡʔγϣϯςϯϓϨʔτͷ࢖༻

    $ sudo lxc-create -n ct01 -t ubuntu

    μ΢ϯϩʔυςϯϓϨʔτͷ࢖༻

    $ sudo lxc-create -n ct01 -t download -- \
    > -d ubuntu -r trusty -a amd64

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 93 / 108

    View full-size slide

  94. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 94 / 108

    View full-size slide

  95. LXCσϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 95 / 108

    View full-size slide

  96. σϞ ʙ LXCίϯςφͷىಈ
    1 γεςϜίϯςφͷىಈ

    $ sudo lxc-start -n ct01 -d

    2 ΞϓϦέʔγϣϯίϯςφͷىಈ

    $ sudo lxc-start -d -n ct01 -- \
    > /usr/sbin/apache2ctl -D FOREGROUND

    3 ίϯςφՔಇͷ֬ೝ

    $ sudo lxc-ls --fancy

    4 Քಇதͷίϯςφ಺ʹೖΔ (ೖΓίϚϯυΛ࣮ߦ) (ssh ࢖Θͳ
    ͍!!)

    $ sudo lxc-attach -n ct01

    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 96 / 108

    View full-size slide

  97. σϞ ʙίϯςφͷΫϩʔϯ
    1 ී௨ʹΫϩʔϯ

    $ sudo lxc-clone -o ct01 -n ct02
    Created container ct02 as copy of ct01
    $ sudo lxc-ls -f
    NAME STATE IPV4 IPV6 GROUPS AUTOSTART
    --------------------------------------------
    ct01 STOPPED - - - NO
    ct02 STOPPED - - - NO

    2 overlayfs ͰΫϩʔϯ

    $ sudo lxc-clone -o ct01 -n ct02 -B overlayfs -s
    Created container ct02 as snapshot of ct01

    ίϯςφσΟϨΫτϦͷ֬ೝ
    Ϋϩʔϯݩ͸ফͤͳ͍
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 97 / 108

    View full-size slide

  98. σϞ ʙLXCͰඇಛݖίϯςφ
    1 ઃఆϑΝΠϧͷ֬ೝ
    /etc/subuid, /etc/subgid,
    $HOME/.config/lxc/default.conf,
    /etc/lxc/lxc-usernet
    2 ίϯςφ࡞੒ (download ςϯϓϨʔτΛ࢖༻͢Δඞཁ͕
    ͋Δ)

    $ lxc-create -t download -n ct01 -- -d ubuntu -r trusty -a amd64

    3 ίϯςφىಈ
    4 Ϛοϐϯάͷ֬ೝɺίϯςφ಺֎Ͱͷࠩ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 98 / 108

    View full-size slide

  99. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 99 / 108

    View full-size slide

  100. LXD
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 100 / 108

    View full-size slide

  101. LXD
    REST API Λఏڙ͢ΔσʔϞϯ (lxd ίϚϯυ)
    ίϚϯυϥΠϯΫϥΠΞϯτ (lxc ίϚϯυ)
    OpenStack Nova ϓϥάΠϯ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 101 / 108

    View full-size slide

  102. LXD
    APIɺίϚϯυϥΠϯπʔϧΛ࢖ͬͯωοτϫʔΫ্ͷίϯ
    ςφͷ؅ཧ͕Մೳ
    Πϝʔδϕʔε
    ϥΠϒϚΠάϨʔγϣϯ
    LXD ϗετΛ OpenStack ίϯϐϡʔτϊʔυʹ
    σϑΥϧτ͸ඇಛݖίϯςφ
    LXC ͷ Go όΠϯσΟϯάΛ࢖ͬͯ liblxc ܦ༝ͰίϯςφΛૢ࡞
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 102 / 108

    View full-size slide

  103. ຊ೔ͷޙ൒ͷ಺༰
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 103 / 108

    View full-size slide

  104. LXDσϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 104 / 108

    View full-size slide

  105. σϞ ʙLXD
    1 ίϯςφͷىಈ (ϩʔΧϧɺϦϞʔτ)
    2 ίϯςφ্ͷίϚϯυͷ࣮ߦ
    3 ϚΠάϨʔγϣϯ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 105 / 108

    View full-size slide

  106. ޙ൒ͷ·ͱΊ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 106 / 108

    View full-size slide

  107. ޙ൒ͷ·ͱΊ
    Linux ʹ͓͚Δίϯςφ࣮૷
    Namespace ͷσϞ
    LXC
    LXC σϞ
    LXD
    LXD σϞ
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 107 / 108

    View full-size slide

  108. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
    Ճ౻ହจ ୈ 7 ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ 2015-06-20 108 / 108

    View full-size slide