Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using LXC on Production

46839cf590a549efe13547c17a6b2fde?s=47 Isao Shimizu
September 06, 2014

Using LXC on Production

第4回 コンテナ型仮想化の情報交換会@東京

46839cf590a549efe13547c17a6b2fde?s=128

Isao Shimizu

September 06, 2014
Tweet

Transcript

  1. Using LXC on Production ୈ4ճ ίϯςφܕԾ૝Խͷ৘ใަ׵ձˏ౦ژ 2014.9.6 Isao SHIMIZU @isaoshimizu

  2. ࠓճͷݩωλ http://alpha.mixi.co.jp/entry/2014/12171/ ͜ͷΤϯτϦΛগ͠ΞϨϯδ͓ͯ͠࿩͠·͢ɻ 

  3. ࣗݾ঺հ ਗ਼ਫ ܄ ʢIsao SHIMIZUʣ ! גࣜձࣾϛΫγΟ ϞϯετελδΦॴଐ ! ݱࡏ4೥໨ʢ2011೥ೖࣾʣ

    mixiͷΠϯϑϥӡ༻ ϞϯελʔετϥΠΫͷΠϯϑϥɾαʔόӡ༻ʢݱࡏʣ ΤϯδχΞϒϩάࣥචʢFedoraɺOpenStackɺLXCͳͲʣ ! લ৬: 2003ʙ2011೥ SIerͰاըɺ։ൃɺΠϯϑϥӡ༻ ૊ΈࠐΈɺWebɺεϚϑΥΞϓϦɺಈը഑৴ͳͲ 
  4. ࠓ೔ͷൃදͷܦҢ 

  5. LXCಋೖʹࢸΔ·Ͱ 

  6. mixiʹ͓͚ΔԾ૝Խ؀ڥ ͍··Ͱ͸͜͏ͩͬͨ 

  7. KVM Kernel-based Virtual Machine 

  8. Ծ૝Խ؀ڥͰKVMΛଟ༻͍ͯͨ࣌͠୅ • Ծ૝Խ؀ڥ͸KVM͔͠࢖͍ͬͯͳ͔ͬͨ • ༻్͸։ൃ؀ڥɺεςʔδϯά؀ڥ͕΄ͱΜͲ • ߏங͸ࣗ࡞ͷγΣϧεΫϦϓτͰ • ϒϦοδΠϯλϑΣʔεͷ࡞੒ •

    Cobblerͱͷ࿈ܞʢϗετ໊ͷ࿈൪Խ΍IPͷॏෳ๷ࢭʣ • virt-install, Kickstart ! • جຊख࡞ۀͰ໘౗͍͘͞ 
  9. KVMͷ͍͍ͱ͜Ζ • ܰ౓ͷར༻Ͱ͸े෼ͳύϑΥʔϚϯε͕ग़Δ • ήετOSʹϚγϯͱಉ͡ѻ͍͕Ͱ͖Δ • ϊ΢ϋ΢ͨ͘͞Μ • Ϋϥ΢υܥͷπʔϧ͕ॆ࣮͍ͯ͠Δ 

  10. KVMͷͭΒ͍ͱ͜Ζ • Ծ૝ԽʹΑΔϘτϧωοΫ͕େ͖͍ʢͱ͘ʹσΟεΫIOʣ • σΟεΫ༰ྔΛଟ͘ফඅ͢ΔʢOS෼͕େ͖͍ʣ • BIOSઃఆͷґଘʢIntel VTͱ͔AMD-Vͱ͔ʣ 

  11. OpenStackͷಋೖ 

  12. OpenStack • 2013೥य़ࠒʹݕূ։࢝ • Version͸Grizzly 2013.01 • ಉ೥ͷՆࠒʹຊ൪ಋೖ • ༻్͸ࣾ಺ϓϩμΫτ޲͚ͷPaaSʢGizmoͱݺ͹ΕΔʣ

    • ΞϓϦαʔό͸ಠࣗͷσϓϩΠπʔϧΛ࢖ͬͯ • ϛυϧ΢ΣΞͷߏ੒͸ChefͰ • MySQLɺRedisɺJenkinsͳͲ • μογϡϘʔυʢHorizonʣศར • ͷͪʹ։ൃ؀ڥʹ΋ల։ 
  13. mixiʹ͓͚ΔԾ૝Խ؀ڥ ͍·ͷӡ༻ 

  14. LXC Linux Containers 

  15. LXCΛ࢖͏લͷҹ৅ • ͱʹ͔͍ܰ͘Β͍͠ʢͰ΋Α͘Θ͔ͬͯͳ͍ʣ • KVMͱ͔ͱԿ͕ҧ͏ͷ͔͍·͍ͪΘ͔ͬͯͳ͍ • LXCͷόʔδϣϯ͕͕͖͍͋ͬͯͯͯͦΖͦΖ͍͍ײ͔͡΋ʁ • Kernelগ্͛͠Ε͹࢖͑ͦ͏ •

    ৽͍ٕ͠ज़ؾʹͳΔɺ࢖ͬͯΈ͍ͨ • ຊ൪Ͱ࢖͍ͬͯΔ࿩͸΄ͱΜͲͳ͍ • ͪΐ͏ͲLinuxCon Japan 2013ͰLXCͷ࿩Λฉ͍ͨ • Ͱ΋ࣾ಺Ͱ͸୭΋৮ͬͯͳ͔ͬͨ 
  16. LXCʹ͍ͭͯ • KVMͷΑ͏ʹϋʔυ΢ΣΞͳͲͷΤϛϡϨʔγϣϯͷ্ʹԾ ૝ϚγϯΛಈ࡞ͤ͞ΔͷͰ͸ͳ͍ • ϓϩηε΍ωοτϫʔΫɺϢʔβʔۭؒͳͲΛ෼཭ͯ͠ɺԾ ૝తͳ؀ڥΛఏڙ • KernelͷػೳΛ࢖ͬͯ෼཭͞Εͨ؀ڥ •

    KVMͰى͖͍ͯͨΑ͏ͳɺCPU΍σΟεΫIOͳͲͷύϑΥʔ ϚϯεྼԽ͕جຊతʹൃੜ͠ͳ͍ • ىಈ͕଎͍ʢinitҎ߱ͷىಈ͚ͩʣ • ΋ͪΖΜΦʔϓϯιʔε 
  17.  LXCͷϨϙδτϦ https://github.com/lxc/lxc

  18.  LXCͷίϛοτਪҠ https://github.com/lxc/lxc

  19. KVMͱLXC 

  20. LXCͷݕূΛ࢝ΊΔ • όʔδϣϯ0.8.0ʢ2012.11.11ϦϦʔεʣ͔Β0.9.0
 ʢ2013.4.5ϦϦʔεʣ΁Ξοϓσʔτ͞Ε͍ͯͨ • 1.0.0͸2014೥2݄Λ༧ఆ͍ͯͨ͠ʢ଴ͯͳ͍ʣ • ·ͣ͸ɺ0.9.0Λݕূͯ͠Έ͍ͨ • ·ͣ΍ͬͨ͜ͱ

    • templatesʹ͋ΔFedoraͷγΣϧεΫϦϓτΛ࢖ͬͯ
 LXCͷΠϝʔδ࡞Γ • ͢ΜͳΓಈ͔ͳ͍ • Fedora޲͚ʹϝϯς͞Εͯͳ͍ͷ͔͍Ζ͍Ζमਖ਼ • ͱΓ͋͑ͣࢼߦࡨޡͯ͠ಈ͍ͨʢख࡞ۀίϚϯυϨϕϧʣ 
  21. ͦΜͳݕূΛ͍ͯ͠Δ͏ͪʹ ΞϨ͕࿩୊ʹ ! ౰࣌2013೥10݄ࠒ 

  22. 

  23. DockerͷτϨϯυ -9$ͷݕূ࢝Ίͨࠒ 

  24. ؾʹͳ͍ͬͯͨDockerͷଘࡏ • ౰࣌ͷόʔδϣϯ 0.6.xʢݱࡏ͸1.2.0ʣ • AUFSؾʹͳΔ • Docker Registryศརͦ͏ •

    GoͷϙʔλϏϦςΟ͢͹Β͍͠ ! • IPϚεΧϨʔυ͸ͪΐͬͱ໘౗͍͘͞ • ίϯςφʹIPΛݸผʹৼͬͯɺԾ૝ϚγϯͷΑ͏ʹѻ͍͍ͨ ʢmacvlan࢖͍͍ͨʣ • taggedVLANͷ؀ڥͰ΋໰୊ͳ͘࢖͍͍ͨ • όʔδϣϯΞοϓ͕ܹ͍͠ 
  25. ಠࣗπʔϧͷ։ൃ΁ 

  26. trailer ʢτϨΠϥʔʣ 

  27. trailerͱ͸ • Ruby੡ͷࣗࣾͰ։ൃͨ͠πʔϧ • LXCͷϥούʔ • ӡ༻ʹඞཁͳػೳͷΈΛ࣮૷ • IPɺMACΞυϨεͷ࠾൪ʢARMͱݺ͹ΕΔαʔόͱ࿈ܞʣ •

    ίϯςφΠϝʔδΛμ΢ϯϩʔυͯ͠ల։͢Δ • ىಈதͷίϯςφ͔ΒΠϝʔδΛ࡞Δ • Trailerfileͱݺ͹ΕΔίϯςφఆٛ 
  28. trailerͷߏ੒ 

  29. trailerΛ࢖ͬͨίϯςφىಈϑϩʔ ᶃ͋Β͔͡Ί࡞ΒΕͨΠϝʔδΛϨϙδτϦαʔό͔Βμ΢ϯϩʔυʢtrailer pullʣ ᶄΠϝʔδΛىಈʢtrailer startʣ (1)ϩʔΧϧʹμ΢ϯϩʔυ͞ΕͨΠϝʔδΛΠϯελϯε༻ͷσΟϨΫτϦʹల։ (2)ARMͱݺ͹ΕΔ಺੡ͷΞυϨε؅ཧπʔϧʹରͯ͠APIΞΫηε͠ɺIPΞυϨεͱ MACΞυϨε͕෷͍ग़͞ΕΔʢARM͸APIΞΫηεՄೳͳDHCPαʔόͷΑ͏ͳ΋ͷʣ (3)औಘͨ͠IPΞυϨεͱMACΛΠϯελϯεʹઃఆ
 ʢmacvlan

    bridgeϞʔυʣͯ͠ɺinitʢsystemdʣΛىಈ (4)trailer start࣮ߦ͔ΒsshͰ઀ଓՄೳʹͳΔ·Ͱʹ͔͔Δ࣌ؒ͸10ඵఔ౓
 ʢΠϝʔδαΠζʹΑͬͯଟগͷมಈ͋Γʣ 
  30. trailerΛ࢖ͬͨίϯςφఀࢭϑϩʔ ᶃఀࢭίϚϯυΛ࣮ߦʢtrailer stopʣ (1)LXCͷϓϩηεͷఀࢭɺσΟϨΫτϦͷ࡟আ 

  31. trailerΛ࢖ͬͨΠϝʔδͷ࡞੒ͱ ϨϙδτϦαʔό΁ͷΞοϓϩʔυͷϑϩʔ ᶃϕʔεͱͳΔΠϝʔδΛىಈʢtrailer startʣ ᶄΠϯελϯεʹରͯ͠ChefͰϨγϐΛద༻ʢknife-soloΛར༻ʣ ᶅϧʔτϑΝΠϧγεςϜʢσΟϨΫτϦπϦʔʣΛѹॖ͢Δʢtrailer snapshotʣ ᶆΠϝʔδ৘ใ͕ॻ͔ΕͨyamlϑΝΠϧͱrootfs.gzΛtarballʹ͢Δʢtrailer archiveʣ ᶇϨϙδτϦαʔό΁Ξοϓϩʔυʢtrailer

    pushʣ 
  32. LXC޲͚ʹ༻ҙͯ͋͠ΔΠϝʔδ •ϕʔεΠϝʔδ •Reverse Proxy (mod_proxy) •Varnish •Q4M (Job Queue) •Application

    Server (mod_perl) •Tokyo Tyrant •Memcached 
  33. LXCΛӡ༻͢Δ্ͰؾΛ͚ͭΔ͜ͱ 

  34. εϨου਺ɺPID਺্ݶ •kernel.threads-max •kernel.pid_max •vm.max_map_count •/etc/security/limits.d/90-nproc.conf Λunlimited ʹ •༻్ʹԠͯ͡File Descriptor਺΍ɺTCP/IPपΓͷKernelύϥϝʔλ ͷௐ੔͕ඞཁɻ

    •ΠϯελϯεଆͰ͸ઃఆͰ͖ͳ͍Kernelύϥϝʔλ͕͋ͬͨΓ͢ ΔͷͰɺsysctl΍echoͳͲͰઃఆ͢Δࡍʹཁ஫ҙɻ 
  35. ͦͷଞɺؾΛ͚ͭΔ͜ͱ wར༻Ϧιʔεͷ༧ଌɺݟੵ΋Γ wଞͷίϯςφʹѱӨڹΛٴ΅͞ͳ͍ઃܭ͕ඞཁ wσΟεΫ༰ྔ w༰ྔ੍ݶ͸Ͱ͖ͳ͍ wϞχλϦϯά wάϥϑେࣄ 

  36. trailerͷσϞ 

  37. ࢀߟࢿྉ 

  38. •OpenStackͱLXCΛಋೖͨ͠࿩ - mixi Engineers' Blog •http://alpha.mixi.co.jp/entry/2014/12171/ •LXCͰֶͿίϯςφೖ໳ ʵܰྔԾ૝Խ؀ڥΛ࣮ݱ͢Δٕज़ •http://gihyo.jp/admin/serial/01/linux_containers •Lxc

    Ͱ࢝ΊΔένένԾ૝Խੜ׆ʁʂ - SlideShare •http://www.slideshare.net/enakai/lxc-8300191 •LXC - Linux Containers •https://linuxcontainers.org/jp/ •LXC(Linux Container) •http://events.linuxfoundation.org/sites/events/files/cojp13_feng.pdf •DockerΛࢧ͑Δٕज़ •http://www.slideshare.net/enakai/docker-34668707 •GitHub - lxc/lxc •https://github.com/lxc/lxc