[JANOG35.5] WhiteBox SW検証 ~サーバサイド編~

ANOG35.5 WhiteBox SW検証 ~サーバサイド編~
グリー株式会社 中野和貴

※JANOG35.5での発表資料です
https://www.janog.gr.jp/meeting/janog35.5/program1

パネルセッション：検証してみて感じたホワイトボックススイッチの未来

登壇者
黒河内 倫 (グリー株式会社)
馬淵 俊弥 (ビッグローブ株式会社)
中野 和貴 (グリー株式会社)
石田 渉 (NTTソフトウェアイノベーションセンタ)

Transcript

1. Copyright © GREE, Inc. All Rights Reserved. JANOG35.5 WhiteBox SWݕূ ʙαʔόαΠυฤʙ άϦʔגࣜձࣾ த໺࿨و
2. Copyright © GREE, Inc. All Rights Reserved. • த໺ ࿨و (ͳ͔ͷ ͔͖ͣ) • FB, twitter: kakky0312 • ۀ຿಺༰ • Ϋϥ΢υ؀ڥ΁ͷαʔόҠߦαϙʔτ • ωοτϫʔΫۀ຿ͷ͓ख఻͍ • ӡ༻πʔϧͷ։ൃ • झຯ / ޷͖ͳ͜ͱ • ҿΈɺ͓ళ։୓ • ͍͍͓ళ͋ͬͨΒڭ͑ͯԼ͍͞(ɾ㱼ɾ) • υϥΠϒ • MTं͕ཉ͍͠ࠓ೔͜ͷࠒ • ԻָΛฉ͍ͨΓԋ૗ͨ͠Γ ͓·͑ͩΕΑ 2
3. Copyright © GREE, Inc. All Rights Reserved. • OS͕جຊతʹ௨ৗͷLinux͕ϕʔεͱͳ͍ͬͯΔ • ࣗ෼ͷ޷͖ͳύοέʔδ͕ೖΕΒΕΔ • ௨ৗͷLinuxαʔόͱಉ͡Α͏ʹѻ͏ࣄ͕Ͱ͖Δ • ᝑମͱOSͷ૊Έ߹ΘͤΛࣗ෼ͨͪͰબ୒Ͱ͖Δ • ONIE (Open Network Install Environment)ʹରԠ͍ͯ͠Δ
 ᝑମͱOSͰ͋Ε͹޷͖ͳ༷ʹ૊Έ߹Θͤͯ࢖͑Δ • ར༻OS͸ͦͷ··ͰΑΓ͍҆ᝑମΛબ΂Δඅ༻໘ͷϝϦοτ • ӡ༻φϨοδू໿ͷϝϦοτ վΊͯWhiteBox SWͷར఺ 3
4. Copyright © GREE, Inc. All Rights Reserved. • ONIEΛར༻ͨ͠OSΠϯετʔϧ • αʔόͷPXEϒʔτ૬౰ͳΠϯετʔϧ͕࣮ݱͰ͖Δ͔ • ࣮ݱͰ͖Ε͹ɺҰ౓ʹଟ͘ͷSW΁OSΠϯετʔϧΛָʹग़དྷΔ • ChefΛ࢖ͬͨઃఆมߋ • αʔόͱಉ༷ʹଟ͘ͷػث΁ಉ࣌ʹઃఆΛྲྀ͠ࠐΊΔԼ஍͕͋Δ͔ • ՄೳͰ͋Ε͹ɺαʔόӡ༻ͰͷφϨοδΛ͋Δఔ౓ద༻ग़དྷΔ • OpenStackίϯϙʔωϯτͷΠϯετʔϧ • εΠονࣗମʹωοτϫʔΫϊʔυΛಋೖ͢Δ͜ͱ͕ग़དྷΔ͔ • ՄೳͳΒ͹ɺύέοτંΓฦ͠ΛݮΒ͢ࣄ͕ՄೳͰ͸ͳ͍͔ ݕূͯ͠Έͨ͜ͱ 4
5. Copyright © GREE, Inc. All Rights Reserved. ONIEΛར༻ͨ͠OSΠϯετʔϧ
6. Copyright © GREE, Inc. All Rights Reserved. • OCP (Open Compute Project)͕ఏএ͢ΔOSΠϯετʔϧͷن֨ • HTTP΍TFTPɺUSBϝϞϦܦ༝ͰͷOSॳظಋೖͱΞοϓσʔτ͕Մೳ • BusyboxͰᝑମϒʔτɺ͔ͦ͜ΒOSΠϯετʔϧΛ࣮ࢪ͢Δ ONIEΛར༻ͨ͠OSΠϯετʔϧ 6 αϒλΠτϧςΩετ ONIEͱ͸ Busybox Get Image OS Install BusyboxͰىಈ͠ɺOSΠϯετʔϧ४උঢ়ଶʹ DHCPͰIPΛऔಘɺHTTP ͰOSΠϝʔδΛऔಘ औಘͨ͠OSΠϝʔδΛ࢖ͬͯɺOSΠϯετʔϧ
7. Copyright © GREE, Inc. All Rights Reserved. • Πϝʔδऔಘ༻αʔόΛ४උ • DHCPͱHTTP͕࢖͑ΔΑ͏ʹdhcpdͱnginxΛΠϯετʔϧ • DHCPʹ͸ҎԼͷઃఆΛೖΕΔ • IPΞυϨεͷऔಘઃఆ • next-hopઃఆʹHTTPܦ༝ͰOSΠϝʔδΛऔಘ͢ΔΑ͏ʹ • Ұ౓ɺᝑମ͔ΒOSΛ࡟আ͢Δ • ࠶౓ᝑମΛىಈͤ͞ɺOSΠϯετʔϧΛ࣮ࢪ • Πϯετʔϧޙʹਖ਼ৗʹىಈͰ͖Δ͔Λ֬ೝ ONIEΛར༻ͨ͠OSΠϯετʔϧ 7 αϒλΠτϧςΩετ ࠓճࢼͨ͜͠ͱ
8. Copyright © GREE, Inc. All Rights Reserved. • ਖ਼ৗʹDHCP͔ΒIPΛऔಘ͠ɺOSΠϯετʔϧ·Ͱग़དྷͨ • ن֨ʹԊͬͨܗͰ࣮૷͞Ε͍ͯΔͨΊɺOSϝʔΧʔͷυΩϡϝϯτ௨ Γʹઃఆ͢Δ͜ͱͰେ͖ͳ໰୊΋ͳ͘͢ΜͳΓೖͬͨ ONIEΛར༻ͨ͠OSΠϯετʔϧ 8 αϒλΠτϧςΩετ ݁Ռ Welcome to GRUB! ONIE: OS Install Mode ... GRUB loading. Version : 2014.08.0.0.3 Info: Mounting kernel filesystems... done. Info: Mounting LABEL=ONIE-BOOT on /mnt/onie-boot ... Running Celestica Redstone-XP platform init post_arch routines... Info: Using eth0 MAC address: xx:xx:xx:xx:xx:xx Info: eth0: Checking link... up. Info: Trying DHCPv4 on interface: eth0 ONIE: Using DHCPv4 addr: eth0: 90.90.90.25 / 255.255.255.0 (தུ) Info: Fetching http://90.90.90.5/CumulusLinux-2.2.2-amd64.bin ... ONIE: Executing installer: http://90.90.90.5/CumulusLinux-2.2.2-amd64.bin Verifying image checksum ... OK. Preparing image archive ... OK.
9. Copyright © GREE, Inc. All Rights Reserved. ChefΛ࢖ͬͨઃఆมߋ
10. Copyright © GREE, Inc. All Rights Reserved. • αʔόߏஙΛࣗಈԽ͢ΔϑϨʔϜϫʔΫ • ઃఆϑΝΠϧΛهड़ͦ͠ΕΛ࣮ߦ͢Δ͜ͱͰɺ͋Β͔͡Ίఆٛͨ͠ঢ় ଶ΁αʔόΛࣗಈతʹߏங͢Δ͜ͱ͕ग़དྷΔ • Ϣʔβ௥Ճ/࡟আ࡞ۀ • ύοέʔδΠϯετʔϧ࡞ۀ • ίϯϑΟάϑΝΠϧͷ഑ஔ/ฤू • etc… • ࠓճ͸αʔόΫϥΠΞϯτܕͷchef
 Ͱ͸ͳ͘ɺΫϥΠΞϯτ୯ମͰಈ͘
 chef-soloΛར༻ͨ͠ • Version 11.6.2 ChefΛ࢖ͬͨઃఆมߋ 10 αϒλΠτϧςΩετ Chefͱ͸
11. Copyright © GREE, Inc. All Rights Reserved. • ϩάΠϯϢʔβͷ௥Ճ/࡟আ • greeͱ͍͏ϩάΠϯϢʔβͷ௥Ճٴͼ࡟আ • NTPαʔόͷมߋ • /etc/ntp.confϑΝΠϧΛฤूͯ͠ར༻͢ΔNTPαʔόΛมߋ • ωοτϫʔΫΠϯλʔϑΣʔεૢ࡞ • ৽نʹVLANͱSVIΛ࡞੒ • SVI΁IP AddressΛઃఆ • ෺ཧϙʔτ΁VLANΛΞαΠϯ • ping͕௨ΔΑ͏ʹͳͬͨ͜ͱΛ֬ೝ ChefΛ࢖ͬͨઃఆมߋ 11 αϒλΠτϧςΩετ ࠓճࢼͨ͜͠ͱ
12. Copyright © GREE, Inc. All Rights Reserved. • ໰୊ͳ͘௥Ճͱ࡟আ͕Ͱ͖ɺϩάΠϯՄෆՄ͕มΘΔࣄΛ֬ೝͰ͖ͨ ChefΛ࢖ͬͨઃఆมߋ 12 αϒλΠτϧςΩετ ݁Ռ (Ϣʔβ௥Ճ/࡟আ) cumulus@cumulus-3$ sudo chef-solo -c solo.rb -o useradd Starting Chef Client, version 11.6.2 [2001-04-01T22:06:41+00:00] WARN: Run List override has been provided. [2001-04-01T22:06:41+00:00] WARN: Original Run List: [] [2001-04-01T22:06:41+00:00] WARN: Overridden Run List: [recipe[useradd]] Compiling Cookbooks... Converging 2 resources Recipe: useradd::default * user[gree] action create - alter user user[gree] * user[gree] action manage (up to date) * user[cumulus] action manage - manage user user[cumulus] Chef Client finished, 2 resources updated
13. Copyright © GREE, Inc. All Rights Reserved. • ntp.confΛࢦఆͷ΋ͷʹஔ͖׵͑ΒΕͨ͜ͱΛ֬ೝ ChefΛ࢖ͬͨઃఆมߋ 13 αϒλΠτϧςΩετ ݁Ռ (NTPαʔόͷมߋ) cumulus@cumulus-3$ sudo chef-solo -c solo.rb -o ntp (தུ) Recipe: ntp::default * service[ntp] action nothing (skipped due to action :nothing) * cookbook_file[/etc/ntp.conf] action create - update content in file /etc/ntp.conf from fa1a33 to 142b4d --- /etc/ntp.conf 2014-11-15 17:38:08.000000000 +0000 +++ /tmp/.ntp.conf20010401-13643-1ckz0em 2001-04-01 21:41:43.936993945 +0000 @@ -18,10 +18,9 @@ -server 0.cumulusnetworks.pool.ntp.org iburst -server 1.cumulusnetworks.pool.ntp.org iburst -server 2.cumulusnetworks.pool.ntp.org iburst -server 3.cumulusnetworks.pool.ntp.org iburst +server ntp1.jst.mfeed.ad.jp +server ntp2.jst.mfeed.ad.jp +server ntp3.jst.mfeed.ad.jp # Access control configuration; see /usr/share/doc/ntp-doc/html/accopt.html for * service[ntp] action restart - restart service service[ntp] Chef Client finished, 2 resources updated
14. Copyright © GREE, Inc. All Rights Reserved. • ࡞੒ͨ͠εΫϦϓτΛChefܦ༝Ͱ࣮ߦ͠VLAN௥Ճ/࡟আ͕ग़དྷͨ • εΫϦϓτ͸OSͷυΩϡϝϯτʹࢦఆ͞Εͨૢ࡞ίϚϯυͷཏྻ • ipίϚϯυ΍brctlίϚϯυ ChefΛ࢖ͬͨઃఆมߋ 14 αϒλΠτϧςΩετ ݁Ռ (ωοτϫʔΫΠϯλʔϑΣʔεૢ࡞) cumulus@cumulus-3$ sudo chef-solo -c solo.rb -o set-vlan Starting Chef Client, version 11.6.2 [2001-04-01T21:15:24+00:00] WARN: Run List override has been provided. [2001-04-01T21:15:24+00:00] WARN: Original Run List: [] [2001-04-01T21:15:24+00:00] WARN: Overridden Run List: [recipe[set-vlan]] Compiling Cookbooks... Converging 2 resources Recipe: set-vlan::default * cookbook_file[/home/cumulus/set-vlan-30.sh] action create (up to date) * bash[set-vlan-30.sh] action run - execute "bash" "/tmp/chef-script20010401-3735-b6lp9r" Chef Client finished, 1 resources updated
15. Copyright © GREE, Inc. All Rights Reserved. • Ұ࣌తͳૢ࡞ʹؔͯ͠͸εΫϦϓτΛྲྀ͠ࠐΉͰରԠͰ͖Δ • ͔͠͠ɺઃఆͷӬଓੑ͸୲อ͞Εͳ͍ • networkingσʔϞϯͷrestartͰઃఆ͕ফ͑ͯ͠·͏ • Ӭଓతͳઃఆ͸/etc/network/interfacesϑΝΠϧʹهࡌ͕ඞཁ • εΫϦϓτͰinterfacesϑΝΠϧΛฤू͢Δͷ͸ා͍ • sed౳Ͱͷஔ͖׵͑࣌ʹଞͷͱ͜Ζ͕มʹҾ͔͔ͬͬͯ͠·͏͔΋ • ୯७ʹͦ͏͍͏ฤूͷ࢓ํ͸ϓϩάϥϜͷ࣮૷͕໘౗͍͘͞ • ΠϯλʔϑΣʔε৘ใ͸DB౳Ͱ؅ཧͯ͠ɺຖճ͔ͦ͜ΒChefܦ༝Ͱಈ తੜ੒ͯ͠ϑΝΠϧΛஔ͖׵͑ͯ͋͛Δ΄͏͕ྑͦ͞͏ ChefΛ࢖ͬͨઃఆมߋ 15 αϒλΠτϧςΩετ ݁Ռ (ωοτϫʔΫΠϯλʔϑΣʔεૢ࡞)
16. Copyright © GREE, Inc. All Rights Reserved. OpenStackίϯϙʔωϯτͷΠϯετʔϧ
17. Copyright © GREE, Inc. All Rights Reserved. • OpenStackͷωοτϫʔΫϊʔυΛεΠον಺ʹΠϯετʔϧ • OpenStack಺ͷIPׂΓৼΓ΍௨৴ʹؔ͢Δϊʔυ • ϑΥϫʔσΟϯά΍NAT௨৴͕͜͜Λ௨ΔࣄͰ֎෦ωοτϫʔΫͱ ΞΫηεՄೳʹͳΔ • ௨ৗ͸αʔόʹߏங͢Δ΋ͷΛεΠον্ʹ௚઀ߏங͢Δ͜ͱͰύέο τͷંΓฦ͠ΛݮΒ͠ɺߴ଎Խ͕ਤΕΔͷ͔Λݕূ • ར༻ͨ͠όʔδϣϯ͸Icehouse OpenStackίϯϙʔωϯτͷΠϯετʔϧ 17 αϒλΠτϧςΩετ ࠓճࢼͨ͜͠ͱ
18. Copyright © GREE, Inc. All Rights Reserved. OpenStackίϯϙʔωϯτͷΠϯετʔϧ 18 αϒλΠτϧςΩετ ࠓճࢼͨ͜͠ͱ OpenStack Controller Compute Node Network Node Management NW Data NW External NW API NW nova-compute neutron-*-plugin-agent Horizon Keystone Neutron Glance nova-api nova-scheduler nova-conductor MySQL / RabbitMQ neutron-*-plugin-agent neutron-l3-agent neutron-dhcp-agent Internet ύέοτͷંΓฦ͠
19. Copyright © GREE, Inc. All Rights Reserved. • Πϯετʔϧࣗମ͕ग़དྷͳ͔ͬͨ • kernel͕ಠࣗϏϧυ͞Εͨ΋ͷͰ͋ΓɺґଘϥΠϒϥϦͷόʔ δϣϯ͕ϕʔεOSͰ͋ΔWheezyͱ݁ߏҧͬͨ • ͦͷͨΊඞཁͳϥΠϒϥϦΛWheezyͷϨϙδτϦΛ௥Ճͯ͠apt ͔ΒΠϯετʔϧ͠Α͏ͱ͕ͨ͠ग़དྷͳ͔ͬͨ • ιʔε͔ΒϏϧυ͢Ε͹Πϯετʔϧग़དྷͨՄೳੑ͸͋Δ • ࠓճݕূ͕࣌ؒͦΕ΄Ͳ௕͘ແ͔ͬͨͨΊͦ͜·Ͱ͸அ೦ • ࠓޙϕϯμʔ͔ΒOpenStackύοέʔδ͕ఏڙ͞Εͯ͘Ε͹
 ͦΕ͕Ұ൪޾ͤͰ࣮֬ OpenStackίϯϙʔωϯτͷΠϯετʔϧ 19 αϒλΠτϧςΩετ ݁Ռͱࢥ͏ॴ
20. Copyright © GREE, Inc. All Rights Reserved. ·ͱΊ
21. Copyright © GREE, Inc. All Rights Reserved. • OSΠϯετʔϧʹؔͯ͠ • ಛʹେ͖ͳ໰୊΋ແ͘ɺΠϯετʔϧͷ΍Γํ΋Կݸ͔͋ΔͷͰ
 ձࣾͷӡ༻্దͨ͠΋ͷΛબ΂͹໰୊ແ͍ • ChefΛ࢖ͬͨࣗಈߏங • جຊతʹͪΌΜͱಈ͕͘ɺChef୯ମͰશͯΛ΍Δͷ͸ਏͦ͏ • దٓDB౳ଞπʔϧͱ࿈ܞͤͯ͞࢖͏ͱΑΓḿΔͷͰ͸ͳ͍͔ • OpenStackίϯϙʔωϯτΠϯετʔϧ • ࣗྗͰೖΕΔͷ͸೉қ౓͕ߴ͍ • OSϕϯμʔ͔Βఏڙ͞ΕΔΑ͏ϦΫΤετ͢Δͷ͕٢ • શମΛ௨ͯ͠ • HW͕PowerPC൛͔ɺx86൛͔Ͱ΋໰୊ͷग़ํ͕ҧͬͨΓͦ͠͏ • ࠓճݕূͨ͠Α͏ͳ࢖͍ํΛ͢Δʹ͸ͦΕͳΓʹαʔόαΠυͷ
 φϨοδ͸ඞཁͰ͋Γಋೖίετ͸ͦΕͳΓʹߴ͍ ·ͱΊ 21