rcbops/chef-cookbooks

第17回 OpenStack 勉強会資料 by 平井伴和

Transcript

1. rcbops / chef-cookbooks Tomokazu HIRAI ( @jedipunkz )
2. ࣗݾ঺հ ໊લ : ฏҪ൐࿨(ͻΒ͍ͱ΋͔ͣ) ID ໊ : jedipunkz ϒϩά : http://jedipunkz.github.io ॴଐ : ೔ຊ OpenStack Ϣʔβձ ࢓ࣄ : R&D, ΠϯϑϥΠϯδχΞ ৬৔ : KDDI ΢Σϒίϛϡχέʔγϣϯζ FF14: ‘jedi master’ Ωϟϥ on Alexander ḉ
3. ࠓ೔࿩͢͜ͱ • osops-utils ͷ֓ཁ ( Chef ͸ෳ਺୆ߏ੒Λѻ͑Δ) • rcbops-cookbooks Ͱෳ਺୆ Havana Λߏ੒ͯ͠ΈΔ • rcbops-cookbooks Ͱෳ਺୆ Swift Λߏ੒ͯ͠ΈΔ
4. લఏ * Chef ͷ஌ࣝ͸લఏͱ͍͖ͤͯͨͩ͞·͢ɻ * Chef αʔόɾϫʔΫεςʔγϣϯͷߏஙํ๏ ͸ׂѪ͍͖ͤͯͨͩ͞·͢ɻ
5. ͱ͸ݴͬͯ΋ Chef ͱ͸ʁ Chef-Server WorkStation Node ʢ̍ʣ ʢ̎ʣ ʢ̏ʣ (1) : cookbooks, roles, data_bag, environments ౳ΛΞοϓϩʔυ (2) : node ʹରͯ͠ ‘bootstrap’ ߦ͍σϓϩΠ։࢝ (3) : cookbooks, roles ౳Λμ΢ϯϩʔυ͠ chef-client ࣮ߦ, ͦͷޙ΋ఆظతʹ࣮ߦ knife Λ࢖͍ૢ࡞
6. rcbops/chef-cookbooks ͱ͸
7. rcbops/chef-cookbooks ͱ͸ʁ#1 rackspace ࣾͷ ‘Private Cloud’ αʔϏεͰ༻͍ΒΕ͍ͯΔ OpenCenter ͕಺෦Ͱ Chef Λ࢖͍ͬͯΔɻ ಛ௃ : * ϒϥ΢βϕʔεͰσϓϩΠ * ෳ਺୆ߏ੒, HA ߏ੒౳
8. rcbops/chef-cookbooks ͱ͸ʁ#2 OpenCenter ಺෦Ͱ༻͍ΒΕ͍ͯΔ Chef Cookbooks ͕ github ্Ͱ Apache ϥΠηϯεͷݩެ։͞Ε͍ͯΔ
9. osops-utils ͷ֓ཁ
10. ·ͣٙ໰ ʮChef Ͱෳ਺୆ߏ੒Λѻ͏ͷ͸ݫ͍͠ʯͱ͍͏ ࿩Λͪΐͪ͘ΐ͘ฉ͘ͷͰ͕͢ຊ౰͔ʁ Chef Ͱෳ਺୆ߏ੒Λѻ͏ͷ͸ ؆୯Ͱ͢ʂ
11. osops-utils ͷ֓ཁ #1 * ෳ਺୆ߏ੒Λѻ͏伴ͱͳΔ Cookbooks * ୯ମͰ͸ಈ͔ͳ͍ * rcbops-cookbooks ͷ֤ Cookbooks ͕ include https://github.com/rcbops-cookbooks/osops-utils ಛ௃ ॴࡏ
12. osops-utils ͷ֓ཁ #2 … snip … ! "override_attributes": { "package_component": "havana", "osops_networks": { "management": "10.200.10.0/24", "public": "10.200.17.0/24", "nova": "10.200.10.0/24" }, ! … snip … Attributes ͷ্ॻ͖Λ Environments Ͱߦ͏ 1 Environment Ͱ OpenStack 1 Ϋϥελ
13. osops-utils ͷ֓ཁ #3 hosts = search(:node, “chef_environment:#{node.chef_environment}") ! … snip … hosts.each do |host| Chef::Log.info("osops-utils/autoetchosts: checking (#{host})") begin ip = ::Chef::Recipe::IPManagement.get_ip_for_net("management", host) stra = String.new("#{ip} #{host["fqdn"]} #{host["hostname"]}\n") hfile << stra rescue Chef::Log.info( "osops-utils/autoetchosts: skipping node (#{ip}) because" + " it doesn't have a network assigned yet") end end Recipe ͷதΛ೷͍ͯΈΔ ࣗΒͱಉ͡ environment ͷϊʔυΛݕࡧ host ͷ IP ΞυϨεΛݕࡧ͍ͯ͠ΔΑ͏ͩ…
14. osops-utils ͷ֓ཁ #4 Chef::Recipe::IPManagement Ϋϥε get_ip_for_net ϝιουΛ೷͘ # network number associated with this network net = IPAddr.new(node[“osops_networks"][network]) ! ! # loop thru node's interfaces and look at addresses node["network"]["interfaces"].each do |interface| # ohai Ͱ interfaces Λऔಘ ! ! Chef::Log.debug("#{ourname} examining interface #{interface[0]}") if interface[1].has_key?("addresses") then # loop thru each address on this interface interface[1]["addresses"].each do |k, v| if v["family"] == "inet6" or (v["family"] == "inet" and v["prefixlen"] != "32") then ! addr=IPAddr.new(k) if net.include?(addr) Chef::Log.debug(ourname + " ===> using #{addr}") return k # found it else Chef::Log.debug(ourname + " - ignoring #{addr}") end environment Ͱࢦఆͨ͠ωοτϫʔΫΞυϨε chef αʔόʹ node ͷશ NW I/F ৘ใΛऔಘ͠ʹ ࢦఆͨ͠ωοτϫʔΫʹଐͨ͠ I/F ͷ IP ΞυϨεΛԠ౴
15. osops-utils ͷ֓ཁ #4 ohai ίϚϯυͷ݁Ռൈਮ "network": { "interfaces": { "lo": { "mtu": "16436", "flags": [ "LOOPBACK", "UP", "LOWER_UP" ], … snip … "eth0": { "type": "eth", …. snip … "10.200.9.100": { "family": "inet", "prefixlen": "24", "netmask": "255.255.255.0", "broadcast": "10.200.9.255", "scope": "Global" }, … snip …
16. osops-utils ͷ֓ཁ #5 query = "#{query_type}s:#{search_string} AND chef_environment:#{current_node.chef_environment}" debug("osops_search query: #{query}") result, _, _ = Chef::Search::Query.new.search(:node, query) * Լهͷ৚݅Ͱݕࡧ͢Δͱ໨తͷϊʔυΛ஌Δ͜ͱ͕Մೳ ‘role, recipe ໊ + ࣗϊʔυͱಉ͡ environmnet’ osops_search ͱ͍͏ϊʔυݕࡧͷͨΊͷϝιουͷൈਮ * ͦͷޙ get_ip_for_net ϝιουʹͯ໨తͷϊʔυͷ IP Λ ஌Δ͜ͱ͕Մೳ ໨తͷϊʔυΛݕࡧ
17. osops-utils ͷ֓ཁ #6 * ࣗϊʔυͱಉ͡ environment ׌ͭ໨తͷ Roles/Recipe ͷϊʔυΛ Chef αʔόͰݕࡧ͢Δ͜ͱ͕Մೳ * ໨తͷϊʔυͷ IP ΞυϨεΛ get_ip_for_net ϝιου Ͱݕࡧ͢Δ͜ͱ͕Մೳ ͓΀ˑͨ͢ͷෳ਺୆ߏ੒Λ Chef Ͱߏ੒ग़དྷΔʂ ·ͱΊ ༻్ʹ͋ͬͨϊʔυͷ I/F ΞυϨε͕औಘ͠߹͑Δ
18. rcbops-cookbooks Ͱ Havana
19. rcbops-cookbooks Ͱ Havana controller network network compute compute workstation external public management guest * 4ͭͷ෺ཧωοτϫʔΫΛલఏ * public ωοτϫʔΫ : ֎෦ API ༻ωοτϫʔΫ * external ωοτϫʔΫ : Πϯελϯε֎෦઀ଓ༻ωοτϫʔΫ * guest ωοτϫʔΫ : Πϯελϯε಺෦༻ωοτϫʔΫ * management ωοτϫʔΫ : ֤ίϯϙʔωϯτ઀ଓ༻ωοτϫʔΫ * public, external ͷΈάϩʔόϧωοτϫʔΫ * controller : 2 nics, network : 4 nics, compute : 3nics ͷߏ੒ * controller ͸γϯάϧߏ੒ * network ϊʔυ͸୆਺֦ுՄೳ, agent ୯ҐͰϊʔυؒҠಈՄೳ * compute ϊʔυ΋୆਺֦ுՄೳ * workstation ͸ chef-repo ͷॴࡏ஍, management ωοτϫʔΫʹॴଐ ಛ௃
20. rcbops-cookbooks Ͱ Havana #2 % git clone https://github.com/rcbops/chef-cookbooks.git % cd chef-cookbooks % git checkout -b v4.2.0 refs/tags/v4.2.0 % # .chef ഑Լͷ४උׂѪɻ֤ Chef αʔό؀ڥʹ߹ΘͤΔ % git submodule init % git submodule sync % git submodule update % knife cookbook upload -o cookbooks -a % knife role from file roles/*.rb
21. rcbops-cookbooks Ͱ Havana #3 { "name": "havana-neutron", "description": "", "cookbook_versions": { }, "json_class": "Chef::Environment", "chef_type": "environment", "default_attributes": { }, "override_attributes": { "package_component": "havana", "osops_networks": { "management": "10.200.10.0/24", "public": "10.200.9.0/24", "nova": "10.200.10.0/24" }, … snip … * ৄ͘͠͸ϒϩάͰɻ http://jedipunkz.github.io/blog/2013/11/17/openstack-havana-chef-deploy/
22. rcbops-cookbooks Ͱ Havana #4 % knife environment from file \ environments/havana-neutron.json
23. rcbops-cookbooks Ͱ Havana #5 controller ϊʔυͷσϓϩΠɻ ! % knife bootstrap <controller_ipaddr> -N <controller_name> \ -r 'role[single-controller]','role[cinder-volume]' \ -E havana-neutron -x <username> —sudo ! network ϊʔυͷσϓϩΠɻ୆਺෼σϓϩΠ͍ͯͩ͘͠͞ɻ ! % knife bootstrap <network_ipaddr> -N <network_name> \ -r 'role[single-network-node]','recipe[nova-network::neutron-l3-agent]' \ -E neutron-havana -x <username> —sudo ! compute ϊʔυͷσϓϩΠɻ୆਺෼σϓϩΠ͍ͯͩ͘͠͞ɻ ! % knife bootstrap <compute_ipaddr> -N <compute_name> \ -r 'role[single-compute]' \ -E havana-neutron -x <username> --sudo
24. rcbops-cookbooks Ͱ Havana #6 % sudo ovs-vsctl add-port br-eth1 eth1 % sudo ovs-vsctl add-port br-ex eth3 % sudo ovs-vsctl add-port br-eth1 eth1 ωοτϫʔΫϊʔυʹͯ ίϯϐϡʔτϊʔυʹͯ
25. rcbops-cookbooks Ͱ Havana #6
26. rcbops-cookboooks Ͱ Swift
27. rcbops-cookbooks Ͱ Swift #1 ߏ੒ swift-storage01 swift-storage02 swift-storage03 swift-account01 swift-account02 swift-account03 chef server chef workstation swift-manage swift-proxy01 swift-proxy02 load balancer proxy network storage network * swift-storage, account ʹ͸༧Ί /dev/sdb Λ઀ଓ * swift-mange ্Ͱ git αʔόՔಇ <- Rings ؅ཧ * proxy01,02 ͸ haproxy, keepalived Ͱ LB, VRRP
28. rcbops-cookbooks Ͱ Swift #2 % git clone https://github.com/rcbops/chef-cookbooks.git % cd chef-cookbooks % git checkout -b v4.1.2 refs/tags/v4.1.2 % git submodule init % git submodule sync % git submodule update % knife cookbook upload -o cookbook -a % knife role from file role/*.rb Cookbooks ͷऔಘͱ Chef αʔό΁ͷΞοϓσʔτ
29. rcbops-cookbooks Ͱ Swift #3 { "name": "swift", "description": "", "cookbook_versions": { }, "json_class": "Chef::Environment", "chef_type": "environment", "default_attributes": { }, "override_attributes": { "package_component": "grizzly", "osops_networks": { "management": "10.200.9.0/24", "public": "10.200.9.0/24", "nova": "10.200.9.0/24", "swift": "10.200.9.0/24" }, Environment ͷ json ϑΝΠϧΛੜ੒
30. rcbops-cookbooks Ͱ Swift #4 % knife environment from file environments/swift.json Environment ͷ json ϑΝΠϧΛΞοϓϩʔυ
31. rcbops-cookbooks Ͱ Swift #6 swift-manage ͷϒʔτετϥοϓ % knife bootstrap <manage_ip_addr> -N swift-manage -r \ ’role[base]','role[mysql-master]','role[keystone]','role[swift-management-server]' -E swift --sudo -x thirai swift-proxyNN ͷϒʔτετϥοϓ % knife bootstrap <proxy01_ip_addr> -N swift-proxy01 -r \ ”role[base]","role[swift-proxy-server]",'role[swift-setup]','role[openstack-ha]' -E swift --sudo -x thirai % knife bootstrap <proxy02_ip_addr> -N swift-proxy02 -r \ ”role[base]","role[swift-proxy-server]",'role[openstack-ha]' -E swift --sudo -x thirai swift-storageNN ͷϒʔτετϥοϓ % knife bootstrap <storage01_ip_addr> -N swift-storage01 -r \ role[base]’,'role[swift-object-server]' -E swift --sudo -x thirai % knife bootstrap <storage02_ip_addr> -N swift-storage02 -r \ ’role[base]','role[swift-object-server]' -E swift --sudo -x thirai % knife bootstrap <storage03_ip_addr> -N swift-storage03 -r \ ’role[base]','role[swift-object-server]' -E swift --sudo -x thirai swift-accountNN ͷϒʔτετϥοϓ % knife bootstrap <account01_ip_addr> -N swift-account01 -r \ ’role[base]','role[swift-account-server]','role[swift-container-server]' -E swift --sudo -x thirai % knife bootstrap <account02_ip_addr> -N swift-account02 -r \ ’role[base]','role[swift-account-server]','role[swift-container-server]' -E swift --sudo -x thirai % knife bootstrap <account03_ip_addr> -N swift-account03 -r \ ’role[base]','role[swift-account-server]','role[swift-container-server]' -E swift --sudo -x thirai knife bootstrap ࣮ߦ
32. rcbops-cookbooks Ͱ Swift #7 % knife bootstrap <ip_swift-proxy01> -N swift-proxy01 -r 'role[ha-swift-controller1]' -E swift-ha --sudo -x jedipunkz % knife bootstrap <ip_swift-proxy02> -N swift-proxy02 -r 'role[ha-swift-controller2]' -E swift-ha --sudo -x jedipunkz ͓·͚ ԼهͷΑ͏ʹ Roles ΛׂΓ౰ͯΔͱ HA ߏ੒ͷ Swift Λߏ੒͢Δ͜ͱ΋Մೳ ಛ௃ * haproxy Ͱϩʔυόϥϯε * keepalived Ͱ VRRP * MySQL HA
33. rcbops-cookbooks Ͱ Swift #8 % knife exec -E "nodes.find(:name => 'swift-storage01') {|n| n.set['swift']['zone'] = '1'; n.save }" % knife exec -E "nodes.find(:name => 'swift-account01') {|n| n.set['swift']['zone'] = '1'; n.save }" % knife exec -E "nodes.find(:name => 'swift-storage02') {|n| n.set['swift']['zone'] = '2'; n.save }" % knife exec -E "nodes.find(:name => 'swift-account02') {|n| n.set['swift']['zone'] = '2'; n.save }" % knife exec -E "nodes.find(:name => 'swift-storage03') {|n| n.set['swift']['zone'] = '3'; n.save }" % knife exec -E "nodes.find(:name => 'swift-account03') {|n| n.set['swift']['zone'] = '3'; n.save }" zone ൪߸ͷ෇༩
34. rcbops-cookbooks Ͱ Swift #9 % knife exec -E \ 'search(:node,"role:swift-object-server OR \ role:swift-account-server \ OR role:swift-container-server") \ { |n| puts "#{n.name}"; \ begin; n[:swift][:state][:devs].each do |d| \ puts "\tdevice #{d[1]["device"]}"; \ end; rescue; puts \ "no candidate drives found"; end; }' swift-storage02 device sdb1 swift-storage03 device sdb1 swift-account01 device sdb1 swift-account02 device sdb1 swift-account03 device sdb1 swift-storage01 device sdb1 disk ͕ݕ஌ग़དྷΔ͔Λ֬ೝ
35. rcbops-cookbooks Ͱ Swift #10 swift-manage% sudo chef-client swift-manage% sudo ${EDITOR} /etc/swift/ring-workspace/generage-rings.sh swift-manage% sudo /etc/swift/ring-workspace/generate-rings.sh swift-manage# cd /etc/swift/ring-workspace/rings swift-manage# git add account.builder container.builder object.builder swift-manage# git add account.ring.gz container.ring.gz object.ring.gz swift-manage# git commit -m "initial commit" swift-manage# git push rings ϑΝΠϧୡͷ git αʔόͰͷ؅ཧ chef-client ࣮ߦͱ rings ϑΝΠϧୡͷੜ੒ rings ϑΝΠϧୡΛ git αʔό΁ push
36. rcbops-cookbooks Ͱ Swift #11 swift-proxy01# chef-client swift-proxy02# chef-client swift-storage01# chef-client swift-storage02# chef-client swift-storage03# chef-client swift-account01# chef-client swift-account02# chef-client swift-account03# chef-client ֤ϊʔυͰͷ chef-client ͷ࣮ߦ rings ϑΝΠϧͷ pull ͱ swift αʔϏεͷ࠶ىಈ
37. ·ͱΊ
38. ৄ͘͠͸ϒϩάͰ http://jedipunkz.github.io/blog/2013/11/17/openstack-havana-chef-deploy/ http://jedipunkz.github.io/blog/2013/10/27/swift-chef/ Swift σϓϩΠํ๏Λهͨ͠ϒϩά Havana OpenStack ෼཭ߏ੒σϓϩΠΛهͨ͠ϒϩά http://jedipunkz.github.io/blog/2013/07/26/swift-ha-chef-deploy/ Swift σϓϩΠํ๏Λهͨ͠ϒϩά(HA ߏ੒)