Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Consul in Production

Consul in Production

Eabad423977cfc6873b8f5df62b848a6?s=128

SHIBATA Hiroshi

August 01, 2016
Tweet

More Decks by SHIBATA Hiroshi

Other Decks in Technology

Transcript

  1. DPOTVMͷ༻్͍Ζ͍Ζ ࣲాതࢤ(.01&1"#0JOD $POTVM$BTVBM5BMLT $POTVMJO1SPEVDUJPO

  2. νʔϑΤϯδχΞ ࣲాതࢤ!ITCU ٕज़෦ٕज़ج൫νʔϜ IUUQTXXXITCUPSH

  3. self.introduce => { github: “@hsbt”, commit_bits: [ “ruby”, “rake”, “rubygems”,

    “rdoc”, “psych”, “syck”, “ruby-build”, “jenkins.rb”… ], sites: [ “hsbt.org”, “ruby-lang.org”, “rubyci.org”, “railsgirls.com”, “railsgirls.jp” ], }
  4. DPOTVMͰ΍ͬͯΔ͜ͱΛ঺հ wNJOOF NJOOFDPN Ͱ΍ͬͯΔ͜ͱJO wNJOOF͸0QFO4UBDLׂɺ"84ׂͷNVMUJ%$ɺ7.͸͘Β͍ɺશ ෦ʹDPOTVMೖΕͯ·͢ wDPOTVMͷED͸࢖ͬͯ·ͤΜɺ0QFO4UBDLͱ"84ͰผͷΫϥελ wDPOTVM EOTNBTRͰ΍Δ಺෦%/4͸লུ wTUSFUDIFSΛ࢖ͬͨσϓϩΠ͸লུ

    wDPOTVMͰUMTTFTTJPOUJDLFUͷߋ৽͸লུ
  5. ͓·͚systemd [Unit] Description=Consul is a tool for service discovery and

    configuration. Consul is distributed, highly available, and extremely scal Documentation=http://www.consul.io After=network-online.target Wants=network-online.target [Service] User=consul Group=consul EnvironmentFile=-/etc/sysconfig/consul ExecStart=/usr/bin/consul $CMD_OPTS ExecReload=/bin/kill -HUP $MAINPID KillSignal=SIGINT Restart=on-failure LimitNOFILE=8192 [Install] WantedBy=multi-user.target
  6. /FX3FMJDͷϗετબఆ

  7. ͜͏͍͏͜ͱΛ΍Γ͍ͨ wNJOOFͰ͸ΞϓϦέʔγϣϯͷύϑΥʔϚϯεϞχλϦϯάʹ/FX3FMJD  IUUQTOFXSFMJDDPN Λ࢖͍ͬͯΔ w/FX3FMJD͸ߴ͍ IPTU  ˠόϧΫσΟεΧ΢ϯτΛަব͢ΔͱIPTU͘Β͍·ͰԼ͕Δ wͦΕͰ΋αʔϏεͷϩʔϧ͕ͭ͋Ε͹࠷௿IPTUʹೖΕΔඞཁ͕͋Δ

    wϩʔϧͷத͔Β͍͍ײ͡ʹ୆͚ͩબΜͰ/FX3FMJDΛ༗ޮʹ͍ͨ͠
  8. ͜͏΍ͬͨ w/FX3FMJDͷઃఆϑΝΠϧ 3BJMT༻ ʹDPOTVM͔ΒϗετΛऔಘͯ͠ϗε τ͚ͩ༗ޮͱ͢ΔΑ͏ͳίʔυΛຒΊࠐΉ wOFXSFMJD@SQNͷHFN͕ϩʔυ͞Εͨ࣌ʹݺͼग़͞Εͯཉ͍͠ͷͰ3VCZͷ ίʔυ͸A3BJMTSPPUKPJO bMJC` Aʹஔ͘ w$POTVM΁ͷΞΫηε͸%JQMPNBU

    IUUQTHJUIVCDPN8F"SF'BSN(FFL EJQMPNBU Λ࢖͏ wΦϦδφϧIUUQTNBLBOESBDBSETDPNNBLBOESBFOBCMF OFXSFMJDNPOJUPSJOHGPSSBJMTPOTQFDJpDIPTUTPOMZ
  9. ͜Μͳײ͡ common: &default_settings (snip) test: <<: *default_settings monitor_mode: false production:

    <<: *default_settings monitor_mode: <%= CustomNewRelicConfiguration.enable_on_current_host? rescue false %>
  10. ͜Μͳײ͡  require 'socket' class CustomNewRelicConfiguration def self.enable_on_current_host? current_hostname =

    Socket.gethostname.strip Rails.logger.info ">>> Checking whether to enable NewRelic on host '#{current_hostname}'" if newrelic_hosts == current_hostname Rails.logger.info ">>> NewRelic should be enabled on this host." true else Rails.logger.info ">>> NewRelic is disabled on this host! Edit config/newrelic.yml to enable." false end end def self.newrelic_hosts require 'diplomat' Diplomat.configuration.url = "http://127.0.0.1:8500" node = Diplomat::Node.get_all. select{|n| n.Node =~ /^www-/ }. sort_by{|n| n.Node }. first.Node "#{node}.minne.com" end end
  11. TJEFLJRTDIFEVMFS

  12. ͜͏͍͏͜ͱΛ΍Γ͍ͨ wDSPO͸WN͕410'ͱͳΓ͕ͪͳͷͰ΍Ί͍ͨ wIUUQTHJUIVCDPNNPPWFJUTJEFLJRTDIFEVMFSΛ࢖͏ͱTJEFLJRʹ QFSJPEJDBMKPC͕࢖͑ΔΑ͏ʹͳΔ TJEFLJRQSPͰ΋Ͱ͖Δ  wTJEFLJRTDIFEVMFS͸୯७ʹ࢖͏ͱXPSLFSશ୆Ͱ༗ޮʹͳͬͯɺδϣϒ͕ॏ ෳͯ͠ࠔΔ wDPOTVMͰৗʹҰ୆͚ͩTJEFLJRTDIFEVMFSΛ༗ޮʹ͍ͨ͠

  13. ֓೦ਤ sidekiq worker sidekiq worker & scheduler redis sidekiq worker

    sidekiq worker sidekiq worker sidekiq worker sidekiq worker redis sidekiq worker sidekiq worker sidekiq worker
  14. ͜͏΍ͬͨ wDPOTVMΛ࢖ͬͯɺҰఆͷϧʔϧͰҰ୆͚ͩTDIFEVMFSΛ༗ޮͱ͢ΔΑ͏ͳϗ ετωʔϜΛऔΓग़͢ wTDIFEVMFS͕͍ͳ͘ͳΔͱࠔΔͷͰɺTDIFEVMFSϗετ͕ϓϥοτϑΥʔϜͷ τϥϒϧͰམͪͨ৔߹͸ࣗಈͰ࠶બग़͢Δ ·ͩͰ͖ͯͳ͍ʜ

  15. ͜Μͳײ͡ Sidekiq.configure_server do |config| if Rails.env.production? # for sidekiq-scheduler config.on(:startup)

    do if Sidekiq::Scheduler.master_server? Sidekiq::Scheduler.register_server Sidekiq::Scheduler.reload_schedule! else Sidekiq::Scheduler.enabled = false end end %i(quiet shutdown).each do |state| config.on(state) do Sidekiq::Scheduler.reset_master_server! end end end end module Sidekiq class Scheduler def self.master_server Diplomat::Node.get_all. select{|n| n.Node =~ /^worker-/ }. sort_by{|n| n.Node }. first.Node end def self.master_server? Socket.gethostname.chomp == self.master_server end def self.register_server Sidekiq.redis {|c| c.set(‘sidekiq:schedules:master’, Socket.gethostname.chomp) } end def self.reset_master_server! if Socket.gethostname.chomp == Sidekiq.redis {|c| c.get('sidekiq:schedules:master')} Sidekiq.redis {|conn| conn.del('sidekiq:schedules:master')} end end end end
  16. ϩʔυόϥϯαͷ%/4ߋ৽

  17. ͜͏͍͏͜ͱΛ΍Γ͍ͨ wMPBECBMBODFSͱͯ͠OHJOYΛO୆ར༻͍ͯ͠Δ wOHJOYͷVQTUSFBNͱͯ͠N୆഑ஔ͠ɺΫϩεϝογϡͰܨ͍Ͱ͍Δ wMPBECBMBODFSΛ%/4ϥ΢ϯυϩϏϯͰ෼ࢄΞΫηε͍ͯ͠Δ wMPBECBMBODFS͕ϐϤͬͨ࣌ʹ%/4ϥ΢ϯυϩϏϯͷର৅͔ΒࣗಈͰ֎ͨ͠ ͍

  18. ֓೦ਤ application server DNS (Route53) load balancer load balancer application

    server application server ো֐ൃੜ application server load balancer load balancer application server application server ࣗಈִ཭ DNS (Route53)
  19. ͜͏΍ͬͨ wIUUQTHJUIVCDPNVE[VSBDPMMFDUPSΛར༻ wDPOTVMͷTFSWJDFঢ়گΛ؂ࢹ OBHJPTQMVHJOBMM࢖ͬͯ·͕ͨ͠NBDLFSFM ʹҠߦத  wDPOTVMͷTFSWJDF MPBECBMBODFSͷOHJOY ͕μ΢ϯ͍ͯ͠Δ৔߹͸֘౰ͷ DPOTVMOPEFΛDPOTVMXBUDIͱDPMMFDUPSΛ༻͍ͯEOT͔Β੾Γ͸ͣ͠

    wৄࡉ͸ઃఆϑΝΠϧͳͲ͸3&"%.&NEΛΈ͍ͯͩ͘͞ লུ
  20. ͜Μͳײ͡

  21. OHJOYͷVQTUSFBNͷࣗಈߋ৽

  22. ͜͏͍͏͜ͱΛ΍Γ͍ͨ wMPBECBMBODFSͰ࢖͍ͬͯΔOHJOYͷVQTUSFBNΛࣗಈߋ৽͍ͨ͠ wIUUQTHJUIVCDPNIBTIJDPSQDPOTVMUFNQMBUFͰOHJOYͷ DPOpHVSBUJPOΛࣗಈੜ੒ wҟৗ͕͋ΔWN͸VQTUSFBN͔Β֎͢͠ɺ৽্ཱͪ͛ͨ͘͠WN͸ࣗಈͰ MPBECBMBODFSʹKPJO͢Δ w஍ཕIUUQUPNPIJTBPEBDPNQPTUTVTF@TZTUFNE@XJUI@DPOTVM UFNQMBUFIUNM

  23. ֓೦ਤ application server DNS (Route53) load balancer load balancer application

    server application server ো֐ൃੜ application server DNS (Route53) load balancer load balancer application server application server ࣗಈִ཭
  24. ͜͏΍ͬͨ wMPBECBMBODFSʹDPOTVMUFNQMBUFΛೖΕͯૉ๿ʹ࣮ߦ wDPOpHVSBUJPOͷDPNNBOEʹOHJOYSFMPBEͱTMBDL௨஌Λૠೖ w༨ஊ wDPOTVMDMVTUFSʹϧʔςΟϯάؒҧ͑ͨOPEFΛೖΕͯɺΫϥελશମͷௐ ࢠ͕ѱ͘ͳͬͨࡍʹNJOʹճOHJOYΛSFMPBE͢Δ঱ঢ়͕ൃੜ wOHY@NSVCZʹSFMPBE͢ΔͱϝϞϦϦʔΫ͢Δόά͕͋ΓMPBECBMBODFS ่͕յʜ IJCPNB͕WBMHSJOEͰൃੜՕॴΛಛఆͯ͠ใࠂͨ͠

  25. ಉ྅͕௚ͯ͘͠Εͨ IUUQTHJUIVCDPNNBUTVNPUPSOHY@NSVCZQVMM

  26. ·ͱΊ wDPOTVMΛ࢖͏͜ͱͰWNΛଋͶͯҰͭͷγεςϜͱͯ͠ѻ͏͜ͱ͕Մೳͱͳ Δ wDPOTVMNBJOUΛ׆༻Ͱ͖ͯͳ͍ͷͰ͍͍ײ͡ʹWNΛୀආ͢ΔΈ͍ͨͷΛ࡞ Γ͍ͨ wશࣗಈͳΊΒ͔αʔόʔϦιʔε੍ޚ͘ΜɺશࣗಈͳΊΒ͔δϣϒεέδϡʔ ϥΛ࡞͍ͬͯΔͱ͜Ζ w͜͏͍͏ͷΛ΍Γ͍ͨਓ͸ࠓ͙͢๻ͱϖύϥϯνϣϯˠIUUQT QFQBCPDPNSFDSVJUQFQBMVODIFPO