$30 off During Our Annual Pro Sale. View Details »

Consul in Production

Consul in Production

SHIBATA Hiroshi

August 01, 2016
Tweet

More Decks by SHIBATA Hiroshi

Other Decks in Technology

Transcript

  1. DPOTVMͷ༻్͍Ζ͍Ζ
    ࣲాതࢤ(.01&1"#0JOD
    $POTVM$BTVBM5BMLT
    $POTVMJO1SPEVDUJPO

    View Slide

  2. νʔϑΤϯδχΞ
    ࣲాതࢤ!ITCU
    ٕज़෦ٕज़ج൫νʔϜ
    IUUQTXXXITCUPSH

    View Slide

  3. self.introduce
    =>
    {
    github: “@hsbt”,
    commit_bits: [
    “ruby”, “rake”, “rubygems”, “rdoc”,
    “psych”, “syck”, “ruby-build”, “jenkins.rb”…
    ],
    sites: [
    “hsbt.org”, “ruby-lang.org”, “rubyci.org”,
    “railsgirls.com”, “railsgirls.jp”
    ],
    }

    View Slide

  4. DPOTVMͰ΍ͬͯΔ͜ͱΛ঺հ
    wNJOOF NJOOFDPN
    Ͱ΍ͬͯΔ͜ͱJO
    wNJOOF͸0QFO4UBDLׂɺ"84ׂͷNVMUJ%$ɺ7.͸͘Β͍ɺશ
    ෦ʹDPOTVMೖΕͯ·͢
    wDPOTVMͷED͸࢖ͬͯ·ͤΜɺ0QFO4UBDLͱ"84ͰผͷΫϥελ
    wDPOTVMEOTNBTRͰ΍Δ಺෦%/4͸লུ
    wTUSFUDIFSΛ࢖ͬͨσϓϩΠ͸লུ
    wDPOTVMͰUMTTFTTJPOUJDLFUͷߋ৽͸লུ

    View Slide

  5. ͓·͚systemd
    [Unit]
    Description=Consul is a tool for service discovery and configuration. Consul is distributed, highly available, and extremely scal
    Documentation=http://www.consul.io
    After=network-online.target
    Wants=network-online.target
    [Service]
    User=consul
    Group=consul
    EnvironmentFile=-/etc/sysconfig/consul
    ExecStart=/usr/bin/consul $CMD_OPTS
    ExecReload=/bin/kill -HUP $MAINPID
    KillSignal=SIGINT
    Restart=on-failure
    LimitNOFILE=8192
    [Install]
    WantedBy=multi-user.target

    View Slide

  6. /FX3FMJDͷϗετબఆ

    View Slide

  7. ͜͏͍͏͜ͱΛ΍Γ͍ͨ
    wNJOOFͰ͸ΞϓϦέʔγϣϯͷύϑΥʔϚϯεϞχλϦϯάʹ/FX3FMJD
    IUUQTOFXSFMJDDPN
    Λ࢖͍ͬͯΔ
    w/FX3FMJD͸ߴ͍ IPTU

    ˠόϧΫσΟεΧ΢ϯτΛަব͢ΔͱIPTU͘Β͍·ͰԼ͕Δ
    wͦΕͰ΋αʔϏεͷϩʔϧ͕ͭ͋Ε͹࠷௿IPTUʹೖΕΔඞཁ͕͋Δ
    wϩʔϧͷத͔Β͍͍ײ͡ʹ୆͚ͩબΜͰ/FX3FMJDΛ༗ޮʹ͍ͨ͠

    View Slide

  8. ͜͏΍ͬͨ
    w/FX3FMJDͷઃఆϑΝΠϧ 3BJMT༻
    ʹDPOTVM͔ΒϗετΛऔಘͯ͠ϗε
    τ͚ͩ༗ޮͱ͢ΔΑ͏ͳίʔυΛຒΊࠐΉ
    wOFXSFMJD@SQNͷHFN͕ϩʔυ͞Εͨ࣌ʹݺͼग़͞Εͯཉ͍͠ͷͰ3VCZͷ
    ίʔυ͸A3BJMTSPPUKPJO bMJC`
    Aʹஔ͘
    w$POTVM΁ͷΞΫηε͸%JQMPNBU IUUQTHJUIVCDPN8F"SF'BSN(FFL
    EJQMPNBU
    Λ࢖͏
    wΦϦδφϧIUUQTNBLBOESBDBSETDPNNBLBOESBFOBCMF
    OFXSFMJDNPOJUPSJOHGPSSBJMTPOTQFDJpDIPTUTPOMZ

    View Slide

  9. ͜Μͳײ͡
    common: &default_settings
    (snip)
    test:
    <<: *default_settings
    monitor_mode: false
    production:
    <<: *default_settings
    monitor_mode: <%= CustomNewRelicConfiguration.enable_on_current_host? rescue false %>

    View Slide

  10. ͜Μͳײ͡

    require 'socket'
    class CustomNewRelicConfiguration
    def self.enable_on_current_host?
    current_hostname = Socket.gethostname.strip
    Rails.logger.info ">>> Checking whether to enable NewRelic on host '#{current_hostname}'"
    if newrelic_hosts == current_hostname
    Rails.logger.info ">>> NewRelic should be enabled on this host."
    true
    else
    Rails.logger.info ">>> NewRelic is disabled on this host! Edit config/newrelic.yml to enable."
    false
    end
    end
    def self.newrelic_hosts
    require 'diplomat'
    Diplomat.configuration.url = "http://127.0.0.1:8500"
    node = Diplomat::Node.get_all.
    select{|n| n.Node =~ /^www-/ }.
    sort_by{|n| n.Node }.
    first.Node
    "#{node}.minne.com"
    end
    end

    View Slide

  11. TJEFLJRTDIFEVMFS

    View Slide

  12. ͜͏͍͏͜ͱΛ΍Γ͍ͨ
    wDSPO͸WN͕410'ͱͳΓ͕ͪͳͷͰ΍Ί͍ͨ
    wIUUQTHJUIVCDPNNPPWFJUTJEFLJRTDIFEVMFSΛ࢖͏ͱTJEFLJRʹ
    QFSJPEJDBMKPC͕࢖͑ΔΑ͏ʹͳΔ TJEFLJRQSPͰ΋Ͱ͖Δ

    wTJEFLJRTDIFEVMFS͸୯७ʹ࢖͏ͱXPSLFSશ୆Ͱ༗ޮʹͳͬͯɺδϣϒ͕ॏ
    ෳͯ͠ࠔΔ
    wDPOTVMͰৗʹҰ୆͚ͩTJEFLJRTDIFEVMFSΛ༗ޮʹ͍ͨ͠

    View Slide

  13. ֓೦ਤ
    sidekiq
    worker
    sidekiq
    worker
    &
    scheduler
    redis
    sidekiq
    worker
    sidekiq
    worker
    sidekiq
    worker
    sidekiq
    worker
    sidekiq
    worker
    redis
    sidekiq
    worker
    sidekiq
    worker
    sidekiq
    worker

    View Slide

  14. ͜͏΍ͬͨ
    wDPOTVMΛ࢖ͬͯɺҰఆͷϧʔϧͰҰ୆͚ͩTDIFEVMFSΛ༗ޮͱ͢ΔΑ͏ͳϗ
    ετωʔϜΛऔΓग़͢
    wTDIFEVMFS͕͍ͳ͘ͳΔͱࠔΔͷͰɺTDIFEVMFSϗετ͕ϓϥοτϑΥʔϜͷ
    τϥϒϧͰམͪͨ৔߹͸ࣗಈͰ࠶બग़͢Δ ·ͩͰ͖ͯͳ͍ʜ

    View Slide

  15. ͜Μͳײ͡
    Sidekiq.configure_server do |config|
    if Rails.env.production? # for sidekiq-scheduler
    config.on(:startup) do
    if Sidekiq::Scheduler.master_server?
    Sidekiq::Scheduler.register_server
    Sidekiq::Scheduler.reload_schedule!
    else
    Sidekiq::Scheduler.enabled = false
    end
    end
    %i(quiet shutdown).each do |state|
    config.on(state) do
    Sidekiq::Scheduler.reset_master_server!
    end
    end
    end
    end
    module Sidekiq
    class Scheduler
    def self.master_server
    Diplomat::Node.get_all.
    select{|n| n.Node =~ /^worker-/ }.
    sort_by{|n| n.Node }.
    first.Node
    end
    def self.master_server?
    Socket.gethostname.chomp == self.master_server
    end
    def self.register_server
    Sidekiq.redis {|c| c.set(‘sidekiq:schedules:master’, Socket.gethostname.chomp) }
    end
    def self.reset_master_server!
    if Socket.gethostname.chomp == Sidekiq.redis {|c| c.get('sidekiq:schedules:master')}
    Sidekiq.redis {|conn| conn.del('sidekiq:schedules:master')}
    end
    end
    end
    end

    View Slide

  16. ϩʔυόϥϯαͷ%/4ߋ৽

    View Slide

  17. ͜͏͍͏͜ͱΛ΍Γ͍ͨ
    wMPBECBMBODFSͱͯ͠OHJOYΛO୆ར༻͍ͯ͠Δ
    wOHJOYͷVQTUSFBNͱͯ͠N୆഑ஔ͠ɺΫϩεϝογϡͰܨ͍Ͱ͍Δ
    wMPBECBMBODFSΛ%/4ϥ΢ϯυϩϏϯͰ෼ࢄΞΫηε͍ͯ͠Δ
    wMPBECBMBODFS͕ϐϤͬͨ࣌ʹ%/4ϥ΢ϯυϩϏϯͷର৅͔ΒࣗಈͰ֎ͨ͠
    ͍

    View Slide

  18. ֓೦ਤ
    application
    server
    DNS
    (Route53)
    load
    balancer
    load
    balancer
    application
    server
    application
    server
    ো֐ൃੜ
    application
    server
    load
    balancer
    load
    balancer
    application
    server
    application
    server
    ࣗಈִ཭
    DNS
    (Route53)

    View Slide

  19. ͜͏΍ͬͨ
    wIUUQTHJUIVCDPNVE[VSBDPMMFDUPSΛར༻
    wDPOTVMͷTFSWJDFঢ়گΛ؂ࢹ OBHJPTQMVHJOBMM࢖ͬͯ·͕ͨ͠NBDLFSFM
    ʹҠߦத

    wDPOTVMͷTFSWJDF MPBECBMBODFSͷOHJOY
    ͕μ΢ϯ͍ͯ͠Δ৔߹͸֘౰ͷ
    DPOTVMOPEFΛDPOTVMXBUDIͱDPMMFDUPSΛ༻͍ͯEOT͔Β੾Γ͸ͣ͠
    wৄࡉ͸ઃఆϑΝΠϧͳͲ͸3&"%.&NEΛΈ͍ͯͩ͘͞ লུ

    View Slide

  20. ͜Μͳײ͡

    View Slide

  21. OHJOYͷVQTUSFBNͷࣗಈߋ৽

    View Slide

  22. ͜͏͍͏͜ͱΛ΍Γ͍ͨ
    wMPBECBMBODFSͰ࢖͍ͬͯΔOHJOYͷVQTUSFBNΛࣗಈߋ৽͍ͨ͠
    wIUUQTHJUIVCDPNIBTIJDPSQDPOTVMUFNQMBUFͰOHJOYͷ
    DPOpHVSBUJPOΛࣗಈੜ੒
    wҟৗ͕͋ΔWN͸VQTUSFBN͔Β֎͢͠ɺ৽্ཱͪ͛ͨ͘͠WN͸ࣗಈͰ
    MPBECBMBODFSʹKPJO͢Δ
    w஍ཕIUUQUPNPIJTBPEBDPNQPTUTVTF@TZTUFNE@XJUI@DPOTVM
    UFNQMBUFIUNM

    View Slide

  23. ֓೦ਤ
    application
    server
    DNS
    (Route53)
    load
    balancer
    load
    balancer
    application
    server
    application
    server
    ো֐ൃੜ
    application
    server
    DNS
    (Route53)
    load
    balancer
    load
    balancer
    application
    server
    application
    server
    ࣗಈִ཭

    View Slide

  24. ͜͏΍ͬͨ
    wMPBECBMBODFSʹDPOTVMUFNQMBUFΛೖΕͯૉ๿ʹ࣮ߦ
    wDPOpHVSBUJPOͷDPNNBOEʹOHJOYSFMPBEͱTMBDL௨஌Λૠೖ
    w༨ஊ
    wDPOTVMDMVTUFSʹϧʔςΟϯάؒҧ͑ͨOPEFΛೖΕͯɺΫϥελશମͷௐ
    ࢠ͕ѱ͘ͳͬͨࡍʹNJOʹճOHJOYΛSFMPBE͢Δ঱ঢ়͕ൃੜ
    wOHY@NSVCZʹSFMPBE͢ΔͱϝϞϦϦʔΫ͢Δόά͕͋ΓMPBECBMBODFS
    ่͕յʜ IJCPNB͕WBMHSJOEͰൃੜՕॴΛಛఆͯ͠ใࠂͨ͠

    View Slide

  25. ಉ྅͕௚ͯ͘͠Εͨ
    IUUQTHJUIVCDPNNBUTVNPUPSOHY@NSVCZQVMM

    View Slide

  26. ·ͱΊ
    wDPOTVMΛ࢖͏͜ͱͰWNΛଋͶͯҰͭͷγεςϜͱͯ͠ѻ͏͜ͱ͕Մೳͱͳ
    Δ
    wDPOTVMNBJOUΛ׆༻Ͱ͖ͯͳ͍ͷͰ͍͍ײ͡ʹWNΛୀආ͢ΔΈ͍ͨͷΛ࡞
    Γ͍ͨ
    wશࣗಈͳΊΒ͔αʔόʔϦιʔε੍ޚ͘ΜɺશࣗಈͳΊΒ͔δϣϒεέδϡʔ
    ϥΛ࡞͍ͬͯΔͱ͜Ζ
    w͜͏͍͏ͷΛ΍Γ͍ͨਓ͸ࠓ͙͢๻ͱϖύϥϯνϣϯˠIUUQT
    QFQBCPDPNSFDSVJUQFQBMVODIFPO

    View Slide