Save 37% off PRO during our Black Friday Sale! »

それでも僕はzabbixと生きていく

C9cacb1d0e96ab27dcb045177068820c?s=47 youyo
October 07, 2017

 それでも僕はzabbixと生きていく

C9cacb1d0e96ab27dcb045177068820c?s=128

youyo

October 07, 2017
Tweet

Transcript

  1. ͦΕͰ΋๻͸Zabbixͱ ੜ͖͍ͯ͘

  2. ࣗݾ঺հ • ੴᖒ௚ਓ(Naoto Ishizawa) • twitter: @youyo_ • github: https://github.com/youyo

    • Cloud Architect @ heptagon inc. • I love golang, python, ruby • ࠷ۙϋϚ͍ͬͯΔ΋ͷ: 
 υϥΰϯϘʔϧυοΧϯότϧ
 LRଙޛ൧ͷݩΛखʹೖΕͨ͜ͱ͕࠷ۙҰ൪خ͔ͬͨ͠
  3. mackerel޷͖Ͱ͔͢ʁ

  4. mackerelͱ͸ • ΤϯδχΞΛϫΫϫΫͤ͞Δʮ௚ײతαʔόʔ؂ࢹαʔϏεʯ
 https://mackerel.io/ja/ • Monitoring as a service

  5. ๻͔Βݟͨmackerelͷ͍͍ͱ͜Ζ • ͍͍ײ͡ʹcloudwatch͔Βσʔλऔ͖ͬͯͯ͘ΕΔ • ͖Ε͍ͳUI • ϑϧϚωʔδυ • ๛෋ͳNotification

  6. ࠷ߴ

  7. mackerelͷಛʹ͍͍ͱ͜Ζ • CloudWatch APIͱͷγʔϜϨεͳ࿈ܞ IUUQTNBDLFSFMJPKBGFBUVSFTBXTJOUFHSBUJPO

  8. ࠷ߴ

  9. ๻͔ΒݟͨmackerelͷؾʹͳΔͱ͜Ζ • 1,800ԁ ϗετ/month
 ->2,000ϗετ؂ࢹ͢ΔͱͳΔͱ360ສԁ
 ->ࣗલͰzabbixͰ؂ࢹ͢Ε͹30ສ͔͔Βͳ͍(౰ࣾൺ) • ϑϧϚωʔδυ
 ->ָͳΜ͚ͩͲͭ·Βͳ͍ʂ
 ->๻΋mackerelΈ͍ͨͳ؂ࢹγεςϜ࡞Γ͍ͨʂʂ


    (ӡ༻͕࢓ࣄͳΜͰͦͷࠜװͱͳΔ΋ͷ͸ࣗલͰ࣋ͭ΂͖)
  10. mackerelʹ͋ͬͯzabbix ʹͳ͍΋ͷΛิ͑͹͍͍

  11. ๻͔Βݟͨmackerelͷ͍͍ͱ͜Ζ • ͍͍ײ͡ʹcloudwatch͔Βσʔλऔ͖ͬͯͯ͘ΕΔ • ͖Ε͍ͳUI • ϑϧϚωʔδυ • ๛෋ͳNotification

  12. ͜Ε͔Β΋Zabbixͱੜ͖͍ͯͨ͘Ίʹ ZabbixͰ͍͍ײ͡ʹ࢖͑Δ AWS Integration࡞Ζ͏

  13. Zabbixͱ͸

  14. Zabbixͱ͸ • Zabbixͱ͸ɺαʔόʔɺωοτϫʔΫɺΞϓϦέʔγϣϯΛ؂ࢹ͢ΔͨΊ ͷιϑτ΢ΣΞͰ͢ɻZabbix͸ओʹҎԼͷ3ͭͷػೳΛ༗͍ͯ͠·͢ɻ • αʔό΍ωοτϫʔΫʹ઀ଓ͞ΕͨσόΠεΛ؂ࢹ͢Δ؂ࢹػೳ • ऩूͨ͠σʔλΛ΋ͱʹάϥϑԽɺωοτϫʔΫϚοϓͷ࡞੒Λߦ͏ά ϥϑΟΧϧදࣔػೳ •

    ऩूͨ͠σʔλʹᮢ஋Λઃఆ͠ɺো֐/෮چ࣌ʹ؅ཧऀʹ௨஌Λߦ͏ো֐ ݕ஌/௨஌ػೳ IUUQXXX[BCCJYKQEPDVNFOUTXIBU@JT@[BCCJY
  15. े෼ͳػೳΛ͍࣋ͬͯΔ

  16. ZabbixͷؾʹͳΔͱ͜Ζ • Ϋϥ΢υϑΝʔετͳ͜ͷ࣌͝ੈɺ͍͍ͪͪΠϯελϯεΛzabbixʹϗετ ొ࿥ͱ͔ͨ͘͠ͳ͍ • ࣗಈొ࿥΋Ͱ͖Δɻ͚Ͳࣗಈ࡟আ͸… • zabbix-agentΛΠϯετʔϧͰ͖ͳ͍αʔϏεͷ؂ࢹͲ͏͢Δʁ

  17. Ͳ͏΍Δʁ • Ϋϥ΢υϑΝʔετͳ͜ͷ࣌͝ੈɺ͍͍ͪͪΠϯελϯεΛzabbixʹϗετ ొ࿥ͱ͔ͨ͘͠ͳ͍ • ࣗಈొ࿥΋Ͱ͖Δɻ͚Ͳࣗಈ࡟আ͸…
 =>LowLevelDiscovery
 • zabbix-agentΛΠϯετʔϧͰ͖ͳ͍αʔϏεͷ؂ࢹͲ͏͢Δʁ
 =>LowLevelDiscovery

  18. Low Level Discovery

  19. LLD (Low Level Discovery) • “ϩʔϨϕϧσΟεΧόϦʹΑΓɺίϯϐϡʔλʔ্ͷछʑͷཁૉʹରͯ͠ ΞΠςϜɺτϦΨʔɺάϥϑΛࣗಈతʹ࡞੒Ͱ͖·͢ɻྫ͑͹ɺZabbix͸ ࢖༻͍ͯ͠ΔϚγϯ্ͷϑΝΠϧγεςϜ·ͨ͸ωοτϫʔΫΠϯλʔϑΣʔ εͷ؂ࢹΛࣗಈతʹ։࢝Ͱ͖·͢ɻͦͷͨΊʹɺ֤ϑΝΠϧγεςϜ·ͨ͸ ωοτϫʔΫΠϯλʔϑΣʔεʹରͯ͠खಈͰΞΠςϜΛ࡞੒͢Δඞཁ͸͋

    Γ·ͤΜɻ͞Βʹɺఆظతʹ࣮ࢪ͞ΕΔσΟεΧόϦͷ࣮ࡍͷ݁Ռʹج͍ͮ ͯෆཁͳཁૉΛࣗಈతʹ࡟আ͢ΔΑ͏ʹɺZabbixΛઃఆ͢Δ͜ͱ͕Ͱ͖· ͢ɻ” IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMEJTDPWFSZMPX@MFWFM@EJTDPWFSZ
  20. LLD (Low Level Discovery) • ͍͍ײ͡ʹ؂ࢹର৅Λݟ͚ͭͯ • ͍͍ײ͡ʹσʔλूΊͯ • ͍͍ײ͡ʹ؂ࢹର৅ফ͑ͨΒϗετొ࿥࡟আ΋ͯ͘͠ΕΔ


    
 “Ϣʔβʔ͸ɺಛఆͷJSONϓϩτίϧʹै͏ͱ͍͏৚݅ͰɺಠࣗͷλΠϓ ͷσΟεΧόϦΛఆٛͰ͖·͢ɻ” IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMEJTDPWFSZMPX@MFWFM@EJTDPWFSZ
  21. ࠷ߴ

  22. Ͳ͏͢Ε͹͍͍ʁ • UserParameterͰεΫϦϓτΛ࣮ߦ͢Δ • Loadable modulesΛ࡞੒͠ɺݺͼग़͢

  23. UserParameter & Loadale modules • UserParameter
 “ر๬ͷσʔλͷݕࡧΛߦ͏ίϚϯυΛॻ͖ɺΤʔδΣϯτઃఆϑΝΠϧͷ ϢʔβʔύϥϝʔλʢʮUserParameterʯઃఆύϥϝʔλʣʹؚΊΔ͜ͱ͕ Ͱ͖·͢ɻ”
 >UserParameter=<key>,<command>

    • Loadable modules
 “loadable modules͸ɺZabbixͷػೳΛ֦ு͢ΔͨΊͷɺੑೳΛߟྀͨ͠ બ୒ࢶͱͳΓ·͢ɻZabbix 2.2͔Β͸ɺੑೳΛଛͶΔ͜ͱͳ͘ɺZabbixΤʔ δΣϯτ͓ΑͼαʔόʔΛ֦ு͢ΔͨΊͷloadable modules͕αϙʔτ͞ ΕΔΑ͏ʹͳΓ·ͨ͠ɻ” IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMDPOpHJUFNTVTFSQBSBNFUFST IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMDPOpHJUFNTMPBEBCMFNPEVMFT
  24. Loadable modulesͰ AWS Integration Ͱ͖ΔΑ͏ʹ͠Α͏

  25. Ͱ΋ࠔͬͨ͜ͱ͕… • loadable modules͸CݴޠͰॻ͘ඞཁ͕͋Δ • ๻ɺCݴޠ͸ॻ͚ͳ͍…

  26. g2z

  27. What is "g2z”? • https://github.com/cavaliercoder/g2z • loadable modulesΛgolangͰॻ͚Δ. (golangͰshared libraryͱͯ͠ੜ੒)

    • Zabbix version 2.xܥ, 3.0ରԠ package main import ( "strings" "github.com/cavaliercoder/g2z" ) func main() { panic("THIS_SHOULD_NEVER_HAPPEN") } func Echo(request *g2z.AgentRequest) (string, error) { return strings.Join(request.Params, " "), nil } func init() { g2z.RegisterStringItem("go.echo", "Hello world!", Echo) }
  28. উͬͨʂ

  29. Ͱ΋ݱ࣮͸؁͘ͳ͔ͬͨ

  30. ͳ͔ͥ͏·͘ಈ͔ͳ͍

  31. Loadable modulesΛ ݺͼग़͢ͱzabbix-agent ϓϩηε͕ͨ·ʹݻ·Δ

  32. IUUQTHJUIVCDPNDBWBMJFSDPEFSH[JTTVFT

  33. ҙ༁ • ֎෦ʹhttpϦΫΤετΛߦ͏ͱɺݻ·Δ͜ͱ͕͋Δ • golang͸ѱ͘ͳ͍ • zabbix΋ѱ͘ͳ͍ • goroutine͕ؔ܎ͯ͠Δͷ͔ͳʁ •

    single coreͷϚγϯͰͳΒਖ਼ৗʹಈͬ͘Ά͍Αʁ
 ->͍΍ಈ͔ͳ͍.. • ͪΐͬͱαϙʔτ͖͠Εͳ͍Θʔ
  34. ٧Μͩ…

  35. UserParameterʹ ੾Γସ͍͑ͯ͘

  36. UserParameterͰLLD͢Δʹ͸ • ϓϩάϥϜॻ͍ͯ΋shell-scriptͰ΋ԿͰ΋͍͍ • ܾ·ͬͨϑΥʔϚοτ(json)ͰσʔλΛฦ͢ඞཁ͕͋Δ { "data":[ { "{#INSTANCE_ID}":"i-xxxxxxxxxx", "{#INSTANCE_NAME}":"test1"

    }, { "{#INSTANCE_ID}":"i-yyyyyyyyyy", "{#INSTANCE_NAME}":"test2" }, { "{#INSTANCE_ID}":"i-zzzzzzzzzz", "{#INSTANCE_NAME}":"test3" } ] }
  37. UserParameterͰLLDͨ͠ • https://github.com/youyo/zabbix-userparameter-script-aws-integration • EC2, RDS(and cluster)ͷdiscoveryʹରԠ • cloudwatchͷϝτϦΫε΋औΕΔ •

    zabbixͷtemplate͸࡞੒த • loadable modulesͱൺֱͯ͠ফඅϦιʔε΍଎౓ͷ໘ͰྼΔ͕͠ΐ͏͕ͳ ͍
  38. ͍͍͍ͩͨײ͡ʹ ͳ͖ͬͯͨ

  39. ͍ͭͰʹzabbix؀ڥΛ ࡞Γ௚͢͜ͱʹͨ͠

  40. Zabbix3.4 Full containernized on k8s

  41. ·ͱΊ • zabbix͸UserParameterͱLoadale modulesΛ࢖͏͜ͱͰࣗ༝ʹ֦ுͰ͖Δ • ཉ͍͠ػೳ͸ࣗ෼Ͱ࡞͍͚ͬͯ͹͍͍ • ผʹzabbixʹݻࣥ͢Δඞཁ͸ͳ͍ • mackerel࠷ߴͩ͠prometheus΋࠷ߴ

    • zabbix࠷ߴ ࠓޙ΍͍ͬͯ͘ • Infrastracture as code • zabbix΋ίʔυͰ؅ཧ͍ͨ͠ • terraformͷpluginΛॻ͍͍ͯͧ͘
  42. ͓·͚: prometheusࡶײ • prometheus΋zabbixͷΑ͏ͳ؂ࢹπʔϧ • ΠϯετʔϧͳͲ͸όΠφϦஔ͚ͩ͘ͰࡁΉͷͰָ • Service discoveryͷػೳ͕͋Γɺࠓճ࡞੒ͨ͠Α͏ͳػೳ͸ॳΊ͔Β͍࣋ͬͯ ΔɻΫϥ΢υωΠςΟϒɻ

    • ίϯςφͷ؂ࢹ΋͍͍ײ͡ʹߦ͑ΔɻcAdvisorͱͷ࿈ܞ͕ૉ੖Β͍͠ɻ • σʔλͷอଘظؒ͸σϑΥϧτ15೔ɻ௕ظอଘ(long term storage)͸ਪ঑͞ Εͳ͍ɻͨͩ͠࠷ۙ֎෦ετϨʔδͱͯ͠influxdb͕αϙʔτ͞Εͨؾ͕͢Δ ͷͰཁݕূ • Ͱ΋ͦ΋ͦ΋؂ࢹσʔλͷ௕ظอଘ͕ඞཁͳέʔεͬͯ͋·Γͳ͍ؾ͕ͯ͠Δɻ (zabbixʹ׳Ε͗ͨ͢) • Ξϥʔτϧʔϧͷهड़͕େมͦ͏ɻzabbixͷ΄͏͕ݸਓతʹ͸Θ͔Γ΍͍͢ɻ • ΞϥʔτपΓΛ͏·͍͜ͱ΍ΕΔΑ͏ʹͳͬͨΒଈಋೖ͍ͨ͠Ϩϕϧ
 (zabbixͱ྆ํ͏·͘࢖͍෼͚͍͖͍ͯͨ)