それでも僕はzabbixと生きていく

Transcript

1. ͦΕͰ΋๻͸Zabbixͱ ੜ͖͍ͯ͘
2. ࣗݾ঺հ • ੴᖒ௚ਓ(Naoto Ishizawa) • twitter: @youyo_ • github: https://github.com/youyo • Cloud Architect @ heptagon inc. • I love golang, python, ruby • ࠷ۙϋϚ͍ͬͯΔ΋ͷ: 
 υϥΰϯϘʔϧυοΧϯότϧ
 LRଙޛ൧ͷݩΛखʹೖΕͨ͜ͱ͕࠷ۙҰ൪خ͔ͬͨ͠
3. mackerel޷͖Ͱ͔͢ʁ
4. mackerelͱ͸ • ΤϯδχΞΛϫΫϫΫͤ͞Δʮ௚ײతαʔόʔ؂ࢹαʔϏεʯ
 https://mackerel.io/ja/ • Monitoring as a service
5. ๻͔Βݟͨmackerelͷ͍͍ͱ͜Ζ • ͍͍ײ͡ʹcloudwatch͔Βσʔλऔ͖ͬͯͯ͘ΕΔ • ͖Ε͍ͳUI • ϑϧϚωʔδυ • ๛෋ͳNotification
6. ࠷ߴ
7. mackerelͷಛʹ͍͍ͱ͜Ζ • CloudWatch APIͱͷγʔϜϨεͳ࿈ܞ IUUQTNBDLFSFMJPKBGFBUVSFTBXTJOUFHSBUJPO
8. ࠷ߴ
9. ๻͔ΒݟͨmackerelͷؾʹͳΔͱ͜Ζ • 1,800ԁ ϗετ/month
 ->2,000ϗετ؂ࢹ͢ΔͱͳΔͱ360ສԁ
 ->ࣗલͰzabbixͰ؂ࢹ͢Ε͹30ສ͔͔Βͳ͍(౰ࣾൺ) • ϑϧϚωʔδυ
 ->ָͳΜ͚ͩͲͭ·Βͳ͍ʂ
 ->๻΋mackerelΈ͍ͨͳ؂ࢹγεςϜ࡞Γ͍ͨʂʂ
 (ӡ༻͕࢓ࣄͳΜͰͦͷࠜװͱͳΔ΋ͷ͸ࣗલͰ࣋ͭ΂͖)
10. mackerelʹ͋ͬͯzabbix ʹͳ͍΋ͷΛิ͑͹͍͍
11. ๻͔Βݟͨmackerelͷ͍͍ͱ͜Ζ • ͍͍ײ͡ʹcloudwatch͔Βσʔλऔ͖ͬͯͯ͘ΕΔ • ͖Ε͍ͳUI • ϑϧϚωʔδυ • ๛෋ͳNotification
12. ͜Ε͔Β΋Zabbixͱੜ͖͍ͯͨ͘Ίʹ ZabbixͰ͍͍ײ͡ʹ࢖͑Δ AWS Integration࡞Ζ͏
13. Zabbixͱ͸
14. Zabbixͱ͸ • Zabbixͱ͸ɺαʔόʔɺωοτϫʔΫɺΞϓϦέʔγϣϯΛ؂ࢹ͢ΔͨΊ ͷιϑτ΢ΣΞͰ͢ɻZabbix͸ओʹҎԼͷ3ͭͷػೳΛ༗͍ͯ͠·͢ɻ • αʔό΍ωοτϫʔΫʹ઀ଓ͞ΕͨσόΠεΛ؂ࢹ͢Δ؂ࢹػೳ • ऩूͨ͠σʔλΛ΋ͱʹάϥϑԽɺωοτϫʔΫϚοϓͷ࡞੒Λߦ͏ά ϥϑΟΧϧදࣔػೳ • ऩूͨ͠σʔλʹᮢ஋Λઃఆ͠ɺো֐/෮چ࣌ʹ؅ཧऀʹ௨஌Λߦ͏ো֐ ݕ஌/௨஌ػೳ IUUQXXX[BCCJYKQEPDVNFOUTXIBU@JT@[BCCJY
15. े෼ͳػೳΛ͍࣋ͬͯΔ
16. ZabbixͷؾʹͳΔͱ͜Ζ • Ϋϥ΢υϑΝʔετͳ͜ͷ࣌͝ੈɺ͍͍ͪͪΠϯελϯεΛzabbixʹϗετ ొ࿥ͱ͔ͨ͘͠ͳ͍ • ࣗಈొ࿥΋Ͱ͖Δɻ͚Ͳࣗಈ࡟আ͸… • zabbix-agentΛΠϯετʔϧͰ͖ͳ͍αʔϏεͷ؂ࢹͲ͏͢Δʁ
17. Ͳ͏΍Δʁ • Ϋϥ΢υϑΝʔετͳ͜ͷ࣌͝ੈɺ͍͍ͪͪΠϯελϯεΛzabbixʹϗετ ొ࿥ͱ͔ͨ͘͠ͳ͍ • ࣗಈొ࿥΋Ͱ͖Δɻ͚Ͳࣗಈ࡟আ͸…
 =>LowLevelDiscovery
 • zabbix-agentΛΠϯετʔϧͰ͖ͳ͍αʔϏεͷ؂ࢹͲ͏͢Δʁ
 =>LowLevelDiscovery
18. Low Level Discovery
19. LLD (Low Level Discovery) • “ϩʔϨϕϧσΟεΧόϦʹΑΓɺίϯϐϡʔλʔ্ͷछʑͷཁૉʹରͯ͠ ΞΠςϜɺτϦΨʔɺάϥϑΛࣗಈతʹ࡞੒Ͱ͖·͢ɻྫ͑͹ɺZabbix͸ ࢖༻͍ͯ͠ΔϚγϯ্ͷϑΝΠϧγεςϜ·ͨ͸ωοτϫʔΫΠϯλʔϑΣʔ εͷ؂ࢹΛࣗಈతʹ։࢝Ͱ͖·͢ɻͦͷͨΊʹɺ֤ϑΝΠϧγεςϜ·ͨ͸ ωοτϫʔΫΠϯλʔϑΣʔεʹରͯ͠खಈͰΞΠςϜΛ࡞੒͢Δඞཁ͸͋ Γ·ͤΜɻ͞Βʹɺఆظతʹ࣮ࢪ͞ΕΔσΟεΧόϦͷ࣮ࡍͷ݁Ռʹج͍ͮ ͯෆཁͳཁૉΛࣗಈతʹ࡟আ͢ΔΑ͏ʹɺZabbixΛઃఆ͢Δ͜ͱ͕Ͱ͖· ͢ɻ” IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMEJTDPWFSZMPX@MFWFM@EJTDPWFSZ
20. LLD (Low Level Discovery) • ͍͍ײ͡ʹ؂ࢹର৅Λݟ͚ͭͯ • ͍͍ײ͡ʹσʔλूΊͯ • ͍͍ײ͡ʹ؂ࢹର৅ফ͑ͨΒϗετొ࿥࡟আ΋ͯ͘͠ΕΔ
 
 “Ϣʔβʔ͸ɺಛఆͷJSONϓϩτίϧʹै͏ͱ͍͏৚݅ͰɺಠࣗͷλΠϓ ͷσΟεΧόϦΛఆٛͰ͖·͢ɻ” IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMEJTDPWFSZMPX@MFWFM@EJTDPWFSZ
21. ࠷ߴ
22. Ͳ͏͢Ε͹͍͍ʁ • UserParameterͰεΫϦϓτΛ࣮ߦ͢Δ • Loadable modulesΛ࡞੒͠ɺݺͼग़͢
23. UserParameter & Loadale modules • UserParameter
 “ر๬ͷσʔλͷݕࡧΛߦ͏ίϚϯυΛॻ͖ɺΤʔδΣϯτઃఆϑΝΠϧͷ ϢʔβʔύϥϝʔλʢʮUserParameterʯઃఆύϥϝʔλʣʹؚΊΔ͜ͱ͕ Ͱ͖·͢ɻ”
 >UserParameter=<key>,<command> • Loadable modules
 “loadable modules͸ɺZabbixͷػೳΛ֦ு͢ΔͨΊͷɺੑೳΛߟྀͨ͠ બ୒ࢶͱͳΓ·͢ɻZabbix 2.2͔Β͸ɺੑೳΛଛͶΔ͜ͱͳ͘ɺZabbixΤʔ δΣϯτ͓ΑͼαʔόʔΛ֦ு͢ΔͨΊͷloadable modules͕αϙʔτ͞ ΕΔΑ͏ʹͳΓ·ͨ͠ɻ” IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMDPOpHJUFNTVTFSQBSBNFUFST
    IUUQTXXX[BCCJYDPNEPDVNFOUBUJPOKQNBOVBMDPOpHJUFNTMPBEBCMFNPEVMFT
24. Loadable modulesͰ AWS Integration Ͱ͖ΔΑ͏ʹ͠Α͏
25. Ͱ΋ࠔͬͨ͜ͱ͕… • loadable modules͸CݴޠͰॻ͘ඞཁ͕͋Δ • ๻ɺCݴޠ͸ॻ͚ͳ͍…
26. g2z
27. What is "g2z”? • https://github.com/cavaliercoder/g2z • loadable modulesΛgolangͰॻ͚Δ. (golangͰshared libraryͱͯ͠ੜ੒) • Zabbix version 2.xܥ, 3.0ରԠ package main import ( "strings" "github.com/cavaliercoder/g2z" ) func main() { panic("THIS_SHOULD_NEVER_HAPPEN") } func Echo(request *g2z.AgentRequest) (string, error) { return strings.Join(request.Params, " "), nil } func init() { g2z.RegisterStringItem("go.echo", "Hello world!", Echo) }
28. উͬͨʂ
29. Ͱ΋ݱ࣮͸؁͘ͳ͔ͬͨ
30. ͳ͔ͥ͏·͘ಈ͔ͳ͍
31. Loadable modulesΛ ݺͼग़͢ͱzabbix-agent ϓϩηε͕ͨ·ʹݻ·Δ
32. IUUQTHJUIVCDPNDBWBMJFSDPEFSH[JTTVFT
33. ҙ༁ • ֎෦ʹhttpϦΫΤετΛߦ͏ͱɺݻ·Δ͜ͱ͕͋Δ • golang͸ѱ͘ͳ͍ • zabbix΋ѱ͘ͳ͍ • goroutine͕ؔ܎ͯ͠Δͷ͔ͳʁ • single coreͷϚγϯͰͳΒਖ਼ৗʹಈͬ͘Ά͍Αʁ
 ->͍΍ಈ͔ͳ͍.. • ͪΐͬͱαϙʔτ͖͠Εͳ͍Θʔ
34. ٧Μͩ…
35. UserParameterʹ ੾Γସ͍͑ͯ͘
36. UserParameterͰLLD͢Δʹ͸ • ϓϩάϥϜॻ͍ͯ΋shell-scriptͰ΋ԿͰ΋͍͍ • ܾ·ͬͨϑΥʔϚοτ(json)ͰσʔλΛฦ͢ඞཁ͕͋Δ { "data":[ { "{#INSTANCE_ID}":"i-xxxxxxxxxx", "{#INSTANCE_NAME}":"test1" }, { "{#INSTANCE_ID}":"i-yyyyyyyyyy", "{#INSTANCE_NAME}":"test2" }, { "{#INSTANCE_ID}":"i-zzzzzzzzzz", "{#INSTANCE_NAME}":"test3" } ] }
37. UserParameterͰLLDͨ͠ • https://github.com/youyo/zabbix-userparameter-script-aws-integration • EC2, RDS(and cluster)ͷdiscoveryʹରԠ • cloudwatchͷϝτϦΫε΋औΕΔ • zabbixͷtemplate͸࡞੒த • loadable modulesͱൺֱͯ͠ফඅϦιʔε΍଎౓ͷ໘ͰྼΔ͕͠ΐ͏͕ͳ ͍
38. ͍͍͍ͩͨײ͡ʹ ͳ͖ͬͯͨ
39. ͍ͭͰʹzabbix؀ڥΛ ࡞Γ௚͢͜ͱʹͨ͠
40. Zabbix3.4 Full containernized on k8s
41. ·ͱΊ • zabbix͸UserParameterͱLoadale modulesΛ࢖͏͜ͱͰࣗ༝ʹ֦ுͰ͖Δ • ཉ͍͠ػೳ͸ࣗ෼Ͱ࡞͍͚ͬͯ͹͍͍ • ผʹzabbixʹݻࣥ͢Δඞཁ͸ͳ͍ • mackerel࠷ߴͩ͠prometheus΋࠷ߴ • zabbix࠷ߴ ࠓޙ΍͍ͬͯ͘ • Infrastracture as code • zabbix΋ίʔυͰ؅ཧ͍ͨ͠ • terraformͷpluginΛॻ͍͍ͯͧ͘
42. ͓·͚: prometheusࡶײ • prometheus΋zabbixͷΑ͏ͳ؂ࢹπʔϧ • ΠϯετʔϧͳͲ͸όΠφϦஔ͚ͩ͘ͰࡁΉͷͰָ • Service discoveryͷػೳ͕͋Γɺࠓճ࡞੒ͨ͠Α͏ͳػೳ͸ॳΊ͔Β͍࣋ͬͯ ΔɻΫϥ΢υωΠςΟϒɻ • ίϯςφͷ؂ࢹ΋͍͍ײ͡ʹߦ͑ΔɻcAdvisorͱͷ࿈ܞ͕ૉ੖Β͍͠ɻ • σʔλͷอଘظؒ͸σϑΥϧτ15೔ɻ௕ظอଘ(long term storage)͸ਪ঑͞ Εͳ͍ɻͨͩ͠࠷ۙ֎෦ετϨʔδͱͯ͠influxdb͕αϙʔτ͞Εͨؾ͕͢Δ ͷͰཁݕূ • Ͱ΋ͦ΋ͦ΋؂ࢹσʔλͷ௕ظอଘ͕ඞཁͳέʔεͬͯ͋·Γͳ͍ؾ͕ͯ͠Δɻ (zabbixʹ׳Ε͗ͨ͢) • Ξϥʔτϧʔϧͷهड़͕େมͦ͏ɻzabbixͷ΄͏͕ݸਓతʹ͸Θ͔Γ΍͍͢ɻ • ΞϥʔτपΓΛ͏·͍͜ͱ΍ΕΔΑ͏ʹͳͬͨΒଈಋೖ͍ͨ͠Ϩϕϧ
 (zabbixͱ྆ํ͏·͘࢖͍෼͚͍͖͍ͯͨ)