Slide 1

Slide 1 text

ϞϯελʔετϥΠΫͷ؂ࢹπʔϧࠓੲ෺ޠ Լר ࠤ౻ྑ༞ גࣜձࣾϛΫγΟ ։ൃຊ෦ SREάϧʔϓ

Slide 2

Slide 2 text

!2 ࣗݾ঺հ

Slide 3

Slide 3 text

!3 ࣗݾ঺հ ● ࠤ౻ ྑ༞ (͞ͱ͏ Γΐ͏͚͢) ● גࣜձࣾϛΫγΟ։ൃຊ෦SREάϧʔϓ ● 2017೥৽ଔೖࣾ

Slide 4

Slide 4 text

!4 ԼרͰ࿩͢͜ͱ • ؂ࢹ؀ڥΛݱ୅Խ͢ΔͨΊͷݱࡏͷऔΓ૊Έ • ݱঢ়ͷ໰୊఺ʹ͍ͭͯͷ੔ཧ • Nagios͔ΒPrometheus΁ͷmigrationʹ͍ͭͯ • In-Progress ͳ࿩ • Future work

Slide 5

Slide 5 text

!5 ͯ͞ɺΈͳ͞Μ
 ؂ࢹ؀ڥ΁ͷෆຬΛ ࣋ͬͯ·ͤΜ͔?


Slide 6

Slide 6 text

!6 ࢲ͸ͱ͘ʹ ࣋ͬͯ·ͤΜͰͨ͠

Slide 7

Slide 7 text

!7 ͱ͍͏͔ʮ؂ࢹʯͬͯͳʹ?

Slide 8

Slide 8 text

!8 ͦΜͳͱ͖ʹʮೖ໳ ؂ࢹʯൃച https://www.oreilly.co.jp/books/9784873118642/

Slide 9

Slide 9 text

!9 ʮ؂ࢹʯΛงғؾͰ΍͍ͬͯΔ ࣗ෼ʹ͸ݴޠԽ͞ΕΑ͘੔ཧͰ͖ͨ

Slide 10

Slide 10 text

!10 ͞͞Δɻࢗ͞Δͧʙ 8.3 SNMP ୯౛௚ೖʹݴΘ͍ͤͯͩ͘͞ɻ4/.1Λαʔόʔ؂ࢹʹ࢖͏ͷ͸ ΍Ί·͠ΐ͏ ೖ໳؂ࢹ: 8.3 SNMP P.115 ˔֤ϝτϦΫεΛ෼͋Δ͍͸ͦΕΑΓ௕ִ͍ؒͰ؂ࢹ͍ͯ͠Δ ˔ϝτϦΫεͷཤྺΛอଘ͍ͯ͠ͳ͍ /BHJPTɺ͓લͷ͜ͱͩ ೖ໳؂ࢹ: 1.3 Ξϯνύλʔϯ3: νΣοΫϘοΫε؂ࢹ P.11

Slide 11

Slide 11 text

!11 ݱঢ়ͷ໰୊఺

Slide 12

Slide 12 text

!12 ݱঢ়ͷ໰୊఺ ● ؂ࢹαʔόʔ͕εέʔϧ͠ͳ͍ɺՄ༻ੑ͕ͳ͍ ● ؂ࢹ͢Διϑτ΢ΣΞ͕ݹ͍ɺґଘ౓͕େ͖͍ ○ SNMP ○ Nagios ○ libmysqlclientͷґଘ໰୊ ● ϝτϦΫεͷཤྺ͕ͳ͍ ● ϝτϦΫεͷऔಘִ͕ؒ5෼ͱ௕͍

Slide 13

Slide 13 text

!13 ݱঢ়ͷ໰୊఺ ● ؂ࢹαʔόʔ͕εέʔϧ͠ͳ͍ɺՄ༻ੑ͕ͳ͍ ● ؂ࢹ͢Διϑτ΢ΣΞ͕ݹ͍ɺґଘ౓͕େ͖͍ ○ SNMP ○ Nagios ○ libmysqlͷґଘ໰୊ ● ϝτϦΫεͷཤྺ͕ͳ͍ ● ϝτϦΫεͷऔಘִ͕ؒ5෼ͱ௕͍ ೴ཪʹͪΒ͍ͭͯ͘Δ͜ͷݴ༿ /BHJPTɺ͓લͷ͜ͱͩ 4/.1Λαʔόʔ؂ࢹʹ ࢖͏ͷ͸΍Ί·͠ΐ͏

Slide 14

Slide 14 text

!14 ؂ࢹαʔόʔ͕εέʔϧ͠ͳ͍ɺՄ༻ੑ͕ͳ͍ σʔληϯλʔ1 σʔληϯλʔ2

Slide 15

Slide 15 text

!15 ؂ࢹαʔόʔ͕εέʔϧ͠ͳ͍ɺՄ༻ੑ͕ͳ͍ σʔληϯλʔ1 σʔληϯλʔ2 σʔληϯλʔ2 ͷ؂ࢹ͕ࢭ·ͬͯ͠·͏ σʔληϯλʔ2 ͷNagios͕DOWN

Slide 16

Slide 16 text

!16 ؂ࢹαʔόʔ͕εέʔϧ͠ͳ͍ɺՄ༻ੑ͕ͳ͍ σʔληϯλʔ1 σʔληϯλʔ2 σʔληϯλʔ3

Slide 17

Slide 17 text

!17 ؂ࢹαʔόʔ͕εέʔϧ͠ͳ͍ɺՄ༻ੑ͕ͳ͍ σʔληϯλʔ1 σʔληϯλʔ2 σʔληϯλʔ3 σʔληϯλʔ͕૿͑Δͱ ͲΜͲΜ૿͍͑ͯ͘ Nagios

Slide 18

Slide 18 text

!18 ؂ࢹ͢Διϑτ΢ΣΞ͕ݹ͍ɺґଘ౓͕େ͖͍ % snmpget -v1 -c public localhost iso.3.6.1.2.1.1.1.0

Slide 19

Slide 19 text

!19 ؂ࢹ͢Διϑτ΢ΣΞ͕ݹ͍ɺґଘ౓͕େ͖͍ % snmpget -v1 -c public localhost iso.3.6.1.2.1.1.1.0

Slide 20

Slide 20 text

!20 ؂ࢹ͢Διϑτ΢ΣΞ͕ݹ͍ɺґଘ౓͕େ͖͍ % snmpget -v1 -c public localhost iso.3.6.1.2.1.1.1.0 iso.3.6.1.2.1.1.1.0 = STRING: "Linux ip-172-31-12-13 4.15.0-1032-aws #34-Ubuntu SMP Thu Jan 17 15:18:09 UTC 2019 x86_64"

Slide 21

Slide 21 text

!21 ؂ࢹ͢Διϑτ΢ΣΞ͕ݹ͍ɺґଘ౓͕େ͖͍ ● ؂ࢹ͢ΔͨΊʹඞཁͳ libmysqlclient ● OSΛΞοϓάϨʔυ͢Δͱόʔδϣϯ্͕͕ͬͯ͠·͏… ● ࣗ෼ͨͪͷͨΊʹόοΫϙʔτ͢Δ??

Slide 22

Slide 22 text

!22 ͦ͜Ͱ Prometheus ͷಋೖ

Slide 23

Slide 23 text

!23 Ͳ͏ͯ͠Prometheusͳͷ? ● SaaSαʔϏεͰ͸ ○ զʑ͕؂ࢹ͍߲ͨ͠໨਺ΑΓରԠ͕গͳ͔ͬͨ ○ ୆਺͕ଟ͍ͨΊɺίετ͕ߴͭ͘͘... ○ զʑࣗ਎Ͱ͍Ζ͍Ζϝϯς͍͖͍ͯͨ͠ ● ΦϯϓϨɺΫϥ΢υͱִͨΓͳ͘؂ࢹ͍ͨ͠

Slide 24

Slide 24 text

!24 Prometheus ͱ͸

Slide 25

Slide 25 text

!25 Prometheus ● Prometheus https://github.com/prometheus/prometheus ● SoundCloudʹΑͬͯ։ൃɺOSSԽɺCNCFͷϓϩδΣΫτ ● Googleͷ಺෦ͷ؂ࢹγεςϜBorgman͔ΒΠϯεύΠΞ ● ؂ࢹ͢ΔͨΊͷexporter΋ॆ࣮ ● Nagiosͱಉ͘͡Pullܕͷ࢓૊Έ

Slide 26

Slide 26 text

!26 NagiosΛ໓΅ͨ͢Ίʹ

Slide 27

Slide 27 text

!27 NagiosΛ໓΅ͨ͢Ίʹ ● node-exporter / mysqld-exporter ͸ެࣜΛ࢖͏ ● ΧελϜϝτϦΫεܥ͸exporterΛࣗ࡞ ● ؂ࢹ؀ڥͷߴՄ༻ੑ ● AlertmanagerʹΑΔΞϥʔτ؂ࢹͷઃఆ ● Grafana DashboardΛ͏·͍͜ͱ࡞Δ ● Prometheusͷconfigࣗಈੜ੒ ● NagiosͱPrometheusͷซ༻ ● NagiosΛ໓΅͢ ● snmpdΛ໓΅͢

Slide 28

Slide 28 text

!28 NagiosΛ໓΅ͨ͢Ίʹ ● node-exporter / mysqld-exporter ͸ެࣜΛ࢖͏ ● ΧελϜϝτϦΫεܥ͸exporterΛࣗ࡞ ● ؂ࢹ؀ڥͷߴՄ༻ੑ ● AlertmanagerʹΑΔΞϥʔτ؂ࢹͷઃఆ ● Grafana DashboardΛ͏·͍͜ͱ࡞Δ ● Prometheusͷconfigࣗಈੜ੒ ● NagiosͱPrometheusͷซ༻ ● NagiosΛ໓΅͢ ● snmpdΛ໓΅͢

Slide 29

Slide 29 text

!29 ؂ࢹߏ੒ͷߴՄ༻ੑ σʔληϯλʔ1 σʔληϯλʔ2

Slide 30

Slide 30 text

!30 ؂ࢹߏ੒ͷߴՄ༻ੑ σʔληϯλʔ1 σʔληϯλʔ2 ڌ఺͝ͱʹPrometheusΛઃஔ • ڌ఺৑௕ • ઃఆ͸౷Ұ • Alertmaneger͕ॏෳഉআͯ͘͠ΕΔ(Β͍͠)

Slide 31

Slide 31 text

!31 Prometheusͷconfigࣗಈੜ੒ ● ݱঢ়ɺࣗಈతͳαʔϏεσΟεΧόϦ͸ͳ͍ ● چདྷͷ؂ࢹઃఆ͔ΒPrometheus༻ͷϑΝΠϧΛࣗಈੜ੒ servers:
 - config: application.yaml
 label: Application Servers
 hosts:
 - 192.168.0.1 app-1 app-1
 - 192.168.0.2 app-2 app-2
 - config: mysql.yaml
 label: DB
 hosts:
 - 192.168.0.10 db1 mysql master
 - 192.168.0.11 db2 mysql slave - targets:
 - 192.168.0.1:9100
 labels:
 datacenter: sample
 env: production
 hostname: app-1
 service: application ...
 - targets:
 - 192.168.0.10:9100
 - 192.168.0.10:9104
 labels:
 datacenter: sample
 env: production
 hostname: db1
 service: mysql
 αʔόʔʹԠͯ͡ LISTENઌΛม͍͑ͯ͘

Slide 32

Slide 32 text

!32 ͜Ε͔Β...

Slide 33

Slide 33 text

!33 ͜Ε͔Β…. ● طଘͷ؂ࢹ߲໨Λચ͍ग़͠؂ࢹઃఆΛ࠶ߏ੒ ● ऩूͨ͠σʔλͷؙΊ… ● Grafana DashboardΛߏங… ● Service Discovery ͷͨΊʹControl PlaneΛߏங ● etc…

Slide 34

Slide 34 text

!34 Զͨͪͷઓ͍͸ ͜Ε͔Βͩ ✊✊✊

Slide 35

Slide 35 text

!35 Զͨͪͷ஥ؒʹͳͬͯ ͘ΕΔਓΛืूதʂʂ https://career.xflag.com/career/engineer/914/

Slide 36

Slide 36 text

Thank you !36