新卒がZabbix3でサーバ監視を冗長化!?H2Oにも移行した！

by miyazawaiti

Slide 1

Slide 1 text

৽ଔ͕ZABBIX3Ͱαʔό؂ࢹΛ ৑௕Խ!?H2Oʹ΋Ҡߦͨ͠ʂ ITI GROUP INFRA SRE TEAM HIKARU MIYAZAWA

Slide 2

Slide 2 text

ɾ̏ɾ AGENDA 1. ࣗݾ঺հ 2. ZABBIX3.0Ͱ؂ࢹΛ৑௕Խͨ͠ܦҢ 3. H2OΛ࠾༻ͯ͠Α͔ͬͨ͜ͱ 4. ZABBIX×H2O 5. ·ͱΊ

Slide 3

Slide 3 text

ɾ̏ɾ HIKARU MIYAZAWA ɾ2016/10ʙ2017/3  גࣜձࣾITIͷΠϯλʔϯɺ  Πϯϑϥͷۀ຿Λܦݧ ɾ2017/4ʙࠓ·Ͱ  גࣜձࣾITIೖࣾ ΠϯϑϥSREνʔϜ झຯɿυϥϜɺ݈߁ΦλΫɺήʔϜɺ  ޷͖ͳςΩετΤσΟλɿSublime Text

Slide 4

Slide 4 text

ɾ̏ɾ ZABBIX3.0Ͱ؂ࢹΛ৑௕Խͨ͠ܦҢ ▸ ฐࣾ؂ࢹαʔό͸zabbix2.2ɺࣗࣾͰߏங,ӡ༻ ▸ 6݄ʹ͘͞Βͷϝϯςφϯε͕ൃද ▸ ϝϯςφϯεத͸؂ࢹΛܧଓͰ͖ͳ͍ ▸ ͜ͷػձʹ͔ͤͬ͘ͳͷͰ2୆ମ੍Ͱ৑௕Խ

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

ɾ̏ɾ ZABBIX3.0 ▸ ׬શແྉͰΦʔϓϯιʔεͷ ؂ࢹιϑτ΢ΣΞ ‣ LAMP؀ڥʴzabbix-serverͰಈ͘ ▸ ؂ࢹର৅αʔόʹ͸ɺ zabbix-agentΛΠϯετʔϧ ▸ ฐࣾͷ؂ࢹର৅αʔόɿ200୆ʙ

Slide 8

Slide 8 text

ɾ̏ɾ ޷͖ʹ؂ࢹɺ௨஌ʂࣗ༝ʂ εΫϦʔϯͰՄࢹԽ →ϦΞϧλΠϜͰো֐ঢ়گ೺Ѳ Slack΍νϟοτϫʔΫ΁ͷ ௨஌ઃఆ΋؆୯

Slide 9

Slide 9 text

ɾ̏ɾ GMO ConoHa ͷ࠾༻ཧ༝ ▸ AWSଆͷো֐ͷӨڹΛड͚ͳ͍ ▸ ZabbixͷεϖοΫཁ݅Λຬ͍ͨͭͭ҆͠ʂ(2GB,3ίΞ,1700ԁ/݄) ▸ Disk͸SSDͰ50GB ※௥ՃDisk͸ߴ͍Ͱ͢(ຖ݄/2400ԁ)

Slide 10

Slide 10 text

ɾ̏ɾ OSͱϛυϧ΢ΣΞ ▸ CentOS6.9(Final) ▸ Apache2.2.15 ▸ MySQL5.7.19 ▸ PHP7.0.18 ▸ zabbix-server3.0.10-1 ▸ (zabbix-agent3.0.10-1)

Slide 11

Slide 11 text

ZABBIXΠϯετʔϧ͸  ؆୯ͳͷͰ    ׂѪ͍͖ͤͯͨͩ͞·͢ʂ

Slide 12

Slide 12 text

ɾ̏ɾ طଘͷZABBIX؂ࢹαʔόͷઃఆΛ৽αʔόʹ΋൓ө ▸ ZABBIXผόʔδϣϯҠߦ࣌ͷ஫ҙ ▸ ɾτϦΨʔͷ৚݅ࣜͷมߋ ▸ ɾෆ࠾༻ʹͳͬͨΞΠςϜ ▸ ɾো֐࣌ΞΫγϣϯͷઃఆ͕exportग़དྷͳ͍ ▸ ɾexportͨ͠xmlϑΝΠϧ͸ॻ͖׵͕͑ඞཁ

Slide 13

Slide 13 text

ͦͯ͠͏·͘ ͍ͬͨͷ͔ʂʁ

Slide 14

Slide 14 text

▸ ͘͞Βͷϝϯςφϯεલʹ৑௕Խ׬ྃ ▸ ΄ͱΜͲ͋ͩͪΜ͞Μ͕΍Γ·ͨ͠ ▸ ݟ͍͑ͯͨ՝୊ ▸ DBͷఆظతͳόοΫΞοϓ ▸ zabbix͸DB͕ੜ͖͍ͯΕ͹༰қʹ෮چͰ͖Δɻ ɾ̏ɾ ॕʂ؂ࢹ৑௕Խʂʂ

Slide 15

Slide 15 text

ɾ̏ɾ ϝϯςφϯε3೔લͷҰίϚ ▸ ▸ ɹɹɹɹɹɹɹɹɹɹɹ๻ʮఆظతʹDBΛόοΫΞοϓ͢ΔγΣϧ,ॻ͚·ͨ͠ʂʯ ▸ ▸ ɹ͋ͩͪΜ͞Μʮςετͯ͠Έͨʁ  ɹɹɹɹɹɹɹɹͪΐͬͱ֬ೝͯ͠ΈΑ͏ʯ(ΧλΧλ)

Slide 16

Slide 16 text

▸ mysqldump -u user -x -p --all-databases   < mysqldump.sql Μʁ

Slide 17

Slide 17 text

—all-databases   < mysqldump.sql ͋……

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

ɾ̏ɾ ͓෼͔Γ௖͚ͨͩΖ͏͔ ▸ mysqldump -u user -x -p --all-databases < mysqldump.sql ▸ ۭͬΆΛͿͪࠐΉ ۭͬΆ ZABBIXͷDB

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

ɾ̏ɾ ຊདྷ͸͜Ε ▸ mysqldump -u user -x -p --all-databases > mysqldump.sql ۭͬΆ ZABBIXͷDB

Slide 22

Slide 22 text

࣍ͷ೔ɺ͋ͩͪΜ͞Μ͕  ޕલதͰ௚͠·ͨ͠ɻ(ૣ͗͢)  (ࣗ෼͸༗څͰͨ͠ww)

Slide 23

Slide 23 text

ɾ̏ɾ εΫϦʔϯͷ࡞੒ ▸ ཁ๬ɿো֐࣌ʹ؂ࢹσʔλΛάϥϑͰ͙͢ʹݟ͍ͨʂ ▸ ΍Γ·͢ʂ೚ͤͯԼ͍͞ʂ͋Εʁ ▸ ฐࣾͷ؂ࢹର৅ϗετ͸200Ҏ্ ▸ 1ϗετʹ͖ͭ12ݸͷ؂ࢹ߲໨ ▸ खಈͰϙνϙν͢Δ৔߹ ▸ 200ϗετ×12߲໨×4ΫϦοΫʹ9600…໿Ұສճͷ࡞ۀ…

Slide 24

Slide 24 text

ɾ̏ɾ ਖ਼௚͠ΜͲ͍…

Slide 25

Slide 25 text

ɾ̏ɾ ޮ཰Α͘΍Ζ͏ʂ

Slide 26

Slide 26 text

ɾ̏ɾ ZABBIXͷઃఆ͸IMPORT/EXPORTͰ͖Δ ZabbixͷઃఆʢҰ෦ʣ ɹɹɹɹɹɹɹɹɹɹɹɾάϧʔϓɹɹɹɹɹ  ɹɹɹɹɹɹɹɹɹɹɹɹɹɾςϯϓϨʔτ  ɹɹɹɹɹɹɹɹɹɹɾϗετ  ɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɾωοτϫʔΫϚοϓ  ɹɹɹɹɹɹɹɹɹɹɹɾΠϝʔδ  ɹɹɹɹɹɹɹɹɹɹɹɹɾεΫϦʔϯ XMLorJSONܗࣜ

Slide 27

Slide 27 text

ɾ̏ɾ εΫϦʔϯ͜Μͳײ͡Ͱ͢

Slide 28

Slide 28 text

ɾ̏ɾ εΫϦʔϯͷXMLͱ࣮ࡍͷWEB্

Slide 29

Slide 29 text

ɾ̏ɾ XMLϑΝΠϧΛࣗಈੜ੒͢ΔSHELLΛ࡞੒

Slide 30

Slide 30 text

͜ ͜ Ͱ໰୊͕ ʂ

Slide 31

Slide 31 text

ɾ̏ɾ ͬͦ͘஗͍ ▸ ϦΞϧλΠϜͰߋ৽͞ΕΔ120ݸͷάϥϑͷදࣔ͸΍ͬͺΓ ॏ͔ͬͨɻ

Slide 32

Slide 32 text

ͱͳΔͱNGINX ͕͍͍ͷ͔ʁ

Slide 33

Slide 33 text

▸ ݩDeNAͷԞҰึࢯ͕։ൃɻ ▸ HTTP/2ʹରԠͨ͠ര଎ͷWEBαʔόʔͰ͋Δ ▸ ͋ͩͪΜ͞Μਪ͠(ΏΔ;ΘΩϟϯύʔ͞Μ΋) ɾ̏ɾ H2Oͷ࠾༻

Slide 34

Slide 34 text

ɾ̏ɾ HTTP/1.1ͷͲ͕͜ѱ͍ͷʁ ‣ ʮHTTPύΠϓϥΠϯʯ  ˠલճϦΫΤετͷ׬ྃΛ଴ͨͣʹ࣍ͷϦΫΤετΛૹ৴͢Δ͜ͱ͕Մೳ ‣ ʮαʔόʔ͸ϦΫΤετͷॱ൪௨ΓʹϨεϙϯεΛฦ͞ͳ͚Ε͹ͳΒͳ͍ʯ  →݁ہॱ൪଴͕ͪൃੜ͠ɺ  ஗Ԇ͍ͯͨ͠ɻ

Slide 35

Slide 35 text

ɾ̏ɾ HTTP/2ͰԿ͕มΘΔ͔ ‣ ଟॏԽ ɾಉ࣌ʹ100Ҏ্ͷϦΫΤετΛൃߦՄೳ ɾλΠϛϯάɺϨεϙϯεͷॱং͕ؔ܎ͳ͍  ɾෳ਺ฒྻॲཧ →HTTP/1.1Ͱͷॱ൪଴͕ͪͳ͍ ‣ ϔομѹॖ(HTTP/2ͷϔομ͸େ͖͍300~500B)  ྫʣ100ճϦΫΤετ͕དྷͯ΋ɺຖճ300όΠτͷ  ϔομΛૹΒͳͯ͘ࡁΉ →খ͞ͳϦΫΤετΛԿճૹͬͯ΋஗͘ͳΒͳ͍

Slide 36

Slide 36 text

ɾ̏ɾ ͦΜͳHTTP/2Λ࠷େݶʹ׆͔ͤΔ͔ ▸ ࣮͸HTTP/2ʹରԠ͔ͨ͠Βͱ͍ͬͯɺ  αʔόଆͷઃఆ͕ͪΌΜͱ͍ͯ͠ͳ͍ͱ  ͍͍ͱ͜ΖΛ׆͔ͤͳ͍ ▸ྫ͑͹…

Slide 37

Slide 37 text

ɾ̏ɾ ༏ઌ౓੍ޚ͕͍͢͝ ▸ ༏ઌ౓੍ޚ  WEBϖʔδΛදࣔ͢ΔͷʹඞཁͳϑΝΠϧΛͲͷॱ൪Ͱ༏ઌతʹಡΈࠐΉ͔Λ͖Ί ͓ͯ͘  ᶃHTMLᶄCSSᶅJSᶆը૾ᶇը૾ᶈը૾ ▸ ΫϥΠΞϯτଆ  →Firefox͸ݡ͍  →Chrome͸͋Δ͚Ͳશવ׆͔͍ͤͯͳ͍  Safari,IE-Edge,ͦͷଞͷϒϥ΢β͸༏ઌ౓ͷ֓೦͢Βແ͠ɻɻ  αʔόଆͰͳΜͱ͔͢Δɻɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹɹ

Slide 38

Slide 38 text

ɾ̏ɾ ༏ઌ౓੍ޚ͕͍͢͝ ▸ ʮhttp2-reprioritize-blocking-assetsʯ ▸ CSSͱJSΛ࠷༏ઌͰϨεϙϯεΛฦ͢ ▸ HTMLΛฦ͢ʢ͜ͷ࣌఺ͰɺWEBϖʔδͷϨϯμϦϯά͕ ։࢝Ͱ͖Δʣ ▸ ը૾Λฦ͢ʢฒྻॲཧͳͷͰॱ൪ΛؾʹͤͣҰؾʹૹΔʣ ▸ ※nginxͰ΋ؤுΕ͹ಉ͡Α͏ͳઃఆ͕ग़དྷΔΒ͍͠

Slide 39

Slide 39 text

ɾ̏ɾ αʔόϓογϡ ▸ αʔόϓογϡ ▸ ϦΫΤετΛઌಡΈͯ͠ඞཁͳϑΝΠϧΛૹΔ  ྫʣHTMLͷϦΫΤετड͚ͨΒCSS΋·ͱΊͯૹΔ ▸ Ωϟογϡ༗ແΛ֬ೝͰ͖ΔͷͰɺແବʹૹΔ͜ͱ͸ͳ͍

Slide 40

Slide 40 text

Ͳ͏ͤઃఆ͢Δͷ ΊΜͲ͍͘͞Μ͡Ό Ζʁ

Slide 41

Slide 41 text

ɾ̏ɾ ͳΜͱνϡʔχϯά͸σϑΥϧτͰON ▸ ී௨ʹ্ཱͪ͛ͨॠ͔ؒΒര଎ʂ ▸ νϡʔχϯάʹ͔͚ͯͨ࣌ؒΛଞͷλε Ϋʹ࢖͑Δ

Slide 42

Slide 42 text

ɾ̏ɾ APACHE ઃఆϑΝΠϧ

Slide 43

Slide 43 text

ɾ̏ɾ H2O ઃఆϑΝΠϧ ▸ h2o.conf

Slide 44

Slide 44 text

ɾ̏ɾ ▸ zabbix.conf

Slide 45

Slide 45 text

ɾ̏ɾ ෛՙςετ ▸ AWS EC2Πϯελϯεt2.micro ▸ Apache2.4(νϡʔχϯάσϑΥϧτ) ▸ H2O2.2(νϡʔχϯάσϑΥϧτ) ▸ 30ສϦΫΤετɺಉ࣌઀ଓ1000ϢʔβͰݕূɻ ▸ ςετπʔϧ͸Goad

Slide 46

Slide 46 text

ɾ̏ɾ ෛՙςετ݁Ռ ▸ Apache    1෼3.65ඵͰϑϦʔζ  ▸ H2O    ໿20ඵͰશϦΫΤετΛࡹ͘

Slide 47

Slide 47 text

ɾ̏ɾ ෛՙςετ݁Ռ Apache-2.4 H2O-2.2 300,000Requests 300,000Requests 1,000user 1,000user 1m3.65s 20.0s

Slide 48

Slide 48 text

ɾ̏ɾ ෛՙςετ݁Ռ Apache-2.4 H2O-2.2 300,000Requests 300,000Requests 1,000user 1,000user 1m3.65s 20.0s Freeze… Done!

Slide 49

Slide 49 text

࣮ࡍʹH2Oͷ  ͢͝͞Λ͓ݟͤ͠·͢

Slide 50

Slide 50 text

ɾ̏ɾ H2O×HTTP/2ͷ͗͢͢͝Δฒྻॲཧೳྗ

Slide 51

Slide 51 text

ɾ̏ɾ ZABBIX3.0×H2O×PHP7.0 ▸ H2O  →ฒྻॲཧͰૣ͍ɺ࠷ॳ͔Βνϡʔχϯά͕͞ΕͯΔ  Մಡੑͷߴ͍ઃఆϑΝΠϧ ▸ PHP7.0΋ૣ͍ ▸ ฐࣾͷ୲౰αʔϏεʹ΋ಋೖ͢Δ͖͔͚ͬʹͳͬͨɻ ▸ JoinsJob(δϣΠϯζδϣϒ),ITϓϩδΣΫτ໊ؑ,kiii(डୗ)

Slide 52

Slide 52 text

ɾ̏ɾ ·ͱΊ ▸ ؂ࢹ͸ࣗࣾͰߏங  ˠίετେ෯μ΢ϯˠ͔͠͠ӡ༻͕େม ▸ ؂ࢹ͸AWSҎ֎Ͱ৑௕Խ͢Δͷ͕͓͢͢Ί  ˠAWSଆͷωοτϫʔΫো֐͕ى͖ͯ΋ࠔΔ͜ͱ͸ͳ͍ ▸ ͜Ε͔ΒͷWEBαʔό͸H2OʹมΘ͍͖ͬͯͦ͏ ▸ Prometheusʹ௅ઓ͍ͨ͠

Slide 53

Slide 53 text

͝ਗ਼ௌ  ͋Γ͕ͱ͏͍͟͝· ͨ͠ʂ