はてなが歩んできた "Infrastructure as Code"の道のりとMackerelが果たす役割について

Slide 1

Slide 1 text

Open Cloud Innovation Festa 2016 17 Sep 2016 Hatena Co., Ltd. Hiromichi Sugiyama はてなが歩んできた "Infrastructure as Code"の道のりとが果たす役割について

Slide 2

Slide 2 text

Hiromichi Sugiyama / ਿࢁ ޿௨ 株式会社はてなサービス・システム開発本部 Mackerelチームプロデューサー経歴 - 大手製造業のIT事業会社 - 大規模環境でのインフラエンジニア - オープンソースのモニタリングツール - SaaSシステム - ITILベースのサービスマネジメント - はてなでMackerel事業へ携わる [email protected]

Slide 3

Slide 3 text

ΞδΣϯμ 1. ͸ͯͳʹ͍ͭͯ 2. ͸ͯͳͷΠϯϑϥͱMackerel͕͸ͨ͢໾ׂ 3. Infrastructure as Code 4. Mackerelͷ͝঺հ 5. MackerelͷϏδϣϯ

Slide 4

Slide 4 text

1 ͸ͯͳʹ͍ͭͯ

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

ձࣾ֓ཁ 所在地京都（烏丸御池）東京（表参道）資本⾦ 185,713千円市場東証マザーズ設⽴ 2001年7⽉（16年⽬）創業者近藤淳也社⻑栗栖義⾂

Slide 7

Slide 7 text

͸ͯͳͷϛογϣϯ

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

͸ͯͳͷαʔϏε

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

ଞʹ΋ଟ਺ͷαʔϏε͋Γ

Slide 15

Slide 15 text

2 ͸ͯͳͷΠϯϑϥͱ Mackerel͕Ռͨ͢໾ׂ

Slide 16

Slide 16 text

16೥ͷ ྺ࢙

Slide 17

Slide 17 text

marqs-60 CMX-50 ࣗ࡞αʔόʔ࣌୅ʢͦͷੲɿ਺೥લʣ

Slide 18

Slide 18 text

ϋΠϒϦου؀ڥʢݱࡏʣ ΦϯϓϨ σʔληϯλʔ ύϒϦοΫ Ϋϥ΢υ ਺ઍ୆ͷେن໛؀ڥͰαʔϏεશମΛӡӦ

Slide 19

Slide 19 text

େن໛ͳαʔϏε؀ڥ

Slide 20

Slide 20 text

ΠϯϑϥΤϯδχΞ͸ ਺໊

Slide 21

Slide 21 text

͍͔ʹޮ཰Α͘ ࣋ଓతʹ؅ཧ͢Δ͔ʁ

Slide 22

Slide 22 text

Dynamic Infrastructure Platformʢੈͷைྲྀʣ • ΑΓಈతͰศརʹͳΔΠϯϑϥ؀ڥ

Slide 23

Slide 23 text

ଘࡏ͢Δଟ਺ͷαʔϏε&ϗετ ਺ेͷαʔϏε ਺ઍͷϗετ

Slide 24

Slide 24 text

๲େͳ૊Έ߹Θͤ΁ͷରॲ

Slide 25

Slide 25 text

αʔϏεɾϩʔϧɾϗετ Ұ؏ͨ͠؅ཧ֓೦Λ࠾༻ Service Role Host

Slide 26

Slide 26 text

Service Role Host ͸ͯͳ Bookmark app proxy DB DB1 DB2 app3 app2 app1 proxy1 proxy2 ద༻ྫɿ͸ͯͳBookmark

Slide 27

Slide 27 text

Θ͔Γқ͞ͱརศੑΛ ਺ेͷαʔϏε ਺ඦͷϩʔϧ ਺ઍͷϗετ

Slide 28

Slide 28 text

։ൃ΍ӡ༻ͱͷؔΘΓ͸ʁ

Slide 29

Slide 29 text

DevOps Lifecycle • ։ൃͱӡ༻͕ӬଓతͳαΠΫ ϧΛ੒͢ • ଟछଟ༷ͳޮ཰Խπʔϧ΍ αʔϏε͕ଘࡏ • దࡐదॴ • ޷Έ Mystery of DevOps Devops Lifecycle http://www.edureka.co/blog/mystery-of-devops/

Slide 30

Slide 30 text

DevOps Tools • దࡐదॴɺ༏Εͨπʔϧ͕ଟ਺ 'FFECBDL %FWFMPQ 5FTU %FQMPZ .POJUPS

Slide 31

Slide 31 text

Mackerel͕Ռͨ͢໾ׂ

Slide 32

Slide 32 text

logo (2) ݸผʹ͸ศརͰ͋Δ͕ɻɻɻ • ಈతͳΠϯϑϥ؀ڥΛͦΕͧΕఆٛͯ͠ѻ ͏ͷ͔ʁ Monitoring Server Configuration Deploy ʁ

Slide 33

Slide 33 text

logo (2) ϗετϨδετϦͱͯ͠ͷMackerel • ϗετ৘ใ͸Mackerel΁ू໿ • πʔϧؒΛૄ݁߹ʢศརͳAPIʣ Monitoring Server Configuration Deploy چMackerel

Slide 34

Slide 34 text

logo (2) ߋͳΔਐԽʢݱࡏʣ • MackerelΛ࣠ͱͨ͠ಈత؅ཧΛ࣮ݱ droot Monitoring & Host Registry Server Configuration Deploy Infrastructure Test SSH-Operation Infrastructure DNS Registry tinydns

Slide 35

Slide 35 text

ϗετϨδετϦ͸ඇৗʹ؆୯ ϗετ ᶃ ϗετىಈ࣌ʹ ඞͣαʔϏεͱϩʔϧΛηοτ͠ MackerelΤʔδΣϯτΛىಈ͢Δ ᶄ Mackerelʹϗετ৘ใ͕ొ࿥͞ΕΔ ᶅ APIͰಈతʹϗετ৘ใΛऔΓग़͠ ͯ࠶ར༻ ᶃ ᶄ ᶅ ʁ

Slide 36

Slide 36 text

3 Infrastructure as Code

Slide 37

Slide 37 text

͜Ε·ͰͷಓͷΓΛ ৼΓฦΓ ڞ௨͍ͯͨ͠Ұͭͷ͜ͱ

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

Infrastructure as Code • ΠϯϑϥΛίʔυͰએݴతʹهड़͢Δ͜ͱʹΑΓɺ ͍··Ͱιϑτ΢ΣΞ։ൃͰ༗ޮੑཱ͕ূ͞Ε͖ͯͨ ϕετϓϥΫςΟεΛΠϯϑϥ΁΋ద༻͢Δ Infrastructure as Code Managing Servers in the Cloud By Kief Morris Publisher: O'Reilly Media Final Release Date: June 2016

Slide 40

Slide 40 text

Infrastructure as CodeͷԸܙ • ಈతͳΠϯϑϥ΁ͷରॲ • ਝ଎ʹɺԿ౓Ͱ΋ɺ࣮֬ʹಉ͡มߋ͕ՄೳʢετϨε͕ͳ͍ʣ • ো֐͔Βͷ෮چ଎౓ • ίʔυϨϏϡʔ • ୯७࡞ۀ͔Β཭ΕɺՁ஋͋Δ͜ͱʹ͕͚࣌ؒ͞ΔΑ͏ʹͳΔ

Slide 41

Slide 41 text

ϞμϯͳΠϯϑϥπʔϧͷཁૉ 7$4 5FTU 0SDIFT USBUJPO 5%% $* $%

Slide 42

Slide 42 text

͸ͯͳͰͷҰྫ

Slide 43

Slide 43 text

Dev Ops Service A Service B Service C ͸ͯͳͷ։ൃͱӡ༻ͷମ੍ • Ops͸Ұͭͷ૊৫͕ͩɺαʔϏε͝ͱʹ୲౰ΛΞαΠϯ͍ͯ͠Δ

Slide 44

Slide 44 text

Dev Ops Service A Service B Service C ։ൃͱӡ༻ͷମ੍ • Ops͸Ұͭͷ૊৫͕ͩɺαʔϏε͝ͱʹ୲౰ΛΞαΠϯ͍ͯ͠Δ Webアプリケーションエンジニア

Slide 45

Slide 45 text

Dev Ops Service A Service B Service C ։ൃͱӡ༻ͷମ੍ • Ops͸Ұͭͷ૊৫͕ͩɺαʔϏε͝ͱʹ୲౰ΛΞαΠϯ͍ͯ͠Δ Webアプリケーションエンジニア Webオペレーションエンジニア

Slide 46

Slide 46 text

ίʔυ؅ཧ͢Δར఺ʢҰྫʣ • ίʔυϨϕϧͰՄࢹԽ͞Ε͍ͯΔͷͰɺ؆୯ʹӽڥͰ͖Δ ΞϓϦέʔγϣϯ ϛυϧ΢ΣΞ OS ϓϥοτϑΥʔϜ Dev Ops DBʹਫ਼௨͍ͯ͠ΔOpsΤϯδχΞ͕SQL ͷϨϏϡʔΛͨ͠Γ DevΤϯδχΞ͕ϛυϧ΢ΣΞΛࢼͨ͢ ΊͷChefϨγϐΛॻ͍ͨΓ ϛυϧ΢ΣΞͷΦϓγϣϯߏ੒Λཧղ ͠ͳ͕ΒίʔυΛॻ͍ͨΓ ↓ϙΠϯτ

Slide 47

Slide 47 text

։ൃͱӡ༻ͷ੹೚ൣғ • άϨΠκʔϯΛ࡞Β͓ͣޓ͍ଚॏ͠ͳ͕Βॿ͚߹͍͑ͯΔ ΞϓϦέʔγϣϯ ϛυϧ΢ΣΞ OS ϓϥοτϑΥʔϜ Dev Ops

Slide 48

Slide 48 text

Performance Working Group (͓·͚) • αʔϏε΁ίϛοτͯ͠Δશϝϯόʔ͕ఆظʢεϓϦϯτຖʣʹੑೳ෼ੳΛ࣮ࢪ ʢMackerelΛ׆༻ʣ • த௕ظͷϦϦʔεӨڹͳͲ෼ੳ͠ɺ։ൃ΍ΠϯϑϥͷΩϟύγςΟܭը΁ଈ࣌൓ө ΞϓϦέʔγϣϯ ϛυϧ΢ΣΞ OS ϓϥοτϑΥʔϜ Dev Ops

Slide 49

Slide 49 text

4 Mackerelͷ͝঺հ

Slide 50

Slide 50 text

Mackerelͱ͸ʁ • "SaaSܕ"αʔόʔ؂ࢹαʔϏε • ΤϯδχΞ͕ཉ͔ͬͨ͠zϫΫϫΫz͢ΔΑ ͏ͳػೳΛఏڙ͠ɺѹ౗తͳzޮ཰ԽzΛ࣮ݱ • ͸ͯͳͷओྗαʔϏεΛ͑͞͞ଓ͚͖ͯͨ ʢ࠷ߴͷυοάϑʔσΟϯάʣ

Slide 51

Slide 51 text

2014.9.17ਖ਼ࣜϦϦʔε ʢຊ೔ͪΐ͏Ͳ2प೥ʂʂʣ

Slide 52

Slide 52 text

ΞʔΩςΫνϟʔ • ΤʔδΣϯτ͔Β ϝτϦοΫΛ౤ߘ )5514 3&45"1*

Slide 53

Slide 53 text

௚ײతͳϢʔβʔମݧ

Slide 54

Slide 54 text

؆୯ʹ௚͙͸͡ΊΒΕΔ • ΨΠυʹै͏͚ͩ • ͨͬͨ਺෼ mackerel-agent written in Golang

Slide 55

Slide 55 text

࠷খͷϧʔϧ͕࠷ߴͷརศੑΛ • ΤʔδΣϯτΛ্ཱͪ͛Δ͚ͩͰ • ࣗಈతʹ؂ࢹ͕࢝·Γ • ಉ͡ϩʔϧͷϗετ͸ಉ͡άϥϑ΁

Slide 56

Slide 56 text

ৗʹਐԽ͠ଓ͚ΔαʔϏε 毎週新機能のリリースを継続中陳腐化せず常に進化 ि࿈ଓ

Slide 57

Slide 57 text

બ͹ΕΔཧ༝ • ಋೖʗUI͕؆୯Ͱ෼͔Γ΍͍͢ • ։ൃεϐʔυ • ϚϧνΫϥ΢υͰ࢖͑Δ • ೔ຊޠαϙʔτ͕ॆ࣮ • NewRelicͱͷซ༻

Slide 58

Slide 58 text

αʔϏεϓϥϯ

Slide 59

Slide 59 text

ओͳػೳ

Slide 60

Slide 60 text

1.Ξϥʔτ௨஌ʹάϥϑ • Slack, HipChat, TypeTalk, Yammer

Slide 61

Slide 61 text

2.URL֎ܗ؂ࢹ • ར༻ऀࢹ఺ͷHTTP/HTTPS؂ࢹ • ֎෦͔ΒͷϨεϙϯελΠϜΛՄࢹԽ • εςʔλείʔυɾϨεϙϯελΠϜͰΞϥʔτ • ΩʔϫʔυͷଘࡏνΣοΫ • SSLূ໌ॻͷ༗ޮظݶΛνΣοΫ Web Service URL

Slide 62

Slide 62 text

3.؂ࢹϧʔϧͷίʔυԽ • zInfrastructure as Codez • mkr monitor diff/pull/push • JSONܗࣜͰϧʔϧΛѻ͏

Slide 63

Slide 63 text

4.؂ࢹϧʔϧΛGitͰ؅ཧ

Slide 64

Slide 64 text

5.ϝτϦοΫΛॊೈʹՄࢹԽʢࣜʣ 差分前週との重ねあわせ最大最大・平均・最小合計

Slide 65

Slide 65 text

̒ɽAWS Integration • AWS CloudWatch API͔Β༷ʑͳϝτϦοΫ΍Πϕ ϯτΛऔಘ͠ՄࢹԽ • 1୆ͷϗετͱͯ͠؅ཧ • EC2,ELB,RDS,ElastiCache͔Βॱ֦࣍େ API Amazon CloudWatch Amazon RDS Amazon ELB

Slide 66

Slide 66 text

̓ɽ৬ਓܳΛΞϧΰϦζϜͰࢧԉ 過去の傾向から将来を予測して警告 $SJUJDBM 将来 5SFOET ϱ݄લ 現在

Slide 67

Slide 67 text

5 MackerelͷՌͨ͢໾ׂ ʢࠓޙ΋ʣ

Slide 68

Slide 68 text

Ϋϥ΢υԽͷྲྀΕ • ։ൃπʔϧ͸Ϋϥ΢υ΁ • svn/git → GitHub (GitHub Enterprise) • Jenkins → TravisCI, CircleCI • ࣮ߦ؀ڥ΋Ϋϥ΢υ΁ • ΦϯϓϨ → AWS, GCP, Azure,SoftLayer

Slide 69

Slide 69 text

Mackerel΁ͷྲྀΕ • ӡ༻πʔϧ΋Ϋϥ΢υ΁ • Zabbix, Nagios → Cloudwatch, NewRelic and Mackerel !! • ࢖͍΍͍͢UIʹΑΔɺΑΓ҆ఆͨ͠ӡ༻Λ

Slide 70

Slide 70 text

Infrastructure as Code • ϗετઃఆΛίʔυԽʢChef/puppetʣ • Mackerelͷઃఆ΋CodeͰ؅ཧ • ؂ࢹϧʔϧ/μογϡϘʔυͷίʔυԽ • ॱ࣍ɺAPIԽΛਪਐ

Slide 71

Slide 71 text

Mackerelͷະདྷ • All-in-OneαʔϏεͰ͸ͳ͍ • ༷ʑͳπʔϧ/αʔϏεͱͷ࿈ܞ • Slack/GitHub/Reactio • γεςϜ؅ཧͷίΞπʔϧͱͯ͠ͷ Mackerel

Slide 72

Slide 72 text

DevOpsͷத֩ͱͯ͠ • ϥΠϑαΠΫϧશମͷޮ཰ԽΛଅਐ %FW 0QT

Slide 73

Slide 73 text

MackerelͷϏδϣϯ ΤϯδχΞΛϫΫϫΫͤ͞Δ ʮ௚ײతαʔόʔ؂ࢹαʔϏεʯ

Slide 74

Slide 74 text

͞ΒͳΔਐԽ

Slide 75

Slide 75 text

ϗετϨδετϦͷڧԽ • ϗετͳͲͷ೚ҙͷཁૉʹKey-Value஋ΛAPI౳Ͱઃ ఆՄೳʹ • Πϯετʔϧ͞Εͨύοέʔδ৘ใ • ࢿ࢈؅ཧ൪߸ ۙ೔ϦϦʔε

Slide 76

Slide 76 text

ΞϧΰϦζϜࢧԉ • ౷ܭ΍ػցֶशͷཧ࿦ΛԠ༻ • ҟৗݕ஌ • ༧ଌ • ෼ྨ

Slide 77

Slide 77 text

αʔϏε੒௕܏޲ʢϗετ७૿ʣ ٸ੒௕

Slide 78

Slide 78 text

࣍ੈ୅Mackerel • ੒௕Λࢧ͑Δɺ͋ΒͨͳΞʔΩςΫ νϟ΁ͷมֵʂ • ͝ظ଴Լ͍͞ʂ