KibanaではじめるDashboard

by Harukasan

Slide 1

Slide 1 text

KibanaͰ͸͡ΊΔDashboard Harukasan / MICHII Shunsuke 2014-09-08  サーバ/インフラエンジニア養成読本ログ収集〜可視化編  出版記念！執筆者が語る大講演会！

Slide 2

Slide 2 text

Harukasan / MICHII Shunsuke Infrastructure team in pixiv Inc. since 2012 ! Works  Contents Delivery / Monitoring / Server Management Server side infrastructure  nginx / Apache TrafﬁcServer / PHP / Ruby Data Analytics  Fluentd / Kibana / Google BigQuery @harukasan shunsuke.michii

Slide 3

Slide 3 text

Agenda • pixivʹ͓͚Δσʔλ෼ੳ • DashboardΛͭ͘Ζ͏ • ͔ͭͬͯ͘Βͷ͜ͱ

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

pixiv ͜͜ʹpixivͷઆ໌͕ೖΔ ʮ͓ֆ͔͖͕΋ͬͱָ͘͠ͳΔ৔ॴʯΛ໨ࢦ͍ͯ͠Δ ΠϥετίϛϡχέʔγϣϯαʔϏε

Slide 6

Slide 6 text

pixivʹ͓͚Δσʔλ෼ੳ

Slide 7

Slide 7 text

pixivͷ։ൃαΠΫϧ • σϓϩΠճ਺: ฏۉ17ճ/೔ • ຖिͳʹ͔͠Β৽͍͠ػೳ͕ϦϦʔε͞ΕΔ • ৽͍͠ػೳͷޮՌΛݕূ͍ͨ͠ • ࢥ͍͕͚ͳ͍มԽ͕ͳ͍͔

Slide 8

Slide 8 text

pixivʹ͓͚Δσʔλ෼ੳ • άϩʔενʔϜ • αʔϏεͷάϩʔεͷͨΊʹͳΜͰ΋΍Δ • ༷ʑͳࢪࡦͷݕূͷͨΊʹϩάΛ࢖༻͍ͯ͠Δ ! • pixivͰ࣮ࡍʹى͍ͬͯ͜Δ͜ͱΛௐ΂Δ • ͲͷΑ͏ͳΠϥετ͕Ӿཡ͞Ε͍ͯΔ͔ • ͲͷΑ͏ͳݕࡧ͕ߦΘΕ͍ͯΔ͔

Slide 9

Slide 9 text

pixivʹ͓͚Δσʔλ෼ੳ • ։ൃνʔϜ • σʔλ෼ੳ͸ܧଓతσϦόϦʔͷखஈ • ػೳΛϦϦʔεͨ͠ͱ͖ʹςετ͢Δ • ਖ਼͍࣮͠૷͕ߦ͑ͨͷ͔ • ༧૝֎ͷޮՌ͕ͳ͍ͷ͔

Slide 10

Slide 10 text

pixivʹ͓͚Δσʔλ෼ੳ • ͳͥՄࢹԽΛ͸͡Ίͨͷ͔ • ࣾ಺ͰFluentd/MongoͰՄࢹԽπʔϧΛͭͬͨ͘ • ܾ·ͬͨ஋Λه࿥͚ͭͮ͠Δ • ͦΕͳΓʹ࢖ΘΕ͍ͯͨ

Slide 11

Slide 11 text

pixivʹ͓͚Δσʔλ෼ੳ • ;ͱKibanaΛ஌Δ • ͳΜ͔͓΋͠Ζͦ͏ • σʔλΛྲྀ͠ࠐΜͰΈΔͱ݁ߏ͓΋͠Ζ͍ • ͦͷ͏ͪଞͷਓ͕࢖͍࢝Ίͨ • ো֐͕ى͜ΔͱౖΒΕΔΑ͏ʹͳͬͨ

Slide 12

Slide 12 text

pixivʹ͓͚Δσʔλ෼ੳ • ﬂuent-plugin-elasticsearch͔Βlogstash-*Ҏ֎ͷ  ΠϯσοΫεʹग़ྗग़དྷΔΑ͏ʹͨ͠ • ͳͥlogstash_preﬁxͱ͍͏໊લʹͯ͠͠·ͬͨͷ͔……

Slide 13

Slide 13 text

Kibana

Slide 14

Slide 14 text

Query/Filter and Panels • Query: ϩάΛݕࡧ • Filter: ͢΂ͯͷΫΤϦʹద༻͞ΕΔ • Panel: ΫΤϦͷݕࡧ݁ՌΛ༷ʑͳํ๏Ͱදࣔ͢Δ

Slide 15

Slide 15 text

• ຊͰͨ͘͞Μઆ໌ͨ͠ Panels

Slide 16

Slide 16 text

Dashboard

Slide 17

Slide 17 text

Dashboard • ݟ͍ͨ਺஋͕1ը໘ʹऩ·͍ͬͯΔ • 1ͭͷσʔλʹରͯ͠ෳ਺ͷ໘͔Β෼ੳ͢Δ • ຖ೔/ຖिܧଓͯ͠มԽΛݟΔ͜ͱ͕ग़དྷΔ • ͜ͷσʔλʹ͍ͭͯݟΔʹ͸͜ͷը໘ΛݟΕ͹ྑ͍ ! • ΠϯϑϥΤϯδχΞ΍։ൃऀ͕࢖༻͍ͯͨ͠  ϩάσʔλʹ୭Ͱ΋ΞΫηεͰ͖ΔΑ͏ʹ͢Δ

Slide 18

Slide 18 text

ྫ͑͹

Slide 19

Slide 19 text

ྫ͑͹ • ελϯϓػೳ௥Ճͷࡍʹͭ͘ΒΕͨ • ͲͷΑ͏ͳ༻్Ͱ࢖ΘΕ͍ͯΔ͔ • ίϝϯτ਺ʹ͸ͲͷΑ͏ͳมԽ͕͔͋ͬͨ • ຖिϛʔςΟϯά࣌ʹ֬ೝ͢Δ

Slide 20

Slide 20 text

DashboardΛͭ͘Ζ͏

Slide 21

Slide 21 text

༻ҙ͢Δ΋ͷ • ద౰ͳαʔό: SSD 256GBx3/ϝϞϦ32GB • 5,000,000 records/day ·Ͱ͸༨༟ • EFKελοΫ • Elasticsearch • Fluentd • Kibana • ϩάϑΝΠϧ • ۭ͖࣌ؒ

Slide 22

Slide 22 text

ͭ͘Γ͔ͨ 1. ElasticsearchΛηοτΞοϓ 2. FluentdͰϩάΛૹΓ࢝ΊΔ 3. μογϡϘʔυΛͭ͘Δ  Elasticsearch Fluentd Application Load Balancer Database ΞΫηεϩά Πϕϯτϩά  ੜσʔλ  Τϥʔϩά

Slide 23

Slide 23 text

Ͳ͏͍͏ϩάΛ࢖͏͔ • αʔόʹอଘ͞Ε͍ͯΔΠϕϯτϩά • ूܭࡁΈͷσʔλ͸೉͍͠ • ΠϯϑϥνʔϜͱ͕ه࿥͍ͯͨ͠ϩάΛଞͷ༻్ʹ΋࢖͑ΔΑ͏ʹ ΞΫηεϩά • ϦΫΤετͨ͠URL • ૹ৴ݩIPΞυϨε • ϨεϙϯελΠϜ/ϦΫΤετॲཧ࣌ؒ ! Πϕϯτϩά • Ϣʔβ͕ߦͬͨΠϕϯτ(ϩάΠϯ/౤ߘ/ϒοΫϚʔΫ)ͷৄࡉ৘ใ • Πϕϯτͷछྨຖʹߏ଄Խ͞Ε͍ͯΔ • ࠷ॳʹՄࢹԽը໘Λͭͬͨ͘ͱ͖ʹ༻ҙͨ͠

Slide 24

Slide 24 text

Πϕϯτϩά • ༧Ίߏ଄Խσʔλ͕͋Δͱศར • ΋ͱ΋ͱ෼ੳπʔϧ༻ͷϩά͕͋ͬͨ ! • type • ϦΫΤετ৘ใ • ϦϞʔτΞυϨε • UA • ϦϑΝϥ • ࠃ৘ใ • Πϕϯτͷৄࡉ • ର৅ίϯςϯτͷID

Slide 25

Slide 25 text

ม׵ॲཧ • FluentdͰ࠷௿ݶͷม׵ॲཧΛߦ͓ͬͯ͘ • ूܭॲཧ͸ߦΘͳ͍ ! fluent-plugin-geoip • IPΞυϨεΛࠃ৘ใʹม׵͢Δ fluent-plugin-rewrite-tag-filter • ϩάͷग़ྗઌΠϯσοΫεΛৼΓ෼͚Δ • ෆඞཁͳϩάΛল͘

Slide 26

Slide 26 text

DashboardΛͭ͘Δ ͖ͬ͞ͷίϝϯτμογϡϘʔυΛྫʹߟ͑Δ ! 1. جຊ͸HistogramͱTable 2. ϩάΛϑΟϧλϦϯά 3. ͱΓ͋͑ͣద౰ʹ͍ΖΜͳύωϧΛฒ΂Δ 4. Share͢Δ

Slide 27

Slide 27 text

Histogram / Table • ώετάϥϜͱςʔϒϧΛஔ͍͚ͨͩ • ·ͣ͸ϑΟϧλϦϯάͯ͠ίϝϯτؔ࿈ͷσʔλΛநग़

Slide 28

Slide 28 text

σϓϩΠΛه࿥͢Δ • σϓϩΠεΫϦϓτʹ1ߦ௥Ճ͢Δ͚ͩ • σϓϩΠޙʹͲͷΑ͏ͳมԽ͕͔͋ͬͨΘ͔Γ΍͍͢ curl -XPOST -H 'Content-type: text/json' http://localhost:9200/deploy/deploy/  -d "{'project':'pixiv', '@timestamp':'$(date +"%Y-%m-%dT%H:%I:%S%z")', 'message':'tag'}"

Slide 29

Slide 29 text

Terms • ؾʹͳ͍ͬͯΔ஋Λूܭ্ͯ͠ҐN݅Λදࣔ • ๮άϥϑɾԁάϥϑͰදࣔͰ͖Δ • TableܗࣜͩͱϘλϯͰϑΟϧλϦϯάͰ͖Δ

Slide 30

Slide 30 text

Text • MarkdownͰςΩετΛॻ͚Δ • ஫ҙࣄ߲ͱ͔ɺຌྫͱ͔Λද͓ࣔͯ͘͠ • Ͳ͏͍͏σʔλ͔ɺͲ͏͍͏࢖͍ํΛ͢Δ͔

Slide 31

Slide 31 text

׬੒

Slide 32

Slide 32 text

͔ͭͬͯ͘Βͷ͜ͱ

Slide 33

Slide 33 text

Share͢Δ • ShareϘλϯͰҰ࣌URLΛੜ੒ͯ͠νϟοτͱ͔ʹషΔ • ͪΌΜͱग़དྷͨΒอଘͯٞ͠ࣄ࿥ͱ͔ʹష͓ͬͯ͘

Slide 34

Slide 34 text

͔ͭͬͯ͘Βͷ͜ͱ • ຖ೔/ຖि࣌ؒΛͱͬͯมԽΛ؍࡯͢Δ • ͨͩϦϯΫΛ͸Δ͚ͩͩͱ୭΋ݟͯ͘Εͳ͍ • ΈΜͳͰมԽʹ͍ͭͯٞ࿦͢Δ࣌ؒΛͭ͘Δ • ৽͍͠վળํ๏΍ΞΠσΞʹͭͳ͕Δ • ՄࢹԽج൫ʹର͍ͯ͠ΖΜͳจ͕۟Ͱͯ͘Δ

Slide 35

Slide 35 text

࢖ͬͯΔͱग़ͯ͘Δ໰୊ • PVΛͦͷ··ಥͬࠐΉͱܭࢉϦιʔε͕଍Γͳ͘ͳΔ • TermsɺTrendsͷύωϧΛฒ΂Δͱ͔ͳΓॏ͍ • ௕ظؒʹ౉ΔσʔλΛ࢖͍͍ͨ • ෳࡶͳूܭॲཧ΍ϑΟϧλϦϯάΛ࢖͍͍ͨ

Slide 36

Slide 36 text

Kibana with BigQuery • BigQueryͰूܭ/ϑΟϧλϦϯάΛߦ͏ • σʔλΛBulk APIͰElasticsearchʹग़ྗ • JenkinsΛ࢖ͬͯఆظతʹ࣮ߦ͢Δ  Elasticsearch Fluentd  Google BigQuery Jenkins

Slide 37

Slide 37 text

ଞͷ෼ੳπʔϧΛ࢖͏ • ·ͣ͸KibanaͰ͸͡ΊͯΈΔ • ਺िؒ෼ͷσʔλͰ͋Ε͹؆୯ʹՄࢹԽͰ͖Δ • Kibana͕ۤखͳ͜ͱ΋Θ͔ͬͯ͘Δ • ͨ͘͞Μͷ࣠ʹ౉Δ౷ܭ৘ใ • ௕ظؒʹ౉Δσʔλ • େྔͷσʔλͷूܭॲཧ • KibanaͷͨΊʹͭͬͨ͘ΠϯϑϥΛଞͷπʔϧʹల։

Slide 38

Slide 38 text

ଞͷ෼ੳπʔϧʹల։  Elasticsearch Fluentd  Google BigQuery Jenkins Tableau HRForecast

Slide 39

Slide 39 text

·ͱΊ • ͱΓ͋͑ͣKibanaΛ࢖͍࢝ΊͯΈΔ • ՄࢹԽΛ͸͡ΊΔͱ͍ΖΜͳ໰୊఺͕ग़ͯ͘Δ • αʔϏεʹ͓͚Δ໰୊఺ʹ͋Θͤͯ෼ੳπʔϧΛల։