Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
teratailの解析基盤をEFKで作っていろいろ楽しい話
Search
ikuwow
March 04, 2016
Technology
0
780
teratailの解析基盤をEFKで作っていろいろ楽しい話
teratailの解析基盤をEFKで作っていろいろ楽しい話 @ ゆとりエンジニア交流会
ikuwow
March 04, 2016
Tweet
Share
More Decks by ikuwow
See All by ikuwow
Elasticsearch on EC2からAmazon Elasticsearch Serviceに 移行してだいぶ楽になった話
ikuwow
0
3.3k
意外と使える! Alibaba Cloud
ikuwow
0
220
UNIXという考え方
ikuwow
1
1.8k
技術書紹介 パーフェクトPHP
ikuwow
0
2k
みんなもMiddlemanで技術ブログ作って幸せになろう!
ikuwow
0
910
PHPサイバーテロの技法 書籍紹介
ikuwow
0
830
Other Decks in Technology
See All in Technology
Tellus の衛星データを見てみよう #mf_fukuoka
kongmingstrap
0
190
Terraformあれやこれ/terraform-this-and-that
emiki
8
1.4k
Azure Container Apps + Bicep 〜 こんな感じで運用しています
kaz29
2
480
Janus
bkuhlmann
1
490
Azure犬駆動開発の記録/GlobalAzureFukuoka2024_20240420
nina01
1
210
複雑な構成要素を持つUIとの向き合い方 〜新・支出グラフでの実例〜 / B43 TECH TALK
nakamuuu
0
140
[新卒向け研修資料] テスト文字列に「うんこ」と入れるな(2024年版)
infiniteloop_inc
4
15k
ChatworkのSRE部って実は 半分くらいPlatform Engineering部かもしれない
saramune
0
160
KubeCon EU 2024 Recap “Kubernetes Policy Time Machine: Where to Next?”
ryysud
0
220
DevOpsメトリクスとアウトカムの接続にトライ!開発プロセスを通して計測できるメトリクスの活用方法
ham0215
2
240
Reducing Cross-Zone Egress at Spotify with Custom gRPC Load Balancing Recap
koh_naga
0
200
生産性向上チームの紹介
cybozuinsideout
PRO
1
870
Featured
See All Featured
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
221
21k
Building Your Own Lightsaber
phodgson
99
5.7k
Learning to Love Humans: Emotional Interface Design
aarron
267
39k
Become a Pro
speakerdeck
PRO
11
4.5k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
Code Reviewing Like a Champion
maltzj
514
39k
GraphQLとの向き合い方2022年版
quramy
32
12k
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
322
20k
Adopting Sorbet at Scale
ufuk
68
8.6k
How to name files
jennybc
65
93k
Principles of Awesome APIs and How to Build Them.
keavy
121
16k
Making the Leap to Tech Lead
cromwellryan
124
8.5k
Transcript
teratailͷղੳج൫Λ EFKͰ࡞ͬͯ ͍Ζ͍Ζָ͍͠ @ikuwow ϨόϨδʔζגࣜձࣾɹςΫϊϩδʔϝσΟΞϥϘ ΏͱΓੈΤϯδχΞަྲྀձʢ2016/03/04ʣ
ࣗݾհ • ϨόϨδʔζגࣜձࣾɺςΫϊϩδʔϝ σΟΞϥϘɺteratailͷ։ൃͯ͠Δਓɻ • ֶੜͷ࣌εϩʔΨϯגࣜձࣾͰ1.5͙ Β͍Πϯλʔϯͯͨ͠ • ίʔυॻ͘ͱ͖PHPͰ͕͢ɺϑϩϯτ ΠϯϑϥͬͨΓ͍Ζ͍ΖΓ·
͢ • ࠷ۙͬͨ͜ͱɿteratailͷϩάղੳج ൫࡞Δ @ikuwow
teratail ͬͯΔਓʙʁ
teratail • ΤϯδχΞɾϓϩάϥ ϚͷͨΊͷQ&AαΠτ • ຖ࣭͕70-80݅ • ճ93% • 3/17ʹϢʔβʔձʮू
·ͬtailʯୈ࢛ճ։࠵༧ ఆ
ࠓ͢͜ͱ • teratailͷϢʔβʔߦಈϩάΛEFKελοΫ (Elasticsearch, Fluentd, KibanaʣͰՄࢹԽ͢ ΔΈ࡞ͬͨ • ָ͍͠ʂ •
ਏ͍ʂʂ
ϢʔβʔͷߦಈΛݟ͍ͨʂ 1. ϦΞϧλΠϜʹࢹͯ͠ϦεΫݕͨ͠Γɺ ΧδϡΞϧʹ࠷ۙͷϢʔβʔͷಈ͖Λͬͨ Γ͍ͨ͠ʂ 2. KPIΛݟΔͷʹ࠷దԽͨ͠ܗͰσʔλΛ࣋ͬ ͯਂ͘ૣ͘ՄࢹԽ͍ͨ͠ ʢ͋ͱHiveQLॻ͘ͷΊΜͲ͍͍͔͘͢͠͝Βૣ͍ͷʹ͍ͨ͠ɾɾɾʣ
࡞ͬͨج൫ Amazon S3 Amazon Redshift ҹϩάͷྲྀΕ 1. ϦΞϧλΠϜՄࢹԽ 2. ਂ͘ՄࢹԽ
͏গ͚ͩ͠ৄ͘͠ node.master: false node.data: false node.master: true node.data: true node.master:
false node.data: false node.master: true node.data: true Amazon Redshift Amazon S3 teratailͷதͷਓ ४ϦΞϧλΠϜՄࢹԽ KPIΛਂ͘ՄࢹԽ όονॲཧ
Fluentdͱ • ϩάͷύʔεɺूΛ͢Δπʔ ϧ • TreasuredataʢຊͰΘΓͱ ਓؾʣ • Α͘Logstashͱൺֱ͞ΕΔ •
όοϑΝϦϯάݡͯ͘ɺ5͙ Β͍ࢭΊͯશ͘ͳ͍
Elasticsearchͱ • ࠷ۙྲྀߦΓͷશจݕࡧΤϯδϯɻ2ܥ ͕࠷৽ɻ • ElasticࣾʢLogstashͱಉ͡ʣ • ͖Ε͍ʹRESTfulͳAPIͰѻ͍͍͢ • ͱΓ͋͑ͣಉ͡ωοτϫʔΫʹஔ͍
͓͚ͯΫϥελ࡞ͬͯ͘ΕΔ • ࠷ۙAWS͕Elasticsearch Serviceͱ ͍͏ͷΛग़ͨ͠Γ
Kibanaͱ • ElasticsearchΛόοΫͱ͠ ͯɺͦΕΒͷσʔλΛ͔ͬ ͜ྑ͘ՄࢹԽ͢Δπʔϧ • nodeΞϓϦέʔγϣϯͳͷ Ͱಋೖָ͕͘͢͝ • ϚεϙνϙνͰϩά͕ݟΒ
ΕΔ
EFKελοΫͷಛ • Πϯετʔϧཧ͕ൺֱతΧϯλϯ • FluentdϫϯϥΠφʔ͚ͩͰ͍͚Δ • Elasticsearchউखʹ͏·͍͜ͱΫϥελ࡞ͬͯ͘ΕΔ • KibanaೖΕΔͷ؆୯ͩ͠ݟͨΒ͍͍ͩͨ͑Δ •
ͦͦ͜͜ރΕ͖ͯͨײ͋Δʁ • ࢼͯ͠ΈΔͱ͍͕͙͢͢͞
࡞ͬͯԿ͕มΘ͔ͬͨʁ • ϩά͕؆୯ʹૣ͔ͬ͘͜Α͘ݟΒΕΔ༷ʹͳͬͨ • ࣌ؒͷॖ • νʔϜશһʹɺ͍ܰؾ࣋ͪͰ͍͍͢͢ϩάΛूܭɾՄࢹԽɾੳ͢ Δश׳͕͍ͭͯɺΠϕϯτࣄͷͨͼʹߦಈྔ૿͑ͨΓ͢Δͷ͕Έͯ ָ͍͠ •
ϩάʹײҠೖͰ͖ΔΑ͏ʹͳͬͨʂ • ͓͍߹Θͤ࣌ʹࠔͬͯΔϢʔβʔͷߦಈΛ͑ΔΑ͏ʹͳͬͨ • όάͷݪҼ͕ɺϩά͔ΒϢʔβʔͷಈ͖Λ࠶ݱͯ͠ΈͨΒ໌ͨ͠
ָ͍͠ʂ
΄͔ʹΓ͍ͨ͜ͱ • ApacheͷΤϥʔϩάɺΞΫηεϩάͷՄࢹԽɾੳ • fluentdͰTemplate͕༻ҙ͞Ε͍ͯΔͷͰ؆୯ʹͰ͖Δ • ϨεϙϯελΠϜͱ͔ग़͓ͯ͘͠ͱͬͱָ͍͠ • ΞϓϦέʔγϣϯϑϨʔϜϫʔΫͷΤϥʔϩά •
Fluentdෳߦϩά͍͚Δ • slow queryͷϩάݟͯΨϯΨϯѱ͍ΫΤϦΛ௵͢ ϦΞϧλΠϜੑ͕ٻΊΒΕΔใΛݟ͍͔͢Β͘͢͝Ԡ༻ར͘
ਏ͔ͬͨ͜ͱ • HadoopʹೖΕ͍ͯͨಠࣗͷϑΥʔϚοτΛਖ਼ن දݱͰॻ͘ͷͭΒ͍ • ϩά͕1.3%͙Β͍ܽଛ͢Δ => ࣏ͬͨ • Index
template͚ͭͨΒಡΊͳ͍ͬͯݴΘΕΔ • Autoscaling͕ݡ͗ͯͬͯͨ͢ͷterminate͞Εͨ
<source> @type tail path /home/ikuo.degawa/hogehoge.logs pos_file /tmp/hogehoge.logs.pos format /^(?<dt>[^\t]+)\t(?<site_id>[^\t]*)\t(?<action>[^\t]*)\t(? <option>[^\t]*)\t(?<user_id>[^\t]*)\t(?<session_cookie>[^\t]*)\t(?
<storage_cookie>[^\t]*)\t(?<view_type>[^\t]*)\t(?<user_agent>[^\t]*)\t(? <page_id>[^\t]*)\t(?<url>[^\t]*)\t(?<time>[^\t]*)\t(?<ip>[^\t]*)\t(? <segment>[^\t]*)\t(?<var>[^\t]*)\t(?<view>[^\t]*)\t(?<act>[^\t]*)\t(?<post0>[^ \u0001]*)\u0001(?<post1>[^\u0001]*)\u0001(?<post2>[^\t]*)\t(?<search0>[^ \u0001]*)\u0001(?<search1>[^\u0001]*)\u0001(?<search2>[^\u0001]*)\u0001(? <search3>[^\u0001]*)\u0001(?<search4>[^\u0001]*)\u0001(?<search5>[^\u0001]*) \u0001(?<search6>[^\u0001]*)\u0001(?<search7>[^\t]*)\t(?<user0>[^\u0001]*) \u0001(?<user1>[^\u0001]*)\u0001(?<user2>[^\u0001]*)\u0001(?<user3>[^\t]*)\t(? <other0>[^\u0001]*)\u0001(?<other1>[^\u0001]*)\u0001(?<other2>.*)$/ tag mogmog-logs.gerogero </source> HadoopʹೖΕ͍ͯͨಠࣗͷϑΥʔ ϚοτΛਖ਼نදݱͰॻ͘ͷͭΒ͍
ϩά͕1.3%͙Β͍ܽଛ͢Δ => ࣏ͬͨ • Kibanaͷ݅ͱɺcat hoge.log | wc -l ͨ݁͠Ռ
͕ҧ͏ʂʂ • lotateͨ͠ઌͷϑΝΠϧΛ ಡΈ࢝ΊΔλΠϛϯά͕ ͍ͱ͍͏༷Λൃݟ • read_from_headΛͬͨ Β࣏ͬͨ લͷ ࣍ͷ ͜ͷล͔ΒಡΜͰͨ
Index template͚ͭͨΒಡΊͳ ͍ͬͯݴΘΕΔ • index template: elasticsearchʹೖΔ ࣌ͷmappingΛࢦ ఆͰ͖Δ •
index໊Λ݅ʹܕ ΛܾΊΒΕΔ { "templates": “awesomelog-*", "settings": { "number_of_shards" : 1 }, "mappings": { "awesomelogs" : { "properties" : { "@timestamp" : { "type" : "date", "format" : "strict_date_optional_time||epoch_millis" }, "act0" : { "type" : "integer" }, "act1" : { "type" : "integer" }, "act10" : { "type" : "string", "index": "not_analyzed" }, "act11" : { "type" : "string" }, "act2" : { "type" : "integer" }, "act3" : { "type" : "integer" }, "act4" : { "type" : "string" }, "act5" : { "type": "multi_field", "fields": {
ύϑΥʔϚϯε্͕Δͱࢥͬͨ Βɾɾɾ { "templates": “awesomelog-*", "settings": { "number_of_shards" : 1
}, "mappings": { "awesomelogs" : { "properties" : { "@timestamp" : { "type" : "date", "format" : "strict_date_optional_time||epoch_millis" }, "act0" : { "type" : "integer" }, "act1" : { "type" : "integer" }, "act10" : { "type" : "string", "index": "not_analyzed" }, "act11" : { "type" : "string" }, "act2" : { "type" : "integer" }, "act3" : { "type" : "integer" }, "act4" : { "type" : "string" }, "act5" : { "type": "multi_field", "fields": { • ࣮intΛظ͍ͯ͠Δͱ͜ ΖʹstringඈΜͰ͖ͨΓ͠ ͯͨʢϩάͷ࣮ϛεʣ • ϩά͕ೖͬͨͱ͖ʹΤϥʔ ు͍ͯͯɺfluentdͷόο ϑΝʹཷ·Γଓ͚ͯͨ • ݁ہnot_analyzedΛ͚ͭͨ ͷΈ
Autoscaling͕ݡ͗ͯ͢terminate ͞Εͨ ʂʁ
ʮavailability zone͕Ճ͞Ε͔ͨΒɺόϥϯε Αͯ͘͠Մ༻ੑ͋͛ΔͨΊʹ͍ͬ͜ফͯ࣍͠ͷ ݐͯΔΑʂʯ
ڭ܇ɾɾɾ • Fluentd͓ੈগͳͯ͘ࡁΉ͕ɺϩάͷಡΈ ํΛͬͱ͚ • ElasticsearchElasticʹ͓͍ͯͨ͠΄͏͕͍͍ • Auto Scaling Groupݡ͍
·ͱΊ • KibanaͰϩάΛ͔ͬ͜Α͘ݟΒΕΔͱσʔλ ʹײҠೖͰ͖ΔΑ͏ʹͳΓɺνʔϜશһ͕ ϢʔβʔͷߦಈΛݟΒΕΔਓʹͳΕΔ • ָ͍͠
ฐࣾͰΤϯδχΞΛืूதͰ͢ ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠
͜ͷຊʹ͓ੈʹͳΓ·ͨ͠ • ͍͍ຊͰ͢
@ikuwow