Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Elastic Stackの紹介 in 福岡
Search
Jun Ohtani
April 27, 2016
Technology
3
4k
Elastic Stackの紹介 in 福岡
「Elasticsearch勉強会 in 福岡」で使用したスライド
Jun Ohtani
April 27, 2016
Tweet
Share
More Decks by Jun Ohtani
See All by Jun Ohtani
Elastic Stackでマイクロサービス運用を 楽にするには? / Monitoring Microservices with Elastic Stack
johtani
5
2.9k
様々なメトリクスやログを集めてシステム解析 - Elastic Stackの入門と活用 - / Getting started Elastic Stack for logging/metrics
johtani
4
1.1k
え?SQLで入門?する ElasticsearchとElastic Stack / Getting started Elastic Stack with SQL
johtani
4
1k
Elastic Stack 入門 2018.09 / Getting started Elastic Stack 2018.09
johtani
3
2.8k
What's new in Elastic Stack 6.3
johtani
2
2.2k
Elastic Stackで始めるJavaアプリのパフォーマンス監視 / Intro Elastic Stack and Elastic APM Java
johtani
5
2.5k
様々なメトリクスやログを集めてシステム解析 - Elastic Stackの入門と活用 - / Intro Elastic Stack
johtani
0
130
Intro Elastic Stack at Telemetry WG
johtani
0
240
What's new in Elastic Stack 6.1?
johtani
0
630
Other Decks in Technology
See All in Technology
Claude Code Actionを使ったコード品質改善の取り組み
potix2
PRO
6
2.3k
5min GuardDuty Extended Threat Detection EKS
takakuni
0
140
Microsoft Build 2025 技術/製品動向 for Microsoft Startup Tech Community
torumakabe
2
270
Understanding_Thread_Tuning_for_Inference_Servers_of_Deep_Models.pdf
lycorptech_jp
PRO
0
120
250627 関西Ruby会議08 前夜祭 RejectKaigi「DJ on Ruby Ver.0.1」
msykd
PRO
2
290
M3 Expressiveの思想に迫る
chnotchy
0
100
Agentic Workflowという選択肢を考える
tkikuchi1002
1
510
Абьюзим random_bytes(). Фёдор Кулаков, разработчик Lamoda Tech
lamodatech
0
340
A2Aのクライアントを自作する
rynsuke
1
170
rubygem開発で鍛える設計力
joker1007
2
200
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
26k
Observability infrastructure behind the trillion-messages scale Kafka platform
lycorptech_jp
PRO
0
140
Featured
See All Featured
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
48
5.4k
Responsive Adventures: Dirty Tricks From The Dark Corners of Front-End
smashingmag
252
21k
Building Flexible Design Systems
yeseniaperezcruz
328
39k
4 Signs Your Business is Dying
shpigford
184
22k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
Intergalactic Javascript Robots from Outer Space
tanoku
271
27k
How GitHub (no longer) Works
holman
314
140k
Testing 201, or: Great Expectations
jmmastey
42
7.5k
Learning to Love Humans: Emotional Interface Design
aarron
273
40k
Large-scale JavaScript Application Architecture
addyosmani
512
110k
Agile that works and the tools we love
rasmusluckow
329
21k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
45
7.4k
Transcript
‹#› 2016/04/26 Evangelist at Elastic Jun Ohtani @johtani Elastic stackͷհ
in Ԭ
‹#›
ΞδΣϯμ • Elastic stackհ • Logstash - Logऩू • Elasticsearch
- ݕࡧɾղੳ • Kibana - ՄࢹԽ • ༻ϓϥάΠϯհ 3
about • Me, Jun Ohtani / Technical Adovocate ‒ lucene-gosenίϛολʔ
‒ ElasticSearch Serverຊޠ൛ͷ༁ ‒ elasticsearch-extended-analysisͷ։ൃ ‒ http://blog.johtani.info • Elasticsearch, founded in 2012 ‒ Products: Elasticsearch, Logstash, Kibana, Marvel, Sheild Professional services: Support & development subscriptions ‒ Trainings 4
5 ElasticελοΫ
ElasticελοΫʢOpen Sourceʣ 6 Kibana Elasticsearch
Logstash Beats
ElasticελοΫ 7 Elastic Cloud
X-Pack Kibana Elasticsearch ! " Logstash Beats +
Elastic stackʹΑΔσʔλੳ 8 σʔλ Import/Parse /Export Store/Search Visualize
9 Logstash
Elastic stackʹΑΔσʔλੳ 10 σʔλ Import/Parse /Export Store/Search Visualize
Logstash in 10 seconds • ϩάɾσʔλͷऩूɾཧ • ऩूɺύʔεɾՃɺૹग़ • ΦʔϓϯιʔεɿApache
License 2.0 • Ruby app (JRuby) 11
Logstash architecture 12 Input Output Filter ? ? collect and
split alter and enrich store and visualize
ઃఆ 13 input { … } filter { … }
output { … }
ઃఆɿinput 14 input { file { path => “/Users/johtani/sample/*_log" start_position
=> "beginning" } }
1ߦ1σʔλ 189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/ 1.1"
404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0" 15
ઃఆɿfilter 16 filter { grok { match => { "message"
=> "%{COMBINEDAPACHELOG}" } break_on_match => false } date { match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"] locale => en } geoip { source => ["clientip"] } useragent { source => "agent" target => "useragent" } }
ύʔε 17 189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/1.1"
404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0" {… "@timestamp": "2015-04-10T09:07:49.325Z", "clientip": "189.120.xx.xx", "ident": "-", "auth": "-", "timestamp": "02/Dec/2014:12:18:29 +0900", "verb": "GET", "request": "/manager/html", … "agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/
ઃఆɿfilter 18 filter { grok { match => { "message"
=> "%{COMBINEDAPACHELOG}" } break_on_match => false } date { match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"] locale => en } geoip { source => ["clientip"] } useragent { source => "agent" target => "useragent" } }
ͷύʔε 19 {… "@timestamp": "2015-04-10T09:07:49.325Z", … "timestamp": "02/Dec/2014:12:18:29 +0900", …
} {… "@timestamp": "2014-12-02T03:18:29.000Z", … "timestamp": "02/Dec/2014:12:18:29 +0900", … }
ઃఆɿfilter 20 filter { grok { match => { "message"
=> "%{COMBINEDAPACHELOG}" } break_on_match => false } date { match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"] locale => en } geoip { source => ["clientip"] } useragent { source => "agent" target => "useragent" } }
IP͔ΒҢܦͳͲ༩ 21 "clientip": "189.120.xx.xx", "clientip": "189.120.xx.xx", "geoip": { "ip": “189.120.xxx.xxx”,
… "country_name": "Brazil", "continent_code": "SA", "region_name": "27", "city_name": "São Paulo", "latitude":
ઃఆɿfilter 22 filter { grok { match => { "message"
=> "%{COMBINEDAPACHELOG}" } break_on_match => false } date { match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"] locale => en } geoip { source => ["clientip"] } useragent { source => "agent" target => "useragent" } }
ϢʔβΤʔδΣϯτͷύʔε 23 "agent": "\"Mozilla/5.0 (Windows NT 5.1; rv: 5.0) Gecko/20100101
Firefox/5.0\"" "agent": "\"Mozilla/5.0 (Windows NT 5.1; rv: 5.0) Gecko/20100101 Firefox/5.0\"" "useragent": { "name": "Firefox", "os": "Windows XP", "os_name": "Windows XP", "device": "Other", "major": "5", "minor": "0"
ઃఆɿoutput 24 output { elasticsearch { hosts => ["localhost"] index
=> “demo_access_log-%{+YYYY.MM.dd}” } }
25 Elasticsearch
‹#› ݕࡧͱͯ͠ͷElasticsearch
‹#› Elasticsearchͱʁ
ϑϦʔϫʔυݕࡧ 28
ߜΓࠐΈ 29
ϋΠϥΠτ 30
ιʔτ 31
ϖʔδϯά 32
ूܭ 33
αδΣετ 34
Elasticsearch in 10 seconds • εΩʔϚϑϦʔɺࢄυΩϡϝϯτετΞɺREST & JSON • Φʔϓϯιʔε:
Apache License 2.0 • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ • JavaͰ࣮ɻ֦ு༰қ 35
Powerful Search at Scale 36
‹#› ؆୯ͳCRUD
σʔλొ 38 curl -XPUT localhost:9200/books/book/1 -d ' { "title" :
"Elasticsearch - The definitive guide", "authors" : "Clinton Gormley", "started" : "2013-02-04", "pages" : 230 }'
σʔλߋ৽ 39 curl -XPUT localhost:9200/books/book/1 -d ' { "title" :
"Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : "2013-02-04", "pages" : 230 }'
σʔλআ 40 curl -X DELETE localhost:9200/books/book/1 σʔλͷऔಘ curl —X GET
localhost:9200/books/book/1 curl —X GET localhost:9200/books/book/1/_source
ݕࡧ 41 curl -XGET localhost:9200/books/_search?q=elasticsearch { "took" : 2, "timed_out"
: false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.076713204, "hits" : [ { "_index" : “books", "_type" : “book", "_id" : "1", "_score" : 0.076713204, "_source" : { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : “2013-02-04", "pages" : 230 } } ]
ݕࡧ - Query DSL 42 curl -XGET ‘localhost:9200/books/book/_search' -d '{
"query": { "filtered" : { "query" : { "match": { "text" : { "query" : “To Be Or Not To Be", "cutoff_frequency" : 0.01 } } }, "filter" : { "range": { "price": { "gte": 20.0 "lte": 50.0
‹#› ࢄߏ εέʔϧ
Basic terms • ΠϯσοΫε ‒ σʔλͷཧతͳू߹ɻ RDBͷσʔλϕʔεͷΑ͏ͳͷLogical • ϨϓϦέʔγϣϯ •
ಡΈࠐΈͷεέʔϥϏϦςΟ্ • SPOFͷղফ • γϟʔσΟϯά • ෳϚγϯσʔλΛׂ ॻ͖ࠐΈͷεέʔϥϏϦςΟ্ σʔλϑϩʔ੍ޚ 44
γϟʔυͱϨϓϦΧ 45 node 1 orders products 1 4 1 2
2 2 curl -X PUT localhost:9200/orders -d '{ "settings.index.number_of_shards" : 4 "settings.index.number_of_replicas" : 1 }' curl -X PUT localhost:9200/products -d '{ "settings.index.number_of_shards" : 2 "settings.index.number_of_replicas" : 0 }'
γϟʔυͱϨϓϦΧ 46 node 1 orders products 1 4 1 node
2 orders products 2 2 3 4 1 2 3
ࣗಈతͳࢄ 47 node 1 orders products 2 1 4 1
node 2 orders products 2 2 node 3 orders products 3 4 1 3
‹#› શจݕࡧͱʁ
શจݕࡧͱʁ • શจݕࡧʢFull text searchʣͱɺίϯϐϡʔλʹ͓͍ͯɺෳͷจॻ ʢϑΝΠϧʣ͔ΒಛఆͷจࣈྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯ ʮ୯ҰϑΝΠϧͷจࣈྻݕࡧʯͱҟͳΓɺʮෳจॻʹ·͕ͨͬͯɺจ ॻʹؚ·ΕΔશจΛରͱͨ͠ݕࡧʯͱ͍͏ҙຯͰ༻͞ΕΔɻ ʢWikipediaΑΓʣ 49
༻ޠ • ΠϯσοΫε ݕࡧΤϯδϯ͕ݕࡧʹ༻͢Δσʔλͷอଘઌ • υΩϡϝϯτʢจॻʣ ‒ ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ • ϑΟʔϧυ
‒ υΩϡϝϯτʹؚ·ΕΔଐੑ • ΫΤϦ ‒ ݕࡧ݅ɺݕࡧࣜ 50
༻ޠ • εΩʔϚ ‒ υΩϡϝϯτͷߏΛఆٛ͢Δͷ • λʔϜʢTermʣɺτʔΫϯʢTokenʣ ‒ ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ ‒
จষΛҰఆͷ๏ଇͰ۠ͬͨ୯ޠ ‒ ୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲؚΉ 51
υΩϡϝϯτͷొ 52 1 2 ΧπΦαβΤͷఋ αβΤϫΧϝͷ࢞ υΩϡϝϯτͷొ
υΩϡϝϯτͷొ 53 1 2 ΧπΦαβΤͷఋ αβΤϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ
ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ υΩϡϝϯτͷొ ୯ޠʹׂ
υΩϡϝϯτͷొ 54 1 2 ΧπΦαβΤͷఋ αβΤϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ
ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ ΧπΦ αβΤ 1 1 2 ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 υΩϡϝϯτͷొ ୯ޠʹׂ ୯ޠ͔Βidͷྻ͕ Ҿ͚ΔΑ͏ʹ
ݕࡧ 55 ΧπΦ αβΤ 1 1 2 ͷ ࢞
ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧ݅ೖྗ ΧπΦɹαβΤ
ݕࡧ 56 ΧπΦ αβΤ 1 1 2 ͷ ࢞
ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ
ݕࡧ 57 ΧπΦ αβΤ 1 1 2 ͷ ࢞
ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ
ݕࡧ 58 ΧπΦ αβΤ 1 1 2 ͷ ࢞
ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ
ݕࡧ 59 ΧπΦ αβΤ 1 1 2 ͷ ࢞
ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ
ݕࡧ 60 ΧπΦ αβΤ 1 1 2 ͷ ࢞
ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ
ݕࡧ 61 ΧπΦ αβΤ 1 1 2 ͷ ࢞
ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ݅ೖྗ ݕࡧ݅ͷύʔε ݕࡧΫΤϦԽ ΧπΦɹαβΤ
୯ޠͷ۠Γํ • ӳޠͷ߹ I am speaking Introduction Elasticsearch.
• ຊޠͷ߹ ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ 62
୯ޠͷ۠Γํ • ӳޠͷ߹ I am speaking Introduction Elasticsearch.
εϖʔε͕ΕͱΘ͔Δ • ຊޠͷ߹ ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ Ͳ͜Ͱ۠ΕΑ͍ʁ 63
N-Gramͱܗଶૉղੳ • సஔΠϯσοΫεͷΩʔͷ࡞Γํ ‒ ຊޠ୯ޠͷΕ͕Θ͔Βͳ͍ͷͰɺసஔΠϯσοΫεͷΩʔ ओʹ࣍ͷ̎ͭͷख๏Ͱ࡞ • N-Gram ‒ NจࣈͣͭจষΛ۠Δ
• ܗଶૉղੳ ‒ ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠Δ 64
ܗଶૉղੳ • ϝϦοτɿ ‒ ҙຯͷ͋Δ୯ޠͷΕ ࢺใΛݩʹՃॲཧ͕ՄೳʢޠװมͳͲʣ • σϝϦοτɿ ‒ ৽ޠʢະޠʣʹऑ͍→ࣙॻϕʔεͷ߹ɺࣙॻʹͳ͍୯ޠݕग़ෆ
ೳɻ 65 ΧπΦαβΤͷఋ ΧπΦ ͷ αβΤ ఋ
N-Gram • ϝϦοτɿ ‒ ະޠʹରԠՄೳ • σϝϦοτɿ ‒ ΠϯσοΫεංେԽ ‒
ࢺใʹجͮ͘ॲཧ͕ෆՄೳ 66 ΧπΦαβΤͷఋ Χπ πΦ Φ α αβ βΤ Τͷ ͷఋ
‹#› Analysis & Mappings
Analysisͱʁ • సஔΠϯσοΫεͷ୯ޠʢTermʣΛυΩϡϝϯτ͔Βநग़͢Δॲཧ • ϑΟʔϧυຖʹࢦఆ͞ΕͨΞφϥΠβ͕ॲཧ • ΞφϥΠβCharFilterɺTokenizerɺTokenFilter͔Βߏ • ΠϯσοΫε࣌ɺݕࡧ࣌ʹ࣮ߦ 68
Analysisͷ֓ཁ 69 Analysis υΩϡϝϯτ Term Id ΧπΦ 1 αβΤ 1ɺ2
ϫΧϝ 2 … ... ΠϯσοΫε
Analysisͷ֓ཁ 70 Analysis ݕࡧ Ωʔϫʔυ ΠϯσοΫε ΠϯσοΫε ݕࡧॲཧ ݕࡧ݁Ռ Term
Id ΧπΦ 1 αβΤ 1ɺ2 ϫΧϝ 2 … ...
Analysisͷߏ 71 Analyzer Text Tokens char_filter tokenizer token_filter char_filter token_filter
Analyzerͷઃఆ 72 {"index":{ “analysis":{ "analyzer" : { "my_analyzer" : {
"type" : "custom", "tokenizer" : “kuromoji_tokenizer”, “char_filter" : [“char_filter1”,…], “filter" : [“token_filter1”,…] } }
Char Filter • ೖྗจࣈྻΛจࣈ୯ҐͰॲཧ • ྫɿhtml_strip ‒ ೖྗɿ ‒ ग़ྗɿ
73 <title>Elasticsearch is not a service of AWS</title> Elasticsearch is not a service of AWS
Tokenizer • ೖྗจࣈྻΛτʔΫϯྻʹׂ • τʔΫϯͷׂ͢ΔϩδοΫτʔΫφΠβʹґଘ • ྫɿstandard ‒ ೖྗɿ ‒
ग़ྗɿ 74 Elasticsearch is not a service of AWS Elasticsearch is not a service of AWS
Tokenizer • ྫɿkeyword ‒ ग़ྗɿ • ྫɿkuromoji_tokenizer ‒ ೖྗɿ ‒
ग़ྗɿ 75 ण࢘ ͕ ඒຯ͔ͬ͠ ͨ Elasticsearch is not a service of AWS ण͕࢘ඒຯ͔ͬͨ͠
TokenFilter • TokenizerʹΑΓग़ྗ͞ΕͨTokenྻʹରͯ͠ॲཧ • ྫɿlowercase ‒ ೖྗɿ ‒ ग़ྗɿ •
ྫɿstop ‒ ೖྗɿ ‒ ग़ྗɿ 76 Elasticsearch is not a service of AWS elasticsearch is not a service of aws Elasticsearch is not a service of AWS Elasticsearch service AWS
TokenFilter • ྫɿkuromoji_baseform ‒ ೖྗɿ ‒ ग़ྗɿ • ྫɿkuromoji_readingform ‒
ೖྗɿ ‒ ग़ྗɿ 77 ण࢘ ͕ ඒຯ͔ͬ͠ ͨ ण࢘ ͕ ඒຯ͍͠ ͨ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ sushi ga oishika ta
Mapping • ΠϯσοΫεͷߏʢschemaʣΛఆٛ • Schema less!͚ͩͲ… • ܗࣜΛࢦఆʢanalyze͢Δ/͠ͳ͍ɺ/গɺͳͲʣ 78
ྫɿWikipediaͷMappingͷҰ෦ 79 "mappings": { "page": { "properties": { "link": {
"type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} } }, "text": { "type": "string"}, "title": { "type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} }…
‹#› ͦͷଞͷػೳ
elasticsearch ͞·͟·ͳܗࣜͷσʔλͰ GeoݕࡧՄೳ ҢܦɺGeoHashɺ GeoShape… GEO
Percolator • υΩϡϝϯτͰͳ͘ΫΤϦΛొ • _percolate APIʹυΩϡϝϯτΛૹ৴ • ొ͞ΕͨΫΤϦʹϚονͨ͠߹ɺ Ϛονͨ͠ΫΤϦΛฦ͢ 82
Snapshot/Restore • 1.0͔Βಋೖ • ΠϯσοΫε୯ҐͰSnapshot/RestoreՄೳ • อଘઌʢRepositoryʣ ‒ Shard FSɺS3ɺHDFSɺAzure…
83
Ecosystem • Plugins ‒ ϓϥάΠϯʹΑΔػೳͷՃ • ΫϥΠΞϯτϥΠϒϥϦ • Ruby, python,
php, perl, javascript, .NET • Scala, clojure, go • Hadoop integration 84
Elasticsearch - The Definitive guide http://www.elastic.co/guide/en/ elasticsearch/guide/current/index.html 85 ৄ͘͠Γ͍ͨํ
‹#› ղੳͱͯ͠ͷElasticsearch
‹#› aggregation
Aggregationͱ • 1.0͔Βಋೖ • FacetΑΓڧྗͳूܭͳͲ͕Մೳ • ֊తͳूܭɺάϧʔϓԽ ಈతͳूܭɺάϧʔϓԽ • େ͖͘2छྨ
• BucketɹυΩϡϝϯτΛ͝ͱʹ݁ՌΛάϧʔϐϯά • Metricɹ υΩϡϝϯτͷ࣋ͭΛूܭ 88
ྫɿݴޠ͓ΑͼҬͷूܭ 89 curl -XGET twitter-2014.08.22/_search -d ' { "aggs": {
"lang": { "terms": {"field": "lang" }, "aggs": { "place": { "terms": { "field": “place.full_name", "size": 10 } } } } } }
ྫɿݴޠ͓ΑͼҬͷूܭ 90 "aggregations": { "lang": { "buckets": [{…}, { "key":
"ja", "doc_count": 980145, "place": { "buckets": [ { "key": "ژࢢ෬ݟ۠, ژ", "doc_count":252 }, { "key": "ઍా۠, ౦ژ", "doc_count": 39 },…
91 KibanaͰՄࢹԽ
Kibana 4 • ElasticsearchͷσʔλΛՄࢹԽ • Node.js server & JavaScript •
Apache License 2.0 • Elastic Stackͷ૭ͷׂ • ༷ʑͳGUIΛPluginͱ͍ͯެ։ • MarvelɺSenseɺTimelionͳͲ 92
Kibana 4 93
σϞ for Kibana4 Access Log 94
Combining Search and Analytics 95
‹#› ͦͷଞͷ ElasticελοΫ
beats
Capture the Packet Packetbeat
Capture the Packet Packetbeat
Unleash the Beats libbeat Beat 1 libbeat Beat 2 Beat
3 +
It was only supposed to be a demo topbeat
It was only supposed to be a demo topbeat
To tail a File filebeat + logstash
To tail a File filebeat + logstash
Welcome to 1998 winlogbeat
Now winlogbeat
elasticsearch-hadoop 107 - • D E H • PD ecd
ER • g D • CH • Ca M DMS D FERC
‹#› ༻ϓϥάΠϯ
‹#› Shield Shield
Shieldͷಛ • User Authentication ‒ LDAP/Active Directory/ϑΝΠϧϕʔε • Authorization ‒
ϩʔϧϕʔεͷΞΫηείϯτϩʔϧ ‒ ΠϯσοΫε͝ͱɺΞΫγϣϯ͝ͱͷઃఆ͕Մೳ ‒ υΩϡϝϯτɾϑΟʔϧυ͝ͱͷઃఆՄೳʹ • ηΩϡΞͳ௨৴ ‒ ElasticsearchϊʔυؒͷSSL/TLSɺIPϑΟϧλϦϯά • ࠪϩά 110
ΨΠυͳͲ • ϓϩμΫτϖʔδ ‒ https://www.elastic.co/products/shield • ΨΠυ ‒ https://www.elastic.co/guide/en/shield/current/quick-getting- started.html
111
‹#› Watcher Watcher
Watcherͷಛ • ΫΤϦʹΑΔWatch ‒ ElasticsearchͷΫΤϦΛར༻ͯ͠σʔλͷࢹ • ݅ͷઃఆ ‒ ΞΫγϣϯΛ࣮ߦ͢Δ͔Ͳ͏͔ͷઃఆ •
εέδϡʔϧ ‒ ΫΤϦΛ࣮ߦ͠ɺ݅ΛνΣοΫ͢Δසͷࢦఆ • ΞΫγϣϯͷఆٛ ‒ ϝʔϧͷૹ৴ɺଞγεςϜͷσʔλૹ৴ͳͲͷಈ࡞Λઃఆ • ཤྺͷอଘ 113
ΨΠυͳͲ • ϓϩμΫτϖʔδ ‒ https://www.elastic.co/products/watcher • ΨΠυ ‒ https://www.elastic.co/guide/en/watcher/current/index.html
114
cloud
Elastic Cloud • Elasticsearch as a Service • Elasticsearchͷػೳ͕ར༻Մೳ •
ϓϥάΠϯͷར༻Մೳ • ΞοϓάϨʔυ؆୯ • ΧελϜࣙॻɺϓϥάΠϯར༻Մೳ • 14ؒͷ͓ࢼ͠ظؒ͋Γ • SLAϕʔεͷαϒεΫϦϓγϣϯ+༻ϓϥάΠϯ ͕ར༻ՄೳͳαʔϏε 116
ࢀߟจݙ • Elasticsearch - The Definitive guide ‒ http://www.elastic.co/guide/en/elasticsearch/guide/current/index.html •
ॻ੶ʢຊޠʣ ‒ ElasticSearchServerຊޠ൛ αʔό/ΠϯϑϥΤϯδχΞ ɹཆಡຊɹϩάऩू 117
ࢀߟαΠτ • Ϣʔεέʔε • https://www.elastic.co/use-cases • DiscussʢWebϑΥʔϥϜʣ • https://discuss.elastic.co •
Elastic{ON}ͷϏσΦͱࢿྉ • https://www.elastic.co/elasticon/videos • αϙʔτϝχϡʔ • https://www.elastic.co/subscriptions 118
Thanks for listening! Q & A 119 We’re hiring! https://www.elastic.co/about/careers/
We’re helping! https://www.elastic.co/subscriptions http://training.elastic.co