Slide 1
Slide 1 text
‹#›
2016/04/26
Evangelist at Elastic
Jun Ohtani @johtani
Elastic stackͷհ in Ԭ
Slide 2
Slide 2 text
‹#›
Slide 3
Slide 3 text
ΞδΣϯμ
• Elastic stackհ
• Logstash - Logऩू
• Elasticsearch - ݕࡧɾղੳ
• Kibana - ՄࢹԽ
• ༻ϓϥάΠϯհ
3
Slide 4
Slide 4 text
about
• Me, Jun Ohtani / Technical Adovocate
‒ lucene-gosenίϛολʔ
‒ ElasticSearch Serverຊޠ൛ͷ༁
‒ elasticsearch-extended-analysisͷ։ൃ
‒ http://blog.johtani.info
• Elasticsearch, founded in 2012
‒ Products: Elasticsearch, Logstash, Kibana, Marvel, Sheild
Professional services: Support & development subscriptions
‒ Trainings
4
Slide 5
Slide 5 text
5
ElasticελοΫ
Slide 6
Slide 6 text
ElasticελοΫʢOpen Sourceʣ
6
Kibana
Elasticsearch
Logstash Beats
Slide 7
Slide 7 text
ElasticελοΫ
7
Elastic Cloud
X-Pack
Kibana
Elasticsearch
!
"
Logstash Beats
+
Slide 8
Slide 8 text
Elastic stackʹΑΔσʔλੳ
8
σʔλ Import/Parse
/Export
Store/Search Visualize
Slide 9
Slide 9 text
9
Logstash
Slide 10
Slide 10 text
Elastic stackʹΑΔσʔλੳ
10
σʔλ Import/Parse
/Export
Store/Search Visualize
Slide 11
Slide 11 text
Logstash in 10 seconds
• ϩάɾσʔλͷऩूɾཧ
• ऩूɺύʔεɾՃɺૹग़
• ΦʔϓϯιʔεɿApache License 2.0
• Ruby app (JRuby)
11
Slide 12
Slide 12 text
Logstash architecture
12
Input Output
Filter
? ?
collect and split alter and enrich store and visualize
Slide 13
Slide 13 text
ઃఆ
13
input {
…
}
filter {
…
}
output {
…
}
Slide 14
Slide 14 text
ઃఆɿinput
14
input {
file {
path => “/Users/johtani/sample/*_log"
start_position => "beginning"
}
}
Slide 15
Slide 15 text
1ߦ1σʔλ
189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/
1.1" 404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101
Firefox/5.0"
15
Slide 16
Slide 16 text
ઃఆɿfilter
16
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 17
Slide 17 text
ύʔε
17
189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/1.1"
404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0"
{…
"@timestamp": "2015-04-10T09:07:49.325Z",
"clientip": "189.120.xx.xx",
"ident": "-",
"auth": "-",
"timestamp": "02/Dec/2014:12:18:29 +0900",
"verb": "GET",
"request": "/manager/html",
…
"agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/
Slide 18
Slide 18 text
ઃఆɿfilter
18
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 19
Slide 19 text
ͷύʔε
19
{…
"@timestamp": "2015-04-10T09:07:49.325Z",
…
"timestamp": "02/Dec/2014:12:18:29 +0900",
…
}
{…
"@timestamp": "2014-12-02T03:18:29.000Z",
…
"timestamp": "02/Dec/2014:12:18:29 +0900",
…
}
Slide 20
Slide 20 text
ઃఆɿfilter
20
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 21
Slide 21 text
IP͔ΒҢܦͳͲ༩
21
"clientip": "189.120.xx.xx",
"clientip": "189.120.xx.xx",
"geoip": {
"ip": “189.120.xxx.xxx”,
…
"country_name": "Brazil",
"continent_code": "SA",
"region_name": "27",
"city_name": "São Paulo",
"latitude":
Slide 22
Slide 22 text
ઃఆɿfilter
22
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 23
Slide 23 text
ϢʔβΤʔδΣϯτͷύʔε
23
"agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:
5.0) Gecko/20100101 Firefox/5.0\""
"agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:
5.0) Gecko/20100101 Firefox/5.0\""
"useragent": {
"name": "Firefox",
"os": "Windows XP",
"os_name": "Windows XP",
"device": "Other",
"major": "5",
"minor": "0"
Slide 24
Slide 24 text
ઃఆɿoutput
24
output {
elasticsearch {
hosts => ["localhost"]
index => “demo_access_log-%{+YYYY.MM.dd}”
}
}
Slide 25
Slide 25 text
25
Elasticsearch
Slide 26
Slide 26 text
‹#›
ݕࡧͱͯ͠ͷElasticsearch
Slide 27
Slide 27 text
‹#›
Elasticsearchͱʁ
Slide 28
Slide 28 text
ϑϦʔϫʔυݕࡧ
28
Slide 29
Slide 29 text
ߜΓࠐΈ
29
Slide 30
Slide 30 text
ϋΠϥΠτ
30
Slide 31
Slide 31 text
ιʔτ
31
Slide 32
Slide 32 text
ϖʔδϯά
32
Slide 33
Slide 33 text
ूܭ
33
Slide 34
Slide 34 text
αδΣετ
34
Slide 35
Slide 35 text
Elasticsearch in 10 seconds
• εΩʔϚϑϦʔɺࢄυΩϡϝϯτετΞɺREST & JSON
• Φʔϓϯιʔε: Apache License 2.0
• ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ
• JavaͰ࣮ɻ֦ு༰қ
35
Slide 36
Slide 36 text
Powerful Search at Scale
36
Slide 37
Slide 37 text
‹#›
؆୯ͳCRUD
Slide 38
Slide 38 text
σʔλొ
38
curl -XPUT localhost:9200/books/book/1 -d '
{
"title" : "Elasticsearch - The definitive guide",
"authors" : "Clinton Gormley",
"started" : "2013-02-04",
"pages" : 230
}'
Slide 39
Slide 39 text
σʔλߋ৽
39
curl -XPUT localhost:9200/books/book/1 -d '
{
"title" : "Elasticsearch - The definitive guide",
"authors" : [ "Clinton Gormley", "Zachary Tong" ],
"started" : "2013-02-04",
"pages" : 230
}'
Slide 40
Slide 40 text
σʔλআ
40
curl -X DELETE localhost:9200/books/book/1
σʔλͷऔಘ
curl —X GET localhost:9200/books/book/1
curl —X GET localhost:9200/books/book/1/_source
Slide 41
Slide 41 text
ݕࡧ
41
curl -XGET localhost:9200/books/_search?q=elasticsearch
{
"took" : 2, "timed_out" : false,
"_shards" : { "total" : 5, "successful" : 5, "failed" : 0 },
"hits" : {
"total" : 1, "max_score" : 0.076713204,
"hits" : [ {
"_index" : “books", "_type" : “book", "_id" : "1",
"_score" : 0.076713204, "_source" : {
"title" : "Elasticsearch - The definitive guide",
"authors" : [ "Clinton Gormley", "Zachary Tong" ],
"started" : “2013-02-04", "pages" : 230
}
} ]
Slide 42
Slide 42 text
ݕࡧ - Query DSL
42
curl -XGET ‘localhost:9200/books/book/_search' -d '{
"query": {
"filtered" : {
"query" : {
"match": {
"text" : {
"query" : “To Be Or Not To Be",
"cutoff_frequency" : 0.01
}
}
},
"filter" : {
"range": {
"price": {
"gte": 20.0
"lte": 50.0
Slide 43
Slide 43 text
‹#›
ࢄߏ
εέʔϧ
Slide 44
Slide 44 text
Basic terms
• ΠϯσοΫε
‒ σʔλͷཧతͳू߹ɻ
RDBͷσʔλϕʔεͷΑ͏ͳͷLogical
• ϨϓϦέʔγϣϯ
• ಡΈࠐΈͷεέʔϥϏϦςΟ্
• SPOFͷղফ
• γϟʔσΟϯά
• ෳϚγϯσʔλΛׂ
ॻ͖ࠐΈͷεέʔϥϏϦςΟ্
σʔλϑϩʔ੍ޚ
44
Slide 45
Slide 45 text
γϟʔυͱϨϓϦΧ
45
node 1
orders
products
1
4
1 2
2
2
curl -X PUT localhost:9200/orders -d '{
"settings.index.number_of_shards" : 4
"settings.index.number_of_replicas" : 1
}'
curl -X PUT localhost:9200/products -d '{
"settings.index.number_of_shards" : 2
"settings.index.number_of_replicas" : 0
}'
Slide 46
Slide 46 text
γϟʔυͱϨϓϦΧ
46
node 1
orders
products
1
4
1
node 2
orders
products
2
2
3 4
1 2
3
Slide 47
Slide 47 text
ࣗಈతͳࢄ
47
node 1
orders
products
2
1
4
1
node 2
orders
products
2
2
node 3
orders
products
3 4
1
3
Slide 48
Slide 48 text
‹#›
શจݕࡧͱʁ
Slide 49
Slide 49 text
શจݕࡧͱʁ
• શจݕࡧʢFull text searchʣͱɺίϯϐϡʔλʹ͓͍ͯɺෳͷจॻ
ʢϑΝΠϧʣ͔ΒಛఆͷจࣈྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯ
ʮ୯ҰϑΝΠϧͷจࣈྻݕࡧʯͱҟͳΓɺʮෳจॻʹ·͕ͨͬͯɺจ
ॻʹؚ·ΕΔશจΛରͱͨ͠ݕࡧʯͱ͍͏ҙຯͰ༻͞ΕΔɻ
ʢWikipediaΑΓʣ
49
Slide 50
Slide 50 text
༻ޠ
• ΠϯσοΫε
ݕࡧΤϯδϯ͕ݕࡧʹ༻͢Δσʔλͷอଘઌ
• υΩϡϝϯτʢจॻʣ
‒ ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ
• ϑΟʔϧυ
‒ υΩϡϝϯτʹؚ·ΕΔଐੑ
• ΫΤϦ
‒ ݕࡧ݅ɺݕࡧࣜ
50
Slide 51
Slide 51 text
༻ޠ
• εΩʔϚ
‒ υΩϡϝϯτͷߏΛఆٛ͢Δͷ
• λʔϜʢTermʣɺτʔΫϯʢTokenʣ
‒ ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ
‒ จষΛҰఆͷ๏ଇͰ۠ͬͨ୯ޠ
‒ ୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲؚΉ
51
Slide 52
Slide 52 text
υΩϡϝϯτͷొ
52
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
υΩϡϝϯτͷొ
Slide 53
Slide 53 text
υΩϡϝϯτͷొ
53
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
1
2
ΧπΦ
αβΤ
ͷ
ͷ
αβΤ
ϫΧϝ
ఋ
࢞
υΩϡϝϯτͷొ
୯ޠʹׂ
Slide 54
Slide 54 text
υΩϡϝϯτͷొ
54
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
1
2
ΧπΦ
αβΤ
ͷ
ͷ
αβΤ
ϫΧϝ
ఋ
࢞
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
υΩϡϝϯτͷొ
୯ޠʹׂ
୯ޠ͔Βidͷྻ͕
Ҿ͚ΔΑ͏ʹ
Slide 55
Slide 55 text
ݕࡧ
55
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ݕࡧ݅ೖྗ
ΧπΦɹαβΤ
Slide 56
Slide 56 text
ݕࡧ
56
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 57
Slide 57 text
ݕࡧ
57
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 58
Slide 58 text
ݕࡧ
58
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 59
Slide 59 text
ݕࡧ
59
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 60
Slide 60 text
ݕࡧ
60
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 61
Slide 61 text
ݕࡧ
61
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 62
Slide 62 text
୯ޠͷ۠Γํ
• ӳޠͷ߹
I am speaking Introduction Elasticsearch.
• ຊޠͷ߹
ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ
62
Slide 63
Slide 63 text
୯ޠͷ۠Γํ
• ӳޠͷ߹
I am speaking Introduction Elasticsearch.
εϖʔε͕ΕͱΘ͔Δ
• ຊޠͷ߹
ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ
Ͳ͜Ͱ۠ΕΑ͍ʁ
63
Slide 64
Slide 64 text
N-Gramͱܗଶૉղੳ
• సஔΠϯσοΫεͷΩʔͷ࡞Γํ
‒ ຊޠ୯ޠͷΕ͕Θ͔Βͳ͍ͷͰɺసஔΠϯσοΫεͷΩʔ
ओʹ࣍ͷ̎ͭͷख๏Ͱ࡞
• N-Gram
‒ NจࣈͣͭจষΛ۠Δ
• ܗଶૉղੳ
‒ ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠Δ
64
Slide 65
Slide 65 text
ܗଶૉղੳ
• ϝϦοτɿ
‒ ҙຯͷ͋Δ୯ޠͷΕ
ࢺใΛݩʹՃॲཧ͕ՄೳʢޠװมͳͲʣ
• σϝϦοτɿ
‒ ৽ޠʢະޠʣʹऑ͍→ࣙॻϕʔεͷ߹ɺࣙॻʹͳ͍୯ޠݕग़ෆ
ೳɻ
65
ΧπΦαβΤͷఋ
ΧπΦ ͷ
αβΤ ఋ
Slide 66
Slide 66 text
N-Gram
• ϝϦοτɿ
‒ ະޠʹରԠՄೳ
• σϝϦοτɿ
‒ ΠϯσοΫεංେԽ
‒ ࢺใʹجͮ͘ॲཧ͕ෆՄೳ
66
ΧπΦαβΤͷఋ
Χπ πΦ Φ α αβ βΤ Τͷ ͷఋ
Slide 67
Slide 67 text
‹#›
Analysis & Mappings
Slide 68
Slide 68 text
Analysisͱʁ
• సஔΠϯσοΫεͷ୯ޠʢTermʣΛυΩϡϝϯτ͔Βநग़͢Δॲཧ
• ϑΟʔϧυຖʹࢦఆ͞ΕͨΞφϥΠβ͕ॲཧ
• ΞφϥΠβCharFilterɺTokenizerɺTokenFilter͔Βߏ
• ΠϯσοΫε࣌ɺݕࡧ࣌ʹ࣮ߦ
68
Slide 69
Slide 69 text
Analysisͷ֓ཁ
69
Analysis
υΩϡϝϯτ
Term Id
ΧπΦ 1
αβΤ 1ɺ2
ϫΧϝ 2
… ...
ΠϯσοΫε
Slide 70
Slide 70 text
Analysisͷ֓ཁ
70
Analysis
ݕࡧ
Ωʔϫʔυ
ΠϯσοΫε
ΠϯσοΫε
ݕࡧॲཧ
ݕࡧ݁Ռ
Term Id
ΧπΦ 1
αβΤ 1ɺ2
ϫΧϝ 2
… ...
Slide 71
Slide 71 text
Analysisͷߏ
71
Analyzer
Text Tokens
char_filter tokenizer token_filter
char_filter token_filter
Slide 72
Slide 72 text
Analyzerͷઃఆ
72
{"index":{
“analysis":{
"analyzer" : {
"my_analyzer" : {
"type" : "custom",
"tokenizer" : “kuromoji_tokenizer”,
“char_filter" : [“char_filter1”,…],
“filter" : [“token_filter1”,…]
}
}
Slide 73
Slide 73 text
Char Filter
• ೖྗจࣈྻΛจࣈ୯ҐͰॲཧ
• ྫɿhtml_strip
‒ ೖྗɿ
‒ ग़ྗɿ
73
Elasticsearch is not a service of AWS
Elasticsearch is not a service of AWS
Slide 74
Slide 74 text
Tokenizer
• ೖྗจࣈྻΛτʔΫϯྻʹׂ
• τʔΫϯͷׂ͢ΔϩδοΫτʔΫφΠβʹґଘ
• ྫɿstandard
‒ ೖྗɿ
‒ ग़ྗɿ
74
Elasticsearch is not a service of AWS
Elasticsearch is not a service of AWS
Slide 75
Slide 75 text
Tokenizer
• ྫɿkeyword
‒ ग़ྗɿ
• ྫɿkuromoji_tokenizer
‒ ೖྗɿ
‒ ग़ྗɿ
75
ण࢘ ͕ ඒຯ͔ͬ͠ ͨ
Elasticsearch is not a service of AWS
ण͕࢘ඒຯ͔ͬͨ͠
Slide 76
Slide 76 text
TokenFilter
• TokenizerʹΑΓग़ྗ͞ΕͨTokenྻʹରͯ͠ॲཧ
• ྫɿlowercase
‒ ೖྗɿ
‒ ग़ྗɿ
• ྫɿstop
‒ ೖྗɿ
‒ ग़ྗɿ
76
Elasticsearch is not a service of AWS
elasticsearch is not a service of aws
Elasticsearch is not a service of AWS
Elasticsearch service AWS
Slide 77
Slide 77 text
TokenFilter
• ྫɿkuromoji_baseform
‒ ೖྗɿ
‒ ग़ྗɿ
• ྫɿkuromoji_readingform
‒ ೖྗɿ
‒ ग़ྗɿ
77
ण࢘ ͕ ඒຯ͔ͬ͠ ͨ
ण࢘ ͕ ඒຯ͍͠ ͨ
ण࢘ ͕ ඒຯ͔ͬ͠ ͨ
sushi ga oishika ta
Slide 78
Slide 78 text
Mapping
• ΠϯσοΫεͷߏʢschemaʣΛఆٛ
• Schema less!͚ͩͲ…
• ܗࣜΛࢦఆʢanalyze͢Δ/͠ͳ͍ɺ/গɺͳͲʣ
78
Slide 79
Slide 79 text
ྫɿWikipediaͷMappingͷҰ෦
79
"mappings": {
"page": {
"properties": {
"link": {
"type": "string",
"fields": {
"raw": {"type": "string",
"index": "not_analyzed"} } },
"text": { "type": "string"},
"title": {
"type": "string",
"fields": {
"raw": {"type": "string",
"index": "not_analyzed"} }…
Slide 80
Slide 80 text
‹#›
ͦͷଞͷػೳ
Slide 81
Slide 81 text
elasticsearch
͞·͟·ͳܗࣜͷσʔλͰ
GeoݕࡧՄೳ
ҢܦɺGeoHashɺ
GeoShape…
GEO
Slide 82
Slide 82 text
Percolator
• υΩϡϝϯτͰͳ͘ΫΤϦΛొ
• _percolate APIʹυΩϡϝϯτΛૹ৴
• ొ͞ΕͨΫΤϦʹϚονͨ͠߹ɺ
Ϛονͨ͠ΫΤϦΛฦ͢
82
Slide 83
Slide 83 text
Snapshot/Restore
• 1.0͔Βಋೖ
• ΠϯσοΫε୯ҐͰSnapshot/RestoreՄೳ
• อଘઌʢRepositoryʣ
‒ Shard FSɺS3ɺHDFSɺAzure…
83
Slide 84
Slide 84 text
Ecosystem
• Plugins
‒ ϓϥάΠϯʹΑΔػೳͷՃ
• ΫϥΠΞϯτϥΠϒϥϦ
• Ruby, python, php, perl, javascript, .NET
• Scala, clojure, go
• Hadoop integration
84
Slide 85
Slide 85 text
Elasticsearch - The Definitive guide
http://www.elastic.co/guide/en/
elasticsearch/guide/current/index.html
85
ৄ͘͠Γ͍ͨํ
Slide 86
Slide 86 text
‹#›
ղੳͱͯ͠ͷElasticsearch
Slide 87
Slide 87 text
‹#›
aggregation
Slide 88
Slide 88 text
Aggregationͱ
• 1.0͔Βಋೖ
• FacetΑΓڧྗͳूܭͳͲ͕Մೳ
• ֊తͳूܭɺάϧʔϓԽ
ಈతͳूܭɺάϧʔϓԽ
• େ͖͘2छྨ
• BucketɹυΩϡϝϯτΛ͝ͱʹ݁ՌΛάϧʔϐϯά
• Metricɹ υΩϡϝϯτͷ࣋ͭΛूܭ
88
Slide 89
Slide 89 text
ྫɿݴޠ͓ΑͼҬͷूܭ
89
curl -XGET twitter-2014.08.22/_search -d '
{
"aggs": {
"lang": {
"terms": {"field": "lang" },
"aggs": {
"place": {
"terms": {
"field": “place.full_name", "size": 10
}
}
}
}
}
}
Slide 90
Slide 90 text
ྫɿݴޠ͓ΑͼҬͷूܭ
90
"aggregations": {
"lang": {
"buckets": [{…}, {
"key": "ja",
"doc_count": 980145,
"place": {
"buckets": [
{ "key": "ژࢢ෬ݟ۠, ژ",
"doc_count":252 },
{ "key": "ઍా۠, ౦ژ",
"doc_count": 39 },…
Slide 91
Slide 91 text
91
KibanaͰՄࢹԽ
Slide 92
Slide 92 text
Kibana 4
• ElasticsearchͷσʔλΛՄࢹԽ
• Node.js server & JavaScript
• Apache License 2.0
• Elastic Stackͷ૭ͷׂ
• ༷ʑͳGUIΛPluginͱ͍ͯެ։
• MarvelɺSenseɺTimelionͳͲ
92
Slide 93
Slide 93 text
Kibana 4
93
Slide 94
Slide 94 text
σϞ for Kibana4
Access Log
94
Slide 95
Slide 95 text
Combining Search and Analytics
95
Slide 96
Slide 96 text
‹#›
ͦͷଞͷ
ElasticελοΫ
Slide 97
Slide 97 text
beats
Slide 98
Slide 98 text
Capture the
Packet
Packetbeat
Slide 99
Slide 99 text
Capture the
Packet
Packetbeat
Slide 100
Slide 100 text
Unleash the
Beats
libbeat
Beat 1
libbeat
Beat 2 Beat 3 +
Slide 101
Slide 101 text
It was only
supposed to
be a demo
topbeat
Slide 102
Slide 102 text
It was only
supposed to
be a demo
topbeat
Slide 103
Slide 103 text
To tail a
File
filebeat + logstash
Slide 104
Slide 104 text
To tail a
File
filebeat + logstash
Slide 105
Slide 105 text
Welcome
to 1998
winlogbeat
Slide 106
Slide 106 text
Now
winlogbeat
Slide 107
Slide 107 text
elasticsearch-hadoop
107
-
• D E H
• PD ecd
ER
• g D
•
CH
• Ca M DMS
D FERC
Slide 108
Slide 108 text
‹#›
༻ϓϥάΠϯ
Slide 109
Slide 109 text
‹#›
Shield
Shield
Slide 110
Slide 110 text
Shieldͷಛ
• User Authentication
‒ LDAP/Active Directory/ϑΝΠϧϕʔε
• Authorization
‒ ϩʔϧϕʔεͷΞΫηείϯτϩʔϧ
‒ ΠϯσοΫε͝ͱɺΞΫγϣϯ͝ͱͷઃఆ͕Մೳ
‒ υΩϡϝϯτɾϑΟʔϧυ͝ͱͷઃఆՄೳʹ
• ηΩϡΞͳ௨৴
‒ ElasticsearchϊʔυؒͷSSL/TLSɺIPϑΟϧλϦϯά
• ࠪϩά
110
Slide 111
Slide 111 text
ΨΠυͳͲ
• ϓϩμΫτϖʔδ
‒ https://www.elastic.co/products/shield
• ΨΠυ
‒ https://www.elastic.co/guide/en/shield/current/quick-getting-
started.html
111
Slide 112
Slide 112 text
‹#›
Watcher
Watcher
Slide 113
Slide 113 text
Watcherͷಛ
• ΫΤϦʹΑΔWatch
‒ ElasticsearchͷΫΤϦΛར༻ͯ͠σʔλͷࢹ
• ݅ͷઃఆ
‒ ΞΫγϣϯΛ࣮ߦ͢Δ͔Ͳ͏͔ͷઃఆ
• εέδϡʔϧ
‒ ΫΤϦΛ࣮ߦ͠ɺ݅ΛνΣοΫ͢Δසͷࢦఆ
• ΞΫγϣϯͷఆٛ
‒ ϝʔϧͷૹ৴ɺଞγεςϜͷσʔλૹ৴ͳͲͷಈ࡞Λઃఆ
• ཤྺͷอଘ
113
Slide 114
Slide 114 text
ΨΠυͳͲ
• ϓϩμΫτϖʔδ
‒ https://www.elastic.co/products/watcher
• ΨΠυ
‒ https://www.elastic.co/guide/en/watcher/current/index.html
114
Slide 115
Slide 115 text
cloud
Slide 116
Slide 116 text
Elastic Cloud
• Elasticsearch as a Service
• Elasticsearchͷػೳ͕ར༻Մೳ
• ϓϥάΠϯͷར༻Մೳ
• ΞοϓάϨʔυ؆୯
• ΧελϜࣙॻɺϓϥάΠϯར༻Մೳ
• 14ؒͷ͓ࢼ͠ظؒ͋Γ
• SLAϕʔεͷαϒεΫϦϓγϣϯ+༻ϓϥάΠϯ
͕ར༻ՄೳͳαʔϏε
116
Slide 117
Slide 117 text
ࢀߟจݙ
• Elasticsearch - The Definitive guide
‒ http://www.elastic.co/guide/en/elasticsearch/guide/current/index.html
• ॻ੶ʢຊޠʣ
‒ ElasticSearchServerຊޠ൛
αʔό/ΠϯϑϥΤϯδχΞ
ɹཆಡຊɹϩάऩू
117
Slide 118
Slide 118 text
ࢀߟαΠτ
• Ϣʔεέʔε
• https://www.elastic.co/use-cases
• DiscussʢWebϑΥʔϥϜʣ
• https://discuss.elastic.co
• Elastic{ON}ͷϏσΦͱࢿྉ
• https://www.elastic.co/elasticon/videos
• αϙʔτϝχϡʔ
• https://www.elastic.co/subscriptions
118
Slide 119
Slide 119 text
Thanks for listening!
Q & A
119
We’re hiring!
https://www.elastic.co/about/careers/
We’re helping!
https://www.elastic.co/subscriptions
http://training.elastic.co