Slide 1
Slide 1 text
Elastic StackͰ͡ΊΔ
ϩάղੳೖ
Jun Ohtani
@johtani
2016/01/29
Slide 2
Slide 2 text
No content
Slide 3
Slide 3 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
3
• Elastic stackհ
• Logstash - Logऩू
• Elasticsearch - ݕࡧɾղੳ
• Kibana - ՄࢹԽ
• ༻ϓϥάΠϯհ
ΞδΣϯμ
Slide 4
Slide 4 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
4
about
• Me, Jun Ohtani / Technical Adovocate
– lucene-gosenίϛολʔ
– ElasticSearch Serverຊޠ൛ͷ༁
– elasticsearch-extended-analysisͷ։ൃ
– http://blog.johtani.info
• Elasticsearch, founded in 2012
– Products: Elasticsearch, Logstash, Kibana, Marvel, Sheild
Professional services: Support & development subscriptions
– Trainings
Slide 5
Slide 5 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
5
ElasticελοΫ
ElasticελοΫ
Slide 6
Slide 6 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
6
ElasticελοΫ
ML
K
B LE
a
B K
B
SWFB
(
M
)
W
)
Slide 7
Slide 7 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
7
ElasticελοΫɿOpen Source෦
ML
K
B LE
a
B K
B
SWFB
(
M
)
W
)
Slide 8
Slide 8 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
8
Elastic stack
Data Import/Parse/
Export
Store/Search
Visualize
Slide 9
Slide 9 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
9
Logstash
Logstash
Slide 10
Slide 10 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
10
Elastic stack
Import/Parse/
Export
Store/Search
Visualize
Data
Slide 11
Slide 11 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
11
Logstash in 10 seconds
• ϩάɾσʔλͷऩूɾཧ
• ऩूɺύʔεɾՃɺૹग़
• ΦʔϓϯιʔεɿApache License 2.0
• Ruby app (JRuby)
• ࠷৽൛ɿ2.1.1
Slide 12
Slide 12 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
12
Logstash architecture
Logstash
Input Output
Filter
? ?
collect and split alter and enrich store and visualize
Slide 13
Slide 13 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
13
ઃఆ
input {
…
}
filter {
…
}
output {
…
}
Slide 14
Slide 14 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
14
ઃఆɿinput
input {
file {
path => “/Users/johtani/sample/*_log"
start_position => "beginning"
}
}
Slide 15
Slide 15 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
15
1ߦ1σʔλ
189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/
1.1" 404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101
Firefox/5.0"
Slide 16
Slide 16 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
16
ઃఆɿfilter
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 17
Slide 17 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
17
ύʔε
189.120.xx.xx - - [02/Dec/2014:12:18:29 +0900] "GET /manager/html HTTP/
1.1" 404 274 "-" "Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101
Firefox/5.0"
{…
"@timestamp": "2015-04-10T09:07:49.325Z",
"clientip": "189.120.xx.xx",
"ident": "-",
"auth": "-",
"timestamp": "02/Dec/2014:12:18:29 +0900",
"verb": "GET",
"request": "/manager/html",
…
"agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0\""
}
Slide 18
Slide 18 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
18
ઃఆɿfilter
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 19
Slide 19 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
19
ͷύʔε
{…
"@timestamp": "2015-04-10T09:07:49.325Z",
…
"timestamp": "02/Dec/2014:12:18:29 +0900",
…
}
{…
"@timestamp": "2014-12-02T03:18:29.000Z",
…
"timestamp": "02/Dec/2014:12:18:29 +0900",
…
}
Slide 20
Slide 20 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
20
ઃఆɿfilter
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 21
Slide 21 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
21
IP͔ΒҢܦͳͲ༩
{…
"clientip": "189.120.xx.xx",
…}
{…
"clientip": "189.120.xx.xx",
…
"geoip": {
"ip": “189.120.xxx.xxx”,
…
"country_name": "Brazil",
"continent_code": "SA",
"region_name": "27",
"city_name": "São Paulo",
"latitude": -23.473299999999995,
"longitude": -46.66579999999999,
…
Slide 22
Slide 22 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
22
ઃఆɿfilter
filter {
grok {
match => { "message" => "%{COMBINEDAPACHELOG}" }
break_on_match => false
}
date {
match => ["timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]
locale => en
}
geoip { source => ["clientip"] }
useragent {
source => "agent"
target => "useragent"
}
}
Slide 23
Slide 23 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
23
ϢʔβΤʔδΣϯτͷύʔε
{…
"agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0\""
…}
{…
"agent": "\"Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0\""
…
"useragent": {
"name": "Firefox",
"os": "Windows XP",
"os_name": "Windows XP",
"device": "Other",
"major": "5",
"minor": "0"
}
…
Slide 24
Slide 24 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
24
ઃఆɿoutput
output {
elasticsearch {
hosts => ["localhost"]
index => “demo_access_log-%{+YYYY.MM.dd}”
}
}
Slide 25
Slide 25 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
25
Elasticsearch
Elasticsearch
Slide 26
Slide 26 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
26
ݕࡧͱͯ͠ͷ
Elasticsearch
ݕࡧͱͯ͠ͷElasticsearch
Slide 27
Slide 27 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
27
Elasticsearchͱʁ
Elasticsearchͱʁ
Slide 28
Slide 28 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
28
ϑϦʔϫʔυݕࡧ
Slide 29
Slide 29 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
29
ߜΓࠐΈ
Slide 30
Slide 30 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
30
ϋΠϥΠτ
Slide 31
Slide 31 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
31
ιʔτ
Slide 32
Slide 32 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
32
ϖʔδϯά
Slide 33
Slide 33 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
33
ूܭ
Slide 34
Slide 34 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
34
αδΣετ
Slide 35
Slide 35 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
35
Elasticsearch in 10 seconds
• εΩʔϚϑϦʔɺࢄυΩϡϝϯτετΞɺREST & JSON
• Φʔϓϯιʔε: Apache License 2.0
• ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ
• JavaͰ࣮ɻ֦ு༰қ
Slide 36
Slide 36 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
36
Powerful Search at Scale
Slide 37
Slide 37 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
37
؆୯ͳCRUD
؆୯ͳCRUD
Slide 38
Slide 38 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
38
» curl -XPUT localhost:9200/books/book/1 -d '
{
"title" : "Elasticsearch - The definitive guide",
"authors" : "Clinton Gormley",
"started" : "2013-02-04",
"pages" : 230
}'
σʔλొ
Slide 39
Slide 39 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
39
» curl -XPUT localhost:9200/books/book/1 -d '
{
"title" : "Elasticsearch - The definitive guide",
"authors" : [ "Clinton Gormley", "Zachary Tong" ],
"started" : "2013-02-04",
"pages" : 230
}'
σʔλߋ৽
Slide 40
Slide 40 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
40
σʔλআ
» curl -X DELETE localhost:9200/books/book/1
σʔλͷऔಘ
» curl —X GET localhost:9200/books/book/1
» curl —X GET localhost:9200/books/book/1/_source
Slide 41
Slide 41 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
41
ݕࡧ
» curl -XGET localhost:9200/books/_search?q=elasticsearch
{
"took" : 2, "timed_out" : false,
"_shards" : { "total" : 5, "successful" : 5, "failed" : 0 },
"hits" : {
"total" : 1, "max_score" : 0.076713204,
"hits" : [ {
"_index" : “books", "_type" : “book", "_id" : "1",
"_score" : 0.076713204, "_source" : {
"title" : "Elasticsearch - The definitive guide",
"authors" : [ "Clinton Gormley", "Zachary Tong" ],
"started" : “2013-02-04", "pages" : 230
}
} ]
}
}
Slide 42
Slide 42 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
42
» curl -XGET ‘localhost:9200/books/book/_search' -d '{
"query": {
"filtered" : {
"query" : {
"match": {
"text" : {
"query" : “To Be Or Not To Be",
"cutoff_frequency" : 0.01
}
}
},
"filter" : {
"range": {
"price": {
"gte": 20.0
"lte": 50.0
...
}
}'
ݕࡧ - Query DSL
Slide 43
Slide 43 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
43
ࢄߏ
εέʔϧ
ࢄߏɺεέʔϧ
Slide 44
Slide 44 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
44
Basic terms
• ΠϯσοΫε
– σʔλͷཧతͳू߹ɻ
RDBͷσʔλϕʔεͷΑ͏ͳͷLogical
• ϨϓϦέʔγϣϯ
• ಡΈࠐΈͷεέʔϥϏϦςΟ্
• SPOFͷղফ
• γϟʔσΟϯά
• ෳϚγϯσʔλΛׂ
ॻ͖ࠐΈͷεέʔϥϏϦςΟ্
σʔλϑϩʔ੍ޚ
Slide 45
Slide 45 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
45
γϟʔυͱϨϓϦΧ
node 1
orders
products
1
4
1 2
2
2
curl -X PUT localhost:9200/orders -d '{
"settings.index.number_of_shards" : 4
"settings.index.number_of_replicas" : 1
}'
curl -X PUT localhost:9200/products -d '{
"settings.index.number_of_shards" : 2
"settings.index.number_of_replicas" : 0
}'
Slide 46
Slide 46 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
46
γϟʔυͱϨϓϦΧ
node 1
orders
products
1
4
1
node 2
orders
products
2
2
3 4
1 2
3
Slide 47
Slide 47 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
47
ࣗಈతͳࢄ
node 1
orders
products
2
1
4
1
node 2
orders
products
2
2
node 3
orders
products
3 4
1
3
Slide 48
Slide 48 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
48
શจݕࡧͱʁ
શจݕࡧͱʁ
Slide 49
Slide 49 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
49
શจݕࡧͱʁ
• શจݕࡧʢFull text searchʣͱɺίϯϐϡʔλʹ͓͍ͯɺෳͷจ
ॻʢϑΝΠϧʣ͔ΒಛఆͷจࣈྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕ
ࡧʯʮ୯ҰϑΝΠϧͷจࣈྻݕࡧʯͱҟͳΓɺʮෳจॻʹ·ͨ
͕ͬͯɺจॻʹؚ·ΕΔશจΛରͱͨ͠ݕࡧʯͱ͍͏ҙຯͰ༻͞
ΕΔɻ
ʢWikipediaΑΓʣ
Slide 50
Slide 50 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
50
༻ޠ
• ΠϯσοΫε
ݕࡧΤϯδϯ͕ݕࡧʹ༻͢Δσʔλͷอଘઌ
• υΩϡϝϯτʢจॻʣ
– ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ
• ϑΟʔϧυ
– υΩϡϝϯτʹؚ·ΕΔଐੑ
• ΫΤϦ
– ݕࡧ݅ɺݕࡧࣜ
Slide 51
Slide 51 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
51
༻ޠ
• εΩʔϚ
– υΩϡϝϯτͷߏΛఆٛ͢Δͷ
• λʔϜʢTermʣɺτʔΫϯʢTokenʣ
– ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ
– จষΛҰఆͷ๏ଇͰ۠ͬͨ୯ޠ
– ୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲؚΉ
Slide 52
Slide 52 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
52
υΩϡϝϯτͷొ
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
υΩϡϝϯτͷొ
Slide 53
Slide 53 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
53
υΩϡϝϯτͷొ
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
1
2
ΧπΦ
αβΤ
ͷ
ͷ
αβΤ
ϫΧϝ
ఋ
࢞
υΩϡϝϯτͷొ
୯ޠʹׂ
Slide 54
Slide 54 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
54
υΩϡϝϯτͷొ
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
1
2
ΧπΦ
αβΤ
ͷ
ͷ
αβΤ
ϫΧϝ
ఋ
࢞
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
υΩϡϝϯτͷొ
୯ޠʹׂ
୯ޠ͔Βidͷྻ͕
Ҿ͚ΔΑ͏ʹ
Slide 55
Slide 55 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
55
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ݕࡧ݅ೖྗ
ΧπΦɹαβΤ
Slide 56
Slide 56 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
56
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 57
Slide 57 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
57
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 58
Slide 58 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
58
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 59
Slide 59 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
59
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 60
Slide 60 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
60
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 61
Slide 61 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
61
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 62
Slide 62 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
62
ͦͷଞͷػೳ
ͦͷଞͷػೳ
Slide 63
Slide 63 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
63
Geo
• ͞·͟·ͳܗࣜͷσʔλͰGeoݕࡧՄೳ
• σʔλɿ
• ҢܦɺGeoHashɺGeoShape
• ݕࡧɿ
• geo shape queryɺgeo shape filterɺgeo distance filterɺ
• geo distance range filterɺgeo bounding box filterɺ
• geo polygon filter
Slide 64
Slide 64 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
64
Percolator
• υΩϡϝϯτͰͳ͘ΫΤϦΛొ
• _percolate APIʹυΩϡϝϯτΛૹ৴
• ొ͞ΕͨΫΤϦʹϚονͨ͠߹ɺ
Ϛονͨ͠ΫΤϦΛฦ͢
Slide 65
Slide 65 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
65
Snapshot/Restore
• 1.0͔Βಋೖ
• ΠϯσοΫε୯ҐͰSnapshot/RestoreՄೳ
• อଘઌʢRepositoryʣ
– Shard FSɺS3ɺHDFSɺAzure…
Slide 66
Slide 66 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
66
Ecosystem
• Plugins
– ϓϥάΠϯʹΑΔػೳͷՃ
• ΫϥΠΞϯτϥΠϒϥϦ
• Ruby, python, php, perl, javascript, .NET
• Scala, clojure, go
• Hadoop integration
Slide 67
Slide 67 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
67
ৄ͘͠Γ͍ͨํ
• Elasticsearch - The Definitive guide
– http://www.elastic.co/guide/en/elasticsearch/guide/current/
index.html
Slide 68
Slide 68 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
68
ղੳͱͯ͠ͷ
Elasticsearch
ղੳͱͯ͠ͷElasticsearch
Slide 69
Slide 69 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
69
aggregation
aggregation
Slide 70
Slide 70 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
70
Aggregationͱ
• 1.0͔Βಋೖ
• FacetΑΓڧྗͳूܭͳͲ͕Մೳ
• ֊తͳूܭɺάϧʔϓԽ
ಈతͳूܭɺάϧʔϓԽ
• େ͖͘2छྨ
• BucketɹυΩϡϝϯτΛ͝ͱʹ݁ՌΛάϧʔϐϯά
• Metricɹ υΩϡϝϯτͷ࣋ͭΛूܭ
Slide 71
Slide 71 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
71
ྫɿݴޠ͝ͱͷϑΝηοτ
» curl -XGET twitter-2014.08.22/_search -d '
GET twitter-2014.08.22/_search
{
"facets": {
"lang_facet": {
"terms": {"field": "lang"}
}
}
}
Slide 72
Slide 72 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
72
ྫɿݴޠ͝ͱͷϑΝηοτ
"facets": {
"lang_facet": {
"_type": "terms",
"missing": 0,
"total": 4521075,
"other": 320228,
"terms": [
{
"term": "en",
"count": 1471819
},
{
"term": "ja",
"count": 980145
},
Slide 73
Slide 73 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
73
ྫɿݴޠ͓ΑͼҬͷूܭ
» curl -XGET twitter-2014.08.22/_search -d '
{
"aggs": {
"lang": {
"terms": {"field": "lang" },
"aggs": {
"place": {
"terms": {
"field": “place.full_name", "size": 10
}
}
}
}
}
}
Slide 74
Slide 74 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
74
ྫɿݴޠ͓ΑͼҬͷूܭ
"aggregations": {
"lang": {
"buckets": [{…}, {
"key": "ja",
"doc_count": 980145,
"place": {
"buckets": [
{ "key": "ژࢢ෬ݟ۠, ژ",
"doc_count":252 },
{ "key": "ઍా۠, ౦ژ",
"doc_count": 39 },…
Slide 75
Slide 75 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
75
bucket
• filter
• filters (>=1.4.0)
• missing
• nested
• reverse nested (>=1.2.0)
• children
• terms
• significant terms (>=1.1.0)
• range
• date range
• ipv4 range
• histogram
• date histogram
• geo distance
• geohash grid
Slide 76
Slide 76 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
76
metric
• min
• max
• sum
• avg
• stats
• extended_stats
• value count
• percentiles (>=1.1.0)
• percentile ranks (>=1.3)
• cardinality (>=1.1.0)
• geo bounds (>=1.3.0)
• top_hits (>=1.3.0)
• scripted metric
(>= 1.4.0)
Slide 77
Slide 77 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
77
KibanaͰՄࢹԽ
KibanaͰՄࢹԽ
Slide 78
Slide 78 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
78
Kibana4
• ElasticsearchͷσʔλΛՄࢹԽ
• Node.js server & JavaScript
• Apache License 2.0
Slide 79
Slide 79 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
79
Kibana 4
Slide 80
Slide 80 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
80
σϞ for Kibana4
Access Log
demo for Kibana4
Slide 81
Slide 81 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
81
Combining Search and Analytics
Slide 82
Slide 82 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
82
ͦͷଞͷ
ElasticελοΫ
ͦͷଞͷElasticελοΫ
Slide 83
Slide 83 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
83
Beats
19
#!"
a ab
U EC W e
Pc i b c W e
g
b b ol st
hU g
c st r
L
Slide 84
Slide 84 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
84
elasticsearch-hadoop
20
-
• D E H
• PD ecd
ER
• g D
•
CH
• Ca M DMS
D FERC
Slide 85
Slide 85 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
85
༻ϓϥάΠϯ
༻ϓϥάΠϯ
Slide 86
Slide 86 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
86
Shield
Shield
Slide 87
Slide 87 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
87
Shieldͷಛ
• User Authentication
• LDAP/Active Directory
• ϑΝΠϧϕʔε
• Authorization
• ϩʔϧϕʔεͷΞΫηείϯτϩʔϧ
• ΠϯσοΫε͝ͱɺΞΫγϣϯ͝ͱͷઃఆ͕Մೳ
• υΩϡϝϯτɾϑΟʔϧυ͝ͱͷઃఆՄೳʹ
• ηΩϡΞͳ௨৴
• ElasticsearchϊʔυؒͷSSL/TLS
• IPϑΟϧλϦϯά
• ࠪϩά
Slide 88
Slide 88 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
88
ΨΠυͳͲ
• ϓϩμΫτϖʔδ
– https://www.elastic.co/products/shield
• ΨΠυ
– https://www.elastic.co/guide/en/shield/current/quick-getting-
started.html
Slide 89
Slide 89 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
89
Watcher
Watcher
Slide 90
Slide 90 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
90
Watcherͷಛ
• ΫΤϦʹΑΔWatch
– ElasticsearchͷΫΤϦΛར༻ͯ͠σʔλͷࢹ
• ݅ͷઃఆ
– ΞΫγϣϯΛ࣮ߦ͢Δ͔Ͳ͏͔ͷઃఆ
• εέδϡʔϧ
– ΫΤϦΛ࣮ߦ͠ɺ݅ΛνΣοΫ͢Δසͷࢦఆ
• ΞΫγϣϯͷఆٛ
– ϝʔϧͷૹ৴ɺଞγεςϜͷσʔλૹ৴ͳͲͷಈ࡞Λઃఆ
• ཤྺͷอଘ
– ElasticsearchͷΠϯσοΫεͱ࣮ͯ͠ߦ݁ՌΛอଘ
Slide 91
Slide 91 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
91
ΨΠυͳͲ
• ϓϩμΫτϖʔδ
– https://www.elastic.co/products/watcher
• ΨΠυ
– https://www.elastic.co/guide/en/watcher/current/index.html
Slide 92
Slide 92 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
92
found.no
Slide 93
Slide 93 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
93
found.no
• Elasticsearch as a Service
• Elasticsearchͷػೳ͕ར༻Մೳ
• ϓϥάΠϯͷར༻Մೳ
• ΞοϓάϨʔυ؆୯
• ΧελϜࣙॻɺϓϥάΠϯར༻Մೳ
• 14ؒͷ͓ࢼ͠ظؒ͋Γ
Slide 94
Slide 94 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
94
found premium
• SLAϕʔεͷαϙʔτ
– ElasticͷαϙʔτϝϯόʔʹΑΔαϙʔτ
• Shield
• Watcher
• Marvel
Slide 95
Slide 95 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
95
ࢀߟจݙ
• Elasticsearch - The Definitive guide
– http://www.elastic.co/guide/en/elasticsearch/guide/current/
index.html
• ॻ੶ʢຊޠʣ
– ElasticSearchServerຊޠ൛
αʔό/ΠϯϑϥΤϯδχΞ
ɹཆಡຊɹϩάऩू
Slide 96
Slide 96 text
www.elastic.co Copyright Elastic 2015 Copying, publishing and/or
distributing without written permission is strictly prohibited
96
ࢀߟαΠτ
• Ϣʔεέʔε
• https://www.elastic.co/use-cases
• DiscussʢWebϑΥʔϥϜʣ
• https://discuss.elastic.co
• Elastic{ON}ͷϏσΦͱࢿྉ
• https://www.elastic.co/elasticon/videos
• αϙʔτϝχϡʔ
• https://www.elastic.co/subscriptions
Slide 97
Slide 97 text
Thanks for listening!
Q & A
We’re hiring!
https://www.elastic.co/about/careers/
We’re helping!
https://www.elastic.co/subscriptions
http://purchases.elastic.co