Slide 1
Slide 1 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Jun Ohtani
@johtani
2014/11/15
ElasticsearchͱKibanaͰ͡ΊΔ
ݕࡧˍΞφϦςΟΫε
Slide 2
Slide 2 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ΞδΣϯμ
• ݕࡧͱͯ͠ͷElasticsearch
• ղੳͱͯ͠ͷElasticsearch
• KibanaͰՄࢹԽ
Slide 3
Slide 3 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
about
• Me, Jun Ohtani / Technical Adovocate
lucene-gosenίϛολʔ
ElasticSearch Serverຊޠ൛ͷ༁
elasticsearch-extended-analysisͷ։ൃ
http://blog.johtani.info
• Elasticsearch, founded in 2012
Products: Elasticsearch, Logstash, Kibana, Marvel
Professional services: Support & development subscriptions
Trainings
Slide 4
Slide 4 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧͱͯ͠ͷElasticsearch
Slide 5
Slide 5 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Elasticsearchͱʁ
Slide 6
Slide 6 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ϑϦʔϫʔυݕࡧ
Slide 7
Slide 7 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ߜΓࠐΈ
Slide 8
Slide 8 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ϋΠϥΠτ
Slide 9
Slide 9 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ιʔτ
Slide 10
Slide 10 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ϖʔδϯά
Slide 11
Slide 11 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ूܭ
Slide 12
Slide 12 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
αδΣετ
Slide 13
Slide 13 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Elasticsearch in 10 seconds
• εΩʔϚϑϦʔɺࢄυΩϡϝϯτετΞɺREST
& JSON
• Φʔϓϯιʔε: Apache License 2.0
• ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ
• JavaͰ࣮ɻ֦ு༰қ
Slide 14
Slide 14 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Powerful Search at Scale
Slide 15
Slide 15 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Πϯετʔϧ͔Βݕࡧ·Ͱ
Slide 16
Slide 16 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
μϯϩʔυͱىಈ
$ wget https://download.elasticsearch.org/...
$ tar -xf elasticsearch-1.4.0.tar.gz
$ ./elasticsearch-1.4.0/bin/elasticsearch
...
[2014-11-12 16:07:13,346][INFO ][node] [Araña] started
...
Also puppet modules and RPM/DEB
Slide 17
Slide 17 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ىಈͷ֬ೝ
» curl localhost:9200
{
"status" : 200,
"name" : "Araña"
"cluster_name" : "elasticsearch",
"version" : {
"number" : "1.4.0",
"build_hash" : "bc94bd81298f81c656893ab1ddddd30a99356066",
"build_timestamp" : "2014-11-05T14:26:12Z",
"build_snapshot" : false,
"lucene_version" : "4.10.2"
},
"tagline" : "You Know, for Search"
}
Slide 18
Slide 18 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
» curl -XPUT localhost:9200/books/book/1 -d '
{
"title" : "Elasticsearch - The definitive guide",
"authors" : "Clinton Gormley",
"started" : "2013-02-04",
"pages" : 230
}'
σʔλొ
Slide 19
Slide 19 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
» curl -XPUT localhost:9200/books/book/1 -d '
{
"title" : "Elasticsearch - The definitive guide",
"authors" : [ "Clinton Gormley", "Zachary Tong" ],
"started" : "2013-02-04",
"pages" : 230
}'
σʔλߋ৽
Slide 20
Slide 20 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
σʔλআ
» curl -X DELETE localhost:9200/books/book/1
σʔλͷऔಘ
» curl —X GET localhost:9200/books/book/1
» curl —X GET localhost:9200/books/book/1/_source
Slide 21
Slide 21 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
» curl -XGET localhost:9200/books/_search?q=elasticsearch
{
"took" : 2, "timed_out" : false,
"_shards" : { "total" : 5, "successful" : 5, "failed" : 0 },
"hits" : {
"total" : 1, "max_score" : 0.076713204,
"hits" : [ {
"_index" : “books", "_type" : “book", "_id" : "1",
"_score" : 0.076713204, "_source" : {
"title" : "Elasticsearch - The definitive guide",
"authors" : [ "Clinton Gormley", "Zachary Tong" ],
"started" : “2013-02-04", "pages" : 230
}
} ]
}
}
Slide 22
Slide 22 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
» curl -XGET ‘localhost:9200/books/book/_search' -d '{
"query": {
"filtered" : {
"query" : {
"match": {
"text" : {
"query" : “To Be Or Not To Be",
"cutoff_frequency" : 0.01
}
}
},
"filter" : {
"range": {
"price": {
"gte": 20.0
"lte": 50.0
...
}
}'
ݕࡧ - Query DSL
Slide 23
Slide 23 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
σϞɿىಈ͔Βݕࡧ·Ͱ
Slide 24
Slide 24 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ࢄߏɺεέʔϧ
Slide 25
Slide 25 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Basic terms
• ΠϯσοΫε
σʔλͷཧతͳू߹ɻ
RDBͷσʔλϕʔεͷΑ͏ͳͷLogical
• ϨϓϦέʔγϣϯ
ಡΈࠐΈͷεέʔϥϏϦςΟ্
SPOFͷղফ
• γϟʔσΟϯά
ෳϚγϯσʔλΛׂ
ॻ͖ࠐΈͷεέʔϥϏϦςΟ্
σʔλϑϩʔ੍ޚ
Slide 26
Slide 26 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
γϟʔυͱϨϓϦΧ
node 1
orders
products
1
4
1 2
2
2
curl
-‐X
PUT
localhost:9200/orders
-‐d
'{
"settings.index.number_of_shards"
:
4
"settings.index.number_of_replicas"
:
1
}'
curl
-‐X
PUT
localhost:9200/products
-‐d
'{
"settings.index.number_of_shards"
:
2
"settings.index.number_of_replicas"
:
0
}'
Slide 27
Slide 27 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
γϟʔυͱϨϓϦΧ
node 1
orders
products
1
4
1
node 2
orders
products
2
2
3 4
1 2
3
Slide 28
Slide 28 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ࣗಈతͳࢄ
node 1
orders
products
2
1
4
1
node 2
orders
products
2
2
node 3
orders
products
3 4
1
3
Slide 29
Slide 29 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
σϞ for Elasticsearch
ຊޠWikipedia
Slide 30
Slide 30 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
શจݕࡧͱʁ
Slide 31
Slide 31 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
શจݕࡧͱʁ
• શจݕࡧʢFull text searchʣͱɺίϯϐϡʔλʹ
͓͍ͯɺෳͷจॻʢϑΝΠϧʣ͔Βಛఆͷจࣈ
ྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯʮ୯Ұ
ϑΝΠϧͷจࣈྻݕࡧʯͱҟͳΓɺʮෳจॻ
ʹ·͕ͨͬͯɺจॻʹؚ·ΕΔશจΛରͱͨ͠
ݕࡧʯͱ͍͏ҙຯͰ༻͞ΕΔɻ
ʢWikipediaΑΓʣ
Slide 32
Slide 32 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
༻ޠ
• ΠϯσοΫε
ݕࡧΤϯδϯ͕ݕࡧʹ༻͢Δσʔλͷอଘઌ
• υΩϡϝϯτʢจॻʣ
ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ
• ϑΟʔϧυ
υΩϡϝϯτʹؚ·ΕΔଐੑ
• ΫΤϦ
ݕࡧ݅ɺݕࡧࣜ
Slide 33
Slide 33 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
༻ޠ
• εΩʔϚ
υΩϡϝϯτͷߏΛఆٛ͢Δͷ
• λʔϜʢTermʣɺτʔΫϯʢTokenʣ
ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ
จষΛҰఆͷ๏ଇͰ۠ͬͨ୯ޠ
୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲؚΉ
Slide 34
Slide 34 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
υΩϡϝϯτͷొ
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
υΩϡϝϯτͷొ
Slide 35
Slide 35 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
υΩϡϝϯτͷొ
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
1
2
ΧπΦ
αβΤ
ͷ
ͷ
αβΤ
ϫΧϝ
ఋ
࢞
υΩϡϝϯτͷొ
୯ޠʹׂ
Slide 36
Slide 36 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
υΩϡϝϯτͷొ
1
2
ΧπΦαβΤͷఋ
αβΤϫΧϝͷ࢞
1
2
ΧπΦ
αβΤ
ͷ
ͷ
αβΤ
ϫΧϝ
ఋ
࢞
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
υΩϡϝϯτͷొ
୯ޠʹׂ
୯ޠ͔Βidͷྻ͕
Ҿ͚ΔΑ͏ʹ
Slide 37
Slide 37 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ݕࡧ݅ೖྗ
ΧπΦɹαβΤ
Slide 38
Slide 38 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 39
Slide 39 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 40
Slide 40 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 41
Slide 41 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 42
Slide 42 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 43
Slide 43 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 44
Slide 44 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧ
ΧπΦ
αβΤ
1
1 2
ͷ
࢞
ϫΧϝ 2
1 2
1 2
1
ఋ
2
ΧπΦ αβΤ
AND
ݕࡧ݅ೖྗ
ݕࡧ݅ͷύʔε
ݕࡧΫΤϦԽ
ΧπΦɹαβΤ
Slide 45
Slide 45 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
୯ޠͷ۠Γํ
• ӳޠͷ߹
I am speaking Introduction Elasticsearch.
ɹ
ɹ
• ຊޠͷ߹
ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ
Slide 46
Slide 46 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
୯ޠͷ۠Γํ
• ӳޠͷ߹
I am speaking Introduction Elasticsearch.
εϖʔε͕ΕͱΘ͔Δ
• ຊޠͷ߹
ࢲೖElasticsearchʹ͍͍ͭͯͯ͠Δɻ
Ͳ͜Ͱ۠ΕΑ͍ʁ
Slide 47
Slide 47 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
N-Gramͱܗଶૉղੳ
• సஔΠϯσοΫεͷΩʔͷ࡞Γํ
ຊޠ୯ޠͷΕ͕Θ͔Βͳ͍ͷͰɺసஔΠϯσοΫεͷ
Ωʔओʹ࣍ͷ̎ͭͷख๏Ͱ࡞
• N-Gram
NจࣈͣͭจষΛ۠Δ
• ܗଶૉղੳ
ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠Δ
Slide 48
Slide 48 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ܗଶૉղੳ
• ϝϦοτɿ
ҙຯͷ͋Δ୯ޠͷΕ
ࢺใΛݩʹՃॲཧ͕ՄೳʢޠװมͳͲʣ
• σϝϦοτɿ
৽ޠʢະޠʣʹऑ͍ˠࣙॻϕʔεͷ߹ɺࣙॻʹͳ͍୯ޠݕ
ग़ෆೳɻ
ΧπΦαβΤͷఋ
ΧπΦ ͷ
αβΤ ఋ
Slide 49
Slide 49 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
N-Gram
• ϝϦοτɿ
ະޠʹରԠՄೳ
• σϝϦοτɿ
ΠϯσοΫεංେԽ
ࢺใʹجͮ͘ॲཧ͕ෆՄೳ
ΧπΦαβΤͷఋ
Χπ πΦ Φ α αβ βΤ Τͷ ͷఋ
Slide 50
Slide 50 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Analysisͱʁ
• సஔΠϯσοΫεͷ୯ޠʢTermʣΛυΩϡϝϯτ
͔Βநग़͢Δॲཧ
• ϑΟʔϧυຖʹࢦఆ͞ΕͨΞφϥΠβ͕ॲཧ
• ΞφϥΠβCharFilterɺTokenizerɺTokenFilter͔
Βߏ
• ΠϯσοΫε࣌ɺݕࡧ࣌ʹ࣮ߦ
Slide 51
Slide 51 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Analysisͷ֓ཁ
Analysis
υΩϡϝϯτ
Term Id
ΧπΦ 1
αβΤ 1ɺ2
ϫΧϝ 2
… ...
ΠϯσοΫε
Slide 52
Slide 52 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Analysisͷ֓ཁ
Analysis
ݕࡧ
Ωʔϫʔυ
ΠϯσοΫε
ΠϯσοΫε
ݕࡧॲཧ
ݕࡧ݁Ռ
Term Id
ΧπΦ 1
αβΤ 1ɺ2
ϫΧϝ 2
… ...
Slide 53
Slide 53 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Analyzer
Analysisͷߏ
char_filter tokenizer
char_filter
token_
filter
token_
filter
Text Tokens
Slide 54
Slide 54 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Analyzerͷઃఆ
{"index":{
“analysis":{
"analyzer" : {
"my_analyzer" : {
"type" : "custom",
"tokenizer" : “kuromoji_tokenizer”,
“char_filter" : [“char_filter1”,…],
“filter" : [“token_filter1”,…]
}
}
Slide 55
Slide 55 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Char Filter
• ೖྗจࣈྻΛจࣈ୯ҐͰॲཧ
• ྫɿhtml_strip
ೖྗɿ
ग़ྗɿ
Elasticsearch is not a service of AWS
Elasticsearch is not a service of AWS
Slide 56
Slide 56 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Tokenizer
• ೖྗจࣈྻΛτʔΫϯྻʹׂ
• τʔΫϯͷׂ͢ΔϩδοΫτʔΫφΠβʹ
ґଘ
• ྫɿstandard
ೖྗɿ
ग़ྗɿ Elasticsearch is not a service of AWS
Elasticsearch is not a service of AWS
Slide 57
Slide 57 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Tokenizer
• ྫɿkeyword
ग़ྗɿ
• ྫɿkuromoji_tokenizer
ೖྗɿ
ग़ྗɿ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ
Elasticsearch is not a service of AWS
ण͕࢘ඒຯ͔ͬͨ͠
Slide 58
Slide 58 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
TokenFilter
• TokenizerʹΑΓग़ྗ͞ΕͨTokenྻʹରͯ͠ॲཧ
• ྫɿlowercase
ೖྗɿ
ग़ྗɿ
• ྫɿstop
ೖྗɿ
ग़ྗɿ
Elasticsearch is not a service of AWS
elasticsearch is not a service of aws
Elasticsearch is not a service of AWS
Elasticsearch service AWS
Slide 59
Slide 59 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
TokenFilter
• ྫɿkuromoji_baseform
ೖྗɿ
ग़ྗɿ
• ྫɿkuromoji_readingform
ೖྗɿ
ग़ྗɿ
ण࢘ ͕ ඒຯ͔ͬ͠ ͨ
ण࢘ ͕ ඒຯ͍͠ ͨ
ण࢘ ͕ ඒຯ͔ͬ͠ ͨ
sushi ga oishika ta
Slide 60
Slide 60 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Mapping
• ΠϯσοΫεͷߏʢschemaʣΛఆٛ
• Schema less!͚ͩͲ…
• ܗࣜΛࢦఆʢanalyze͢Δ/͠ͳ͍ɺ/গɺ
ͳͲʣ
Slide 61
Slide 61 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿWikipediaͷMappingͷҰ෦
"mappings": {
"page": {
"properties": {
"link": {
"type": "string",
"fields": {
"raw": {"type": "string",
"index": "not_analyzed"} } },
"text": { "type": "string"},
"title": {
"type": "string",
"fields": {
"raw": {"type": "string",
"index": "not_analyzed"} }…
Slide 62
Slide 62 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ͦͷଞͷػೳ
Slide 63
Slide 63 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Geo
• ͞·͟·ͳܗࣜͷσʔλͰGeoݕࡧՄೳ
• σʔλɿ
ҢܦɺGeoHashɺGeoShape
• ݕࡧɿ
geo shape queryɺgeo shape filterɺgeo distance filterɺ
geo distance range filterɺgeo bounding box filterɺ
geo polygon filter
Slide 64
Slide 64 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Percolator
• υΩϡϝϯτͰͳ͘ΫΤϦΛొ
• _percolate APIʹυΩϡϝϯτΛૹ৴
• ొ͞ΕͨΫΤϦʹϚονͨ͠߹ɺ
Ϛονͨ͠ΫΤϦΛฦ͢
Slide 65
Slide 65 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Snapshot/Restore
• 1.0͔Βಋೖ
• ΠϯσοΫε୯ҐͰSnapshot/RestoreՄೳ
• อଘઌʢRepositoryʣ
Shard FSɺS3ɺHDFSɺAzure…
Slide 66
Slide 66 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Ecosystem
• Plugins
ϓϥάΠϯʹΑΔػೳͷՃ
• ΫϥΠΞϯτϥΠϒϥϦ
Ruby, python, php, perl, javascript, .NET
Scala, clojure, go
• Hadoop integration
Slide 67
Slide 67 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ৄ͘͠Γ͍ͨํ
• Elasticsearch - The Definitive guide
http://www.elasticsearch.com/guide/en/elasticsearch/guide/
current/index.html
Slide 68
Slide 68 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ղੳͱͯ͠ͷElasticsearch
Slide 69
Slide 69 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ELK stack
Slide 70
Slide 70 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ELK stack
Data Import/Parse/
Export
Store/Search
Visualize
Slide 71
Slide 71 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ELK stack
Import/Parse/
Export
Store/Search
Visualize
Data
Slide 72
Slide 72 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Logstash
Slide 73
Slide 73 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Logstash in 10 seconds
• ϩάɾσʔλͷऩूɾཧ
• ऩूɺύʔεɾՃɺૹग़
• Apache License 2.0
• Ruby app (JRuby)
Slide 74
Slide 74 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Logstash architecture
Logstash
Input Output
Filter
? ?
collect and split alter and enrich store and visualize
Slide 75
Slide 75 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ELK stack
Import/Parse/
Export
Store/Search
Visualize
Data
Slide 76
Slide 76 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
aggregation
Slide 77
Slide 77 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregationͱ
• 1.0͔Βಋೖ
• FacetΑΓڧྗͳूܭͳͲ͕Մೳ
֊తͳूܭɺάϧʔϓԽ
ಈతͳूܭɺάϧʔϓԽ
• େ͖͘2छྨ
BucketɹυΩϡϝϯτΛ͝ͱʹ݁ՌΛάϧʔϐϯά
Metricɹ υΩϡϝϯτͷ࣋ͭΛूܭ
Slide 78
Slide 78 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿݴޠ͝ͱͷϑΝηοτ
» curl -XGET twitter-2014.08.22/_search -d '
GET twitter-2014.08.22/_search
{
"facets": {
"lang_facet": {
"terms": {"field": "lang"}
}
}
}
Slide 79
Slide 79 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿݴޠ͝ͱͷϑΝηοτ
"facets": {
"lang_facet": {
"_type": "terms",
"missing": 0,
"total": 4521075,
"other": 320228,
"terms": [
{
"term": "en",
"count": 1471819
},
{
"term": "ja",
"count": 980145
},
Slide 80
Slide 80 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿݴޠ͓ΑͼҬͷूܭ
» curl -XGET twitter-2014.08.22/_search -d '
{
"aggs": {
"lang": {
"terms": {"field": "lang" },
"aggs": {
"place": {
"terms": {
"field": “place.full_name", "size": 10
}
}
}
}
}
}
Slide 81
Slide 81 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿݴޠ͓ΑͼҬͷूܭ
"aggregations": {
"lang": {
"buckets": [{…}, {
"key": "ja",
"doc_count": 980145,
"place": {
"buckets": [
{ "key": "ژࢢ෬ݟ۠, ژ",
"doc_count":252 },
{ "key": "ઍా۠, ౦ژ",
"doc_count": 39 },…
Slide 82
Slide 82 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
bucket
• filter
• filters (>=1.4.0)
• missing
• nested
• reverse nested
(>=1.2.0)
• children
• terms
• significant terms
(>=1.1.0)
• range
• date range
• ipv4 range
• histogram
• date histogram
• geo distance
• geohash grid
Slide 83
Slide 83 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
metric
• min
• max
• sum
• avg
• stats
• extended_stats
• value count
• percentiles (>=1.1.0)
• percentile ranks (>=1.3)
• cardinality (>=1.1.0)
• geo bounds (>=1.3.0)
• top_hits (>=1.3.0)
• scripted metric
(>= 1.4.0)
Slide 84
Slide 84 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
σϞɿAggregation
Slide 85
Slide 85 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
KibanaͰՄࢹԽ
Slide 86
Slide 86 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Kibana3
• ElasticsearchͷσʔλΛՄࢹԽ
• In-browserʢJavaScriptɿAngularJSɺD3ͳͲʣ
• Apache License 2.0
Slide 87
Slide 87 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Kibana3
Slide 88
Slide 88 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Kibana3
Slide 89
Slide 89 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Kibana3
Slide 90
Slide 90 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
σϞ for Kibana3
Twitter Stream
Slide 91
Slide 91 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Slide 92
Slide 92 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Kibana4 - Beta
• BetaϦϦʔε൛ʂ
• ElasticsearchͷσʔλΛՄࢹԽ
• JRuby server & JavaScript
• Apache License 2.0
Slide 93
Slide 93 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
σϞ for Kibana4
Access log
Slide 94
Slide 94 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Combining Search and Analytics
Slide 95
Slide 95 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Marvel
Slide 96
Slide 96 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Overview
Slide 97
Slide 97 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Node statistics
Slide 98
Slide 98 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Index statistics
Slide 99
Slide 99 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Cluster Pulse
Slide 100
Slide 100 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Sense
Slide 101
Slide 101 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ࢀߟจݙ
• Elasticsearch - The Definitive guide
http://www.elasticsearch.com/guide/en/elasticsearch/guide/
current/index.html
• ॻ੶ʢຊޠʣ
ElasticSearchServerຊޠ൛
αʔό/ΠϯϑϥΤϯδχΞ
ɹཆಡຊɹϩάऩू
Slide 102
Slide 102 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Thanks for listening!
Q & A
We’re hiring!
http://elasticsearch.com/about/jobs
We’re helping!
http://elasticsearch.com/support
http://elasticsearch.com/training