Slide 1
Slide 1 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Jun Ohtani
@johtani
2014-09-16
Aggregation͋Ε͜Ε
Slide 2
Slide 2 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
about
• Me, Jun Ohtani / Technical Adovocate
lucene-gosenίϛολʔ
ElasticSearch Serverຊޠ൛ͷ༁
elasticsearch-extended-analysisͷ։ൃ
http://blog.johtani.info
• Elasticsearch, founded in 2012
Products: Elasticsearch, Logstash, Kibana, Marvel
Professional services: Support & development subscriptions
Trainings
Slide 3
Slide 3 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ΞδΣϯμ
• Elasticsearch?
• Aggregationͱ
• AggregationͷΈ
• Aggregationͷछྨ
!
Slide 4
Slide 4 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Elasticsearchʁ
Slide 5
Slide 5 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ϑϦʔϫʔυݕࡧ
Slide 6
Slide 6 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ߜΓࠐΈ
Slide 7
Slide 7 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ϋΠϥΠτ
Slide 8
Slide 8 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ιʔτ
Slide 9
Slide 9 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ϖʔδϯά
Slide 10
Slide 10 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
ूܭ
Slide 11
Slide 11 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited
αδΣετ
Slide 12
Slide 12 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Elasticsearch in 10 seconds
• εΩʔϚϑϦʔɺࢄυΩϡϝϯτετΞɺREST
& JSON
• Φʔϓϯιʔε: Apache License 2.0
• ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ
• JavaͰ࣮ɻ֦ு༰қ
Slide 13
Slide 13 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregation
Slide 14
Slide 14 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregationͱ
• 1.0͔Βར༻Մೳ
• ݕࡧ݁Ռʹରͯ͠ɺσʔλͷղੳ͕༰қʹ
• FacetΑΓڧྗͳूܭͳͲ͕Մೳ
֊తͳूܭɺάϧʔϓԽ
ಈతͳूܭɺάϧʔϓԽ
• Facetɺdeprecatedɻ2.xܥͰͳ͘ͳΔ
!
Slide 15
Slide 15 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregationͱ
• SQLͩͱ
!
!
• GROUP BY colorɿBucket
• COUNT(color)ɿMetric
SELECT
COUNT(color)
FROM
table
GROUP
BY
color
Slide 16
Slide 16 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregations
Documents
Slide 17
Slide 17 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregations
Documents
Query
Slide 18
Slide 18 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregations
Documents
Query
Buckets
Slide 19
Slide 19 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregations
Documents
Query
Buckets
Slide 20
Slide 20 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregations
Documents
Query
Buckets
Metrics 123
123
243 185
Slide 21
Slide 21 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregationͱ
• Aggregationେ͖̎͘छྨ
• Bucket
• ݕࡧ݁ՌͷυΩϡϝϯτΛ
ࢦఆͨ݅͠ʢόέπʣʹྨ
• Metric
• BucketͷυΩϡϝϯτͷσʔλΛ
ͱʹ͍Ζ͍Ζܭࢉ
Slide 22
Slide 22 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Bucket
• Bucketͷྫɿ
• Ϣʔβͷੑผɿʮஉੑʯʮঁੑʯ
• πΠʔτͷॴɿʮ౦ژʯʮژʯ
• πΠʔτͷݴޠɿʮຊޠʯʮӳޠʯ
• ϒϥβɿʮIEʯʮChromeʯ
Slide 23
Slide 23 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Metric
• Metricͷྫɿ
• BucketͷυΩϡϝϯτͷ
• πΠʔτจࣈͷฏۉ
• ച্ߴͷ࠷େ
• ϦΫΤετॲཧ࣌ؒͷ࠷େɺ95ύʔηϯλΠϧ
Slide 24
Slide 24 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
αϯϓϧ
{ "price": 10000, "color": "red", "make": "honda",
ɹɹ"sold": "2014-10-28" }
{ "price": 20000, "color": "red", "make": "honda",
ɹɹ"sold": "2014-11-05" }
{ "price": 30000, "color": "green", "make": "ford",
ɹɹ"sold": "2014-05-18" }
{ "price": 15000, "color": "blue", "make": "toyota",
ɹɹ"sold": "2014-07-02" }
{ "price": 12000, "color": "green", "make": "toyota",
ɹɹ"sold": "2014-08-19" }...
Slide 25
Slide 25 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:৭Ͱूܭ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"popular_colors" : {
"terms" : {
"field" : "color"
}
}
}
}
Slide 26
Slide 26 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:৭Ͱूܭ
"aggregations": {
"popular_colors": {
"buckets": [
{
"key": "red",
"doc_count": 4
},{
"key": "blue",
"doc_count": 2
}, {
"key": "green",
"doc_count": 2
}…
Slide 27
Slide 27 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:৭Ͱྨͯ͠Ձ֨ͷฏۉ
» GET /cars/transactions/_search?search_type=count
{
"aggs": {
"colors": {
"terms": {
"field": "color"
},
"aggs": {
"avg_price": {
"avg": {
"field": "price"
}
} } } } }
Slide 28
Slide 28 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:৭Ͱྨͯ͠Ձ֨ͷฏۉ
» GET /cars/transactions/_search?search_type=count
{
"aggs": {
"colors": {
"terms": {
"field": "color"
},
"aggs": {
"avg_price": {
"avg": {
"field": "price"
}
}
}
} } }
Slide 29
Slide 29 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:৭Ͱྨͯ͠Ձ֨ͷฏۉ
"aggregations": {
"colors": {
"buckets": [
{ "key": "red",
"doc_count": 4,
"avg_price": {
"value": 32500
}
}, {"key": "blue",
"doc_count": 2,
"avg_price": {
"value": 20000
}
},,…
Slide 30
Slide 30 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:ϝʔΧʔΛՃ
» GET /cars/transactions/_search?search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color" },
"aggs": {
"avg_price": {
"avg": { "field": “price" }
},
"make": {
"terms": { "field": "make" }
}
}
}
}}
Slide 31
Slide 31 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:ϝʔΧʔΛՃ
» GET /cars/transactions/_search?search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color" },
"aggs": {
"avg_price": {
"avg": { "field": “price" }
},
"make": {
"terms": { "field": "make" }
}
}
}
}}
Slide 32
Slide 32 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:ϝʔΧʔΛՃ
"aggregations": {
"colors": {
"buckets": [
{ "key": "red",
"doc_count": 4,
"make": {
"buckets": [
{ "key": "honda",
"doc_count": 3 },
{ "key": "bmw",
"doc_count": 1 } ] },
"avg_price": {
"value": 32500
}…
Slide 33
Slide 33 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:ϝʔΧʔ͝ͱͷ࠷খ/࠷େՁ֨
» GET /cars/transactions/_search?search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color” },
"aggs": {
"avg_price": { "avg": { "field": "price" } },
"make" : {
"terms" : { "field" : “make" },
"aggs" : {
"min_price" : { "min": { "field": "price"} },
"max_price" : { "max": { "field": "price"} }
} } } } }
}
Slide 34
Slide 34 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:ϝʔΧʔ͝ͱͷ࠷খ/࠷େՁ֨
» GET /cars/transactions/_search?search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color” },
"aggs": {
"avg_price": { "avg": { "field": "price" } },
"make" : {
"terms" : { "field" : “make" },
"aggs" : {
"min_price" : { "min": { "field": "price"} },
"max_price" : { "max": { "field": "price"} }
} } } } }
}
Slide 35
Slide 35 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫ:ϝʔΧʔ͝ͱͷ࠷খ/࠷େՁ֨
{ "key": "red",
"doc_count": 4,
"make": {
"buckets": [
{ "key": "honda",
"doc_count": 3,
"min_price": { "value": 10000 },
"max_price": { "value": 20000 }
}, {
"key": “bmw”,
…
} ] },
"avg_price": {
"value": 32500
}…
Slide 36
Slide 36 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
AggregationͷΈ
Slide 37
Slide 37 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
AggregationͷΠϝʔδ
» GET /cars/transactions/_search?
search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color" },
"aggs": {
"avg_price": {
"avg": { "field": “price" }
},
"make": {
"terms": { "field": "make" }
}
}
}
}
}
root
terms
(color)
terms
(make)
avg
(price)
Slide 38
Slide 38 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
AggregationͷΠϝʔδ
» GET /cars/transactions/_search?
search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color" },
"aggs": {
"avg_price": {
"avg": { "field": “price" }
},
"make": {
"terms": { "field": "make" }
}
}
}
}
}
root
terms
(color)
terms
(make)
avg
(price)
Slide 39
Slide 39 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
AggregationͷΠϝʔδ
» GET /cars/transactions/_search?
search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color" },
"aggs": {
"avg_price": {
"avg": { "field": “price" }
},
"make": {
"terms": { "field": "make" }
}
}
}
}
}
root
terms
(color)
terms
(make)
avg
(price)
Slide 40
Slide 40 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
AggregationͷΠϝʔδ
» GET /cars/transactions/_search?
search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color" },
"aggs": {
"avg_price": {
"avg": { "field": “price" }
},
"make": {
"terms": { "field": "make" }
}
}
}
}
}
root
terms
(color)
terms
(make)
avg
(price)
Slide 41
Slide 41 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
AggregationͷΠϝʔδ
» GET /cars/transactions/_search?
search_type=count
{
"aggs": {
"colors": {
"terms": { "field": “color" },
"aggs": {
"avg_price": {
"avg": { "field": “price" }
},
"make": {
"terms": { "field": "make" }
}
}
}
}
}
root
terms
(color)
terms
(make)
avg
(price)
Slide 42
Slide 42 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggsͷࢄॲཧ
Slide 43
Slide 43 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ݕࡧॲཧ
Node
Shard
1
Shard
2
Shard
0
Query
Slide 44
Slide 44 text
Shard
2
Node
Query
Shard
1
Shard
0
Query
Query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ΫΤϦͷड͚औΓ
Slide 45
Slide 45 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
γϟʔυʹݕࡧ
Node
Shard
1
Shard
2
Shard
0
Query
Query
Query
Slide 46
Slide 46 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
γϟʔυͷ݁ՌΛϚʔδ
Node
Shard
1
Shard
2
Shard
0
Results 2
Results 1
Results 0
Results
Slide 47
Slide 47 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
݁ՌΛΫϥΠΞϯτ
Node
Shard
1
Shard
2
Shard
0
Results
Slide 48
Slide 48 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregationಉ༷
Node
Shard
1
Shard
2
Shard
0
Query
Slide 49
Slide 49 text
Shard
2
Shard
1
Shard
0
Node
Query
Query
Query
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ΫΤϦͱAggsͷड͚औΓ
Slide 50
Slide 50 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
γϟʔυʹࢄ(ΫΤϦޙʹAggs)
Node
Shard
1
Shard
2
Shard
0
Query
Query
Query
Slide 51
Slide 51 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ΫΤϦͱAggsͷ݁ՌͷϚʔδ
Node
Shard
1
Shard
2
Shard
0
Results 2
Results 0
Results 1
Results
Slide 52
Slide 52 text
Node
Results
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
݁ՌΛΫϥΠΞϯτʹ
Shard
1
Shard
2
Shard
0
Slide 53
Slide 53 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ॲཧγʔέϯεʢ·ͱΊʣ
• AggregationݕࡧͷΫΤϦϑΣʔζͰ࣮ߦ
ݕࡧΫΤϦͱϑΣονͷϑΣʔζ͕͋Δ
• γϟʔυ୯ҐͰॲཧΛ࣮ߦ
• Ϛʔδ͢ΔϊʔυͰɺ
֤γϟʔυͷAggsͷ݁ՌΛϚʔδ͢Δ
Slide 54
Slide 54 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
post_filterͱaggs
• post_filterΛར༻ͨ͠߹ɺpost_filterͰࢦఆͨ͠
ϑΟϧλΛద༻͢Δʮલʯͷݕࡧ݁Ռʹରͯ͠
AggsॲཧΛ࣮ߦ
• Aggsॲཧʹରͯ͠ϑΟϧλΛద༻͢Δ߹filter
aggregationΛར༻
Slide 55
Slide 55 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
Aggregationͷछྨ
Slide 56
Slide 56 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
bucket
• filter
• filters (>=1.4.0)
• missing
• nested
• reverse nested
(>=1.2.0)
• children
• terms
• significant terms
(>=1.1.0)
• range
• date range
• ipv4 range
• histogram
• date histogram
• geo distance
• geohash grid
Slide 57
Slide 57 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
terms
• ΠϯσοΫεʢϑΟʔϧυʣͷ୯ޠ͝ͱʹ
BucketΛੜ
• औಘ݅ΛsizeͰɺιʔτ݅ΛorderͰࢦఆՄೳ
• ݁Ռۙࣅ
sizeࢦఆ͞Ε͕ͨ݅γϟʔυ͔Βฦ٫͞ΕΔͨΊ
ͨͩ͠ɺorder͕_termͷ߹ۙࣅͰͳ͍
shard_sizeΛར༻͢Δͱγϟʔυ͔Βͷฦ٫Λผ్ɺࢦఆՄೳ
Slide 58
Slide 58 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿtermsͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"popular_colors" : {
"terms" : {
"field" : "color"
}
}
}
}
Slide 59
Slide 59 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿtermsͷ݁Ռ
"aggregations": {
"popular_colors": {
"buckets": [
{
"key": "red",
"doc_count": 4
},{
"key": "blue",
"doc_count": 2
}, {
"key": "green",
"doc_count": 2
}…
Slide 60
Slide 60 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
significant terms
• Uncommonly common
• ҟৗݕɺϨίϝϯυͳͲʹར༻
• ྫɿ
өըͷϨίϝϯυ
ΫϨδοτΧʔυͷར༻ཤྺͰͷҟৗͷݕग़
Ҭͷಛతͳ൜ࡑ
Slide 61
Slide 61 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿsignificant termsͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{
"query" : {
"terms" : {"force" : [ "British Transport Police" ]}
},
"aggregations" : {
"significantCrimeTypes" : {
"significant_terms" : { "field" : "crime_type" }
}
}
}
Slide 62
Slide 62 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿsignificant termsͷ݁Ռ
"aggregations" : {
"significantCrimeTypes" : {
"doc_count": 47347,
"buckets" : [
{
"key": "Bicycle theft",
"doc_count": 3640,
"score": 0.371235374214817,
"bg_count": 66799
}
...
]
}
Slide 63
Slide 63 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
range
• BucketΛൣғͰఆٛͯ͠ࢦఆ
• Range Facetͱಉ༷
• ͦͷଞʹσʔλʹಛԽͨ͠ͷ
date rangeɿσʔλ͚ɻdate format͕ར༻Մ
ipv4 rangeɿIPv4σʔλ͚ɻCIDRར༻Մ
geo distanceɿཧใ͚ɻڑʹΑΔൣғ
!
Slide 64
Slide 64 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿrangeͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{"aggs":{
"price_ranges":{
"range":{
"field":"price",
"ranges":[
{“to":50 },
{"from":50,"to":100},
{"from":100}
]
} } } }
Slide 65
Slide 65 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿrangeͷ݁Ռ
"aggregations": {
"price_ranges" : {
"buckets": [
{ "to": 50,
"doc_count": 2
},{
"from": 50,
"to": 100,
"doc_count": 4
}, {
"from": 100,
"doc_count": 4
}
]
} }
Slide 66
Slide 66 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿkeyedΫΤϦ
{
"aggs" : {
"price_ranges" : {
"range" : {
"field" : "price",
"keyed" : true,
"ranges" : [
{"key": "cheap", "to": 50 },
{"key": "average", "from": 50, "to": 100 },
{"key": "expensive", "from": 100 }
]
} } } }
Slide 67
Slide 67 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
histogram
• Bar Chart͚
• ෯ʢintervalʣΛࢦఆ
• Λ෯ຖʹ۠ΒΕͨBucketʹྨ
• min_doc_countʹ0Λઃఆ͢ΔͱɺۭͷBucketऔ
ಘՄೳ
Slide 68
Slide 68 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿhistogramͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"prices" : {
"histogram" : {
"field" : "price",
"interval" : 50,
“min_doc_count” : 0,
}
}
}
}
Slide 69
Slide 69 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿhistogramͷ݁Ռ
"aggregations": {
"prices" : {
"buckets": [
{ "key": 0,
"doc_count": 2
}, {
"key": 50,
"doc_count": 4
}, {
"key": 100,
"doc_count":0
}, {
"key": 150,
"doc_count": 3
} ] } }
Slide 70
Slide 70 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
metric
• min
• max
• sum
• avg
• stats
• extended_stats
• value count
• percentiles (>=1.1.0)
• percentile ranks (>=1.3)
• cardinality (>=1.1.0)
• geo bounds (>=1.3.0)
• top_hits (>=1.3.0)
• scripted metric
(>= 1.4.0)
Slide 71
Slide 71 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
min/max/sum/avg/stats
• Bucketͷmin/max/sum/avgΛฦ͢
• statsmin/max/sum/avg͓ΑͼcountΛฦ͢
• sum_of_squaresɺvarianceɺstd_deviationΛՃ
ͨ͠extended_stats͋Γ
Slide 72
Slide 72 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿminͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"min_price" : {
"min" : { "field" : "price" }
}
}
}
Slide 73
Slide 73 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿminͷ݁Ռ
"aggregations": {
"min_price": {
"value": 10
}
}
Slide 74
Slide 74 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿstatsͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"grades_stats" : {
"stats" : { "field" : "grade" }
}
}
}
Slide 75
Slide 75 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿstatsͷ݁Ռ
"aggregations": {
"grades_stats": {
"count": 6,
"min": 60,
"max": 98,
"avg": 78.5,
"sum": 471
}
}
Slide 76
Slide 76 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
cardinality
• SQLͷDISTINCT
• ۙࣅʢHyperLogLog++ʣ
precision_thresholdʹͯਫ਼ΛࢦఆՄೳ
࠷େ40,000·ͰࢦఆՄೳ
precision_threshold * 8 όΠτͷϝϞϦ͕ඞཁ
!
• ࢿྉɿhttp://www.elasticsearch.org/guide/en/elasticsearch/
reference/1.x/search-aggregations-metrics-cardinality-
aggregation.html
Slide 77
Slide 77 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿcardinalityͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"distinct_colors" : {
"cardinality" : {
"field" : "color"
}
}
}
}
Slide 78
Slide 78 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿcardinalityͷ݁Ռ
"aggregations": {
"distinct_colors": {
"value": 3
}
}
Slide 79
Slide 79 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿprecision_thresholdͷࢦఆ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"distinct_colors" : {
"cardinality" : {
"field" : "color",
"precision_threshold" : 100
}
}
}
}
Slide 80
Slide 80 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
percentiles
• ύʔηϯλΠϧܭࢉ
• percentsଐੑͰܭࢉ͢ΔൣғΛࢦఆՄೳ
• σϑΥϧτ1ɺ5ɺ25ɺ50ɺ75ɺ95ɺ99
• compressionͰϝϞϦ༻ྔΛௐՄೳ
Slide 81
Slide 81 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿpercentilesͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{
"aggs" : {
"load_time_outlier" : {
"percentiles" : {
"field" : "load_time"
}
}
}
}
Slide 82
Slide 82 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿtermsͷ݁Ռ
"aggregations": {
"load_time_outlier": {
"values" : {
"1.0": 15,
"5.0": 20,
"25.0": 23,
"50.0": 25,
"75.0": 29,
"95.0": 60,
"99.0": 150
}
}
}
Slide 83
Slide 83 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
top-hits
• άϧʔϐϯά͞Εͨݕࡧ݁ՌʢField Collapsingʣ
• BucketͷυΩϡϝϯτΛ݁Ռͱͯ͠ग़ྗ
• ҎԼͷݕࡧػೳ͕ར༻Մ
ϋΠϥΠτɺNamed FilterɺQueryɺSource FilteringɺScript
Field
• ࢀߟɿhttps://medium.com/hello-elasticsearch/
top-hits-aggregation-f091986cb66e
Slide 84
Slide 84 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿtop_hitsͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{"aggs": {
"top-tags": {
"terms": {
"field": "tags",
"size": 3
},
"aggs": {
"top_tag_hits": {
"top_hits": {
"sort": [ {
"last_activity_date": { "order": “desc" }
} ],
"_source": { "include": [ "title" ] },
"size" : 1
} } } } }}
Slide 85
Slide 85 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿtermsͷ݁Ռ
"aggregations": {
"top-tags": {
"buckets": [
{ "key": "windows-7",
"doc_count": 25365,
"top_tags_hits": {
"hits": {
"total": 25365,
"max_score": 1,
"hits": [
{ "_index": "stack",
"_type": "question",
"_id": "602679",
"_score": 1,
"_source": {"title": "Windows port opening"},
"sort": [ 1370143231177 ]
…
Slide 86
Slide 86 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
scripted metrics
• 1.4.0Ͱಋೖ༧ఆ
• scriptʹΑΔMetricࢦఆ͕Մೳ
• ҎԼͷεςʔδຖʹهड़
init_script
map_script
combine_script
reduce_script
Slide 87
Slide 87 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ྫɿscripted metricͷΫΤϦ
» GET /cars/transactions/_search?search_type=count
{“aggs": {
"profit": {
"scripted_metric": {
"init_script" : "_agg['transactions'] = []",
"map_script" :
"if (doc['type'].value == \"sale\") {
_agg.transactions.add(doc['amount'].value)
} else {
_agg.transactions.add(-1 * doc['amount'].value)
}",
"combine_script" :
"profit = 0; for (t in _agg.transactions) {
profit += t }; return profit",
"reduce_script" :
"profit = 0; for (a in _aggs) { profit += a };
return profit" } } }
Slide 88
Slide 88 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
·ͱΊ
• Aggregationͷ֓ཁ
BucketͱMetric
• Aggregationͷಈ࡞
ݕࡧ࣌ʹಉ࣌ʹ࣮ߦ͠ɺ1ύεͰॲཧ
• Aggregationͷछྨ
͍Ζ͍Ζͳ༻్ͷAggs͕͋Δ
Slide 89
Slide 89 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ࢀߟ
• The Significant Terms Aggregation (Slide)
https://speakerdeck.com/elasticsearch/the-significant-terms-
aggregation
• elasticsearc - aggregations (Slide & Video)
Berlin Buzzwords 2014
http://berlinbuzzwords.de/session/elasticsearch-aggregations
• Elasticsearch - The Definitive Guide
http://shop.oreilly.com/product/0636920028505.do
Slide 90
Slide 90 text
Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited
ࢀߟ
• Github
https://github.com/elasticsearch
• ΨΠυ
http://www.elasticsearch.org/guide/
• αϙʔτ
http://www.elasticsearch.com/support/