Aggregationあれこれ

第6回elasticsearch勉強会発表スライド。

#elasticsearchjp #elasticsearch

Transcript

1. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Jun Ohtani @johtani 2014-09-16 Aggregation͋Ε͜Ε
2. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited about • Me, Jun Ohtani / Technical Adovocate lucene-gosenίϛολʔ ElasticSearch Server೔ຊޠ൛ͷ຋༁ elasticsearch-extended-analysisͷ։ൃ http://blog.johtani.info • Elasticsearch, founded in 2012 Products: Elasticsearch, Logstash, Kibana, Marvel
 Professional services: Support & development subscriptions Trainings
3. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ΞδΣϯμ • Elasticsearch? • Aggregationͱ͸ • Aggregationͷ࢓૊Έ • Aggregationͷछྨ !
4. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearchʁ
5. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϑϦʔϫʔυݕࡧ
6. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ߜΓࠐΈ
7. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϋΠϥΠτ
8. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ιʔτ
9. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϖʔδϯά
10. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ूܭ
11. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited αδΣετ
12. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Elasticsearch in 10 seconds • εΩʔϚϑϦʔɺ෼ࢄυΩϡϝϯτετΞɺREST & JSON • Φʔϓϯιʔε: Apache License 2.0 • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ • JavaͰ࣮૷ɻ֦ு΋༰қ
13. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregation
14. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregationͱ͸ • 1.0͔Βར༻Մೳ • ݕࡧ݁Ռʹରͯ͠ɺσʔλͷղੳ͕༰қʹ • FacetΑΓ΋ڧྗͳूܭͳͲ͕Մೳ ֊૚తͳूܭɺάϧʔϓԽ
 ಈతͳूܭɺάϧʔϓԽ • Facet͸ɺdeprecatedɻ2.xܥͰͳ͘ͳΔ !
15. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregationͱ͸ • SQLͩͱ ! ! • GROUP BY colorɿBucket • COUNT(color)ɿMetric SELECT  COUNT(color)     FROM  table   GROUP  BY  color
16. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregations Documents
17. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregations Documents Query
18. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregations Documents Query Buckets
19. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregations Documents Query Buckets
20. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregations Documents Query Buckets Metrics 123 123 243 185
21. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregationͱ͸ • Aggregation͸େ͖̎͘छྨ • Bucket • ݕࡧ݁ՌͷυΩϡϝϯτΛ
 ࢦఆͨ͠৚݅ʢόέπʣʹ෼ྨ • Metric • Bucket಺ͷυΩϡϝϯτͷσʔλΛ
 ΋ͱʹ͍Ζ͍Ζܭࢉ
22. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Bucket • Bucketͷྫɿ • Ϣʔβͷੑผɿʮஉੑʯʮঁੑʯ • πΠʔτͷ৔ॴɿʮ౦ژʯʮژ౎ʯ • πΠʔτͷݴޠɿʮ೔ຊޠʯʮӳޠʯ • ϒϥ΢βɿʮIEʯʮChromeʯ
23. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Metric • Metricͷྫɿ • Bucket಺ͷυΩϡϝϯτͷ਺ • πΠʔτจࣈ਺ͷฏۉ • ച্ߴͷ࠷େ஋ • ϦΫΤετॲཧ࣌ؒͷ࠷େ஋ɺ95ύʔηϯλΠϧ
24. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited αϯϓϧ { "price": 10000, "color": "red", "make": "honda", ɹɹ"sold": "2014-10-28" } { "price": 20000, "color": "red", "make": "honda", ɹɹ"sold": "2014-11-05" } { "price": 30000, "color": "green", "make": "ford", ɹɹ"sold": "2014-05-18" } { "price": 15000, "color": "blue", "make": "toyota", ɹɹ"sold": "2014-07-02" } { "price": 12000, "color": "green", "make": "toyota", ɹɹ"sold": "2014-08-19" }...
25. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:৭Ͱूܭ » GET /cars/transactions/_search?search_type=count { "aggs" : { "popular_colors" : { "terms" : { "field" : "color" } } } }
26. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:৭Ͱूܭ "aggregations": { "popular_colors": { "buckets": [ { "key": "red", "doc_count": 4 },{ "key": "blue", "doc_count": 2 }, { "key": "green", "doc_count": 2 }…
27. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:৭Ͱ෼ྨͯ͠Ձ֨ͷฏۉ » GET /cars/transactions/_search?search_type=count { "aggs": { "colors": { "terms": { "field": "color" }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } }
28. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:৭Ͱ෼ྨͯ͠Ձ֨ͷฏۉ » GET /cars/transactions/_search?search_type=count { "aggs": { "colors": { "terms": { "field": "color" }, "aggs": { "avg_price": { "avg": { "field": "price" } } } } } }
29. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:৭Ͱ෼ྨͯ͠Ձ֨ͷฏۉ "aggregations": { "colors": { "buckets": [ { "key": "red", "doc_count": 4, "avg_price": { "value": 32500 } }, {"key": "blue", "doc_count": 2, "avg_price": { "value": 20000 } },,…
30. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:ϝʔΧʔΛ௥Ճ » GET /cars/transactions/_search?search_type=count { "aggs": { "colors": { "terms": { "field": “color" }, "aggs": { "avg_price": { "avg": { "field": “price" } }, "make": { "terms": { "field": "make" } } } } }}
31. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:ϝʔΧʔΛ௥Ճ » GET /cars/transactions/_search?search_type=count { "aggs": { "colors": { "terms": { "field": “color" }, "aggs": { "avg_price": { "avg": { "field": “price" } }, "make": { "terms": { "field": "make" } } } } }}
32. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:ϝʔΧʔΛ௥Ճ "aggregations": { "colors": { "buckets": [ { "key": "red", "doc_count": 4, "make": { "buckets": [ { "key": "honda", "doc_count": 3 }, { "key": "bmw", "doc_count": 1 } ] }, "avg_price": { "value": 32500 }…
33. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:ϝʔΧʔ͝ͱͷ࠷খ/࠷େՁ֨ » GET /cars/transactions/_search?search_type=count { "aggs": { "colors": { "terms": { "field": “color” }, "aggs": { "avg_price": { "avg": { "field": "price" } }, "make" : { "terms" : { "field" : “make" }, "aggs" : { "min_price" : { "min": { "field": "price"} }, "max_price" : { "max": { "field": "price"} } } } } } } }
34. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:ϝʔΧʔ͝ͱͷ࠷খ/࠷େՁ֨ » GET /cars/transactions/_search?search_type=count { "aggs": { "colors": { "terms": { "field": “color” }, "aggs": { "avg_price": { "avg": { "field": "price" } }, "make" : { "terms" : { "field" : “make" }, "aggs" : { "min_price" : { "min": { "field": "price"} }, "max_price" : { "max": { "field": "price"} } } } } } } }
35. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫ:ϝʔΧʔ͝ͱͷ࠷খ/࠷େՁ֨ { "key": "red", "doc_count": 4, "make": { "buckets": [ { "key": "honda", "doc_count": 3, "min_price": { "value": 10000 }, "max_price": { "value": 20000 } }, { "key": “bmw”, … } ] }, "avg_price": { "value": 32500 }…
36. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregationͷ࢓૊Έ
37. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited AggregationͷΠϝʔδ » GET /cars/transactions/_search? search_type=count { "aggs": { "colors": { "terms": { "field": “color" }, "aggs": { "avg_price": { "avg": { "field": “price" } }, "make": { "terms": { "field": "make" } } } } } } root terms
 (color) terms
 (make) avg
 (price)
38. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited AggregationͷΠϝʔδ » GET /cars/transactions/_search? search_type=count { "aggs": { "colors": { "terms": { "field": “color" }, "aggs": { "avg_price": { "avg": { "field": “price" } }, "make": { "terms": { "field": "make" } } } } } } root terms
 (color) terms
 (make) avg
 (price)
39. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited AggregationͷΠϝʔδ » GET /cars/transactions/_search? search_type=count { "aggs": { "colors": { "terms": { "field": “color" }, "aggs": { "avg_price": { "avg": { "field": “price" } }, "make": { "terms": { "field": "make" } } } } } } root terms
 (color) terms
 (make) avg
 (price)
40. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited AggregationͷΠϝʔδ » GET /cars/transactions/_search? search_type=count { "aggs": { "colors": { "terms": { "field": “color" }, "aggs": { "avg_price": { "avg": { "field": “price" } }, "make": { "terms": { "field": "make" } } } } } } root terms
 (color) terms
 (make) avg
 (price)
41. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited AggregationͷΠϝʔδ » GET /cars/transactions/_search? search_type=count { "aggs": { "colors": { "terms": { "field": “color" }, "aggs": { "avg_price": { "avg": { "field": “price" } }, "make": { "terms": { "field": "make" } } } } } } root terms
 (color) terms
 (make) avg
 (price)
42. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggsͷ෼ࢄॲཧ
43. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ݕࡧॲཧ Node Shard
 1 Shard
 2 Shard
 0 Query
44. Shard
 2 Node Query Shard
 1 Shard
 0 Query Query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ΫΤϦͷड͚औΓ
45. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited γϟʔυʹݕࡧ Node Shard
 1 Shard
 2 Shard
 0 Query Query Query
46. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited γϟʔυͷ݁ՌΛϚʔδ Node Shard
 1 Shard
 2 Shard
 0 Results 2 Results 1 Results 0 Results
47. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ݁ՌΛΫϥΠΞϯτ΁ Node Shard
 1 Shard
 2 Shard
 0 Results
48. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregation΋ಉ༷ Node Shard
 1 Shard
 2 Shard
 0 Query
49. Shard
 2 Shard
 1 Shard
 0 Node Query Query Query Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ΫΤϦͱAggsͷड͚औΓ
50. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited γϟʔυʹ෼ࢄ(ΫΤϦޙʹAggs) Node Shard
 1 Shard
 2 Shard
 0 Query Query Query
51. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ΫΤϦͱAggsͷ݁ՌͷϚʔδ Node Shard
 1 Shard
 2 Shard
 0 Results 2 Results 0 Results 1 Results
52. Node Results Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ݁ՌΛΫϥΠΞϯτʹ Shard
 1 Shard
 2 Shard
 0
53. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ॲཧγʔέϯεʢ·ͱΊʣ • Aggregation͸ݕࡧͷΫΤϦϑΣʔζͰ࣮ߦ ݕࡧ͸ΫΤϦͱϑΣονͷϑΣʔζ͕͋Δ • γϟʔυ୯ҐͰॲཧΛ࣮ߦ • Ϛʔδ͢Δϊʔυ಺Ͱɺ
 ֤γϟʔυͷAggsͷ݁ՌΛϚʔδ͢Δ
54. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited post_filterͱaggs • post_filterΛར༻ͨ͠৔߹ɺpost_filterͰࢦఆͨ͠ ϑΟϧλΛద༻͢Δʮલʯͷݕࡧ݁Ռʹରͯ͠ AggsॲཧΛ࣮ߦ • Aggsॲཧʹରͯ͠ϑΟϧλΛద༻͢Δ৔߹͸filter aggregationΛར༻
55. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited Aggregationͷछྨ
56. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited bucket • filter • filters (>=1.4.0) • missing • nested • reverse nested (>=1.2.0) • children • terms • significant terms (>=1.1.0) • range • date range • ipv4 range • histogram • date histogram • geo distance • geohash grid
57. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited terms • ΠϯσοΫεʢϑΟʔϧυʣͷ୯ޠ͝ͱʹ
 BucketΛੜ੒ • औಘ݅਺ΛsizeͰɺιʔτ৚݅ΛorderͰࢦఆՄೳ • ݁Ռ͸ۙࣅ஋ sizeࢦఆ͞Εͨ݅਺͕γϟʔυ͔Βฦ٫͞ΕΔͨΊ ͨͩ͠ɺorder͕_termͷ৔߹͸ۙࣅͰ͸ͳ͍
 shard_sizeΛར༻͢Δͱγϟʔυ͔Βͷฦ٫஋Λผ్ɺࢦఆՄೳ
58. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿtermsͷΫΤϦ » GET /cars/transactions/_search?search_type=count { "aggs" : { "popular_colors" : { "terms" : { "field" : "color" } } } }
59. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿtermsͷ݁Ռ "aggregations": { "popular_colors": { "buckets": [ { "key": "red", "doc_count": 4 },{ "key": "blue", "doc_count": 2 }, { "key": "green", "doc_count": 2 }…
60. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited significant terms • Uncommonly common • ҟৗݕ஌ɺϨίϝϯυͳͲʹར༻ • ྫɿ өըͷϨίϝϯυ ΫϨδοτΧʔυͷར༻ཤྺͰͷҟৗ஋ͷݕग़ ஍Ҭͷಛ௃తͳ൜ࡑ
61. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿsignificant termsͷΫΤϦ » GET /cars/transactions/_search?search_type=count { "query" : { "terms" : {"force" : [ "British Transport Police" ]} }, "aggregations" : { "significantCrimeTypes" : { "significant_terms" : { "field" : "crime_type" } } } }
62. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿsignificant termsͷ݁Ռ "aggregations" : { "significantCrimeTypes" : { "doc_count": 47347, "buckets" : [ { "key": "Bicycle theft", "doc_count": 3640, "score": 0.371235374214817, "bg_count": 66799 } ... ] }
63. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited range • BucketΛൣғͰఆٛͯ͠ࢦఆ • Range Facetͱಉ༷ • ͦͷଞʹσʔλʹಛԽͨ͠΋ͷ date rangeɿ೔෇σʔλ޲͚ɻdate format͕ར༻Մ ipv4 rangeɿIPv4σʔλ޲͚ɻCIDR΋ར༻Մ geo distanceɿ஍ཧ৘ใ޲͚ɻڑ཭ʹΑΔൣғ !
64. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿrangeͷΫΤϦ » GET /cars/transactions/_search?search_type=count {"aggs":{ "price_ranges":{ "range":{ "field":"price", "ranges":[ {“to":50 }, {"from":50,"to":100}, {"from":100} ] } } } }
65. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿrangeͷ݁Ռ "aggregations": { "price_ranges" : { "buckets": [ { "to": 50, "doc_count": 2 },{ "from": 50, "to": 100, "doc_count": 4 }, { "from": 100, "doc_count": 4 } ] } }
66. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿkeyedΫΤϦ { "aggs" : { "price_ranges" : { "range" : { "field" : "price", "keyed" : true, "ranges" : [ {"key": "cheap", "to": 50 }, {"key": "average", "from": 50, "to": 100 }, {"key": "expensive", "from": 100 } ] } } } }
67. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited histogram • Bar Chart޲͚ • ෯ʢintervalʣΛࢦఆ • ஋Λ෯ຖʹ۠੾ΒΕͨBucketʹ෼ྨ • min_doc_countʹ0Λઃఆ͢ΔͱɺۭͷBucket΋औ ಘՄೳ
68. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿhistogramͷΫΤϦ » GET /cars/transactions/_search?search_type=count { "aggs" : { "prices" : { "histogram" : { "field" : "price", "interval" : 50, “min_doc_count” : 0, } } } }
69. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿhistogramͷ݁Ռ "aggregations": { "prices" : { "buckets": [ { "key": 0, "doc_count": 2 }, { "key": 50, "doc_count": 4 }, { "key": 100, "doc_count":0 }, { "key": 150, "doc_count": 3 } ] } }
70. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited metric • min • max • sum • avg • stats • extended_stats • value count • percentiles (>=1.1.0) • percentile ranks (>=1.3) • cardinality (>=1.1.0) • geo bounds (>=1.3.0) • top_hits (>=1.3.0) • scripted metric 
 (>= 1.4.0)
71. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited min/max/sum/avg/stats • Bucket಺ͷmin/max/sum/avgΛฦ͢ • stats͸min/max/sum/avg͓ΑͼcountΛฦ͢ • sum_of_squaresɺvarianceɺstd_deviationΛ௥Ճ ͨ͠extended_stats΋͋Γ
72. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿminͷΫΤϦ » GET /cars/transactions/_search?search_type=count { "aggs" : { "min_price" : { "min" : { "field" : "price" } } } }
73. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿminͷ݁Ռ "aggregations": { "min_price": { "value": 10 } }
74. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿstatsͷΫΤϦ » GET /cars/transactions/_search?search_type=count { "aggs" : { "grades_stats" : { "stats" : { "field" : "grade" } } } }
75. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿstatsͷ݁Ռ "aggregations": { "grades_stats": { "count": 6, "min": 60, "max": 98, "avg": 78.5, "sum": 471 } }
76. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited cardinality • SQLͷDISTINCT • ۙࣅ஋ʢHyperLogLog++ʣ precision_thresholdʹͯਫ਼౓ΛࢦఆՄೳ ࠷େ40,000·ͰࢦఆՄೳ precision_threshold * 8 όΠτͷϝϞϦ͕ඞཁ ! • ࢿྉɿhttp://www.elasticsearch.org/guide/en/elasticsearch/ reference/1.x/search-aggregations-metrics-cardinality- aggregation.html
77. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿcardinalityͷΫΤϦ » GET /cars/transactions/_search?search_type=count { "aggs" : { "distinct_colors" : { "cardinality" : { "field" : "color" } } } }
78. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿcardinalityͷ݁Ռ "aggregations": { "distinct_colors": { "value": 3 } }
79. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿprecision_thresholdͷࢦఆ » GET /cars/transactions/_search?search_type=count { "aggs" : { "distinct_colors" : { "cardinality" : { "field" : "color", "precision_threshold" : 100 } } } }
80. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited percentiles • ύʔηϯλΠϧܭࢉ • percentsଐੑͰܭࢉ͢ΔൣғΛࢦఆՄೳ • σϑΥϧτ͸1ɺ5ɺ25ɺ50ɺ75ɺ95ɺ99 • compressionͰϝϞϦ࢖༻ྔΛௐ੔Մೳ
81. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿpercentilesͷΫΤϦ » GET /cars/transactions/_search?search_type=count { "aggs" : { "load_time_outlier" : { "percentiles" : { "field" : "load_time" } } } }
82. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿtermsͷ݁Ռ "aggregations": { "load_time_outlier": { "values" : { "1.0": 15, "5.0": 20, "25.0": 23, "50.0": 25, "75.0": 29, "95.0": 60, "99.0": 150 } } }
83. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited top-hits • άϧʔϐϯά͞Εͨݕࡧ݁ՌʢField Collapsingʣ • Bucket಺ͷυΩϡϝϯτΛ݁Ռͱͯ͠ग़ྗ • ҎԼͷݕࡧػೳ͕ར༻Մ ϋΠϥΠτɺNamed FilterɺQueryɺSource FilteringɺScript Field • ࢀߟɿhttps://medium.com/hello-elasticsearch/ top-hits-aggregation-f091986cb66e
84. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿtop_hitsͷΫΤϦ » GET /cars/transactions/_search?search_type=count {"aggs": { "top-tags": { "terms": { "field": "tags", "size": 3 }, "aggs": { "top_tag_hits": { "top_hits": { "sort": [ { "last_activity_date": { "order": “desc" } } ], "_source": { "include": [ "title" ] }, "size" : 1 } } } } }}
85. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿtermsͷ݁Ռ "aggregations": { "top-tags": { "buckets": [ { "key": "windows-7", "doc_count": 25365, "top_tags_hits": { "hits": { "total": 25365, "max_score": 1, "hits": [ { "_index": "stack", "_type": "question", "_id": "602679", "_score": 1, "_source": {"title": "Windows port opening"}, "sort": [ 1370143231177 ] …
86. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited scripted metrics • 1.4.0Ͱಋೖ༧ఆ • scriptʹΑΔMetricࢦఆ͕Մೳ • ҎԼͷεςʔδຖʹهड़ init_script map_script combine_script reduce_script
87. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ྫɿscripted metricͷΫΤϦ » GET /cars/transactions/_search?search_type=count {“aggs": { "profit": { "scripted_metric": { "init_script" : "_agg['transactions'] = []", "map_script" : "if (doc['type'].value == \"sale\") { _agg.transactions.add(doc['amount'].value) } else { _agg.transactions.add(-1 * doc['amount'].value) }", "combine_script" : "profit = 0; for (t in _agg.transactions) { profit += t }; return profit", "reduce_script" : "profit = 0; for (a in _aggs) { profit += a }; return profit" } } }
88. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ·ͱΊ • Aggregationͷ֓ཁ BucketͱMetric • Aggregationͷಈ࡞ ݕࡧ࣌ʹಉ࣌ʹ࣮ߦ͠ɺ1ύεͰॲཧ • Aggregationͷछྨ ͍Ζ͍Ζͳ༻్ͷAggs͕͋Δ
89. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ࢀߟ • The Significant Terms Aggregation (Slide) https://speakerdeck.com/elasticsearch/the-significant-terms- aggregation
 • elasticsearc - aggregations (Slide & Video) Berlin Buzzwords 2014
 http://berlinbuzzwords.de/session/elasticsearch-aggregations • Elasticsearch - The Definitive Guide http://shop.oreilly.com/product/0636920028505.do
90. Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited ࢀߟ • Github https://github.com/elasticsearch • ΨΠυ http://www.elasticsearch.org/guide/ • αϙʔτ http://www.elasticsearch.com/support/