Elasticsearch 2.0の紹介

第12回Elasticsearch勉強会で使用した資料です。
Elasticsearch2.0の変更点などの紹介をしました。

発表時にはブラウザで表示したMigrationプラグインの画面を追加してあります。

Transcript

1. Elasticsearch 2.0ͷ঺հ Jun Ohtani @johtani 2015/10/08
3. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 3 about • Me, Jun Ohtani / Technical Adovocate – lucene-gosenίϛολʔ – ElasticSearch Server೔ຊޠ൛ͷ຋༁ – elasticsearch-extended-analysisͷ։ൃ – http://blog.johtani.info • Elasticsearch, founded in 2012 – OSS: Elasticsearch, Logstash, Kibana, Beats – Commercial plugins: Marvel, Sheild, Watcher – Hosted Elasticsearch: Found
 Professional services: Support & development subscriptions – Trainings
4. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 4 2.0? Elasticsearch 2.0
5. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 5 ͬ͘͟Γ • Lucent 5ϕʔε • 2500Ҏ্΋ͷPull Request • ͏ͪ850͕৽نͷ΋ͷ • 496ίϛολʔ • RC1ʂʂʂ • RC1൛͸ຊ൪Ͱ͸ར༻͠ͳ͍Ͱ͍ͩ͘͞ʂ
6. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 6 Breaking changes https://www.elastic.co/guide/en/elasticsearch/reference/2.0/breaking-changes-2.0.html
7. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 7 ಛ௃ • Simplification • Security • Resiliency • Features
8. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 8 Simplification Simplification
9. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 9 ഇࢭ͞Εͨػೳ • River – LogstashͳͲͷར༻Λ • Facet – aggregation • _shutdown API – ϓϥοτϑΥʔϜͷػߏΛ • Thrift / Memcached protocol – αϙʔτ • Bulk UDP – ௨ৗͷbulk API͔ɺLogstashʹUDPͰܦ༝Ͱ
10. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 10 ϓϥάΠϯʹͳͬͨػೳ • Delete by query – ./bin/plugin install delete-by-query • murmur3 – ./bin/plugin install mapper-murmur3 • size – ./bin/plugin/install mapper-size • Multicast discovery – ./bin/plugin install discovery-multicast
11. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 11 ओཁϓϥάΠϯΛಉࠝ • ͍͔ͭ͘ͷϓϥάΠϯΛelasticsearchͷϦϙδτϦʹҠಈ – analysis • icu, kuromoji, phonetic, smartcn, stempel – cloud • aws, azure, gce – delete-by-query – discovery • multicast – lang • expression, groovy, javascript, python – mapper-murmur3, -size
12. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 12 ϓϥάΠϯͷΠϯετʔϧ • ಉࠝϓϥάΠϯͷΠϯετʔϧ͕؆қԽ • ͦΕҎ֎͸͜Ε·Ͱ௨Γ $ bin/plugin install [plugin_name] $ bin/plugin install analysis-icu $ bin/plugin install [org]/[user|component]/[version] $ bin/plugin install lmenezes/elasticsearch-kopf $ bin/plugin install lmenezes/elasticsearch-kopf/1.x
13. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 13 Mappingͷมߋ • Conflicting field mappings • short nameͰͷϑΟʔϧυࢦఆͷېࢭ • Type name prefixͷഇࢭ • ϑΟʔϧυ໊ʹυοτ͕࢖༻ෆՄ • υοτͰ࢝·ΔλΠϓ໊Λېࢭ • 255จࣈҎ্ͷλΠϓ໊Λېࢭ • λΠϓͷ࡟আΛېࢭ • _analyzerɺindex_analyzerͷഇࢭ • date formatͷมߋ • ͳͲͳͲ
14. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 14 MappingͷίϯϑϦΫτͱ͸ʁ PUT my_index { "mappings": { "type_one": { "properties": { "name": { "type": "string" } ❶ } }, "type_two": { "properties": { "name": { "type": "string", "analyzer": "english" } ❷ } } } }
15. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 15 Fieldͷshort name͸ͳͥېࢭʁ • ʮtitleʯͱࢦఆͨ͠৔߹ɺᶄͱᶆͲͬͪʁ • ʮname.titleʯͱࢦఆͨ͠৔߹ɺᶃ-ᶄͱᶅ-ᶆͲͬͪʁ • ʮname.name.titleʯͱࢦఆͨ͠Βʁ PUT my_index { "mappings": { "name": { ❶ "properties": { "title": { "type": "string" }, ❷ "name": { ❸ "properties": { "title": { "type": "string" } ❹ } } } } } }
16. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 16 2.0ͰͷLookup • ʮtitleʯͱࢦఆͨ͠৔߹ɺᶄ • ʮname.titleʯͱࢦఆͨ͠৔߹ɺᶅ-ᶆ • ʮname.name.titleʯͱࢦఆͨ͠ΒɺΤϥʔ PUT my_index { "mappings": { "name": { ❶ "properties": { "title": { "type": "string" }, ❷ "name": { ❸ "properties": { "title": { "type": "string" } ❹ } } } } } }
17. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 17 ࢀߟɿ • ࢀߟɿhttps://www.elastic.co/blog/great-mapping-refactoring
18. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 18 QueryDSL • QueryͱFilterͷ౷߹ – 2.0Ҏલ • QueryɿείΞͷର৅ɺΩϟογϡෆՄ • FilterɿείΞͷର৅֎ɺΩϟογϡՄೳ – 2.0Ҏ߱ • Queryʹ౷߹ɻίϯςΩετʹΑͬͯಈ࡞͕ҟͳΔ
19. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 19 QueryDSL
20. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 20 Security Security
21. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 21 ωοτϫʔΫઃఆͷมߋ • σϑΥϧτͷมߋ – Unicast • Multicast͸ϓϥάΠϯʹ – localhostʹόΠϯυ • IPv4ͱIPv6 • ΑΓηΩϡΞʹ͢ΔͨΊ – transport.tcp.portͷ࠷ॳͷ5ϙʔτ͕σϑΥϧτ • ͜Ε·Ͱ௨ΓखݩͰىಈͨ͠৔߹ʹ؆୯ʹΫϥελߏ੒Մೳ
22. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 22 Security Managerͷಋೖ • ElasticsearchͷϓϩηεىಈʹJavaͷSecurityManagerΛ࢖༻ • Elasticsearch͕ΞΫηεͰ͖ΔݖݶΛϊʔυͷىಈ࣌ʹࢦఆ
23. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 23 Resiliency Resiliency
24. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 24 Indexૢ࡞ΛΑΓ৑௕ʹ • 2.0Ҏલ – 5ඵ͓͖ʹtransaction logΛfsync • 2.0Ҏ߱ – ֤ૢ࡞͝ͱʹfsync – ઃఆՄೳ
25. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 25 ෳ਺data.pathʹΑΔετϥΠϐϯά • 2.0Ҏલ – શͯͷύεʹରͯ͠શͯͷγϟʔυΛετϥΠϐϯά – shardͷϑΝΠϧ͕શͯͷύεʹ෼ࢄ • 2.0Ҏ߱ – ύε͝ͱʹͦΕͧΕͷγϟʔυΛ഑ஔ – 1γϟʔυͷϑΝΠϧ͸ಉ͡ύεʹॻ͖ࠐΈ
26. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 26 ෳ਺data.path
27. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 27 2.0Ҏલ
28. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 28 2.0Ҏ߱
29. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 29 ͦͷଞ • ClusterStateͷࠩ෼ߋ৽ • doc values͕σϑΥϧτ – 2.0ͰΠϯσοΫεΛ࡞੒ͨ͠৔߹ – not_analyzedͳindexedͷϑΟʔϧυ͸σϑΥϧτͰର৅ • ࣌ؒ΍όΠτͷ஋ͷઃఆʹ୯Ґ͕ඞਢ
30. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 30 Fetures Features
31. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 31 Pipeline Aggregations • Derivatives • Moving average • Holt Winters(prediction / anomaly detection) • Stats: Min/Max/Avg • Time-series math • ࢀߟɿhttps://www.elastic.co/blog/out-of-this-world-aggregations
32. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 32 Index Compression • ΠϯσοΫεαΠζ͕খ͘͞ʢ10%͔Β30ˋʣ • ΠϯσΩγϯά΍ϚʔδͷॲཧʹӨڹ͋Γ • ؀ڥʹ߹ΘͤͨσʔλѹॖΛબ୒ • ࣌ܥྻΠϯσοΫεʹ • ࢀߟɿhttps://www.elastic.co/blog/store-compression-in-lucene-and- elasticsearch
33. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 33 2.0΁ͷҠߦ 2.0΁ͷҠߦ
34. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 34 Upgrade • ϩʔϦϯάΞοϓάϨʔυ͸ෆՄ – Clusterͷఀࢭ͕ඞཁ • μ΢ϯάϨʔυෆՄ • όοΫΞοϓͱΓ·͠ΐ͏ • ·ͣ͸ςετΛʂ
35. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 35 Migration Plugin • 1.xܥͷϓϥάΠϯͱͯ͠Πϯετʔϧͯ͠໰୊఺ΛνΣοΫ • SiteϓϥάΠϯͳͷͰɺ࠶ىಈͤͣʹར༻Մೳ • https://github.com/elastic/elasticsearch-migration • σϞʂ
37. 12/16(⽔) @東京 ユーザーカンファレンス 開催決定！ { プロダクトアップデート } { 製品デモンストレーション } { カスタマーケーススタディ } { ネットワーキング } https://www.elastic.co/elasticon/tour/2015 お問い合わせはこちらまで elastic-japan@elastic.co とうろくは11/17までです。 Webサイトより事前のご登録お願い致します。 アジェンダ・場所など詳細が決まり次第、 ご登録アドレスへご連絡させて頂きます。
38. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 38 ࢀߟจݙ • Elasticsearch - The Definitive guide – http://www.elastic.co/guide/en/elasticsearch/guide/current/ index.html • ॻ੶ʢ೔ຊޠʣ – ElasticSearchServer೔ຊޠ൛
 αʔό/ΠϯϑϥΤϯδχΞ
 ɹཆ੒ಡຊɹϩάऩू
39. www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 39 ࢀߟαΠτ • Ϣʔεέʔε • https://www.elastic.co/use-cases • DiscussʢWebϑΥʔϥϜʣ • https://discuss.elastic.co • Elastic{ON}ͷϏσΦͱࢿྉ • https://www.elastic.co/elasticon/2015/sf/videos • αϙʔτϝχϡʔ • https://www.elastic.co/subscriptions
40. Thanks for listening! Q & A We’re hiring! https://www.elastic.co/about/careers/ We’re helping! https://www.elastic.co/subscriptions http://purchases.elastic.co