Elasticsearch の基礎 / Get Started with Elasticsearch

Transcript

1. Elasticsearch ͷجૅ 2019-04-23 1
2. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !2
3. Elasticsearch ͱ͸ ಛ௃ (ʮσʔλ෼ੳج൫ߏஙೖ໳ʯΑΓɺͨͩ͠ଠࣈ͸ಠࣗ) • OSS (Apache Lisence v2) • υΩϡϝϯτࢦ޲ • ෼ࢄγεςϜ • Ϛϧνςφϯτ • RESTful API • (near) ϦΞϧλΠϜ !3
4. υΩϡϝϯτࢦ޲ • υΩϡϝϯτ (document) • Elasticsearch Ͱѻ͏σʔλͷ࠷খ୯Ґ • JSON • id Λ࣋ͭ • (εΩʔϚϨεʹ࢖͑ΔͷͰ) ॊೈͳσʔλͷొ࿥͕Մೳ !4
5. ෼ࢄγεςϜ • σʔλΛෳ਺ node ؒͰ෼ࢄͯ͠อ࣋ɺݕࡧ͢Δ • εέʔϧΞ΢τΛ૝ఆͨ͠ઃܭ !5
6. RESTful API • ͋ΒΏΔૢ࡞͕ REST API Ͱఏڙ͞Ε͍ͯΔ • document ͷ௥Ճɺߋ৽ɺ࡟আ • document ͷݕࡧ • ઃఆͷ֬ೝɺมߋ • ϝτϦΫεͷऔಘ !6
7. RESTful API • ྫ. document ͷ௥Ճ readonly ES_INDEX=index1 readonly ES_TYPE=_doc request=$(cat <<EOF { "name": "Alice", "age": 21, "registered_at": "2019-04-23T03:00:00Z" } EOF ) curl -XPOST -H "Content-Type: application/json" \ http://$ES_HOST:$ES_PORT/$ES_INDEX/$ES_TYPE \ -d "$request" !7
8. (near) ϦΞϧλΠϜ • ొ࿥ͨ͠σʔλ͸΄΅ϦΞϧλΠϜʹݕࡧ݁Ռʹ൓ө • σϑΥϧτઃఆͩͱ 1 ඵޙʹ͸൓ө͞ΕΔͬΆ͍ !8
9. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !9
10. Elasticsearch ͷجຊ֓೦ • Elasticsearch ͸ 1 ୆Ҏ্ͷ node Ͱ cluster Λߏ੒͢Δ • cluster ʹ͸ 0 Ҏ্ͷ index ͕ଘࡏ͢Δ • index ʹ͸ 1 ͭҎ্ͷ shard ͕ଘࡏ͢Δ • document ͸͍ͣΕ͔ͷ shard ʹอଘ͞ΕΔ • client ͸ index Λࢦఆͯ͠อଘ͢Δ • อଘઌͷ shard ͸ id (ਖ਼֬ʹ͸ routing) ʹΑܾͬͯ·Δ !10
11. Cluster 1 node, 1 index, 1 shard Node1 0 primary shard Index1 !11
12. Cluster 3 node ߏ੒ʹ Index1 Node1 Node2 Node3 0 primary shard !12
13. Cluster shard ਺Λ 1 -> 4 ʹ Index1 Node1 Node2 Node3 0 1 2 3 primary shard !13
14. Cluster ֤ shard ʹ 1 replica ௥Ճ Index1 Node1 Node2 Node3 0 1 2 3 0 1 2 3 primary shard replica shard !14
15. Cluster index ௥Ճ Index1 Node1 Node2 Node3 0 1 2 3 0 1 2 3 primary shard replica shard Index2 !15
16. ՔಇதͷγεςϜͰ͸ʁ • 9 node Ͱ cluster Λߏ੒ • 1 index ͷΈ, 128 primary shards • ద੾ͳ primary shard ਺͸ѻ͏σʔλ౳ʹΑͬͯҟͳΔ • ಈతʹ primary shard ਺Λมߋ͢Δ͜ͱ͸Ͱ͖ͳ͍ • replica ਺͸ 1 (֤ primary shard ʹ 1 replica ଘࡏ) • id, routing ͸ಠࣗʹઃఆ !16
17. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !17
18. master node ͱ data node • ֤ node ͸ىಈ࣌ʹ໾ׂΛઃఆ͞ΕΔ • ໾ׂ1: master eligible (master ީิ) • ͜ͷ node ͷத͔Β master ͕Ұ୆બ͹ΕΔ • ໾ׂ2: data (document ͷอଘઌ) • ׂΓ౰ͯΒΕͨ shard ΁ͷ document ͷอଘɺݕࡧΛ୲౰ • master eligible, data ͸݉೚Ͱ͖Δ !18
19. Cluster master node ͱ data node Index1 Node1 Node2 Node3 0 1 2 3 0 1 2 3 Index2 master eligible master data !19
20. ՔಇதͷγεςϜͰ͸ʁ • master eligible node ͕ 3 ୆ • data node ͕ 6 ୆ • master eligible ͱ data Λ݉೚ͨ͠ node ͸͍ͳ͍ • ຊ൪؀ڥͰ͸෼͚Δ͜ͱ͕ਪ঑͞Ε͍ͯΔ • master ʹ༨ܭͳෛՙΛ͔͚ͳ͍Α͏ʹ !20
21. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !21
22. mapping • index ઃఆͷҰ෦ • document ΛͲͷΑ͏ʹอଘ͢Δ͔ɺindex ͢Δ͔ • ϑΟʔϧυͷܕ (ྫ. text, keyword, long, date) • analyzer ͷઃఆ • document ʹԠͯࣗ͡ಈͰੜ੒͞ΕΔ • ࣄલʹఆ͓ٛͯ͘͜͠ͱ΋Ͱ͖Δ !22
23. mapping { "name": "Alice", "age": 21, "registered_at": "2019-04-23T03:00:00Z" } • ࣗಈͰੜ੒͞ΕΔ mapping ͷྫ (Elasticsearch v7.0) { "index1": { "mappings": { "properties": { "age": { "type": "long" }, "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "registered_at": { "type": "date" } } } } } Elasticsearch POST /index1/doc GET /index1/mappings !23
24. mapping • ࣗಈͰੜ੒͞ΕΔ mapping ͷྫ (Elasticsearch v7.0) { "index1": { "mappings": { "properties": { "age": { "type": "long" }, "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "registered_at": { "type": "date" } } } } } age ͷܕ͸ long name ͷܕ͸ text (શจݕࡧ༻) name.keyword Ͱ keyword ͱͯ͠΋ѻ͑Δ !24
25. ՔಇதͷγεςϜͰ͸ʁ • mapping ͸ࣄલʹఆ͍ٛͯ͠Δ • จࣈྻ͸͢΂ͯ string ܕͰ not_analyzed ͱͯ͠อଘ • ݕࡧ࣌ʹ͸อଘͨ͠จࣈྻͷ׬શҰகɺ෦෼Ұகݕࡧ • શจݕࡧͷΑ͏ͳ༻్Λ͢ΔϑΟʔϧυ͸ͳ͍ • (v7.0 Ͱ͸ keyword ܕʹ͋ͨΔઃఆ) !25
26. ՔಇதͷγεςϜͰ͸ʁ • λάʹ͍ͭͯ͸ dynamic_templates Λ࢖༻ { "index1": { "mappings": { "dynamic_templates": [ { "string-double-tags": { "path_match": "tag.*", "mapping": { "fields": { "double": { "type": "double" } }, "type": "keyword" } } } ] } } } !26
27. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !27
28. (near) ϦΞϧλΠϜͱ͸ • Ϣʔβ͕ document ͷอଘΛґཔ͢Δͱ... 1. in-memory buffer ʹॻ͖ࠐΈ (͜͜ͰϨεϙϯεฦ٫) 2. refresh (͜͜Ͱݕࡧʹ൓өɻCRUD ͸΋ͬͱૣ͍ʁ) 3. flush (͜͜ͰӬଓԽ͞ΕΔɻtranslog ΛΫϦΞ) 4. merge (աڈͷ segment Λ·ͱΊͯσΟεΫ࡟ݮ) ஫ҙ: v2.x Ͱͷઆ໌Λࢀߟʹ͍ͯ͠Δ !28
29. refresh_interval • index ઃఆͷ refresh_interval Ͱ refresh ִؒΛઃఆՄೳ # GET /index1/_settings { "index1": { "settings": { "index": { "refresh_interval": "30s", "number_of_shards": "3", "provided_name": "index1", "creation_date": "1555723945152", "number_of_replicas": "0", "uuid": "3lJEo2YZQWmNzSKjZKDNFw", "version": { "created": "7000099" } } } } } !29
30. ࢿྉ·ͱΊ • https://github.com/tiqwab/slides/tree/master/ elasticsearch !30