Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Elasticsearch の基礎 / Get Started with Elasticsearch

Elasticsearch の基礎 / Get Started with Elasticsearch

Naohisa Murakami

April 23, 2019
Tweet

Other Decks in Programming

Transcript

  1. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node

    ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !2
  2. Elasticsearch ͱ͸ ಛ௃ (ʮσʔλ෼ੳج൫ߏஙೖ໳ʯΑΓɺͨͩ͠ଠࣈ͸ಠࣗ) • OSS (Apache Lisence v2) •

    υΩϡϝϯτࢦ޲ • ෼ࢄγεςϜ • Ϛϧνςφϯτ • RESTful API • (near) ϦΞϧλΠϜ !3
  3. υΩϡϝϯτࢦ޲ • υΩϡϝϯτ (document) • Elasticsearch Ͱѻ͏σʔλͷ࠷খ୯Ґ • JSON •

    id Λ࣋ͭ • (εΩʔϚϨεʹ࢖͑ΔͷͰ) ॊೈͳσʔλͷొ࿥͕Մೳ !4
  4. RESTful API • ͋ΒΏΔૢ࡞͕ REST API Ͱఏڙ͞Ε͍ͯΔ • document ͷ௥Ճɺߋ৽ɺ࡟আ

    • document ͷݕࡧ • ઃఆͷ֬ೝɺมߋ • ϝτϦΫεͷऔಘ !6
  5. RESTful API • ྫ. document ͷ௥Ճ readonly ES_INDEX=index1 readonly ES_TYPE=_doc

    request=$(cat <<EOF { "name": "Alice", "age": 21, "registered_at": "2019-04-23T03:00:00Z" } EOF ) curl -XPOST -H "Content-Type: application/json" \ http://$ES_HOST:$ES_PORT/$ES_INDEX/$ES_TYPE \ -d "$request" !7
  6. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node

    ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !9
  7. Elasticsearch ͷجຊ֓೦ • Elasticsearch ͸ 1 ୆Ҏ্ͷ node Ͱ cluster

    Λߏ੒͢Δ • cluster ʹ͸ 0 Ҏ্ͷ index ͕ଘࡏ͢Δ • index ʹ͸ 1 ͭҎ্ͷ shard ͕ଘࡏ͢Δ • document ͸͍ͣΕ͔ͷ shard ʹอଘ͞ΕΔ • client ͸ index Λࢦఆͯ͠อଘ͢Δ • อଘઌͷ shard ͸ id (ਖ਼֬ʹ͸ routing) ʹΑܾͬͯ·Δ !10
  8. Cluster shard ਺Λ 1 -> 4 ʹ Index1 Node1 Node2

    Node3 0 1 2 3 primary shard !13
  9. Cluster ֤ shard ʹ 1 replica ௥Ճ Index1 Node1 Node2

    Node3 0 1 2 3 0 1 2 3 primary shard replica shard !14
  10. Cluster index ௥Ճ Index1 Node1 Node2 Node3 0 1 2

    3 0 1 2 3 primary shard replica shard Index2 !15
  11. ՔಇதͷγεςϜͰ͸ʁ • 9 node Ͱ cluster Λߏ੒ • 1 index

    ͷΈ, 128 primary shards • ద੾ͳ primary shard ਺͸ѻ͏σʔλ౳ʹΑͬͯҟͳΔ • ಈతʹ primary shard ਺Λมߋ͢Δ͜ͱ͸Ͱ͖ͳ͍ • replica ਺͸ 1 (֤ primary shard ʹ 1 replica ଘࡏ) • id, routing ͸ಠࣗʹઃఆ !16
  12. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node

    ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !17
  13. master node ͱ data node • ֤ node ͸ىಈ࣌ʹ໾ׂΛઃఆ͞ΕΔ •

    ໾ׂ1: master eligible (master ީิ) • ͜ͷ node ͷத͔Β master ͕Ұ୆બ͹ΕΔ • ໾ׂ2: data (document ͷอଘઌ) • ׂΓ౰ͯΒΕͨ shard ΁ͷ document ͷอଘɺݕࡧΛ୲౰ • master eligible, data ͸݉೚Ͱ͖Δ !18
  14. Cluster master node ͱ data node Index1 Node1 Node2 Node3

    0 1 2 3 0 1 2 3 Index2 master eligible master data !19
  15. ՔಇதͷγεςϜͰ͸ʁ • master eligible node ͕ 3 ୆ • data

    node ͕ 6 ୆ • master eligible ͱ data Λ݉೚ͨ͠ node ͸͍ͳ͍ • ຊ൪؀ڥͰ͸෼͚Δ͜ͱ͕ਪ঑͞Ε͍ͯΔ • master ʹ༨ܭͳෛՙΛ͔͚ͳ͍Α͏ʹ !20
  16. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node

    ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !21
  17. mapping • index ઃఆͷҰ෦ • document ΛͲͷΑ͏ʹอଘ͢Δ͔ɺindex ͢Δ͔ • ϑΟʔϧυͷܕ

    (ྫ. text, keyword, long, date) • analyzer ͷઃఆ • document ʹԠͯࣗ͡ಈͰੜ੒͞ΕΔ • ࣄલʹఆ͓ٛͯ͘͜͠ͱ΋Ͱ͖Δ !22
  18. mapping { "name": "Alice", "age": 21, "registered_at": "2019-04-23T03:00:00Z" } •

    ࣗಈͰੜ੒͞ΕΔ mapping ͷྫ (Elasticsearch v7.0) { "index1": { "mappings": { "properties": { "age": { "type": "long" }, "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "registered_at": { "type": "date" } } } } } Elasticsearch POST /index1/doc GET /index1/mappings !23
  19. mapping • ࣗಈͰੜ੒͞ΕΔ mapping ͷྫ (Elasticsearch v7.0) { "index1": {

    "mappings": { "properties": { "age": { "type": "long" }, "name": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "registered_at": { "type": "date" } } } } } age ͷܕ͸ long name ͷܕ͸ text (શจݕࡧ༻) name.keyword Ͱ keyword ͱͯ͠΋ѻ͑Δ !24
  20. ՔಇதͷγεςϜͰ͸ʁ • mapping ͸ࣄલʹఆ͍ٛͯ͠Δ • จࣈྻ͸͢΂ͯ string ܕͰ not_analyzed ͱͯ͠อଘ

    • ݕࡧ࣌ʹ͸อଘͨ͠จࣈྻͷ׬શҰகɺ෦෼Ұகݕࡧ • શจݕࡧͷΑ͏ͳ༻్Λ͢ΔϑΟʔϧυ͸ͳ͍ • (v7.0 Ͱ͸ keyword ܕʹ͋ͨΔઃఆ) !25
  21. ՔಇதͷγεςϜͰ͸ʁ • λάʹ͍ͭͯ͸ dynamic_templates Λ࢖༻ { "index1": { "mappings": {

    "dynamic_templates": [ { "string-double-tags": { "path_match": "tag.*", "mapping": { "fields": { "double": { "type": "double" } }, "type": "keyword" } } } ] } } } !26
  22. τϐοΫε 1. Elasticsearch ͱ͸ 2. Elasticsearch ͷجຊ֓೦ 3. master node

    ͱ data node 4. mapping 5. (near) ϦΞϧλΠϜͱ͸ !27
  23. (near) ϦΞϧλΠϜͱ͸ • Ϣʔβ͕ document ͷอଘΛґཔ͢Δͱ... 1. in-memory buffer ʹॻ͖ࠐΈ

    (͜͜ͰϨεϙϯεฦ٫) 2. refresh (͜͜Ͱݕࡧʹ൓өɻCRUD ͸΋ͬͱૣ͍ʁ) 3. flush (͜͜ͰӬଓԽ͞ΕΔɻtranslog ΛΫϦΞ) 4. merge (աڈͷ segment Λ·ͱΊͯσΟεΫ࡟ݮ) ஫ҙ: v2.x Ͱͷઆ໌Λࢀߟʹ͍ͯ͠Δ !28
  24. refresh_interval • index ઃఆͷ refresh_interval Ͱ refresh ִؒΛઃఆՄೳ # GET

    /index1/_settings { "index1": { "settings": { "index": { "refresh_interval": "30s", "number_of_shards": "3", "provided_name": "index1", "creation_date": "1555723945152", "number_of_replicas": "0", "uuid": "3lJEo2YZQWmNzSKjZKDNFw", "version": { "created": "7000099" } } } } } !29