Introduction Elasticsearch and ELK - Elasticsearch勉強会 in Nagoya

Slide 1

Slide 1 text

Introduction Elasticsearch Jun Ohtani @johtani 2015/04/04

Slide 2

Slide 2 text

No content

Slide 3

Slide 3 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 3 • ݕࡧͱͯ͠ͷElasticsearch • ղੳͱͯ͠ͷElasticsearch • ELK stack • KibanaͰՄࢹԽ ΞδΣϯμ

Slide 4

Slide 4 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 4 about • Me, Jun Ohtani / Technical Adovocate – lucene-gosenίϛολʔ – ElasticSearch Server೔ຊޠ൛ͷ຋༁ – elasticsearch-extended-analysisͷ։ൃ – http://blog.johtani.info • Elasticsearch, founded in 2012 – Products: Elasticsearch, Logstash, Kibana, Marvel, Sheild  Professional services: Support & development subscriptions – Trainings

Slide 5

Slide 5 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 5 ݕࡧͱͯ͠ͷ Elasticsearch ݕࡧͱͯ͠ͷElasticsearch

Slide 6

Slide 6 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 6 Elasticsearchͱ͸ʁ Elasticsearchͱ͸ʁ

Slide 7

Slide 7 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 7 ϑϦʔϫʔυݕࡧ

Slide 8

Slide 8 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 8 ߜΓࠐΈ

Slide 9

Slide 9 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 9 ϋΠϥΠτ

Slide 10

Slide 10 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 10 ιʔτ

Slide 11

Slide 11 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 11 ϖʔδϯά

Slide 12

Slide 12 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 12 ूܭ

Slide 13

Slide 13 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 13 αδΣετ

Slide 14

Slide 14 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 14 Elasticsearch in 10 seconds • εΩʔϚϑϦʔɺ෼ࢄυΩϡϝϯτετΞɺREST & JSON • Φʔϓϯιʔε: Apache License 2.0 • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ • JavaͰ࣮૷ɻ֦ு΋༰қ

Slide 15

Slide 15 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 15 Powerful Search at Scale

Slide 16

Slide 16 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 16 Πϯετʔϧ͔Β ݕࡧ·Ͱ Πϯετʔϧ͔Βݕࡧ·Ͱ

Slide 17

Slide 17 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 17 μ΢ϯϩʔυͱىಈ $ wget https://download.elasticsearch.org/... $ tar -xf elasticsearch-1.5.0.tar.gz $ ./elasticsearch-1.5.0/bin/elasticsearch ... [2015-04-04 10:02:17,278][INFO ][node] [Joseph] started ... Also puppet modules and RPM/DEB

Slide 18

Slide 18 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 18 ىಈͷ֬ೝ » curl localhost:9200 { "status" : 200, "name" : "Joseph" "cluster_name" : "elasticsearch", "version" : { "number" : "1.5.0", "build_hash" : "544816042d40151d3ce4ba4f95399d7860dc2e92", "build_timestamp" : "2015-03-23T14:30:58Z", "build_snapshot" : false, "lucene_version" : "4.10.4" }, "tagline" : "You Know, for Search" }

Slide 19

Slide 19 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 19 » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : "Clinton Gormley", "started" : "2013-02-04", "pages" : 230 }' σʔλొ࿥

Slide 20

Slide 20 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 20 » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : "2013-02-04", "pages" : 230 }' σʔλߋ৽

Slide 21

Slide 21 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 21 σʔλ࡟আ » curl -X DELETE localhost:9200/books/book/1 σʔλͷऔಘ » curl —X GET localhost:9200/books/book/1 » curl —X GET localhost:9200/books/book/1/_source

Slide 22

Slide 22 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 22 ݕࡧ » curl -XGET localhost:9200/books/_search?q=elasticsearch { "took" : 2, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.076713204, "hits" : [ { "_index" : “books", "_type" : “book", "_id" : "1", "_score" : 0.076713204, "_source" : { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : “2013-02-04", "pages" : 230 } } ] } }

Slide 23

Slide 23 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 23 » curl -XGET ‘localhost:9200/books/book/_search' -d '{ "query": { "filtered" : { "query" : { "match": { "text" : { "query" : “To Be Or Not To Be", "cutoff_frequency" : 0.01 } } }, "filter" : { "range": { "price": { "gte": 20.0 "lte": 50.0 ... } }' ݕࡧ - Query DSL

Slide 24

Slide 24 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 24 σϞɿىಈ͔Βݕࡧ·Ͱ σϞ

Slide 25

Slide 25 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 25 ෼ࢄߏ੒ εέʔϧ ෼ࢄߏ੒ɺεέʔϧ

Slide 26

Slide 26 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 26 Basic terms • ΠϯσοΫε – σʔλͷ࿦ཧతͳू߹ɻ  RDBͷσʔλϕʔεͷΑ͏ͳ΋ͷLogical • ϨϓϦέʔγϣϯ • ಡΈࠐΈͷεέʔϥϏϦςΟ޲্ • SPOFͷղফ • γϟʔσΟϯά • ෳ਺Ϛγϯ΁σʔλΛ෼ׂ  ॻ͖ࠐΈͷεέʔϥϏϦςΟ޲্  σʔλϑϩʔ੍ޚ

Slide 27

Slide 27 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 27 γϟʔυͱϨϓϦΧ node 1 orders products 1 4 1 2 2 2 curl -‐X PUT localhost:9200/orders -‐d '{ "settings.index.number_of_shards" : 4 "settings.index.number_of_replicas" : 1 }' curl -‐X PUT localhost:9200/products -‐d '{ "settings.index.number_of_shards" : 2 "settings.index.number_of_replicas" : 0 }'

Slide 28

Slide 28 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 28 γϟʔυͱϨϓϦΧ node 1 orders products 1 4 1 node 2 orders products 2 2 3 4 1 2 3

Slide 29

Slide 29 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 29 ࣗಈతͳ෼ࢄ node 1 orders products 2 1 4 1 node 2 orders products 2 2 node 3 orders products 3 4 1 3

Slide 30

Slide 30 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 30 Ϋϥελ؅ཧ • ୯ҰϚελ – Ϋϥελͷঢ়ଶʢϊʔυ৘ใɺϚοϐϯάʣΛ؅ཧ • ϚϧνΩϟετʹΑΔϊʔυ୳ࡧ  ɹ (ϢχΩϟετ΋Մೳ) • ඞཁͳઃఆ – Ϋϥελ໊  ࠷খϚελϊʔυ਺ • Tips: 3ͭͷϚελϊʔυΛ༻ҙ͠ɺσʔλΛอ࣋͠ͳ͍

Slide 31

Slide 31 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 31 ΫϥελɺϊʔυͷαΠδϯά • σʔλྔͱϦΫΤετʹґଘ – υΩϡϝϯτ਺͸ʁϑΟʔϧυ਺͸ʁΫΤϦͷׂ߹͸ʁ  Facet΍AggregationɺιʔτɺΧελϜείΞͷར༻͸ʁ  ॻ͖ࠐΈͷׂ߹͸ʁυΩϡϝϯτ͸࡟আ/ߋ৽͢Δʁ  ࣌ؒʹґଘͨ͠σʔλʁ • 1ϊʔυɺϨϓϦΧແ͠Ͱςετ – γϟʔυαΠζɺJVMώʔϓ࢖༻ྔɺGCස౓ɺγϟʔυ͓Αͼϊ ʔυ਺ɺγϟʔυ͋ͨΓͷυΩϡϝϯτ਺ɺCPU࢖༻཰ɺσΟεΫ ࢖༻཰ɺΠϯσοΫεͷύλʔϯ • Tips: 30 GB heap

Slide 32

Slide 32 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 32 Ϋϥελ͓ΑͼΞϓϦͷߏ੒ • ϩʔΧϧσΟεΫ; SAN OK • Tips: ϨΠςϯγͷߴ͍WAN্ͷΫϥελ͸ඇਪ঑ɻCross-zone in EC2 is OK. Your app ES Data 1 ES Data N ... ES Master; no data Your app ... ES Master; no data ES Master; no data High Speed Network ES Node Client ES Node Client

Slide 33

Slide 33 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 33 σϞ for Elasticsearch ೔ຊޠWikipedia σϞ for Elasticsearch ೔ຊޠWikipedia

Slide 34

Slide 34 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 34 શจݕࡧͱ͸ʁ શจݕࡧͱ͸ʁ

Slide 35

Slide 35 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 35 શจݕࡧͱ͸ʁ • શจݕࡧʢFull text searchʣͱ͸ɺίϯϐϡʔλʹ͓͍ͯɺෳ਺ͷจ ॻʢϑΝΠϧʣ͔ΒಛఆͷจࣈྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕ ࡧʯ΍ʮ୯ҰϑΝΠϧ಺ͷจࣈྻݕࡧʯͱҟͳΓɺʮෳ਺จॻʹ·ͨ ͕ͬͯɺจॻʹؚ·ΕΔશจΛର৅ͱͨ͠ݕࡧʯͱ͍͏ҙຯͰ࢖༻͞ ΕΔɻ  ʢWikipediaΑΓʣ

Slide 36

Slide 36 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 36 ༻ޠ • ΠϯσοΫε ݕࡧΤϯδϯ͕ݕࡧʹ࢖༻͢Δσʔλͷอଘઌ • υΩϡϝϯτʢจॻʣ – ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ • ϑΟʔϧυ – υΩϡϝϯτʹؚ·ΕΔଐੑ • ΫΤϦ – ݕࡧ৚݅ɺݕࡧࣜ

Slide 37

Slide 37 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 37 ༻ޠ • εΩʔϚ – υΩϡϝϯτͷߏ଄Λఆٛ͢Δ΋ͷ • λʔϜʢTermʣɺτʔΫϯʢTokenʣ – ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ – จষΛҰఆͷ๏ଇͰ۠੾ͬͨ୯ޠ – ୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲ΋ؚΉ

Slide 38

Slide 38 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited ΧπΦ͸αβΤͷఋ 38 υΩϡϝϯτͷొ࿥ 1 2 αβΤ͸ϫΧϝͷ࢞ υΩϡϝϯτͷొ࿥

Slide 39

Slide 39 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 39 υΩϡϝϯτͷొ࿥ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͸ ͸ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ υΩϡϝϯτͷొ࿥ ୯ޠʹ෼ׂ

Slide 40

Slide 40 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 40 υΩϡϝϯτͷొ࿥ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͸ ͸ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 υΩϡϝϯτͷొ࿥ ୯ޠʹ෼ׂ ୯ޠ͔Βidͷ഑ྻ͕ Ҿ͚ΔΑ͏ʹ

Slide 41

Slide 41 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 41 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧ৚݅ೖྗ ΧπΦɹαβΤ

Slide 42

Slide 42 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 42 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

Slide 43

Slide 43 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 43 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

Slide 44

Slide 44 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 44 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

Slide 45

Slide 45 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 45 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

Slide 46

Slide 46 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 46 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

Slide 47

Slide 47 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 47 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

Slide 48

Slide 48 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 48 ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

Slide 49

Slide 49 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 49 ୯ޠͷ۠੾Γํ • ӳޠͷ৔߹ I am speaking Introduction Elasticsearch.     • ೔ຊޠͷ৔߹ ࢲ͸ೖ໳Elasticsearchʹ͍ͭͯ࿩͍ͯ͠Δɻ   

Slide 50

Slide 50 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 50 ୯ޠͷ۠੾Γํ • ӳޠͷ৔߹ I am speaking Introduction Elasticsearch.     εϖʔε͕੾Ε໨ͱΘ͔Δ • ೔ຊޠͷ৔߹ ࢲ͸ೖ໳Elasticsearchʹ͍ͭͯ࿩͍ͯ͠Δɻ    Ͳ͜Ͱ۠੾Ε͹Α͍ʁ

Slide 51

Slide 51 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 51 N-Gramͱܗଶૉղੳ • సஔΠϯσοΫεͷΩʔͷ࡞Γํ – ೔ຊޠ͸୯ޠͷ੾Ε໨͕Θ͔Βͳ͍ͷͰɺసஔΠϯσοΫεͷΩ ʔ͸ओʹ࣍ͷ̎ͭͷख๏Ͱ࡞੒ • N-Gram – NจࣈͣͭจষΛ۠੾Δ • ܗଶૉղੳ – ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠੾Δ

Slide 52

Slide 52 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 52 ܗଶૉղੳ • ϝϦοτɿ – ҙຯͷ͋Δ୯ޠͷ੾Ε໨  ඼ࢺ৘ใΛݩʹ௥Ճॲཧ͕Մೳʢޠװม׵ͳͲʣ • σϝϦοτɿ – ৽ޠʢະ஌ޠʣʹऑ͍ˠࣙॻϕʔεͷ৔߹ɺࣙॻʹͳ͍୯ޠ͸ݕ ग़ෆೳɻ ΧπΦ͸αβΤͷఋ ΧπΦ ͸ ͷ αβΤ ఋ

Slide 53

Slide 53 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 53 N-Gram • ϝϦοτɿ – ະ஌ޠʹରԠՄೳ • σϝϦοτɿ – ΠϯσοΫεංେԽ – ඼ࢺ৘ใʹجͮ͘ॲཧ͕ෆՄೳ ΧπΦ͸αβΤͷఋ Χπ πΦ Φ͸ ͸α αβ βΤ Τͷ ͷఋ

Slide 54

Slide 54 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 54 Analysis & Mappings Analysis & Mappings

Slide 55

Slide 55 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 55 Analysisͱ͸ʁ • సஔΠϯσοΫεͷ୯ޠʢTermʣΛυΩϡϝϯτ͔Βநग़͢Δॲཧ • ϑΟʔϧυຖʹࢦఆ͞ΕͨΞφϥΠβ͕ॲཧ • ΞφϥΠβ͸CharFilterɺTokenizerɺTokenFilter͔Βߏ੒ • ΠϯσοΫε࣌ɺݕࡧ࣌ʹ࣮ߦ

Slide 56

Slide 56 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 56 Analysisͷ֓ཁ Analysis υΩϡϝϯτ Term Id ΧπΦ 1 αβΤ 1ɺ2 ϫΧϝ 2 … ... ΠϯσοΫε

Slide 57

Slide 57 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 57 Analysisͷ֓ཁ Analysis ݕࡧ  Ωʔϫʔυ ΠϯσοΫε ΠϯσοΫε  ݕࡧॲཧ ݕࡧ݁Ռ Term Id ΧπΦ 1 αβΤ 1ɺ2 ϫΧϝ 2 … ...

Slide 58

Slide 58 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 58 Analyzer Analysisͷߏ੒ Text Tokens char_filt er tokenizer char_filt er token_  filter token_  filter

Slide 59

Slide 59 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 59 Analyzerͷઃఆ {"index":{ “analysis":{ "analyzer" : { "my_analyzer" : { "type" : "custom", "tokenizer" : “kuromoji_tokenizer”, “char_filter" : [“char_filter1”,…], “filter" : [“token_filter1”,…] } }

Slide 60

Slide 60 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 60 Char Filter • ೖྗจࣈྻΛจࣈ୯ҐͰॲཧ • ྫɿhtml_strip – ೖྗɿ – ग़ྗɿ Elasticsearch is not a service of AWS Elasticsearch is not a service of AWS

Slide 61

Slide 61 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 61 Tokenizer • ೖྗจࣈྻΛτʔΫϯྻʹ෼ׂ • τʔΫϯ΁ͷ෼ׂ͢ΔϩδοΫ͸τʔΫφΠβʹґଘ • ྫɿstandard – ೖྗɿ – ग़ྗɿ Elasticsearch is not a service of AWS Elasticsearch is not a service of AWS

Slide 62

Slide 62 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 62 Tokenizer • ྫɿkeyword – ग़ྗɿ • ྫɿkuromoji_tokenizer – ೖྗɿ – ग़ྗɿ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ Elasticsearch is not a service of AWS ण͕࢘ඒຯ͔ͬͨ͠

Slide 63

Slide 63 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 63 TokenFilter • TokenizerʹΑΓग़ྗ͞ΕͨTokenྻʹରͯ͠ॲཧ • ྫɿlowercase – ೖྗɿ – ग़ྗɿ • ྫɿstop – ೖྗɿ – ग़ྗɿ Elasticsearch is not a service of AWS elasticsearch is not a service of aws Elasticsearch is not a service of AWS Elasticsearch service AWS

Slide 64

Slide 64 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 64 TokenFilter • ྫɿkuromoji_baseform – ೖྗɿ – ग़ྗɿ • ྫɿkuromoji_readingform – ೖྗɿ – ग़ྗɿ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ ण࢘ ͕ ඒຯ͍͠ ͨ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ sushi ga oishika ta

Slide 65

Slide 65 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 65 Mapping • ΠϯσοΫεͷߏ଄ʢschemaʣΛఆٛ • Schema less!͚ͩͲ… • ܗࣜΛࢦఆʢanalyze͢Δ/͠ͳ͍ɺ੔਺/গ਺ɺ೔෇ͳͲʣ

Slide 66

Slide 66 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 66 ྫɿWikipediaͷMappingͷҰ෦ "mappings": { "page": { "properties": { "link": { "type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} } }, "text": { "type": "string"}, "title": { "type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} }…

Slide 67

Slide 67 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 67 ͦͷଞͷػೳ ͦͷଞͷػೳ

Slide 68

Slide 68 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 68 Geo • ͞·͟·ͳܗࣜͷσʔλͰGeoݕࡧՄೳ • σʔλɿ • Ң౓ܦ౓ɺGeoHashɺGeoShape • ݕࡧɿ • geo shape queryɺgeo shape filterɺgeo distance filterɺ • geo distance range filterɺgeo bounding box filterɺ • geo polygon filter

Slide 69

Slide 69 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 69 Percolator • υΩϡϝϯτͰ͸ͳ͘ΫΤϦΛొ࿥ • _percolate APIʹυΩϡϝϯτΛૹ৴ • ొ࿥͞ΕͨΫΤϦʹϚονͨ͠৔߹ɺ  Ϛονͨ͠ΫΤϦΛฦ͢

Slide 70

Slide 70 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 70 Snapshot/Restore • 1.0͔Βಋೖ • ΠϯσοΫε୯ҐͰSnapshot/RestoreՄೳ • อଘઌʢRepositoryʣ – Shard FSɺS3ɺHDFSɺAzure…

Slide 71

Slide 71 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 71 Ecosystem • Plugins – ϓϥάΠϯʹΑΔػೳͷ௥Ճ • ΫϥΠΞϯτϥΠϒϥϦ • Ruby, python, php, perl, javascript, .NET • Scala, clojure, go • Hadoop integration

Slide 72

Slide 72 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 72 ৄ͘͠஌Γ͍ͨํ͸ • Elasticsearch - The Definitive guide – http://www.elastic.co/guide/en/elasticsearch/guide/current/ index.html

Slide 73

Slide 73 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 73 ղੳͱͯ͠ͷ Elasticsearch ղੳͱͯ͠ͷElasticsearch

Slide 74

Slide 74 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 74 ELK stack ELK stack

Slide 75

Slide 75 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 75 ELK stack Data Import/Parse/ Export Store/Search Visualize

Slide 76

Slide 76 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 76 ELK stack Import/Parse/ Export Store/Search Visualize Data

Slide 77

Slide 77 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 77 Logstash Logstash

Slide 78

Slide 78 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 78 Logstash in 10 seconds • ϩάɾσʔλͷऩूɾ؅ཧ • ऩूɺύʔεɾՃ޻ɺૹग़ • Apache License 2.0 • Ruby app (JRuby)

Slide 79

Slide 79 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 79 Logstash architecture Logstash Input Output Filter ? ? collect and split alter and enrich store and visualize

Slide 80

Slide 80 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 80 ઃఆαϯϓϧ • http://blog.johtani.info/blog/2014/11/21/import-apache-accesslog- using-logstash/

Slide 81

Slide 81 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 81 ELK stack Import/Parse/ Export Store/Search Visualize Data

Slide 82

Slide 82 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 82 aggregation aggregation

Slide 83

Slide 83 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 83 Aggregationͱ͸ • 1.0͔Βಋೖ • FacetΑΓ΋ڧྗͳूܭͳͲ͕Մೳ • ֊૚తͳूܭɺάϧʔϓԽ  ಈతͳूܭɺάϧʔϓԽ • େ͖͘2छྨ • BucketɹυΩϡϝϯτΛ஋͝ͱʹ݁ՌΛάϧʔϐϯά • Metricɹ υΩϡϝϯτͷ࣋ͭ஋Λूܭ

Slide 84

Slide 84 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 84 ྫɿݴޠ͝ͱͷϑΝηοτ » curl -XGET twitter-2014.08.22/_search -d ' GET twitter-2014.08.22/_search { "facets": { "lang_facet": { "terms": {"field": "lang"} } } }

Slide 85

Slide 85 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 85 ྫɿݴޠ͝ͱͷϑΝηοτ "facets": { "lang_facet": { "_type": "terms", "missing": 0, "total": 4521075, "other": 320228, "terms": [ { "term": "en", "count": 1471819 }, { "term": "ja", "count": 980145 },

Slide 86

Slide 86 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 86 ྫɿݴޠ͓Αͼ஍Ҭͷूܭ » curl -XGET twitter-2014.08.22/_search -d ' { "aggs": { "lang": { "terms": {"field": "lang" }, "aggs": { "place": { "terms": { "field": “place.full_name", "size": 10 } } } } } }

Slide 87

Slide 87 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 87 ྫɿݴޠ͓Αͼ஍Ҭͷूܭ "aggregations": { "lang": { "buckets": [{…}, { "key": "ja", "doc_count": 980145, "place": { "buckets": [ { "key": "ژ౎ࢢ෬ݟ۠, ژ౎", "doc_count":252 }, { "key": "ઍ୅ా۠, ౦ژ", "doc_count": 39 },…

Slide 88

Slide 88 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 88 bucket • filter • filters (>=1.4.0) • missing • nested • reverse nested (>=1.2.0) • children • terms • significant terms (>=1.1.0) • range • date range • ipv4 range • histogram • date histogram • geo distance • geohash grid

Slide 89

Slide 89 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 89 metric • min • max • sum • avg • stats • extended_stats • value count • percentiles (>=1.1.0) • percentile ranks (>=1.3) • cardinality (>=1.1.0) • geo bounds (>=1.3.0) • top_hits (>=1.3.0) • scripted metric   (>= 1.4.0)

Slide 90

Slide 90 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 90 KibanaͰՄࢹԽ KibanaͰՄࢹԽ

Slide 91

Slide 91 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 91 Kibana3 • ElasticsearchͷσʔλΛՄࢹԽ • In-browserʢJavaScriptɿAngularJSɺD3ͳͲʣ • Apache License 2.0

Slide 92

Slide 92 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 92 Kibana3

Slide 93

Slide 93 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 93 Kibana3

Slide 94

Slide 94 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 94 Kibana3

Slide 95

Slide 95 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 95 σϞ for Kibana3 Twitter Stream demo for Kibana3

Slide 96

Slide 96 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 96 Kibana4 • ElasticsearchͷσʔλΛՄࢹԽ • Node.js server & JavaScript • Apache License 2.0

Slide 97

Slide 97 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 97 Kibana 4

Slide 98

Slide 98 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 98 σϞ for Kibana4 Twitter Stream demo for Kibana4

Slide 99

Slide 99 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 99 σϞ for Kibana4 Access Log demo for Kibana4

Slide 100

Slide 100 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 100 Combining Search and Analytics

Slide 101

Slide 101 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 101 Marvel Marvel

Slide 102

Slide 102 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 102 Title Text • Body Level One • Body Level Two • Body Level Three • Body Level Four » Body Level Five Overview

Slide 103

Slide 103 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 103 Title Text • Body Level One • Body Level Two • Body Level Three • Body Level Four » Body Level Five Node statistics

Slide 104

Slide 104 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 104 Title Text • Body Level One • Body Level Two • Body Level Three • Body Level Four » Body Level Five Index statistics

Slide 105

Slide 105 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 105 Title Text • Body Level One • Body Level Two • Body Level Three • Body Level Four » Body Level Five Cluster Pulse

Slide 106

Slide 106 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 106 Title Text • Body Level One • Body Level Two • Body Level Three • Body Level Four » Body Level Five Sense

Slide 107

Slide 107 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 107 found.no

Slide 108

Slide 108 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 108 found.no • Elasticsearch as a Service • Elasticsearchͷػೳ͕ར༻Մೳ • ϓϥάΠϯͷར༻΋Մೳ • ΞοϓάϨʔυ΋؆୯ • ΧελϜࣙॻɺϓϥάΠϯ΋ར༻Մೳ • 14೔ؒͷ͓ࢼ͠ظؒ͋Γ

Slide 109

Slide 109 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 109 ࢀߟจݙ • Elasticsearch - The Definitive guide – http://www.elastic.co/guide/en/elasticsearch/guide/current/ index.html • ॻ੶ʢ೔ຊޠʣ – ElasticSearchServer೔ຊޠ൛  αʔό/ΠϯϑϥΤϯδχΞ  ɹཆ੒ಡຊɹϩάऩू

Slide 110

Slide 110 text

www.elastic.co Copyright Elastic 2015 Copying, publishing and/or distributing without written permission is strictly prohibited 110 ࢀߟαΠτ • Ϣʔεέʔε • https://www.elastic.co/use-cases • Elastic{ON}ͷϏσΦͱࢿྉ • https://www.elastic.co/elasticon/2015/sf/videos • αϙʔτϝχϡʔ • https://www.elastic.co/subscriptions • Elasticsearch as a Service, Found.no • https://www.found.no/

Slide 111

Slide 111 text

Thanks for listening! Q & A We’re hiring! https://www.elastic.co/about/careers/ We’re helping! https://www.elastic.co/subscriptions http://purchases.elastic.co