Elasticsearch & Kibana

Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission
is strictly prohibited Jun Ohtani @johtani 2014/Sep. Elasticsearch & Kibana

is strictly prohibited ΞδΣϯμ • Elasticsearchͷ঺հ • ElasticsearchͷσϞ • KibanaͷσϞ

is strictly prohibited about • Me, Jun Ohtani / Technical Adovocate lucene-gosenίϛολʔ ElasticSearch Server೔ຊޠ൛ͷ຋༁ elasticsearch-extended-analysisͷ։ൃ http://blog.johtani.info • Elasticsearch, founded in 2012 Products: Elasticsearch, Logstash, Kibana, Marvel  Professional services: Support & development subscriptions Trainings

is strictly prohibited શจݕࡧͱ͸ʁ

is strictly prohibited શจݕࡧͱ͸ʁ • શจݕࡧʢFull text searchʣͱ͸ɺίϯϐϡʔλʹ ͓͍ͯɺෳ਺ͷจॻʢϑΝΠϧʣ͔Βಛఆͷจࣈ ྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯ΍ʮ୯Ұ ϑΝΠϧ಺ͷจࣈྻݕࡧʯͱҟͳΓɺʮෳ਺จॻ ʹ·͕ͨͬͯɺจॻʹؚ·ΕΔશจΛର৅ͱͨ͠ ݕࡧʯͱ͍͏ҙຯͰ࢖༻͞ΕΔɻ  ʢWikipediaΑΓʣ

is strictly prohibited ༻ޠ • ΠϯσοΫε ݕࡧΤϯδϯ͕ݕࡧʹ࢖༻͢Δσʔλͷอଘઌ • υΩϡϝϯτʢจॻʣ ݕࡧΤϯδϯʹอଘ͞Εͨσʔλ • ϑΟʔϧυ υΩϡϝϯτʹؚ·ΕΔଐੑ • ΫΤϦ ݕࡧ৚݅ɺݕࡧࣜ

is strictly prohibited ༻ޠ • εΩʔϚ υΩϡϝϯτͷߏ଄Λఆٛ͢Δ΋ͷ • λʔϜʢTermʣɺτʔΫϯʢTokenʣ ΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ จষΛҰఆͷ๏ଇͰ۠੾ͬͨ୯ޠ ୯ޠ͚ͩͰͳ͘ɺ୯ޠͷҐஔͳͲ΋ؚΉ

is strictly prohibited υΩϡϝϯτͷొ࿥ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ υΩϡϝϯτͷొ࿥

is strictly prohibited υΩϡϝϯτͷొ࿥ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͸ ͸ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ υΩϡϝϯτͷొ࿥ ୯ޠʹ෼ׂ

is strictly prohibited υΩϡϝϯτͷొ࿥ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͸ ͸ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 υΩϡϝϯτͷొ࿥ ୯ޠʹ෼ׂ ୯ޠ͔Βidͷ഑ྻ͕ Ҿ͚ΔΑ͏ʹ

is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧ৚݅ೖྗ ΧπΦɹαβΤ

is strictly prohibited ݕࡧ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ΧπΦ αβΤ AND ݕࡧ৚݅ೖྗ ݕࡧ৚݅ͷύʔε  ݕࡧΫΤϦԽ ΧπΦɹαβΤ

is strictly prohibited ୯ޠͷ۠੾Γํ • ӳޠͷ৔߹ I am speaking Introduction Elasticsearch.   ɹ  ɹ • ೔ຊޠͷ৔߹ ࢲ͸ೖ໳Elasticsearchʹ͍ͭͯ࿩͍ͯ͠Δɻ   

is strictly prohibited ୯ޠͷ۠੾Γํ • ӳޠͷ৔߹ I am speaking Introduction Elasticsearch.     εϖʔε͕੾Ε໨ͱΘ͔Δ • ೔ຊޠͷ৔߹ ࢲ͸ೖ໳Elasticsearchʹ͍ͭͯ࿩͍ͯ͠Δɻ    Ͳ͜Ͱ۠੾Ε͹Α͍ʁ

is strictly prohibited N-Gramͱܗଶૉղੳ • సஔΠϯσοΫεͷΩʔͷ࡞Γํ ೔ຊޠ͸୯ޠͷ੾Ε໨͕Θ͔Βͳ͍ͷͰɺసஔΠϯσοΫεͷ Ωʔ͸ओʹ࣍ͷ̎ͭͷख๏Ͱ࡞੒ • N-Gram NจࣈͣͭจষΛ۠੾Δ • ܗଶૉղੳ ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠੾Δ

is strictly prohibited ܗଶૉղੳ ! ! • ϝϦοτɿ ҙຯͷ͋Δ୯ޠͷ੾Ε໨  ඼ࢺ৘ใΛݩʹ௥Ճॲཧ͕Մೳʢޠװม׵ͳͲʣ • σϝϦοτɿ ৽ޠʢະ஌ޠʣʹऑ͍ˠࣙॻϕʔεͷ৔߹ɺࣙॻʹͳ͍୯ޠ͸ݕ ग़ෆೳɻ ΧπΦ͸αβΤͷఋ ΧπΦ ͸ ͷ αβΤ ఋ

is strictly prohibited N-Gram ! ! • ϝϦοτɿ ະ஌ޠʹରԠՄೳ • σϝϦοτɿ ΠϯσοΫεංେԽ ඼ࢺ৘ใʹجͮ͘ॲཧ͕ෆՄೳ ΧπΦ͸αβΤͷఋ Χπ πΦ Φ͸ ͸α αβ βΤ Τͷ ͷఋ

is strictly prohibited Elasticsearchͱ͸ʁ

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϑϦʔϫʔυݕࡧ

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ߜΓࠐΈ

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϋΠϥΠτ

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ιʔτ

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ϖʔδϯά

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited ूܭ

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited αδΣετ

is strictly prohibited Elasticsearch in 10 seconds • εΩʔϚϑϦʔɺ෼ࢄυΩϡϝϯτετΞɺREST & JSON • Φʔϓϯιʔε: Apache License 2.0 • ઃఆͳ͠Ͱ؆୯ʹࢼ͢͜ͱ͕Մೳ • JavaͰ࣮૷ɻ֦ு΋༰қ

is strictly prohibited Πϯετʔϧ͔Βݕࡧ·Ͱ

is strictly prohibited μ΢ϯϩʔυͱىಈ $ wget https://download.elasticsearch.org/... $ tar -xf elasticsearch-1.3.2.tar.gz $ ./elasticsearch-1.3.2/bin/elasticsearch ... [2014-08-29 13:53:07,356][INFO ][node] [Araña] started ... Also puppet modules and RPM/DEB

is strictly prohibited ىಈͷ֬ೝ » curl localhost:9200 { "status" : 200, "name" : "Araña", "version" : { "number" : "1.3.2", "build_hash" : "dee175dbe2f254f3f26992f5d7591939aaefd12f", "build_timestamp" : "2014-08-13T14:29:30Z", "build_snapshot" : false, "lucene_version" : "4.9" }, "tagline" : "You Know, for Search" }

is strictly prohibited » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : "Clinton Gormley", "started" : "2013-02-04", "pages" : 230 }' σʔλొ࿥

is strictly prohibited » curl -XPUT localhost:9200/books/book/1 -d ' { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : "2013-02-04", "pages" : 230 }' σʔλߋ৽

is strictly prohibited σʔλ࡟আ » curl -X DELETE localhost:9200/books/book/1 σʔλͷऔಘ » curl —X GET localhost:9200/books/book/1 » curl —X GET localhost:9200/books/book/1/_source

is strictly prohibited ݕࡧ » curl -XGET localhost:9200/books/_search?q=elasticsearch { "took" : 2, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "hits" : { "total" : 1, "max_score" : 0.076713204, "hits" : [ { "_index" : “books", "_type" : “book", "_id" : "1", "_score" : 0.076713204, "_source" : { "title" : "Elasticsearch - The definitive guide", "authors" : [ "Clinton Gormley", "Zachary Tong" ], "started" : “2013-02-04", "pages" : 230 } } ] } }

is strictly prohibited » curl -XGET ‘localhost:9200/books/book/_search' -d '{ "query": { "filtered" : { "query" : { "match": { "text" : { "query" : “To Be Or Not To Be", "cutoff_frequency" : 0.01 } } }, "filter" : { "range": { "price": { "gte": 20.0 "lte": 50.0 ... } }' ݕࡧ - Query DSL

is strictly prohibited ෼ࢄߏ੒ɺεέʔϧ

is strictly prohibited Basic terms • ΠϯσοΫε σʔλͷ࿦ཧతͳू߹ɻ  RDBͷσʔλϕʔεͷΑ͏ͳ΋ͷLogical • ϨϓϦέʔγϣϯ ಡΈࠐΈͷεέʔϥϏϦςΟ޲্ SPOFͷղফ • γϟʔσΟϯά ෳ਺Ϛγϯ΁σʔλΛ෼ׂ  ॻ͖ࠐΈͷεέʔϥϏϦςΟ޲্  σʔλϑϩʔ੍ޚ

is strictly prohibited γϟʔυͱϨϓϦΧ node 1 orders products 1 4 1 2 2 2 curl -‐X PUT localhost:9200/orders -‐d '{ "settings.index.number_of_shards" : 4 "settings.index.number_of_replicas" : 1 }' curl -‐X PUT localhost:9200/products -‐d '{ "settings.index.number_of_shards" : 2 "settings.index.number_of_replicas" : 0 }'

is strictly prohibited γϟʔυͱϨϓϦΧ node 1 orders products 1 4 1 node 2 orders products 2 2 3 4 1 2 3

is strictly prohibited ࣗಈతͳ෼ࢄ node 1 orders products 2 1 4 1 node 2 orders products 2 2 node 3 orders products 3 4 1 3

is strictly prohibited Ϋϥελ؅ཧ • ୯ҰϚελ Ϋϥελͷঢ়ଶʢϊʔυ৘ใɺϚοϐϯάʣΛ؅ཧ • ϚϧνΩϟετʹΑΔϊʔυ୳ࡧ  ɹ (ϢχΩϟετ΋Մೳ) • ඞཁͳઃఆ Ϋϥελ໊  ࠷খϚελϊʔυ਺ • Tips: 3ͭͷϚελϊʔυΛ༻ҙ͠ɺσʔλΛอ࣋ ͠ͳ͍

is strictly prohibited ΫϥελɺϊʔυͷαΠδϯά • σʔλྔͱϦΫΤετʹґଘ υΩϡϝϯτ਺͸ʁϑΟʔϧυ਺͸ʁΫΤϦͷׂ߹͸ʁ  Facet΍AggregationɺιʔτɺΧελϜείΞͷར༻͸ʁ  ॻ͖ࠐΈͷׂ߹͸ʁυΩϡϝϯτ͸࡟আ/ߋ৽͢Δʁ  ࣌ؒʹґଘͨ͠σʔλʁ • 1ϊʔυɺϨϓϦΧແ͠Ͱςετ γϟʔυαΠζɺJVMώʔϓ࢖༻ྔɺGCස౓ɺγϟʔυ͓Α ͼϊʔυ਺ɺγϟʔυ͋ͨΓͷυΩϡϝϯτ਺ɺCPU࢖༻ ཰ɺσΟεΫ࢖༻཰ɺΠϯσοΫεͷύλʔϯ • Tips: 30 GB heap

is strictly prohibited Ϋϥελ͓ΑͼΞϓϦͷߏ੒ • ϩʔΧϧσΟεΫ; SAN OK • Tips: ϨΠςϯγͷߴ͍WAN্ͷΫϥελ͸ඇਪ ঑ɻCross-zone in EC2 is OK. ! Your app ES Data 1 ES Data N ... ES Master; no data ! Your app ... ES Master; no data ES Master; no data High Speed Network ES Node Client ES Node Client

is strictly prohibited ΠϯσοΫεͷొ࿥ͱݕࡧͷ ಺෦ಈ࡞

is strictly prohibited υΩϡϝϯτొ࿥γʔέϯε • υΩϡϝϯτΛొ࿥ doc1 node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1

is strictly prohibited υΩϡϝϯτొ࿥γʔέϯε • υΩϡϝϯτͷIDͷϋογϡ஋Λܭࢉͯ͠γϟʔ υ਺ͰׂΓɺొ࿥͢ΔγϟʔυΛܾΊΔ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 doc1

is strictly prohibited υΩϡϝϯτొ࿥γʔέϯε • γϟʔυ͕ܾ·ͬͨΒɺϓϥΠϚϦγϟʔυ͕͋ Δϊʔυ͕Ͳ͔͜Λௐ΂Δ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 doc1

is strictly prohibited υΩϡϝϯτొ࿥γʔέϯε • ϓϥΠϚϦ͔ΒϨϓϦΧ΁ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 doc1

is strictly prohibited ݕࡧγʔέϯε • ݕࡧΫΤϦΛ࣮ߦ Query node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1

is strictly prohibited ݕࡧγʔέϯε • ϊʔυ͕̍ड͚औͬͨΫΤϦΛશγϟʔυʹૹ৴ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 Query

is strictly prohibited ݕࡧγʔέϯε • ϊʔυ͕̍ड͚औͬͨΫΤϦΛશγϟʔυʹૹ৴ • γϟʔυຖʹݕࡧ࣮ߦ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 Query Query Query Query Query

is strictly prohibited ݕࡧγʔέϯε • શγϟʔυ͔Βͷ݁ՌΛϚʔδ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 Query ݁Ռ ݁Ռ ݁Ռ ݁Ռ

is strictly prohibited ݕࡧγʔέϯε • શγϟʔυ͔Βͷ݁ՌΛϚʔδ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1 ݁Ռ

is strictly prohibited ݕࡧγʔέϯε • ݁ՌΛૹ৴ ݁Ռ node 1 bookstore 2 1 4 node 2 bookstore 2 3 3 4 1

is strictly prohibited Analysis & Mappings

is strictly prohibited Analysisͱ͸ʁ • సஔΠϯσοΫεͷ୯ޠʢTermʣΛυΩϡϝϯτ ͔Βநग़͢Δॲཧ • ϑΟʔϧυຖʹࢦఆ͞ΕͨΞφϥΠβ͕ॲཧ • ΞφϥΠβ͸CharFilterɺTokenizerɺTokenFilter͔ Βߏ੒ • ΠϯσοΫε࣌ɺݕࡧ࣌ʹ࣮ߦ

is strictly prohibited Analysisͷ֓ཁ Analysis υΩϡϝϯτ Term Id ΧπΦ 1 αβΤ 1ɺ2 ϫΧϝ 2 … ... ΠϯσοΫε

is strictly prohibited Analysisͷ֓ཁ Analysis ݕࡧ  Ωʔϫʔυ ΠϯσοΫε ΠϯσοΫε  ݕࡧॲཧ ݕࡧ݁Ռ Term Id ΧπΦ 1 αβΤ 1ɺ2 ϫΧϝ 2 … ...

is strictly prohibited Analyzer Analysisͷߏ੒ char_filter tokenizer char_filter token_  filter token_  filter Text Tokens

is strictly prohibited Analyzerͷઃఆ {"index":{ “analysis":{ "analyzer" : { "my_analyzer" : { "type" : "custom", "tokenizer" : “kuromoji_tokenizer”, “char_filter" : [“char_filter1”,…], “filter" : [“token_filter1”,…] } }

is strictly prohibited Char Filter • ೖྗจࣈྻΛจࣈ୯ҐͰॲཧ ! • ྫɿhtml_strip ೖྗɿ ! ग़ྗɿ <title>Elasticsearch is not a service of AWS</title> Elasticsearch is not a service of AWS

is strictly prohibited Tokenizer • ೖྗจࣈྻΛτʔΫϯྻʹ෼ׂ • τʔΫϯ΁ͷ෼ׂ͢ΔϩδοΫ͸τʔΫφΠβʹ ґଘ • ྫɿstandard ೖྗɿ ! ग़ྗɿ Elasticsearch is not a service of AWS Elasticsearch is not a service of AWS

is strictly prohibited Tokenizer • ྫɿkeyword ग़ྗɿ • ྫɿkuromoji_tokenizer ೖྗɿ ! ग़ྗɿ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ Elasticsearch is not a service of AWS ण͕࢘ඒຯ͔ͬͨ͠

is strictly prohibited TokenFilter • TokenizerʹΑΓग़ྗ͞ΕͨTokenྻʹରͯ͠ॲཧ • ྫɿlowercase ೖྗɿ ! ग़ྗɿ • ྫɿstop ೖྗɿ ! ग़ྗɿ Elasticsearch is not a service of AWS elasticsearch is not a service of aws Elasticsearch is not a service of AWS Elasticsearch service AWS

is strictly prohibited TokenFilter • ྫɿkuromoji_baseform ೖྗɿ ! ग़ྗɿ • ྫɿkuromoji_readingform ೖྗɿ ! ग़ྗɿ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ ण࢘ ͕ ඒຯ͍͠ ͨ ण࢘ ͕ ඒຯ͔ͬ͠ ͨ sushi ga oishika ta

is strictly prohibited Mapping • ΠϯσοΫεͷߏ଄ʢschemaʣΛఆٛ • Schema less!͚ͩͲ… • ܗࣜΛࢦఆʢanalyze͢Δ/͠ͳ͍ɺ੔਺/গ਺ɺ೔ ෇ͳͲʣ

is strictly prohibited ྫɿWikipediaͷMappingͷҰ෦ "mappings": { "page": { "properties": { "link": { "type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} } }, "text": { "type": "string"}, "title": { "type": "string", "fields": { "raw": {"type": "string", "index": "not_analyzed"} }…

is strictly prohibited aggregation

is strictly prohibited Aggregationͱ͸ • 1.0͔Βಋೖ • FacetΑΓ΋ڧྗͳूܭͳͲ͕Մೳ ֊૚తͳूܭɺάϧʔϓԽ  ಈతͳूܭɺάϧʔϓԽ • େ͖͘2छྨ BucketɹυΩϡϝϯτΛ஋͝ͱʹ݁ՌΛάϧʔϐϯά Metricɹ υΩϡϝϯτͷ࣋ͭ஋Λूܭ

is strictly prohibited ྫɿݴޠ͝ͱͷϑΝηοτ » curl -XGET twitter-2014.08.22/_search -d ' GET twitter-2014.08.22/_search { "facets": { "lang_facet": { "terms": {"field": "lang"} } } }

is strictly prohibited ྫɿݴޠ͝ͱͷϑΝηοτ "facets": { "lang_facet": { "_type": "terms", "missing": 0, "total": 4521075, "other": 320228, "terms": [ { "term": "en", "count": 1471819 }, { "term": "ja", "count": 980145 },

is strictly prohibited ྫɿݴޠ͓Αͼ஍Ҭͷूܭ » curl -XGET twitter-2014.08.22/_search -d ' { "aggs": { "lang": { "terms": {"field": "lang" }, "aggs": { "place": { "terms": { "field": “place.full_name", "size": 10 } } } } } }

is strictly prohibited ྫɿݴޠ͓Αͼ஍Ҭͷूܭ "aggregations": { "lang": { "buckets": [{…}, { "key": "ja", "doc_count": 980145, "place": { "buckets": [ { "key": "ژ౎ࢢ෬ݟ۠, ژ౎", "doc_count":252 }, { "key": "ઍ୅ా۠, ౦ژ", "doc_count": 39 },…

is strictly prohibited bucket • filter • filters (>=1.4.0) • missing • nested • reverse nested (>=1.2.0) • children • terms • significant terms (>=1.1.0) • range • date range • ipv4 range • histogram • date histogram • geo distance • geohash grid

is strictly prohibited metric • min • max • sum • avg • stats • extended_stats • value count • percentiles (>=1.1.0) • percentile ranks (>=1.3) • cardinality (>=1.1.0) • geo bounds (>=1.3.0) • top_hits (>=1.3.0) • scripted metric   (>= 1.4.0)

is strictly prohibited ͦͷଞͷػೳ

is strictly prohibited Geo • ͞·͟·ͳܗࣜͷσʔλͰGeoݕࡧՄೳ • σʔλɿ Ң౓ܦ౓ɺGeoHashɺGeoShape • ݕࡧɿ geo shape queryɺgeo shape filterɺgeo distance filterɺ geo distance range filterɺgeo bounding box filterɺ geo polygon filter

is strictly prohibited ྫɿGeoσʔλొ࿥ ## Ң౓ܦ౓ { "pin" : { "location" : "41.12,-71.34" } } ! ## GeoHash { "pin" : { "location" : "drm3btev3e86" } }

is strictly prohibited ྫɿGeoσʔλొ࿥ ## GeoShape(Polygon) { "location" : { "type" : "polygon", "coordinates" : [[ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ]] } } ## GeoShape(Circle) { "location" : { "type" : "circle", "coordinates" : [-45.0, 45.0], "radius" : "100m" } }

is strictly prohibited ྫɿGeoσʔλݕࡧ { "query": { "geo_shape": { "location": { "shape": { "type": "envelope", "coordinates": [[13, 53],[14, 52]] } } } } }

is strictly prohibited ྫɿڑ཭ʹΑΔϑΟϧλ { "filtered" : { "query" : {"match_all" : {}}, "filter" : { "geo_distance" : { "distance" : "200km", "pin.location" : { "lat" : 40, "lon" : -70 } } } } }

is strictly prohibited Percolator • υΩϡϝϯτͰ͸ͳ͘ΫΤϦΛొ࿥ • _percolate APIʹυΩϡϝϯτΛૹ৴ • ొ࿥͞ΕͨΫΤϦʹϚονͨ͠৔߹ɺ  Ϛονͨ͠ΫΤϦΛฦ͢

is strictly prohibited ྫɿΫΤϦొ࿥ curl -XPUT 'localhost:9200/my-index/.percolator/1' -d '{ "query" : { "match" : { "message" : "bonsai tree" } } }'

is strictly prohibited ྫɿσʔλνΣοΫ curl -XGET 'localhost:9200/my-index/message/_percolate' -d '{ "doc" : { "message" : "A new bonsai tree in the office" } }'

is strictly prohibited ྫɿ݁Ռ { "took" : 19, "_shards" : { "total" : 5, "successful" : 5, "failed" : 0 }, "total" : 1, "matches" : [ { "_index" : "my-index", "_id" : "1" } ] }

is strictly prohibited Snapshot/Restore • 1.0͔Βಋೖ • ΠϯσοΫε୯ҐͰSnapshot/RestoreՄೳ • อଘઌʢRepositoryʣ Shard FSɺS3ɺHDFSɺAzure…

is strictly prohibited ྫɿϦϙδτϦ࡞੒ $ curl -XPUT 'http://localhost:9200/_snapshot/ my_backup' -d '{ "type": "fs", "settings": { "location": "/mount/backups/my_backup", "compress": true } }'

is strictly prohibited ྫɿSnapshotͱϦετΞ ## Snapshot  $ curl -XPUT “localhost:9200/_snapshot/my_backup/ snapshot_1?wait_for_completion=true" ! ! ## Restore $ curl -XPOST "localhost:9200/_snapshot/my_backup/ snapshot_1/_restore"

is strictly prohibited Ecosystem • Plugins ϓϥάΠϯʹΑΔػೳͷ௥Ճ • ΫϥΠΞϯτϥΠϒϥϦ Ruby, python, php, perl, javascript, .NET Scala, clojure, go • Kibana • Logstash

is strictly prohibited Hadoop࿈ܞ&Kibana • ʮαʔόʗΠϯϑϥΤϯδχΞཆ੒ಡຊ ϩάऩ ूʙՄࢹԽฤʯൃചத • Hadoop࿈ܞͷ঺հ • Kibanaͷৄࡉ

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited

is strictly prohibited Tools for sys admins

is strictly prohibited REST-based management • Elasticsearch is full of monitoring APIs Everything is returned as JSON • Humans are not the world’s best JSON parsers • What if elasticsearch had an easy to use interface from the commandline?

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Which node is the master? $ curl "localhost:9200/_cluster/state?pretty&filter_metadata=true&! filter_routing_table=true"! {! "cluster_name" : "elasticsearch",! "master_node" : "GNf0hEXlTfaBvQXKBF300A",! "blocks" : { },! "nodes" : {! "ObdRqLHGQ6CMI5rOEstA5A" : {! "name" : "Triton",! "transport_address" : “inet[/10.0.1.11:9300]”,! "attributes" : { }! },! "4C7pKbfhTvu0slcSy_G4_w" : {! "name" : "Kid Colt",! "transport_address" : "inet[/10.0.1.12:9300]",! "attributes" : { }! },! "GNf0hEXlTfaBvQXKBF300A" : {! "name" : "Lang, Steven",! "transport_address" : "inet[/10.0.1.13:9300]",! "attributes" : { }! }! }! }

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited Which one is the master? (v1.0) $ curl localhost:9200/_cat/master GNf0hEXlTfaBvQXKBF300A 10.0.1.13 Lang, Steven

is strictly prohibited Copyright Elasticsearch 2013. Copying, publishing and/or distributing without written permission is strictly prohibited _cat/* api • /_cat/allocation • /_cat/count • /_cat/health • /_cat/master • /_cat/aliases • /_cat/nodes • /_cat/recovery • /_cat/shards • /_cat/indices • /_cat/thread_pool

is strictly prohibited Monitor your cluster with Marvel • Point in time views are a start • Marvel shows historical trends • Visualize cluster behavior, act before problems ! • Free for development, $500/year for up to 5 nodes

is strictly prohibited Overview

is strictly prohibited Node statistics

is strictly prohibited Index statistics

is strictly prohibited Cluster Pulse

is strictly prohibited Sense

is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Elasticsearch  ೔ຊޠWikipedia

is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Elasticsearch  Twitter Stream

is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Kibana  Twitter Stream

is strictly prohibited Copyright Elasticsearch 2014. Copying, publishing and/or distributing without written permission is strictly prohibited σϞ for Kibana  Git log

is strictly prohibited ࢀߟจݙ • Elasticsearch - The Definitive guide http://www.elasticsearch.com/guide/en/elasticsearch/guide/ current/index.html • ॻ੶ ElasticSearchServer೔ຊޠ൛  αʔό/ΠϯϑϥΤϯδχΞ  ɹཆ੒ಡຊɹϩάऩू

is strictly prohibited Q&A • Training http://training.elasticsearch.com • Support http://www.elasticsearch.com/support/ • Github https://github.com/elasticsearch

Elasticsearch & Kibana

Elasticsearch & Kibana

More Decks by Jun Ohtani

Other Decks in Technology

Featured

Transcript