solrとelasticsearchの比較

elasticsearchの紹介です！
atnd.org/events/33718

Transcript

1. elasticsearchͱSolrͷൺֱ ݉ࢁ ݩଠ @penguinana_ Monday, November 26, 12
2. ࣗݾ঺հ • ݉ࢁ ݩଠ @penguinana_ • ϨγϐݕࡧνʔϜ @ http://cookpad.com/ • Solr4.0 Monday, November 26, 12
3. SolrͷόʔδϣϯΞοϓΛ ݕ౼͍ͯ͠Δͱ͖... Monday, November 26, 12
4. Elasticsearch΋ ௐ΂ͨ΄͏͕͍͍ͷͰ͸ʁ Monday, November 26, 12
5. • Luceneϕʔε • HTTP API • ෼ࢄݕࡧOK • ೔ຊޠOK Monday, November 26, 12
6. • Luceneϕʔε • HTTP API • ෼ࢄݕࡧOK • ೔ຊޠOK طࢹײ Monday, November 26, 12
7. http://solr-vs-elasticsearch.com/ Monday, November 26, 12
8. ײ૝ • ػೳ໘Ͱෆ଍͸ͳ͍ • API͕։ൃऀʹ΍͍͞͠ • ༰қʹशಘͰ͖Δ • େن໛෼ࢄݕࡧҎ֎Ͱ΋༗༻ • SolrΛ࢖ͬͯͳ͚Ε͹ͬͪ͜Λຊ൪ʹ࢖͍ͨ ͍ʂ Monday, November 26, 12
9. αϯϓϧΛ࢖ͬͯ ͻͱ௨Γઆ໌͠·͢ Monday, November 26, 12
10. http://blog.livedoor.jp/techblog/archives/65836960.html Monday, November 26, 12
11. livedoorάϧϝ • Ϩετϥϯ৘ใ(21.4ສళ) • ళ໊ɺѻ͍ͬͯΔྉཧɺॅॴɺҢ౓ ܦ౓ɺΞΫηε਺ɺ࠷دΓฑߦ͖͔ Βͷڑ཭ɺetc... Monday, November 26, 12
12. livedoorάϧϝ • ϨϏϡʔ৘ใ(20.5ສϨϏϡʔ) • ૯߹ධՁʢ5ஈ֊ʣ • งғؾɺ஋ஈɺαʔϏεɺຯ • ϨϏϡʔίϝϯτ Monday, November 26, 12
13. https://github.com/penguinco/ld_gourmet_search Monday, November 26, 12
14. ElasticsearchΛ࢖͏ • 1݅ొ࿥ͯ͠ɺ1݅ݕࡧ • ೔ຊޠͷѻ͍Λఆٛ • εΩʔϚఆٛ • औΓࠐΈ • ݕࡧ • είΞϦϯάͳͲͷௐ੔ Monday, November 26, 12
15. PUT curl -XPUT http://localhost:9200/twitter/tweet/1 -d ' { "user": "kimchy", "post_date": "2012-11-26T20:12:00", "message": "Trying out elasticsearch", "score": 5 } ' index type id Monday, November 26, 12
16. PUT curl -XPUT http://localhost:9200/twitter/user/kimchy -d ' { "name" : "Shay Banon" } ' index type id Monday, November 26, 12
17. GET curl -XGET http://localhost:9200/twitter/tweet/1 { "user": "kimchy", "post_date": "2012-11-26T20:12:00", "message": "Trying out elasticsearch", "score": 5 } } index type id Monday, November 26, 12
18. SEARCH curl -XGET http://localhost:9200/twitter/tweet/_search -d '{ "query" : { "term" : { "user": "kimchy" } } }' index type id { "user": "kimchy", "post_date": "2012-11-26T20:12:00", "message": "Trying out elasticsearch", "score": 5 } Monday, November 26, 12
19. REST API • υΩϡϝϯτͷ௥Ճɾ࡟আ • ઃఆͷ௥Ճɾ࡟আ • શ෦HTTP APIͰͰ͖Δ • εΩʔϚϑϦʔ Monday, November 26, 12
20. ೔ຊޠ $ curl -XGET 'localhost:9200/_analyze?pretty' -d 'ਆઘ' { "tokens" : [ { "token" : "ਆ", "start_offset" : 0, "end_offset" : 1, "type" : "<IDEOGRAPHIC>", "position" : 1 }, { "token" : "ઘ", "start_offset" : 1, "end_offset" : 2, "type" : "<IDEOGRAPHIC>", "position" : 2 } ] } Monday, November 26, 12
21. ೔ຊޠ AnalyzerΛมߋ͢Δ͜ͱͰରԠ kuromoji͕࢖͑·͢ʂ http://www.hirotakaster.com/archives/2012/11/ elasticsearch-kuromoji-plugin.php Monday, November 26, 12
22. kuromoji $ cd elasticsearch $ bin/plugin -install elasticsearch/elasticsearch-analysis-kuromoji/1.0.0 $ git clone git://github.com/elasticsearch/elasticsearch-analysis- kuromoji.git $ cd elasticsearch-analysis-kuromoji/ $ mvn clean package $ cp target/elasticsearch-analysis-kuromoji-1.2.0-SNAPSHOT.jar ../plugins/ analysis-kuromoji/elasticsearch-analysis-kuromoji-1.0.0.jar # restart elasticsearch Monday, November 26, 12
23. add analyzer $ curl -XPUT 'localhost:9200/test/' -d ' { "index":{ "analysis":{ "tokenizer" : { "kuromoji" : { "type":"kuromoji_tokenizer", "mode":"search" } }, "analyzer" : { "kuromoji_analyzer" : { "type" : "custom", "tokenizer" : "kuromoji_tokenizer" } } } } } ‘ Monday, November 26, 12
24. kuromoji $ curl -XGET 'localhost:9200/test/_analyze? analyzer=kuromoji_analyzer&pretty' -d 'ਆઘ' { "tokens" : [ { "token" : "ਆઘ", "start_offset" : 0, "end_offset" : 2, "type" : "word", "position" : 1 } ] } Monday, November 26, 12
25. _analyze $ curl -XGET 'localhost:9200/test/_analyze? analyzer=kuromoji_analyzer&pretty' -d 'ؔ੢ࠃࡍۭߓ' { "tokens" : [ {"token" : "ؔ੢",}, {"token" : "ؔ੢ࠃࡍۭߓ",}, {"token" : "ࠃࡍ",}, {"token" : "ۭߓ",} ] } Monday, November 26, 12
26. kuromojiΛσϑΥϧτʹ • default͍ͬͯ͏໊લͰanalyzerΛએݴ Monday, November 26, 12
27. ಉٛޠ • Solrಉ༷ಉٛޠ͕ϑΝΠϧͰॻ͚Δ • +WordNetܗࣜ΋࢖͑Δ Monday, November 26, 12
28. analyzer Monday, November 26, 12
29. ೔ຊޠͷ৺഑͸͋Δఔ౓ย෇͍ͨʂ Monday, November 26, 12
30. εΩʔϚఆٛ • εΩʔϚϑϦʔʂ • JSONͷܕ͕࠾༻͞ΕΔ • ڧ੍తʹఆٛ΋Ͱ͖Δ(mapping) Monday, November 26, 12
31. mappingྫ $ curl -XPUT 'http://localhost:9200/twitter/tweet/ _mapping' -d ' { "tweet" : { "properties" : { "message" : {"type" : "string", "store" : "yes"} } } } ' Monday, November 26, 12
32. Solrͱͷࠩ෼ • SolrͷDynamicFieldΑΓ΋؆୯ • type • 1ίΞ಺ʹෳ਺छྨͷdocΛೖΕΔ͜ ͱΛ૝ఆͯ͋ͬͯ͠ศར Monday, November 26, 12
33. import(ruby) ratings = [] CSV.foreach("ratings.csv") do |row| ratings << { :id => row[:id].to_i, :restaurant_id => row[:restaurant_id].to_i, :body => row[:body], :type => 'rating' } end Tire.index 'livedoor_gourmet' do import ratings end Monday, November 26, 12
34. ݕࡧ curl -X GET 'http://localhost:9200/livedoor_gourmet/ restaurant/_search?pretty' -d ' { "query":{ "query_string":{ "query":"ϥʔϝϯ" } }, "sort":[{"access_count":"desc"}], "filter":{ "term":{"closed":"0"} } } ' Monday, November 26, 12
35. Solrͱͷࠩ෼ • DSL͕݁ߏҧ͏ • filter, facet, grouping, highlight΋αϙʔτ • είΞϦϯά͸εΫϦϓτݴޠͰఆٛ Ͱ͖Δ Monday, November 26, 12
36. είΞϦϯά • PVॱͰฒ΂ͨΒ͏·͍ͬͨ͆͘ • ݱ࣮ͷ໰୊΋݁ߏ͜͏͍͏͜ͱଟ͍ Monday, November 26, 12
37. είΞϦϯά • ڵຯͷ͋Δํ͸ͥͻ • εΫϦϓτݴޠͰఆٛͰ͖Δ • google: elasticsearch guide scoring Monday, November 26, 12
38. ײ૝ • ػೳ໘Ͱෆ଍͸ͳ͍ • API͕։ൃऀʹ΍͍͞͠ • ༰қʹशಘͰ͖Δ • େن໛෼ࢄݕࡧҎ֎Ͱ΋༗༻ Monday, November 26, 12
39. API Monday, November 26, 12
40. config curl͚ͩͰͰ͖Δ →ΞϓϦέʔγϣϯʹఆٛΛஔ͚Δ Monday, November 26, 12
41. ίΞ௥Ճ curl͚ͩͰͰ͖Δ →։ൃऀͻͱΓͰ׬݁Ͱ͖Δ Monday, November 26, 12
42. ༰қʹशಘͰ͖Δ • ΄ͱΜͲͷૢ࡞͸curlͰ׬݁ • Solrͱڞ௨ͷ஌ࣝ΋ଟ͍ • luceneͷΫΤϦ͕࢖͑Δ • qury DSL͸ͪΐͬͱোน… Monday, November 26, 12
43. ෼ࢄݕࡧ Monday, November 26, 12
44. ෼ࢄݕࡧ • number_of_shards • number_of_replicas • replication • async/sync • write consistency(one, quorum, all) Monday, November 26, 12
45. multi-tenant • open/close index • write I/O throttling • merge policy control • shard allocation • number_of_replicas per index Monday, November 26, 12
46. plugin Monday, November 26, 12
47. plugin $ bin/plugin -install Aconex/elasticsearch-head Monday, November 26, 12
48. ύϑΥʔϚϯε • ࣄྫ͸ଟ͘ݟ͔ͭΔ • foursquare, soundcloud, bugsense ...etc • ΫΤϦΩϟογϡ͕ͳ͍ • nginx, varnishͳͲͰΩϟογϡ͢Δ Monday, November 26, 12
49. ·ͱΊ • ෼ࢄݕࡧΛ࢖͏ͳΒelasticsearch • ෼ࢄݕࡧΛ࢖Θͳͯ͘΋ར఺͕ଟ͍ • ࠓޙ࢖ΘΕΔػձ͕͋Δ͔΋ Monday, November 26, 12
50. see also... • http://www.elasticsearch.org/ • http://www.elasticsearch.org/guide/ • http://solr-vs-elasticsearch.com/ • github.com/elasticsearch • http://blog.sematext.com/ • #elasticsearch Monday, November 26, 12