Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
solrとelasticsearchの比較
Search
genta kaneyama
November 26, 2012
Programming
15
5.3k
solrとelasticsearchの比較
elasticsearchの紹介です!
atnd.org/events/33718
genta kaneyama
November 26, 2012
Tweet
Share
More Decks by genta kaneyama
See All by genta kaneyama
MOSHでの生成AI活用の取り組み
penguinco
0
100
search and community in cookpad 2019
penguinco
2
1.7k
行動ログでプロダクトを改善するには/exploit user behavior for product
penguinco
3
9k
Solr @ CROSS2015 C4
penguinco
1
1.3k
how to improve search
penguinco
8
2k
Other Decks in Programming
See All in Programming
Architectures with Lightweight Stores: New Rules and Options
manfredsteyer
PRO
0
100
CSC307 Lecture 10
javiergs
PRO
0
310
開発部に不満を持っていたCSがエンジニアにジョブチェンしてわかった「勝手に諦めない」ことの大切さ
sakuraikotone
28
16k
MIERUNE BBQにおけるユーザー中心設計()
mierune
PRO
1
110
12年前の『型システム入門』翻訳の思い出話
mame
11
1.2k
Introduction of Happy Eyeballs Version 2 (RFC8305) to the Socket library
coe401_
1
220
Advanced App Shrinking Techniques
cbeyls
2
150
Xcode 16のPreviewModifierと@Previewableを活用した効率的なプレビュー方法の考察
ojun9
2
160
Modern Angular: Renovation for Your Applications
manfredsteyer
PRO
0
140
Folding Cheat Sheet #7
philipschwarz
PRO
0
150
【Go言語】ジェネリクス
tomo1227
0
170
Namespace on read
tagomoris
2
370
Featured
See All Featured
Save Time (by Creating Custom Rails Generators)
garrettdimon
PRO
13
430
Rebuilding a faster, lazier Slack
samanthasiow
78
8.5k
The Invisible Side of Design
smashingmag
294
50k
Intergalactic Javascript Robots from Outer Space
tanoku
266
26k
It's Worth the Effort
3n
181
27k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
26
1.6k
Building Flexible Design Systems
yeseniaperezcruz
323
37k
For a Future-Friendly Web
brad_frost
173
9.2k
A Modern Web Designer's Workflow
chriscoyier
689
190k
Fontdeck: Realign not Redesign
paulrobertlloyd
79
5.1k
Understanding Cognitive Biases in Performance Measurement
bluesmoon
18
1.2k
Designing on Purpose - Digital PM Summit 2013
jponch
113
6.6k
Transcript
elasticsearchͱSolrͷൺֱ ݉ࢁ ݩଠ @penguinana_ Monday, November 26, 12
ࣗݾհ • ݉ࢁ ݩଠ @penguinana_ • ϨγϐݕࡧνʔϜ @ http://cookpad.com/ •
Solr4.0 Monday, November 26, 12
SolrͷόʔδϣϯΞοϓΛ ݕ౼͍ͯ͠Δͱ͖... Monday, November 26, 12
Elasticsearch ௐͨ΄͏͕͍͍ͷͰʁ Monday, November 26, 12
• Luceneϕʔε • HTTP API • ࢄݕࡧOK • ຊޠOK Monday,
November 26, 12
• Luceneϕʔε • HTTP API • ࢄݕࡧOK • ຊޠOK طࢹײ
Monday, November 26, 12
http://solr-vs-elasticsearch.com/ Monday, November 26, 12
ײ • ػೳ໘Ͱෆͳ͍ • API͕։ൃऀʹ͍͞͠ • ༰қʹशಘͰ͖Δ • େنࢄݕࡧҎ֎Ͱ༗༻ •
SolrΛͬͯͳ͚Εͬͪ͜Λຊ൪ʹ͍ͨ ͍ʂ Monday, November 26, 12
αϯϓϧΛͬͯ ͻͱ௨Γઆ໌͠·͢ Monday, November 26, 12
http://blog.livedoor.jp/techblog/archives/65836960.html Monday, November 26, 12
livedoorάϧϝ • Ϩετϥϯใ(21.4ສళ) • ళ໊ɺѻ͍ͬͯΔྉཧɺॅॴɺҢ ܦɺΞΫηεɺ࠷دΓฑߦ͖͔ Βͷڑɺetc... Monday, November 26,
12
livedoorάϧϝ • ϨϏϡʔใ(20.5ສϨϏϡʔ) • ૯߹ධՁʢ5ஈ֊ʣ • งғؾɺஈɺαʔϏεɺຯ • ϨϏϡʔίϝϯτ Monday,
November 26, 12
https://github.com/penguinco/ld_gourmet_search Monday, November 26, 12
ElasticsearchΛ͏ • 1݅ొͯ͠ɺ1݅ݕࡧ • ຊޠͷѻ͍Λఆٛ • εΩʔϚఆٛ • औΓࠐΈ •
ݕࡧ • είΞϦϯάͳͲͷௐ Monday, November 26, 12
PUT curl -XPUT http://localhost:9200/twitter/tweet/1 -d ' { "user": "kimchy", "post_date":
"2012-11-26T20:12:00", "message": "Trying out elasticsearch", "score": 5 } ' index type id Monday, November 26, 12
PUT curl -XPUT http://localhost:9200/twitter/user/kimchy -d ' { "name" : "Shay
Banon" } ' index type id Monday, November 26, 12
GET curl -XGET http://localhost:9200/twitter/tweet/1 { "user": "kimchy", "post_date": "2012-11-26T20:12:00", "message":
"Trying out elasticsearch", "score": 5 } } index type id Monday, November 26, 12
SEARCH curl -XGET http://localhost:9200/twitter/tweet/_search -d '{ "query" : { "term"
: { "user": "kimchy" } } }' index type id { "user": "kimchy", "post_date": "2012-11-26T20:12:00", "message": "Trying out elasticsearch", "score": 5 } Monday, November 26, 12
REST API • υΩϡϝϯτͷՃɾআ • ઃఆͷՃɾআ • શ෦HTTP APIͰͰ͖Δ •
εΩʔϚϑϦʔ Monday, November 26, 12
ຊޠ $ curl -XGET 'localhost:9200/_analyze?pretty' -d 'ਆઘ' { "tokens" :
[ { "token" : "ਆ", "start_offset" : 0, "end_offset" : 1, "type" : "<IDEOGRAPHIC>", "position" : 1 }, { "token" : "ઘ", "start_offset" : 1, "end_offset" : 2, "type" : "<IDEOGRAPHIC>", "position" : 2 } ] } Monday, November 26, 12
ຊޠ AnalyzerΛมߋ͢Δ͜ͱͰରԠ kuromoji͕͑·͢ʂ http://www.hirotakaster.com/archives/2012/11/ elasticsearch-kuromoji-plugin.php Monday, November 26, 12
kuromoji $ cd elasticsearch $ bin/plugin -install elasticsearch/elasticsearch-analysis-kuromoji/1.0.0 $ git
clone git://github.com/elasticsearch/elasticsearch-analysis- kuromoji.git $ cd elasticsearch-analysis-kuromoji/ $ mvn clean package $ cp target/elasticsearch-analysis-kuromoji-1.2.0-SNAPSHOT.jar ../plugins/ analysis-kuromoji/elasticsearch-analysis-kuromoji-1.0.0.jar # restart elasticsearch Monday, November 26, 12
add analyzer $ curl -XPUT 'localhost:9200/test/' -d ' { "index":{
"analysis":{ "tokenizer" : { "kuromoji" : { "type":"kuromoji_tokenizer", "mode":"search" } }, "analyzer" : { "kuromoji_analyzer" : { "type" : "custom", "tokenizer" : "kuromoji_tokenizer" } } } } } ‘ Monday, November 26, 12
kuromoji $ curl -XGET 'localhost:9200/test/_analyze? analyzer=kuromoji_analyzer&pretty' -d 'ਆઘ' { "tokens"
: [ { "token" : "ਆઘ", "start_offset" : 0, "end_offset" : 2, "type" : "word", "position" : 1 } ] } Monday, November 26, 12
_analyze $ curl -XGET 'localhost:9200/test/_analyze? analyzer=kuromoji_analyzer&pretty' -d 'ؔࠃࡍۭߓ' { "tokens"
: [ {"token" : "ؔ",}, {"token" : "ؔࠃࡍۭߓ",}, {"token" : "ࠃࡍ",}, {"token" : "ۭߓ",} ] } Monday, November 26, 12
kuromojiΛσϑΥϧτʹ • default͍ͬͯ͏໊લͰanalyzerΛએݴ Monday, November 26, 12
ಉٛޠ • Solrಉ༷ಉٛޠ͕ϑΝΠϧͰॻ͚Δ • +WordNetܗࣜ͑Δ Monday, November 26, 12
analyzer Monday, November 26, 12
ຊޠͷ৺͋Δఔย͍ͨʂ Monday, November 26, 12
εΩʔϚఆٛ • εΩʔϚϑϦʔʂ • JSONͷܕ͕࠾༻͞ΕΔ • ڧ੍తʹఆٛͰ͖Δ(mapping) Monday, November 26,
12
mappingྫ $ curl -XPUT 'http://localhost:9200/twitter/tweet/ _mapping' -d ' { "tweet"
: { "properties" : { "message" : {"type" : "string", "store" : "yes"} } } } ' Monday, November 26, 12
Solrͱͷࠩ • SolrͷDynamicFieldΑΓ؆୯ • type • 1ίΞʹෳछྨͷdocΛೖΕΔ͜ ͱΛఆͯ͋ͬͯ͠ศར Monday, November
26, 12
import(ruby) ratings = [] CSV.foreach("ratings.csv") do |row| ratings << {
:id => row[:id].to_i, :restaurant_id => row[:restaurant_id].to_i, :body => row[:body], :type => 'rating' } end Tire.index 'livedoor_gourmet' do import ratings end Monday, November 26, 12
ݕࡧ curl -X GET 'http://localhost:9200/livedoor_gourmet/ restaurant/_search?pretty' -d ' { "query":{
"query_string":{ "query":"ϥʔϝϯ" } }, "sort":[{"access_count":"desc"}], "filter":{ "term":{"closed":"0"} } } ' Monday, November 26, 12
Solrͱͷࠩ • DSL͕݁ߏҧ͏ • filter, facet, grouping, highlightαϙʔτ • είΞϦϯάεΫϦϓτݴޠͰఆٛ
Ͱ͖Δ Monday, November 26, 12
είΞϦϯά • PVॱͰฒͨΒ͏·͍ͬͨ͆͘ • ݱ࣮ͷ݁ߏ͜͏͍͏͜ͱଟ͍ Monday, November 26, 12
είΞϦϯά • ڵຯͷ͋Δํͥͻ • εΫϦϓτݴޠͰఆٛͰ͖Δ • google: elasticsearch guide scoring
Monday, November 26, 12
ײ • ػೳ໘Ͱෆͳ͍ • API͕։ൃऀʹ͍͞͠ • ༰қʹशಘͰ͖Δ • େنࢄݕࡧҎ֎Ͱ༗༻ Monday,
November 26, 12
API Monday, November 26, 12
config curl͚ͩͰͰ͖Δ →ΞϓϦέʔγϣϯʹఆٛΛஔ͚Δ Monday, November 26, 12
ίΞՃ curl͚ͩͰͰ͖Δ →։ൃऀͻͱΓͰ݁Ͱ͖Δ Monday, November 26, 12
༰қʹशಘͰ͖Δ • ΄ͱΜͲͷૢ࡞curlͰ݁ • Solrͱڞ௨ͷࣝଟ͍ • luceneͷΫΤϦ͕͑Δ • qury DSLͪΐͬͱোน…
Monday, November 26, 12
ࢄݕࡧ Monday, November 26, 12
ࢄݕࡧ • number_of_shards • number_of_replicas • replication • async/sync •
write consistency(one, quorum, all) Monday, November 26, 12
multi-tenant • open/close index • write I/O throttling • merge
policy control • shard allocation • number_of_replicas per index Monday, November 26, 12
plugin Monday, November 26, 12
plugin $ bin/plugin -install Aconex/elasticsearch-head Monday, November 26, 12
ύϑΥʔϚϯε • ࣄྫଟ͘ݟ͔ͭΔ • foursquare, soundcloud, bugsense ...etc • ΫΤϦΩϟογϡ͕ͳ͍
• nginx, varnishͳͲͰΩϟογϡ͢Δ Monday, November 26, 12
·ͱΊ • ࢄݕࡧΛ͏ͳΒelasticsearch • ࢄݕࡧΛΘͳͯ͘ར͕ଟ͍ • ࠓޙΘΕΔػձ͕͋Δ͔ Monday, November 26,
12
see also... • http://www.elasticsearch.org/ • http://www.elasticsearch.org/guide/ • http://solr-vs-elasticsearch.com/ • github.com/elasticsearch
• http://blog.sematext.com/ • #elasticsearch Monday, November 26, 12