Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
solrとelasticsearchの比較
Search
genta kaneyama
November 26, 2012
Programming
15
5.4k
solrとelasticsearchの比較
elasticsearchの紹介です!
atnd.org/events/33718
genta kaneyama
November 26, 2012
Tweet
Share
More Decks by genta kaneyama
See All by genta kaneyama
MOSHでの生成AI活用の取り組み
penguinco
0
120
search and community in cookpad 2019
penguinco
2
1.8k
行動ログでプロダクトを改善するには/exploit user behavior for product
penguinco
4
9k
Solr @ CROSS2015 C4
penguinco
1
1.3k
how to improve search
penguinco
8
2k
Other Decks in Programming
See All in Programming
GitHub Copilot Workspace で我々のアプリ開発がどう変わるのか?
shuyakinjo
0
850
Flutterアプリを生成AIで生成する勘所
rizumita
0
250
為醫療加裝Python的引擎
cclai999
0
270
Cancel Next.js Page Navigation: Full Throttle
ypresto
1
120
(Deep|Web) Link support with expo-router
mrtry
0
110
Vue :: Better Testing 2024
up1
1
370
フロントエンドの標準仕様をどう追っているか / How I follow the frontend standards specs
petamoriken
2
170
CSC509 Lecture 03
javiergs
PRO
0
130
2024-10-02 dev2next - Application Observability like you've never heard before
jonatan_ivanov
0
160
Beyond the RuboCop Defaults
koic
2
480
M5Stackボードの選び方
tanakamasayuki
0
200
コードレビューと私の過去と未来
jxmtst
0
180
Featured
See All Featured
Design and Strategy: How to Deal with People Who Don’t "Get" Design
morganepeng
125
18k
5 minutes of I Can Smell Your CMS
philhawksworth
202
19k
The Success of Rails: Ensuring Growth for the Next 100 Years
eileencodes
42
6.5k
The Brand Is Dead. Long Live the Brand.
mthomps
53
38k
GraphQLとの向き合い方2022年版
quramy
43
13k
Building Better People: How to give real-time feedback that sticks.
wjessup
360
19k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
48k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
37
1.7k
Stop Working from a Prison Cell
hatefulcrawdad
267
20k
How To Stay Up To Date on Web Technology
chriscoyier
786
250k
Gamification - CAS2011
davidbonilla
80
5k
Visualization
eitanlees
143
15k
Transcript
elasticsearchͱSolrͷൺֱ ݉ࢁ ݩଠ @penguinana_ Monday, November 26, 12
ࣗݾհ • ݉ࢁ ݩଠ @penguinana_ • ϨγϐݕࡧνʔϜ @ http://cookpad.com/ •
Solr4.0 Monday, November 26, 12
SolrͷόʔδϣϯΞοϓΛ ݕ౼͍ͯ͠Δͱ͖... Monday, November 26, 12
Elasticsearch ௐͨ΄͏͕͍͍ͷͰʁ Monday, November 26, 12
• Luceneϕʔε • HTTP API • ࢄݕࡧOK • ຊޠOK Monday,
November 26, 12
• Luceneϕʔε • HTTP API • ࢄݕࡧOK • ຊޠOK طࢹײ
Monday, November 26, 12
http://solr-vs-elasticsearch.com/ Monday, November 26, 12
ײ • ػೳ໘Ͱෆͳ͍ • API͕։ൃऀʹ͍͞͠ • ༰қʹशಘͰ͖Δ • େنࢄݕࡧҎ֎Ͱ༗༻ •
SolrΛͬͯͳ͚Εͬͪ͜Λຊ൪ʹ͍ͨ ͍ʂ Monday, November 26, 12
αϯϓϧΛͬͯ ͻͱ௨Γઆ໌͠·͢ Monday, November 26, 12
http://blog.livedoor.jp/techblog/archives/65836960.html Monday, November 26, 12
livedoorάϧϝ • Ϩετϥϯใ(21.4ສళ) • ళ໊ɺѻ͍ͬͯΔྉཧɺॅॴɺҢ ܦɺΞΫηεɺ࠷دΓฑߦ͖͔ Βͷڑɺetc... Monday, November 26,
12
livedoorάϧϝ • ϨϏϡʔใ(20.5ສϨϏϡʔ) • ૯߹ධՁʢ5ஈ֊ʣ • งғؾɺஈɺαʔϏεɺຯ • ϨϏϡʔίϝϯτ Monday,
November 26, 12
https://github.com/penguinco/ld_gourmet_search Monday, November 26, 12
ElasticsearchΛ͏ • 1݅ొͯ͠ɺ1݅ݕࡧ • ຊޠͷѻ͍Λఆٛ • εΩʔϚఆٛ • औΓࠐΈ •
ݕࡧ • είΞϦϯάͳͲͷௐ Monday, November 26, 12
PUT curl -XPUT http://localhost:9200/twitter/tweet/1 -d ' { "user": "kimchy", "post_date":
"2012-11-26T20:12:00", "message": "Trying out elasticsearch", "score": 5 } ' index type id Monday, November 26, 12
PUT curl -XPUT http://localhost:9200/twitter/user/kimchy -d ' { "name" : "Shay
Banon" } ' index type id Monday, November 26, 12
GET curl -XGET http://localhost:9200/twitter/tweet/1 { "user": "kimchy", "post_date": "2012-11-26T20:12:00", "message":
"Trying out elasticsearch", "score": 5 } } index type id Monday, November 26, 12
SEARCH curl -XGET http://localhost:9200/twitter/tweet/_search -d '{ "query" : { "term"
: { "user": "kimchy" } } }' index type id { "user": "kimchy", "post_date": "2012-11-26T20:12:00", "message": "Trying out elasticsearch", "score": 5 } Monday, November 26, 12
REST API • υΩϡϝϯτͷՃɾআ • ઃఆͷՃɾআ • શ෦HTTP APIͰͰ͖Δ •
εΩʔϚϑϦʔ Monday, November 26, 12
ຊޠ $ curl -XGET 'localhost:9200/_analyze?pretty' -d 'ਆઘ' { "tokens" :
[ { "token" : "ਆ", "start_offset" : 0, "end_offset" : 1, "type" : "<IDEOGRAPHIC>", "position" : 1 }, { "token" : "ઘ", "start_offset" : 1, "end_offset" : 2, "type" : "<IDEOGRAPHIC>", "position" : 2 } ] } Monday, November 26, 12
ຊޠ AnalyzerΛมߋ͢Δ͜ͱͰରԠ kuromoji͕͑·͢ʂ http://www.hirotakaster.com/archives/2012/11/ elasticsearch-kuromoji-plugin.php Monday, November 26, 12
kuromoji $ cd elasticsearch $ bin/plugin -install elasticsearch/elasticsearch-analysis-kuromoji/1.0.0 $ git
clone git://github.com/elasticsearch/elasticsearch-analysis- kuromoji.git $ cd elasticsearch-analysis-kuromoji/ $ mvn clean package $ cp target/elasticsearch-analysis-kuromoji-1.2.0-SNAPSHOT.jar ../plugins/ analysis-kuromoji/elasticsearch-analysis-kuromoji-1.0.0.jar # restart elasticsearch Monday, November 26, 12
add analyzer $ curl -XPUT 'localhost:9200/test/' -d ' { "index":{
"analysis":{ "tokenizer" : { "kuromoji" : { "type":"kuromoji_tokenizer", "mode":"search" } }, "analyzer" : { "kuromoji_analyzer" : { "type" : "custom", "tokenizer" : "kuromoji_tokenizer" } } } } } ‘ Monday, November 26, 12
kuromoji $ curl -XGET 'localhost:9200/test/_analyze? analyzer=kuromoji_analyzer&pretty' -d 'ਆઘ' { "tokens"
: [ { "token" : "ਆઘ", "start_offset" : 0, "end_offset" : 2, "type" : "word", "position" : 1 } ] } Monday, November 26, 12
_analyze $ curl -XGET 'localhost:9200/test/_analyze? analyzer=kuromoji_analyzer&pretty' -d 'ؔࠃࡍۭߓ' { "tokens"
: [ {"token" : "ؔ",}, {"token" : "ؔࠃࡍۭߓ",}, {"token" : "ࠃࡍ",}, {"token" : "ۭߓ",} ] } Monday, November 26, 12
kuromojiΛσϑΥϧτʹ • default͍ͬͯ͏໊લͰanalyzerΛએݴ Monday, November 26, 12
ಉٛޠ • Solrಉ༷ಉٛޠ͕ϑΝΠϧͰॻ͚Δ • +WordNetܗࣜ͑Δ Monday, November 26, 12
analyzer Monday, November 26, 12
ຊޠͷ৺͋Δఔย͍ͨʂ Monday, November 26, 12
εΩʔϚఆٛ • εΩʔϚϑϦʔʂ • JSONͷܕ͕࠾༻͞ΕΔ • ڧ੍తʹఆٛͰ͖Δ(mapping) Monday, November 26,
12
mappingྫ $ curl -XPUT 'http://localhost:9200/twitter/tweet/ _mapping' -d ' { "tweet"
: { "properties" : { "message" : {"type" : "string", "store" : "yes"} } } } ' Monday, November 26, 12
Solrͱͷࠩ • SolrͷDynamicFieldΑΓ؆୯ • type • 1ίΞʹෳछྨͷdocΛೖΕΔ͜ ͱΛఆͯ͋ͬͯ͠ศར Monday, November
26, 12
import(ruby) ratings = [] CSV.foreach("ratings.csv") do |row| ratings << {
:id => row[:id].to_i, :restaurant_id => row[:restaurant_id].to_i, :body => row[:body], :type => 'rating' } end Tire.index 'livedoor_gourmet' do import ratings end Monday, November 26, 12
ݕࡧ curl -X GET 'http://localhost:9200/livedoor_gourmet/ restaurant/_search?pretty' -d ' { "query":{
"query_string":{ "query":"ϥʔϝϯ" } }, "sort":[{"access_count":"desc"}], "filter":{ "term":{"closed":"0"} } } ' Monday, November 26, 12
Solrͱͷࠩ • DSL͕݁ߏҧ͏ • filter, facet, grouping, highlightαϙʔτ • είΞϦϯάεΫϦϓτݴޠͰఆٛ
Ͱ͖Δ Monday, November 26, 12
είΞϦϯά • PVॱͰฒͨΒ͏·͍ͬͨ͆͘ • ݱ࣮ͷ݁ߏ͜͏͍͏͜ͱଟ͍ Monday, November 26, 12
είΞϦϯά • ڵຯͷ͋Δํͥͻ • εΫϦϓτݴޠͰఆٛͰ͖Δ • google: elasticsearch guide scoring
Monday, November 26, 12
ײ • ػೳ໘Ͱෆͳ͍ • API͕։ൃऀʹ͍͞͠ • ༰қʹशಘͰ͖Δ • େنࢄݕࡧҎ֎Ͱ༗༻ Monday,
November 26, 12
API Monday, November 26, 12
config curl͚ͩͰͰ͖Δ →ΞϓϦέʔγϣϯʹఆٛΛஔ͚Δ Monday, November 26, 12
ίΞՃ curl͚ͩͰͰ͖Δ →։ൃऀͻͱΓͰ݁Ͱ͖Δ Monday, November 26, 12
༰қʹशಘͰ͖Δ • ΄ͱΜͲͷૢ࡞curlͰ݁ • Solrͱڞ௨ͷࣝଟ͍ • luceneͷΫΤϦ͕͑Δ • qury DSLͪΐͬͱোน…
Monday, November 26, 12
ࢄݕࡧ Monday, November 26, 12
ࢄݕࡧ • number_of_shards • number_of_replicas • replication • async/sync •
write consistency(one, quorum, all) Monday, November 26, 12
multi-tenant • open/close index • write I/O throttling • merge
policy control • shard allocation • number_of_replicas per index Monday, November 26, 12
plugin Monday, November 26, 12
plugin $ bin/plugin -install Aconex/elasticsearch-head Monday, November 26, 12
ύϑΥʔϚϯε • ࣄྫଟ͘ݟ͔ͭΔ • foursquare, soundcloud, bugsense ...etc • ΫΤϦΩϟογϡ͕ͳ͍
• nginx, varnishͳͲͰΩϟογϡ͢Δ Monday, November 26, 12
·ͱΊ • ࢄݕࡧΛ͏ͳΒelasticsearch • ࢄݕࡧΛΘͳͯ͘ར͕ଟ͍ • ࠓޙΘΕΔػձ͕͋Δ͔ Monday, November 26,
12
see also... • http://www.elasticsearch.org/ • http://www.elasticsearch.org/guide/ • http://solr-vs-elasticsearch.com/ • github.com/elasticsearch
• http://blog.sematext.com/ • #elasticsearch Monday, November 26, 12