Solr入門？

Transcript

1. Solrೖ໳? 2013/06/18 @johtani 13೥6݄19೔ਫ༵೔
2. ࣗݾ঺հ ࢯ໊ɿେ୩ɹ७ Twitterɿ@johtani lucene-gosenͷίϛολʔ چϒϩάɿhttp://johtani.jugem.jp ৽ϒϩάɿhttp://blog.johtani.info ɹ※࠷ۙOctopressͰॻ͍ͯ·͢ 13೥6݄19೔ਫ༵೔
3. ݕࡧΤϯδϯͬͯʁ 13೥6݄19೔ਫ༵೔
4. શจݕࡧͬͯʁ શจݕࡧʢFull text searchʣͱ͸ɺίϯϐϡʔλ ʹ͓͍ͯɺෳ਺ͷจॻʢϑΝΠϧʣ͔ΒಛఆͷจࣈྻΛ ݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯ΍ʮ୯ҰϑΝΠϧ಺ ͷจࣈྻݕࡧʯͱҟͳΓɺʮෳ਺จॻʹ·͕ͨͬͯɺจ ॻʹؚ·ΕΔશจΛର৅ͱͨ͠ݕࡧʯͱ͍͏ҙຯͰ࢖༻ ͞ΕΔɻ ʢWikipediaΑΓʣ 13೥6݄19೔ਫ༵೔
5. ༻ޠ จॻʢυΩϡϝϯτʣ ɹݕࡧΤϯδϯʹอଘ͞ΕͨσʔλɻRDBͰͷϨίʔυʹ૬౰ ΫΤϦ ɹݕࡧ৚݅ɺݕࡧࣜ εΩʔϚ ɹRDBͷςʔϒϧʹ૬౰ ϑΟʔϧυ ɹRDBͷΧϥϜʹ૬౰ λʔϜʢTermʣ/τʔΫϯʢTokenʣ ɹΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ 13೥6݄19೔ਫ༵೔
6. ݕࡧΤϯδϯͷಈ࡞ υΩϡϝϯτొ࿥ͷྲྀΕ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ υΩϡϝϯτͷొ࿥ 13೥6݄19೔ਫ༵೔
7. ݕࡧΤϯδϯͷಈ࡞ υΩϡϝϯτొ࿥ͷྲྀΕ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͸ ͸ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ υΩϡϝϯτͷొ࿥ ୯ޠʹ෼ׂ 13೥6݄19೔ਫ༵೔
8. ݕࡧΤϯδϯͷಈ࡞ υΩϡϝϯτొ࿥ͷྲྀΕ 1 2 ΧπΦ͸αβΤͷఋ αβΤ͸ϫΧϝͷ࢞ 1 2 ΧπΦ αβΤ ͸ ͸ ͷ ͷ αβΤ ϫΧϝ ఋ ࢞ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 υΩϡϝϯτͷొ࿥ ୯ޠʹ෼ׂ ୯ޠ͔Βidͷ഑ྻ͕Ҿ ͚ΔΑ͏ʹ ʹసஔΠϯσοΫε 13೥6݄19೔ਫ༵೔
9. ݕࡧΤϯδϯͷಈ࡞ ݕࡧͷྲྀΕ ΧπΦɹαβΤ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧจࣈೖྗ 13೥6݄19೔ਫ༵೔
10. ݕࡧΤϯδϯͷಈ࡞ ݕࡧͷྲྀΕ ΧπΦɹαβΤ ΧπΦ αβΤ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧจࣈೖྗ ݕࡧจࣈͷύʔε ʴݕࡧΫΤϦԽ AND 13೥6݄19೔ਫ༵೔
11. ݕࡧΤϯδϯͷಈ࡞ ݕࡧͷྲྀΕ ΧπΦɹαβΤ ΧπΦ αβΤ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧจࣈೖྗ ݕࡧจࣈͷύʔε ʴݕࡧΫΤϦԽ సஔΠϯσοΫεΛ ݕࡧ AND 13೥6݄19೔ਫ༵೔
12. ݕࡧΤϯδϯͷಈ࡞ ݕࡧͷྲྀΕ ΧπΦɹαβΤ ΧπΦ αβΤ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧจࣈೖྗ ݕࡧจࣈͷύʔε ʴݕࡧΫΤϦԽ సஔΠϯσοΫεΛ ݕࡧ AND 13೥6݄19೔ਫ༵೔
13. ݕࡧΤϯδϯͷಈ࡞ ݕࡧͷྲྀΕ ΧπΦɹαβΤ ΧπΦ αβΤ ΧπΦ αβΤ 1 1 2 ͸ ͷ ࢞ ϫΧϝ 2 1 2 1 2 1 ఋ 2 ݕࡧจࣈೖྗ ݕࡧจࣈͷύʔε ʴݕࡧΫΤϦԽ సஔΠϯσοΫεΛ ݕࡧ ˠANDݕࡧͳͷͰidͷ ഑ྻͷANDΛͱΔ AND 13೥6݄19೔ਫ༵೔
14. N-gramͱܗଶૉղੳ సஔΠϯσοΫεͷΩʔʢ୯ޠʣͷ࡞Γํ ೔ຊޠ͸୯ޠͷ੾Ε໨͕Θ͔Βͳ͍ͷͰసஔΠϯσοΫ εͷΩʔ͸ओʹͭ͗ͷ2ͭͷख๏Ͱ࡞Δ N-gram Nʢ1Ҏ্ͷ਺ࣈʣจࣈͣͭจষΛ۠੾Δ ܗଶૉղੳ ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠੾Δ 13೥6݄19೔ਫ༵೔
15. N-gram N͕2ͷ࣌ͷྫɿ ϝϦοτɿ ݕࡧ࿙Ε͕ͳ͍ σϝϦοτɿ ਫ਼౓͕ѱ͍"ʮΧπʯͰ΋ώοτͪ͠Ό͏ ΠϯσοΫε͕ංେԽ ޠװʢ୯ޠͷجຊܗͱ͔ʣͰݕࡧෆՄ ΧπΦ͸αβΤͷఋ Χπ πΦ Φ͸ ͸α αβ βΤ Τͷ ͷఋ 13೥6݄19೔ਫ༵೔
16. ܗଶૉղੳ ܗଶૉղੳͷྫɿ ϝϦοτɿ ਫ਼౓͕Α͍ɿʮΧπʯͰݕࡧ"ʮΧπΦʯώοτ͠ͳ͍ ޠװʹΑΔݕࡧ͕ՄೳɿʮҿΈʯ"ʮҿΉʯ͕ݕࡧՄ σϝϦοτɿ ৽ޠʢະ஌ޠʣʹऑ͍"ࣙॻϕʔεͷ৔߹ʹࣙॻʹͳ͍ ୯ޠ͕Θ͔Βͳ͍ɻࣙॻߋ৽ޙ͸ΠϯσοΫε࠶࡞੒ ΧπΦ͸αβΤͷఋ ΧπΦ ͸ ͷ αβΤ ఋ 13೥6݄19೔ਫ༵೔
17. Lucene/Solrͬͯʁ 13೥6݄19೔ਫ༵೔
18. LuceneͱSolrͷؔ܎ LuceneɿશจݕࡧϥΠϒϥϦ ɹʢసஔΠϯσοΫεϥΠϒϥϦʣ http://lucene.apahce.org/core/ SolrɿLuceneΛར༻ͨ͠શจݕࡧαʔό ɹྨࣅͷ΋ͷͱͯ͠ElasticSearchͳͲ http://lucene.apache.org/solr/ 13೥6݄19೔ਫ༵೔
19. SolrͷΞʔΩςΫνϟ &RQWHQW )HHGHU HJ6ROU- &RQWHQW &RQQHFWRUV 0&) &RQWHQW 6RXUFHV ,QGH[ /XFHQH 6ROU:HE$SS VFKHPD VROU FRQILJ 8SGDWH&KDLQ $QDO\VLV 6HDUFK5HTXHVW +DQGOLQJ 4XHU\ 4XHU\ 3DUVHU 8SGDWH5HTXHVW +DQGOLQJ -DYD$SS6HUYHU 6HDUFK&RPS 6ROU&RUH $GPLQ 6WDWV =RR .HHSHU 5HVS :ULWHU $SDFKH6ROU &RQFHSWXDO $UFKLWHFWXUH $GPLQ 8VHU 8SGDWH +DQGOHU ,OOXVWUDWLRQE\&RPLQYHQW$6 <RXPD\IUHHO\XVHLWDV ORQJDV\RXDWWULEXWHZLWK OLQNWRZZZFRPLQYHQWFRP DFFRUGLQJWR&&E\OLFHQVH ZZZFUHDWLYHFRPPRQVRUJ 2ULJLQDOGLDJUDPE\ ZZZFRPLQYHQWFRP &RUH $GPLQ &RO OHFWLRQ $GPLQ ςΩετ Original diagram byɹ http://www.cominvent.com 13೥6݄19೔ਫ༵೔
20. Solrͷߏ੒ύλʔϯ EmbeddedSolrɿJVM಺ͰSolrΛىಈ WebΞϓϦʢ୯ҰʣɿJ2EEαʔό্ͰSolrΛىಈ Solr͸exampleͱͯ͠JettyΛ಺แ ෼ࢄݕࡧɿෳ਺ͷSolrΛ̍ͭͷΠϯσοΫεͱͯ͠ݕ ࡧɻʢݕࡧػೳͷΈɻΠϯσοΫεʣ http://wiki.apache.org/solr/DistributedSearch SolrCloudɿෳ਺ͷJ2EEαʔόͰΫϥελΛߏ੒ http://wiki.apache.org/solr/SolrCloud 13೥6݄19೔ਫ༵೔
21. ߏ੒֓ཁ Solr Web App SolrαʔόɻJ2EEαʔό্Ͱಈ࡞ SolrCore 1ͭͷΠϯσοΫεΛ؅ཧ schema.xmlͱsolrconfig.xmlͰίΞͷڍ ಈɺεΩʔϚΛఆٛ 1ͭͷSolrServer্Ͱɺෳ਺ͷCoreΛىಈͰ͖ ΔɻʢϚϧνίΞʣ ͦΕͧΕͷίΞ͝ͱʹεΩʔϚͷઃఆ͕Մೳ 13೥6݄19೔ਫ༵೔
22. ߏ੒֓ཁ Update Reuqest Handling ߋ৽ॲཧΛUpdateChainͱ͍͏ܗͰఆٛՄೳ SolrଆͰొ࿥લॲཧΛॻ͚Δ ྫɿݴޠ൑ผͯ͠ɺϑΟʔϧυΛܾఆͳͲ http://wiki.apache.org/solr/UpdateRequestProcessor Search Request Handling ݕࡧΫΤϦΛड͚औΓ݁ՌΛग़ྗ͢Δॲཧ SearchComponentͷ૊Έ߹ΘͤʹΑΓఆٛՄೳ http://wiki.apache.org/solr/SolrRequestHandler 13೥6݄19೔ਫ༵೔
23. ߏ੒֓ཁ Analysis υΩϡϝϯτɺΫΤϦΛTokenʹ෼ղ͢Δॲཧ ࠷ऴతͳToken͕సஔΠϯσοΫεͷΩʔ 13೥6݄19೔ਫ༵೔
24. AnalysisͷྲྀΕ http://www.lucenerevolution.org/sites/ default/files/Language%20support%20and %20linguistics%20in%20LuceneSolr %20%26%20its%20eco-system.pdf 13೥6݄19೔ਫ༵೔
25. SolrͷΞʔΩςΫνϟ &RQWHQW )HHGHU HJ6ROU- &RQWHQW &RQQHFWRUV 0&) &RQWHQW 6RXUFHV ,QGH[ /XFHQH 6ROU:HE$SS VFKHPD VROU FRQILJ 8SGDWH&KDLQ $QDO\VLV 6HDUFK5HTXHVW +DQGOLQJ 4XHU\ 4XHU\ 3DUVHU 8SGDWH5HTXHVW +DQGOLQJ -DYD$SS6HUYHU 6HDUFK&RPS 6ROU&RUH $GPLQ 6WDWV =RR .HHSHU 5HVS :ULWHU $SDFKH6ROU &RQFHSWXDO $UFKLWHFWXUH $GPLQ 8VHU 8SGDWH +DQGOHU ,OOXVWUDWLRQE\&RPLQYHQW$6 <RXPD\IUHHO\XVHLWDV ORQJDV\RXDWWULEXWHZLWK OLQNWRZZZFRPLQYHQWFRP DFFRUGLQJWR&&E\OLFHQVH ZZZFUHDWLYHFRPPRQVRUJ 2ULJLQDOGLDJUDPE\ ZZZFRPLQYHQWFRP &RUH $GPLQ &RO OHFWLRQ $GPLQ ͜͜ Original diagram byɹ http://www.cominvent.com 13೥6݄19೔ਫ༵೔
26. CharFilter TokenizerΑΓલͷจࣈྻʹରͯ͠ॲཧ Tokenize͞ΕΔલͷਖ਼نԽॲཧʹར༻ ྫɿʮűƄŖſʯ"ʮϏʔϧʯ ྫɿʮ̍ʯ"ʮ1ʯ http://wiki.apache.org/solr/ AnalyzersTokenizersTokenFilters#Char_Filters 13೥6݄19೔ਫ༵೔
27. Tokenizer/TokenFilter ͍ͬͺ͍͋Γ·͢ɻಠ࣮ࣗ૷΋Մೳ http://wiki.apache.org/solr/ AnalyzersTokenizersTokenFilters 13೥6݄19೔ਫ༵೔
28. ઃఆϑΝΠϧ ओͳઃఆϑΝΠϧ solrconfig.xmlɿαʔόʹؔ͢ΔઃఆΛهࡌ ΩϟογϡઃఆɺόοϑΝαΠζͳͲ schema.xmlɿΠϯσοΫεͷߏ੒ʹؔ͢ΔఆٛΛ هࡌ ϑΟʔϧυఆٛɺϑΟʔϧυλΠϓఆٛͳͲ ͜ΕΒͷઃఆΛมߋͨ͠৔߹͸SoreCoreͷ࠶ىಈ͕ඞ ཁʢSolrαʔόͰ͸ͳ͍ʣ 13೥6݄19೔ਫ༵೔
29. schema.xml ΠϯσοΫεͷߏ੒ΛXMLʹͯఆٛ ϑΟʔϧυఆٛʢ<fields>λά಺ʣ ݕࡧʹར༻͢ΔϑΟʔϧυ໊ͷઃఆ ϑΟʔϧυܕʢ<types>λά಺ʣ ϑΟʔϧυܕʢstringɺlongͳͲʣΛఆٛ ίϐʔϑΟʔϧυʢ<copyField>λάʣ Solr಺෦ͰϑΟʔϧυͷ஋Λίϐʔ͢Δఆٛ 13೥6݄19೔ਫ༵೔
30. FieldType ϑΟʔϧυܕ͸େ͖͘2छྨ ඇςΩετܥ TokenizeॲཧΛߦΘͳ͍ ςΩετܥ TokenizeॲཧΛߦ͏ 13೥6݄19೔ਫ༵೔
31. FieldTypeɿඇςΩετܥ ྫɿจࣈྻܕ <fieldType name="string" class="solr.StrField"/> nameɿϑΟʔϧυܕ໊ɻFieldఆٛ࣌ʹར༻ classɿSolr͕ఏڙ͍ͯ͠ΔΫϥε ※ಠࣗʹ࣮૷ɺఆٛ͢Δ͜ͱ͸΄΅ͳ͍ 13೥6݄19೔ਫ༵೔
32. FieldType:ςΩετܥ ྫɿtext_jaϑΟʔϧυܕ <fieldType name="text_ja" class="solr.TextField" positionIncrementGap="100" autoGeneratePhraseQueries="false"> <analyzer> <tokenizer class="solr.JapaneseTokenizerFactory" mode="search"/> <filter class="solr.JapaneseBaseFormFilterFactory"/> <filter class="solr.JapanesePartOfSpeechStopFilterFactory" tags="lang/stoptags_ja.txt" /> ... </analyzer> </fieldType> nameɺclass͸ඇςΩετܥಉ༷ AnalysisॲཧΛఆٛՄೳ 13೥6݄19೔ਫ༵೔
33. Field લड़ͨ͠FieldܕΛར༻ͯ͠ɺ࣮ࡍʹݕࡧʹར༻͢Δϑ ΟʔϧυΛఆٛ ྫɿ <field name="features" type="text_general" indexed="true" stored="true" multiValued="true"/> nameɿϑΟʔϧυ໊ typeɿϑΟʔϧυܕʢFieldTypeͰఆٛ͞Ε͍ͯΔ΋ͷʣ 13೥6݄19೔ਫ༵೔
34. Fieldఆٛͷઃఆ߲໨ ଐੑ໊ આ໌ σϑΥϧτ indexed ݕࡧର৅ϑΟʔϧυ͔Ͳ͏͔ true stored ϑΟʔϧυ஋Λอଘ͢Δ͔Ͳ͏͔ true required ඞਢϑΟʔϧυ͔Ͳ͏͔ false multiValued ϑΟʔϧυ͕഑ྻܕʢෳ਺஋ʣ͔Ͳ͏͔ false omitNorms norm஋Λอଘ͢Δ false termVectors termPositions termOffsets Termʹؔ͢Δ৘ใΛอଘ͢Δ false 13೥6݄19೔ਫ༵೔
35. ϑΟʔϧυΦϓγϣϯͱϢʔεέʔε http://wiki.apache.org/solr/FieldOptionsByUseCase 13೥6݄19೔ਫ༵೔
36. μΠφϛοΫϑΟʔϧυ ϑΟʔϧυ໊Λొ࿥࣌ʹࢦఆͰ͖Δ࢓૊Έ schema.xmlΛมߋ͠ͳ͍Ͱ΋ϑΟʔϧυΛ௥ՃՄೳ ϑΟʔϧυ໊͔ΒϑΟʔϧυܕΛղܾ ྫɿʮ_dtʯͰऴΘΔ໊લͷ࣌͸೔෇ܕͱͯ͢͠Δ <dynamicField name="*_dt" type="date" indexed="true" stored="true"/> 13೥6݄19೔ਫ༵೔
37. Indexing 13೥6݄19೔ਫ༵೔
38. ొ࿥ɺߋ৽ํ๏ REST API HTTPܦ༝Ͱొ࿥ɺߋ৽Մೳ XMLɺJSONɺCSVܗࣜͰొ࿥Մೳ ίϚϯυπʔϧʢpost.jarɺpost.shʣ XMLܗࣜͷϑΝΠϧΛొ࿥ɺߋ৽Մೳ ಺෦Ͱ͸HTTPܦ༝ͰSolrʹΞΫηε 13೥6݄19೔ਫ༵೔
39. ࣮ࡍͷొ࿥ͷྲྀΕ add Solrʹσʔλ͕ૹΒΕɺసஔΠϯσοΫε࡞੒ σʔλͷमਖ਼΋addʢ಺෦Ͱdelete-insertʣ commitʢhard commitʣ addͨ͠σʔλΛݕࡧՄೳʹ ҰԠrollback΋͋Δ ※ηογϣϯͷ֓೦͕ͳ͍ͨΊɺ୭͕add͔ͨ͠͸൑ผ ͍ͯ͠ͳ͍ɻͦͷͨΊෳ਺ͷΫϥΠΞϯτ͔Βaddͯ͠ ͩΕ͔͕commitͨ͠Βɺ͢΂͕ͯݕࡧՄೳʹͳΔ 13೥6݄19೔ਫ༵೔
40. addσʔλαϯϓϧʢXMLʣ <add> <doc> <field name="authors">Patrick Eagar</field> <field name="subject">Sports</field> <field name="dd">796.35</field> <field name="numpages">128</field> <field name="desc"></field> <field name="price">12.40</field> <field name="title" boost="2.0">Summer of the all-rounder: Test and championship cricket in England 1982</field> <field name="isbn">0002166313</field> <field name="yearpub">1982</field> <field name="publisher">Collins</field> </doc> <doc boost="2.5"> ... </doc> </add> 13೥6݄19೔ਫ༵೔
41. soft commit ϝϞϦ্ͷΠϯσοΫε΁ͷίϛοτ ଈ࣌ݕࡧର৅ͱ͢Δ͜ͱ͕Ͱ͖ΔʢҰ෦੍ݶ͋Γʣ ϑΝΠϧ΁ͷύʔγετ͸࣮ࢪ͠ͳ͍ http://wiki.apache.org/solr/UpdateXmlMessages#A. 22commit.22_and_.22optimize.22 13೥6݄19೔ਫ༵೔
42. auto commit SolrαʔόଆͰఆظతʹίϛοτ͢Δػೳ solrconfig.xmlʹͯλΠϛϯάͷࢦఆ͕Մೳ maxDocsɿҰఆ਺υΩϡϝϯτ͕add͞ΕͨΒίϛ οτ࣮ߦ maxTimesɿ௚લͷadd͔ΒҰఆ͕࣌ؒܦաͨ͠Β ίϛοτ࣮ߦ soft/hardͷͦΕͧΕͰఆٛՄೳ http://wiki.apache.org/solr/SolrConfigXml? #Update_Handler_Section 13೥6݄19೔ਫ༵೔
43. ࡟আ IDࢦఆɺΫΤϦࢦఆͰͷ࡟আ͕Մೳ <delete> <id>0002166313</id> <id>0031745983</id> <query>subject:sport</query> <query>publisher:penguin</query> </delete> 13೥6݄19೔ਫ༵೔
44. segment LuceneͷΠϯσοΫε͸ߋ৽ࠩ෼ΛSegmentͱͯ͠௥ Ճʢ௥هܕσʔλϕʔεΈ͍ͨͳײ͡ʣ ௥Ճɺߋ৽σʔλ͸SegmentʹΠϯσοΫεԽ ࡟আʹؔͯ͠͸ɺ࡟আϑϥάʹͯ؅ཧ ఆظతʢαΠζىҼͳͲʣʹMergeॲཧ͕࣮ߦ͞Ε Segment͕݁߹͞ΕΔ ໌ࣔతͳMergeʹoptimizeʢcommit࣌ʹࢦఆՄೳʣ MergeॲཧͷϙϦγʔ͸solrconfig.xmlʹࢦఆ 13೥6݄19೔ਫ༵೔
45. Search 13೥6݄19೔ਫ༵೔
46. ݕࡧػೳ঺հ ϦΫΤετϋϯυϥ୯ҐͰಈ࡞ ʢઃఆ͸solrconfig.xmlʣ SearchComponentͷνΣΠϯͰݕࡧॲཧΛઃఆՄೳ QueryComponentʢݕࡧ࣮ߦʣɺ FacetComponentʢϑΝηοτܭࢉʣɺ HighlightComponentʢ݁ՌϋΠϥΠτʣͳͲ σϑΥϧτ͸/select 13೥6݄19೔ਫ༵೔
47. SolrͷΞʔΩςΫνϟ &RQWHQW )HHGHU HJ6ROU- &RQWHQW &RQQHFWRUV 0&) &RQWHQW 6RXUFHV ,QGH[ /XFHQH 6ROU:HE$SS VFKHPD VROU FRQILJ 8SGDWH&KDLQ $QDO\VLV 6HDUFK5HTXHVW +DQGOLQJ 4XHU\ 4XHU\ 3DUVHU 8SGDWH5HTXHVW +DQGOLQJ -DYD$SS6HUYHU 6HDUFK&RPS 6ROU&RUH $GPLQ 6WDWV =RR .HHSHU 5HVS :ULWHU $SDFKH6ROU &RQFHSWXDO $UFKLWHFWXUH $GPLQ 8VHU 8SGDWH +DQGOHU ,OOXVWUDWLRQE\&RPLQYHQW$6 <RXPD\IUHHO\XVHLWDV ORQJDV\RXDWWULEXWHZLWK OLQNWRZZZFRPLQYHQWFRP DFFRUGLQJWR&&E\OLFHQVH ZZZFUHDWLYHFRPPRQVRUJ 2ULJLQDOGLDJUDPE\ ZZZFRPLQYHQWFRP &RUH $GPLQ &RO OHFWLRQ $GPLQ ͜͜ Original diagram byɹ http://www.cominvent.com 13೥6݄19೔ਫ༵೔
48. ݕࡧͷྲྀΕ ݕࡧURLͷྫ http://localhost:8983/solr/collection1/select? q=Apache&start=0&rows=5 13೥6݄19೔ਫ༵೔
49. ΫΤϦ qύϥϝʔλͰࢦఆ ΫΤϦύʔαʹΑΓɺύʔε͞ΕͯΠϯσοΫεΛݕࡧ σϑΥϧτ͸luceneΫΤϦύʔα ྫɿtitle:"The Right Way" AND text:go http://wiki.apache.org/solr/ SolrQuerySyntax 13೥6݄19೔ਫ༵೔
50. ग़ྗܗࣜ wtύϥϝʔλʹΑΓࢦఆՄೳ solrconfig.xmlʹ<queryResponseWriter>Ͱઃఆ ҎԼ͸ઃఆͳ͠Ͱར༻Մೳ http://wiki.apache.org/solr/QueryResponseWriter ஋ આ໌ xml xml XMLܗࣜɻσϑΥϧτ json JSONܗࣜ python PythonͷγϦΞϥΠζܗࣜ ruby RubyͷγϦΞϥΠζܗࣜ php PHPͷevalؔ਺ܗࣜ phps PHPͷγϦΞϥΠζܗࣜ csv CSVܗࣜ 13೥6݄19೔ਫ༵೔
51. ͦͷଞݕࡧύϥϝʔλ ओͳύϥϝʔλ ύϥϝʔλ આ໌ σϑΥϧτ start ݁ՌͷελʔτҐஔ 0 rows औಘ݅਺ 10 fl औಘϑΟʔϧυ *,score fq ߜΓࠐΈ৚݅ df σϑΥϧτݕࡧϑΟʔϧυ 13೥6݄19೔ਫ༵೔
52. ϑΝηοτ ݕࡧ݁ՌͷߜΓࠐΈʹΑ͘࢖͏" υϦϧμ΢ϯͱ΋ݺ͹ΕΔ ϑΟʔϧυࢦఆɺΫΤϦࢦఆʹΑΓ ूܭ͕Մೳɻ ྫ http://localhost:8983/solr/collection1/select? q=Apache&fq=&facet=on&facet.field=genre 13೥6݄19೔ਫ༵೔
53. ϋΠϥΠτ Googleݕࡧͷ݁Ռʹग़ͯ͘ΔεχϖοτΛ࡞੒ Solr಺෦Ͱ͸εχϖοτੜ੒ʴڧௐදࣔΛҰॹʹ࣮ߦ ϋΠϥΠτॲཧͷ੍ݶ ର৅ϑΟʔϧυ͸ʮindexed=”true” stored=”true”ʯ ࣮૷ํ͕ࣜ3ͭଘࡏ ʢdefaultɺFastVectorɺPostingsʣ 13೥6݄19೔ਫ༵೔
54. ιʔτ sortύϥϝʔλʹͯࢦఆՄೳ σϑΥϧτιʔτ͸TF-IDFΛϕʔεʹͨ͠είΞ ΧϯϚ۠੾ΓͰෳ਺ࢦఆՄೳ ੍ݶ ର৅ϑΟʔϧυ͸ʮmultiValued=”false”ʯ 13೥6݄19೔ਫ༵೔
55. ෼ࢄݕࡧ ෳ਺ͷSolrΛଋͶͯݕࡧՄೳʢΠϯσοΫε͸ࣗ෼Ͱ ෼ࢄͤ͞Δඞཁ͋Γʣ yokozuna͸͜ͷػೳΛར༻ ػೳʹΑͬͯɺ෼ࢄݕࡧʹରԠ͍ͯ͠ͳ͍ػೳ͕͋Δ ෼ࢄݕࡧ͸2ϑΣʔζ͔Βߏ੒ 1. shardݕࡧ 2. ݕࡧ݁Ռ౷߹ 13೥6݄19೔ਫ༵೔
56. ෼ࢄݕࡧͷ࢓૊Έ 1. shardݕࡧ Solr Core Solr Core Solr Core Solr Core index index index shard title:Apache 13೥6݄19೔ਫ༵೔
57. ෼ࢄݕࡧͷ࢓૊Έ 1. shardݕࡧ Solr Core Solr Core Solr Core Solr Core index index index shard title:Apache title:Apache title:Apache 13೥6݄19೔ਫ༵೔
58. ෼ࢄݕࡧͷ࢓૊Έ 1. shardݕࡧ Solr Core Solr Core Solr Core Solr Core index index index shard title:Apache title:Apache title:Apache 13೥6݄19೔ਫ༵೔
59. ෼ࢄݕࡧͷ࢓૊Έ 1. shardݕࡧ Solr Core Solr Core Solr Core Solr Core index index index shard idͷϦετ idͷϦετ idͷϦετ 13೥6݄19೔ਫ༵೔
60. ෼ࢄݕࡧͷ࢓૊Έ 1. shardݕࡧ Solr Core Solr Core Solr Core Solr Core index index index shard idͷϦετ idͷϦετ idͷϦετ 13೥6݄19೔ਫ༵೔
61. ෼ࢄݕࡧͷ࢓૊Έ 2. ݁Ռ౷߹ Solr Core Solr Core Solr Core Solr Core index index index shard ֤shard͔Β ͷ݁ՌΛείΞ Ͱιʔτ͠ɺඞ ཁͳ෦෼ΛऔΓ ग़͢ 13೥6݄19೔ਫ༵೔
62. ෼ࢄݕࡧͷ࢓૊Έ 2. ݁Ռ౷߹ Solr Core Solr Core Solr Core Solr Core index index index shard ฦ٫σʔλऔಘ ͷͨΊɺ֤ shardʹidͰ໰ ͍߹Θͤ 13೥6݄19೔ਫ༵೔
63. ෼ࢄݕࡧͷ࢓૊Έ 2. ݁Ռ౷߹ Solr Core Solr Core Solr Core Solr Core index index index shard 13೥6݄19೔ਫ༵೔
64. ෼ࢄݕࡧͷ࢓૊Έ 2. ݁Ռ౷߹ Solr Core Solr Core Solr Core Solr Core index index index shard flύϥϝʔλͰࢦఆ͞Εͨ ϑΟʔϧυ৘ใΛฦ٫ 13೥6݄19೔ਫ༵೔
65. ෼ࢄݕࡧͷ࢓૊Έ 2. ݁Ռ౷߹ Solr Core Solr Core Solr Core Solr Core index index index shard 13೥6݄19೔ਫ༵೔
66. ෼ࢄݕࡧͷ࢓૊Έ 2. ݁Ռ౷߹ Solr Core Solr Core Solr Core Solr Core index index index shard ݁ՌΛXMLԽͯ͠ Ϩεϙϯεग़ྗ 13೥6݄19೔ਫ༵೔
67. SolrCloud Ϋϥελߏ੒͕ՄೳͳSolr ZookeeperͰߏ੒ɺઃఆΛ؅ཧ ϨϓϦΧ਺ͳͲΛઃఆՄೳ Solr 4ʙ Cloudera Search͸͜Ε͕ϕʔεʁ 13೥6݄19೔ਫ༵೔
68. ͦͷଞͷػೳ MoreLikeThisɿྨࣅυΩϡϝϯτݕࡧ SpatialSearchɿҐஔ৘ใݕࡧ ResultGroupingɿݕࡧ݁Ռάϧʔϐϯά JoinɿDBͷJoinͷΑ͏ͳ΋ͷ SolrCellɿApache TikaΛར༻ͨ͠υΩϡϝϯτղ ੳʢPDFͳͲ͔Βຊจɺϝλσʔλநग़ʣ DataImportHandlerɿઃఆϑΝΠϧΛॻ͘ͱDB΍ϑ ΝΠϧ͔ΒσʔλΛऔಘͯ͠Solrʹొ࿥Ͱ͖Δػೳ 13೥6݄19೔ਫ༵೔
69. ࢀߟURL SolrຊՈͷWikiɿ ɹhttp://wiki.apache.org/solr/ LucidWorksͷϦϑΝϨϯεΨΠυɿ ɹhttp://docs.lucidworks.com/display/solr/Apache+Solr +Reference+Guide Tokenizerͷಈ࡞ʹ͍ͭͯʢLucene Revolution 2013ͷεϥΠυʣ ɹhttp://www.lucenerevolution.org/sites/default/files/ Language%20support%20and%20linguistics%20in %20LuceneSolr%20%26%20its%20eco-system.pdf Lucene RevolutionͷαΠτ ɹhttp://www.lucenerevolution.org/2013/Lucene-Solr- Revolution-2013-Presentations 13೥6݄19೔ਫ༵೔