ElasticSearch入門

ElasticSearch

Transcript

1. ElasticSearchೖ໳
   2013/08/29
   @johtani
   13೥8݄30೔༵ۚ೔
   

   View Slide

2. ࣗݾ঺հ
   ࢯ໊ɿେ୩ɹ७
   Twitterɿ@johtani
   lucene-gosenͷίϛολʔ
   Solrೖ໳ஶऀͷ1ਓ
   ϒϩάɿhttp://blog.johtani.info
   ॴଐɿגࣜձࣾγʔϚʔΫ
   13೥8݄30೔༵ۚ೔
   

   View Slide

3. ΞδΣϯμ
   ݕࡧΤϯδϯͬͯʁ
   సஔΠϯσοΫεɺN-gramɺܗଶૉͷ࿩
   ElasticSearchͱ͸ʁ
   ΠϯσοΫεͷ࿦ཧߏ੒ͱ෺ཧߏ੒
   ػೳ֓ཁ
   ࢀߟࢿྉͳͲ
   13೥8݄30೔༵ۚ೔
   

   View Slide

4. ݕࡧΤϯδϯͬͯʁ
   ʢElasticSearchͱ͸ؔ܎ͷͳ͍֓ཁతͳ࿩ʣ
   13೥8݄30೔༵ۚ೔
   

   View Slide

5. શจݕࡧͬͯʁ
   શจݕࡧʢFull text searchʣͱ͸ɺίϯϐϡʔλʹ͓͍ͯɺෳ਺ͷจॻʢϑΝΠ
   ϧʣ͔ΒಛఆͷจࣈྻΛݕࡧ͢Δ͜ͱɻʮϑΝΠϧ໊ݕࡧʯ΍ʮ୯ҰϑΝΠϧ
   ಺ͷจࣈྻݕࡧʯͱҟͳΓɺʮෳ਺จॻʹ·͕ͨͬͯɺจॻʹؚ·ΕΔશจΛ
   ର৅ͱͨ͠ݕࡧʯͱ͍͏ҙຯͰ࢖༻͞ΕΔɻ
   ʢWikipediaΑΓʣ
   13೥8݄30೔༵ۚ೔
   

   View Slide

6. ༻ޠ
   จॻʢυΩϡϝϯτʣ
   ɹݕࡧΤϯδϯʹอଘ͞ΕͨσʔλɻRDBͰͷϨίʔυʹ૬౰
   ΫΤϦ
   ɹݕࡧ৚݅ɺݕࡧࣜ
   εΩʔϚ
   ɹRDBͷςʔϒϧʹ૬౰
   ϑΟʔϧυ
   ɹRDBͷΧϥϜʹ૬౰
   λʔϜʢTermʣ/τʔΫϯʢTokenʣ
   ɹΠϯσοΫεͷΩʔʹͳΔ୯ޠʢจࣈྻʣ
   13೥8݄30೔༵ۚ೔
   

   View Slide

7. υΩϡϝϯτొ࿥ͷྲྀΕ
   1
   2
   ΧπΦ͸αβΤͷఋ
   αβΤ͸ϫΧϝͷ࢞
   υΩϡϝϯτͷొ࿥
   13೥8݄30೔༵ۚ೔
   

   View Slide

8. υΩϡϝϯτొ࿥ͷྲྀΕ
   1
   2
   ΧπΦ͸αβΤͷఋ
   αβΤ͸ϫΧϝͷ࢞
   1
   2
   ΧπΦ
   αβΤ
   ͸
   ͸
   ͷ
   ͷ
   αβΤ
   ϫΧϝ
   ఋ
   ࢞
   υΩϡϝϯτͷొ࿥
   ୯ޠʹ෼ׂ
   13೥8݄30೔༵ۚ೔
   

   View Slide

9. υΩϡϝϯτొ࿥ͷྲྀΕ
   1
   2
   ΧπΦ͸αβΤͷఋ
   αβΤ͸ϫΧϝͷ࢞
   1
   2
   ΧπΦ
   αβΤ
   ͸
   ͸
   ͷ
   ͷ
   αβΤ
   ϫΧϝ
   ఋ
   ࢞
   ΧπΦ
   αβΤ
   1
   1 2 ͸
   ͷ
   ࢞
   ϫΧϝ 2
   1 2
   1 2
   1
   ఋ
   2
   υΩϡϝϯτͷొ࿥
   ୯ޠʹ෼ׂ
   ୯ޠ͔Βidͷ഑ྻ͕
   Ҿ͚ΔΑ͏ʹ
   ʹసஔΠϯσοΫε
   13೥8݄30೔༵ۚ೔
   

   View Slide

10. ݕࡧͷྲྀΕ
    ΧπΦɹαβΤ
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ
    ࢞
    ϫΧϝ 2
    1 2
    1 2
    1
    ఋ
    2
    ݕࡧจࣈೖྗ
    13೥8݄30೔༵ۚ೔
    

    View Slide

11. ݕࡧͷྲྀΕ
    ΧπΦɹαβΤ
    ΧπΦ αβΤ
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ
    ࢞
    ϫΧϝ 2
    1 2
    1 2
    1
    ఋ
    2
    AND
    ݕࡧจࣈೖྗ
    ݕࡧจࣈͷύʔε
    ʴݕࡧΫΤϦԽ
    13೥8݄30೔༵ۚ೔
    

    View Slide

12. ݕࡧͷྲྀΕ
    ΧπΦɹαβΤ
    ΧπΦ αβΤ
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ
    ࢞
    ϫΧϝ 2
    1 2
    1 2
    1
    ఋ
    2
    AND
    ݕࡧจࣈೖྗ
    ݕࡧจࣈͷύʔε
    ʴݕࡧΫΤϦԽ
    సஔΠϯσοΫεΛݕࡧ
    13೥8݄30೔༵ۚ೔
    

    View Slide

13. ݕࡧͷྲྀΕ
    ΧπΦɹαβΤ
    ΧπΦ αβΤ
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ
    ࢞
    ϫΧϝ 2
    1 2
    1 2
    1
    ఋ
    2
    AND
    ݕࡧจࣈೖྗ
    ݕࡧจࣈͷύʔε
    ʴݕࡧΫΤϦԽ
    సஔΠϯσοΫεΛݕࡧ
    ˠANDݕࡧͳͷͰidͷ
    ഑ྻͷANDΛͱΔ
    13೥8݄30೔༵ۚ೔
    

    View Slide

14. ݕࡧͷྲྀΕ
    ΧπΦɹαβΤ
    ΧπΦ αβΤ
    ΧπΦ
    αβΤ
    1
    1 2 ͸
    ͷ
    ࢞
    ϫΧϝ 2
    1 2
    1 2
    1
    ఋ
    2
    AND
    ݕࡧจࣈೖྗ
    ݕࡧจࣈͷύʔε
    ʴݕࡧΫΤϦԽ
    సஔΠϯσοΫεΛݕࡧ
    ˠANDݕࡧͳͷͰidͷ
    ഑ྻͷANDΛͱΔ
    13೥8݄30೔༵ۚ೔
    

    View Slide

15. N-gramͱܗଶૉղੳ
    సஔΠϯσοΫεͷΩʔʢ୯ޠʣͷ࡞Γํ
    ೔ຊޠ͸୯ޠͷ੾Ε໨͕Θ͔Βͳ͍ͷͰసஔΠϯσοΫεͷΩʔ͸ओʹͭ͗ͷ2
    ͭͷख๏Ͱ࡞Δ
    N-gram
    Nʢ1Ҏ্ͷ਺ࣈʣจࣈͣͭจষΛ۠੾Δ
    ܗଶૉղੳ
    ࣙॻͳͲΛ༻͍ͯҙຯͷ͋Δ୯ޠͰ۠੾Δ
    13೥8݄30೔༵ۚ೔
    

    View Slide

16. N-gram
    N͕2ͷ࣌ͷྫɿ
    ϝϦοτɿ
    ݕࡧ࿙Ε͕ͳ͍
    σϝϦοτɿ
    ਫ਼౓͕ѱ͍→ʮΧπʯͰ΋ώοτͪ͠Ό͏
    ΠϯσοΫε͕ංେԽ
    ޠװʢ୯ޠͷجຊܗͱ͔ʣͰݕࡧෆՄ
    ΧπΦ͸αβΤͷఋ
    Χπ πΦ Φ͸ ͸α αβ βΤ Τͷ ͷఋ
    13೥8݄30೔༵ۚ೔
    

    View Slide

17. ܗଶૉղੳ
    ܗଶૉղੳͷྫɿ
    ϝϦοτɿ
    ਫ਼౓͕Α͍ɿʮΧπʯͰݕࡧ→ʮΧπΦʯώοτ͠ͳ͍
    ޠװʹΑΔݕࡧ͕ՄೳɿʮҿΈʯ→ʮҿΉʯ͕ݕࡧՄ
    σϝϦοτɿ
    ৽ޠʢະ஌ޠʣʹऑ͍→ࣙॻϕʔεͷ৔߹ʹࣙॻʹͳ͍୯ޠ͕Θ͔Βͳ͍ɻࣙॻ
    ߋ৽ޙ͸ΠϯσοΫε࠶࡞੒
    ΧπΦ͸αβΤͷఋ
    ΧπΦ ͸ ͷ
    αβΤ ఋ
    13೥8݄30೔༵ۚ೔
    

    View Slide

18. υΩϡϝϯτͱϑΟʔϧυ
    υΩϡϝϯτ͸ෳ਺ͷϑΟʔϧυ͔Βߏ੒͞ΕΔ
    ϑΟʔϧυ୯ҐͰసஔΠϯσοΫεΛ࡞੒
    ϑΟʔϧυ୯ҐͰݕࡧ͕Մೳʹ
    13೥8݄30೔༵ۚ೔
    

    View Slide

19. ElasticSearchͱ͸ʁ
    13೥8݄30೔༵ۚ೔
    

    View Slide

20. ຊ೔ͷཁ఺
    13೥8݄30೔༵ۚ೔
    

    View Slide

21. ࠓ೔͸͜Ε͚ͩ͸֮͑ͯؼͬ
    ͍ͯͩ͘͞ɻ
    13೥8݄30೔༵ۚ೔
    

    View Slide

22. ElasticSearch͸
    Amazon Web Servicesͷ
    αʔϏεͱ͸ؔ܎͋Γ·ͤΜ
    13೥8݄30೔༵ۚ೔
    

    View Slide

23. Ҏ্
    13೥8݄30೔༵ۚ೔
    

    View Slide

24. ElasticSearchͱ͸ʁ
    ෼ࢄܕRESTfulݕࡧˍղੳαʔό
    OSSʢApache Software License Version 2.0ʣ
    Apache LuceneΛίΞʹ࠾༻ʢ0.90.3͸Lucene 4.4.0ʹରԠʣ
    ϚϧνςφϯτରԠ
    εΩʔϚϑϦʔˍυΩϡϝϯτࢦ޲
    ָ؍తഉଞ੍ޚ
    ͳͲͳͲ
    13೥8݄30೔༵ۚ೔
    

    View Slide

25. ElasticSearchͷ
    ΠϯσοΫεߏ੒
    ʢ࿦ཧͱ෺ཧʣ
    13೥8݄30೔༵ۚ೔
    

    View Slide

26. ΠϯσοΫεߏ੒ʢ࿦ཧʣ
    ʮΠϯσοΫε/λΠϓʢϚοϐϯάλΠϓʣ/υΩϡϝϯτʯͱ͍͏ߏ੒
    ※Solrʹ͸λΠϓͱ͍͏֓೦͕ͳ͍
    ElasticSearch
    indexɿbooks
    typeɿ
    novel
    typeɿ
    magazine
    doc1
    doc2
    doc3
    indexɿdvds
    13೥8݄30೔༵ۚ೔
    

    View Slide

27. ΠϯσοΫεߏ੒ʢ࿦ཧʣ
    υΩϡϝϯτ
    ɹɹɿRDBͷߦʹ૬౰͢Δ֓೦ɻ
    ɹɹɹߏ଄͕ݻఆͰͳ͍ʹεΩʔϚϑϦʔʢεΩʔϚΛఆٛ͢Δ͜ͱ΋Մೳʣ
    λΠϓʢϚοϐϯάλΠϓʣ
    ɹɹɿRDBͷςʔϒϧʹ૬౰͢Δ֓೦ɻ
    ɹɹɹυΩϡϝϯτΛ·ͱΊΔίϯςφతͳ΋ͷɻ
    ɹɹɹλΠϓຖʹεΩʔϚʢESͰ͸Ϛοϐϯάͱ͍͏ʣΛఆٛՄೳ
    ΠϯσοΫε
    ɹɹɿRDBͷσʔλϕʔεʹ૬౰͢Δ֓೦ɻ
    ɹɹɹ෺ཧతͳϑΝΠϧʹอଘ͞ΕΔ୯Ґ
    ɹɹʢλΠϓ͸͋͘·Ͱ΋֓೦తͳ΋ͷͰɺ
    ɹɹɹ෺ཧతʹ͸ΠϯσοΫεͷ୯ҐͰอଘɻ※෺ཧߏ੒͸ޙड़ʣ
    13೥8݄30೔༵ۚ೔
    

    View Slide

28. ΠϯσοΫεߏ੒ʢ࿦ཧʣ
    ElasticSearchͰ͸࿦ཧߏ੒͕URLͱಉ͡ߏ੒
    υΩϡϝϯτͷొ࿥
    curl -XPUT http://localhost:9200/books/novel/doc1 -d ‘{...}’
    υΩϡϝϯτͷݕࡧʢΠϯσοΫε୯Ґʣ
    curl -XGET http://localhost:9200/books/_search?q=...
    υΩϡϝϯτͷݕࡧʢλΠϓ୯Ґʣ
    curl -XGET http://localhost:9200/books/novel/_search?q=...
    υΩϡϝϯτͷIDʹΑΔऔಘʢυΩϡϝϯτ୯Ґʣ
    curl -XGET http://localhost:9200/books/novel/doc1
    13೥8݄30೔༵ۚ೔
    

    View Slide

29. ΠϯσοΫεߏ੒ʢ࿦ཧʣ
    ෳ਺ͷΠϯσοΫεΛԣஅతʹݕࡧ΋ՄೳʢΧϯϚ۠੾ΓͰOKʣ
    curl -XGET http://localhost:9200/books,dvds/_search?q=...
    શΠϯσοΫεΛԣஅݕࡧ͢Δʹ͸
    curl -XGET http://localhost:9200/_search?q=...
    શΠϯσοΫεͷλΠϓࢦఆͰͷݕࡧ΋Մೳ
    curl -XGET http://localhost:9200/_all/magazine/_search?q=...
    13೥8݄30೔༵ۚ೔
    

    View Slide

30. ΠϯσοΫεߏ੒ʢ෺ཧʣ
    13೥8݄30೔༵ۚ೔
    

    View Slide

31. Ϋϥελɿelasticsearch
    ΠϯσοΫεߏ੒ʢ෺ཧʣ
    ΫϥελͱϊʔυɺΠϯσοΫεͱγϟʔυʢshardʣͷؔ܎
    ϊʔυɿElasticSearchϓϩηε̍ ϊʔυɿElasticSearchϓϩηε̎
    booksɿprimary shard1
    dvdsɿreplica shard1
    dvdsɿreplica shard2
    booksɿreplica shard2
    dvdsɿprimary shard1
    booksɿreplica shard1
    dvdsɿprimary shard2
    booksɿprimary shard2
    13೥8݄30೔༵ۚ೔
    

    View Slide

32. ΠϯσοΫεߏ੒ʢ෺ཧʣ
    Ϋϥελ͸nݸͷϊʔυ͔Βߏ੒ʢcluster.name͕ಉҰͳΒಉҰΫϥελʣ
    ΠϯσοΫε͸nݸͷγϟʔυ͔Βߏ੒͞Εɺγϟʔυ୯ҐͰϊʔυʹ഑ஔ͞Ε
    Δ͜ͱͰ෼ࢄՄೳ
    γϟʔυ୯ҐͰϨϓϦΧΛอ࣋Մೳ
    σϑΥϧτͰɺ5γϟʔυɺ1ϨϓϦΧʢ/1γϟʔυʣ
    ※γϟʔυ਺͸ΠϯσοΫε࡞੒࣌ʹͷΈࢦఆ͕ՄೳɻϨϓϦΧ਺͸มߋՄೳ
    γϟʔυ෼ׂػೳ͕ͳ͍ͨΊɻSolr͸shard splittingػೳ͋Γʢ2013/04ʣ
    ※ಉҰϓϩηε্ʹಉҰγϟʔυͷϨϓϦΧͱϓϥΠϚϦ͕৐Βͳ͍Α͏ʹͳͬ
    ͍ͯΔ
    13೥8݄30೔༵ۚ೔
    

    View Slide

33. Ϋϥελ؅ཧ
    Ϋϥελ΁ͷϊʔυ௥ՃͳͲͷϊʔυ୳ࡧ͸σϑΥϧτZen Discovery
    ϚϧνΩϟετorϢχΩϟετʹΑΔϊʔυ୳ࡧ
    ϚϧνΩϟετɿϚϧνΩϟετΛར༻ͨ͠ϊʔυ୳ࡧ͓ΑͼΫϥελߏங
    ϢχΩϟετɿϗετϦετΛར༻ͨ͠Ϋϥελͷߏங
    ΫϥελىಈޙʹϚελʔϊʔυΛબग़ʢϚϧνϚελ͕Մೳʣ
    Ϛελϊʔυ͕Ϋϥελશମͷো֐ݕ஌Λ࣮ࢪ
    http://www.elasticsearch.org/guide/reference/modules/discovery/zen/
    ※ZooKeeperΛ༻͍ͨϊʔυ୳ࡧͷϓϥάΠϯ΋͋Δ
    13೥8݄30೔༵ۚ೔
    

    View Slide

34. ػೳ֓ཁ
    13೥8݄30೔༵ۚ೔
    

    View Slide

35. ػೳ֓ཁ
    ઃఆؔ࿈
    ܕͷࣗಈܾఆɺಈతϚοϐϯάɺςϯϓϨʔτͳͲ
    σʔλߏ଄
    ೖΕࢠɺ਌ࢠߏ଄ɺ෦෼ߋ৽
    ศརػೳ
    TTLɺεϩʔΫΤϦϩάɺεϩʔΠϯσΫγϯάϩάɺϓϥάΠϯ
    ݕࡧɺొ࿥ػೳ
    Geoɺ PercoratorɺόϧΫΠϯαʔτ
    13೥8݄30೔༵ۚ೔
    

    View Slide

36. ઃఆؔ࿈
    13೥8݄30೔༵ۚ೔
    

    View Slide

37. ઃఆؔ࿈ʢجຊʣ
    ΄ͱΜͲ͢΂ͯͷઃఆʹ͍ͭͯɺREST APIͰઃఆ͕Մೳ
    ઃఆϑΝΠϧͱͯ͠อଘ΋Մೳ
    ֬ೝ΋REST APIͰՄೳ
    Ϋϥελͷঢ়ଶͳͲ΋
    ΠϯσοΫεͷ࡞੒ɺ࡟আ΋
    curl -XPUT 'http://localhost:9200/twitter/'
    curl -XDELETE 'http://localhost:9200/twitter/'
    curl -XGET 'http://localhost:9200/_cluster/health?pretty=true'
    13೥8݄30೔༵ۚ೔
    

    View Slide

38. ઃఆؔ࿈ʢܕͷܾఆػೳʣ
    JSONΦϒδΣΫτ͔ΒࣗಈͰϑΟʔϧυͷܕΛਪଌͯ͠ొ࿥Մೳ
    ʢεΩʔϚϑϦʔͱݺ͹ΕΔॴҎʣ
    ਺஋ɺ೔෇ɺจࣈྻͳͲΛਪଌͯ͠ϑΟʔϧυΛܾఆ
    ਺஋ɿ numeric_detectionʹtrue/falseΛઃఆ
    ೔෇ɿdynamic_date_formatsɿjoda-timeϥΠϒϥϦͷ೔෇ϑΥʔϚοτจ
    ࣈྻΛࢦఆʢෳ਺ࢦఆՄʣ
    13೥8݄30೔༵ۚ೔
    

    View Slide

39. ܕͷࣗಈܾఆͷ஫ҙ఺
    ஫ҙ఺
    ͢ͰʹܕΛܾఆͨ͠ϑΟʔϧυʹҟͳΔܕͷσʔλΛ͍ΕΔͱΤϥʔʹͳͬ
    ͨΓɺ੾ΓࣺͯΒΕͨΓ
    Τϥʔྫɿ24ʢlongܕʣΛొ࿥ޙɺ”hoge”ʢstringܕʣΛొ࿥
    ignore_malformed͕trueͰઃఆ͞Ε͍ͯΕ͹ΤϥʔʹͳΒͳ͍
    ΤϥʔʹͳΒͳ͍ྫɿ24ʢlongܕʣΛొ࿥ޙɺ12.3ʢfloatܕʣΛొ࿥
    σϑΥϧτͰ͸খ਺఺ҎԼ͕੾ΓࣺͯΒΕͯ12Ͱొ࿥͞Εͯ͠·͏
    13೥8݄30೔༵ۚ೔
    

    View Slide

40. ઃఆؔ࿈ʢಈతϚοϐϯάʣ
    ϑΟʔϧυ໊ͳͲʹΑΔϑΟʔϧυܕͷਪఆΛઃఆՄೳ
    SolrͷμΠφϛοΫϑΟʔϧυͷΑΓॊೈͳػೳ
    mappingsఆٛʹdynamic_templatesͷઃఆΛ഑ྻͰ௥Ճ
    {
    "person" : {
    "dynamic_templates" : [
    {
    "template_1" : {
    "match" : "multi*",
    "mapping" : {
    "type" : "multi_field",
    "fields" : {
    "{name}" : {"type": "{dynamic_type}", "index" : "analyzed"},
    "org" : {"type": "{dynamic_type}", "index" : "not_analyzed"}
    }
    }
    }
    },...
    13೥8݄30೔༵ۚ೔
    

    View Slide

41. ઃఆؔ࿈ʢςϯϓϨʔτʣ
    ΠϯσοΫε୯Ґͷઃఆ΋ςϯϓϨʔτԽՄೳ
    ΠϯσοΫε໊ʹtemplate͕Ұகͨ͠΋ͷΛద༻ɻorderͷେ͖͍΋ͷͰ্ॻ͖
    ྫɿجຊઃఆ͸ϨϓϦΧ0͕ͩɺha_Ͱ͸͡·ΔΠϯσοΫε͸ϨϓϦΧ5
    curl -XPUT localhost:9200/_template/main_template -d '
    {
    "template" : "*",
    "order" : "1",
    "settings" : {
    "index.number_of_replicas" : 0
    }
    }
    curl -XPUT localhost:9200/_template/ha_template -d '
    {
    "template" : "ha_*",
    "order" : "10",
    "settings" : {
    "index.number_of_replicas" : 5
    }
    }
    13೥8݄30೔༵ۚ೔
    

    View Slide

42. σʔλߏ଄
    13೥8݄30೔༵ۚ೔
    

    View Slide

43. σʔλߏ଄ʢωετʣ
    ωετɺ਌ࢠɺΦϒδΣΫτͳͲͷσʔλ΋ొ࿥Մೳ
    ʢSolr͸جຊϑϥοτͳσʔλͷΈʣ
    ྫɿTγϟπͷ৭ͱαΠζͷ૊Έ߹Θͤʢωετʣ
    {
    "variation" : [
    {
    "color" : "blue",
    "size" : 4
    },
    {
    "name" : "green",
    "size" : 6
    }
    ]
    }
    {
    "query" : {
    "nested" : {
    "path" : "variation",
    "query" : {
    "bool" : {
    "must" : [
    { "term" : {"variation.name" : "green"} },
    { "term" : {"variation.size" : "6"} }
    ]
    }
    }
    }
    }
    }
    ΫΤϦ
    σʔλ
    13೥8݄30೔༵ۚ೔
    

    View Slide

44. σʔλߏ଄ʢ਌ࢠʣ
    ਌ࢠؔ܎ʹؔ͢Δݕࡧ΋ՄೳʢmappingͰઃఆ΋ඞཁʣ
    ࢠσʔλొ࿥࣌ʹ਌IDΛࢦఆ
    $ curl -XPUT localhost:9200/shop/variation/1?parent=1 -d '{
    "color" : "red",
    "size" : "XL"
    }' ࢠσʔλ
    $ curl -XPUT localhost:9200/shop/cloth/1 -d '{
    "name" : "something"
    }' ਌σʔλ
    $ curl -XPUT localhost:9200/shop/variation/2?parent=1 -d '{
    "color" : "green",
    "size" : "S"
    }' ࢠσʔλ
    {
    "has_child" : {
    "type" : "variation",
    "query" : {
    "bool" : {
    "must" : [
    { "term": {"size" : "XL"} },
    { "term": {"color" : "red"} }
    ]
    }
    }
    }
    }
    ΫΤϦ
    13೥8݄30೔༵ۚ೔
    

    View Slide

45. σʔλߏ଄ʢ෦෼ߋ৽ʣ
    _sourceϑΟʔϧυʹσϑΥϧτͰೖྗ͞ΕͨJSONͷσʔλΛอ࣋
    ʢSolrʹ͸ͳ͍ʣ
    _sourceϑΟʔϧυΛ࢖༻͢Δ෦෼ߋ৽͕Մೳ
    curl -XPOST 'localhost:9200/test/type1/1/_update' -d '{
    "script" : "ctx._source.counter += count",
    "params" : {
    "count" : 4
    }
    }'
    13೥8݄30೔༵ۚ೔
    

    View Slide

46. ศརػೳʢTTLʣ
    TTLʹTime To Live
    σʔλͷੜଘظؒΛࢦఆՄೳʢྫ͸1೔ʣ
    ExpireͷνΣοΫॲཧ͸60sຖʹߦΘΕΔʢมߋՄʣ
    {
    "tweet" : {
    "_ttl" : { "enabled" : true, "default" : "1d" }
    }
    }
    mapping
    13೥8݄30೔༵ۚ೔
    

    View Slide

47. ศརػೳʢϩάʣ
    index slow log/search slow logʢSolrʹ͸ͳ͍ػೳʣ
    ΠϯσοΫε࣌ɺݕࡧ࣌ʹॲཧ͕͖͍࣌ؒ͠஋Λ௒͑ͨΒϩάग़ྗ
    ϩάϨϕϧ͝ͱʹ஋ΛઃఆՄೳ
    #index.search.slowlog.threshold.query.warn: 10s
    #index.search.slowlog.threshold.query.info: 5s
    #index.search.slowlog.threshold.query.debug: 2s
    #index.search.slowlog.threshold.query.trace: 500ms
    #index.search.slowlog.threshold.fetch.warn: 1s
    #index.search.slowlog.threshold.fetch.info: 800ms
    #index.search.slowlog.threshold.fetch.debug: 500ms
    #index.search.slowlog.threshold.fetch.trace: 200ms
    13೥8݄30೔༵ۚ೔
    

    View Slide

48. ศརػೳʢϓϥάΠϯʣ
    ElasticSearchͷ֦ுػೳ
    RiverɿσʔλΛElasticSearchʹྲྀ͠ࠐΉϓϥάΠϯʢྫɿRabbitMQͱ͔ʣ
    AnalysisɿΞφϥΠβͷϓϥάΠϯʢྫɿKuromojiͱ͔ʣ
    SiteɿΫϥελ؅ཧͱ͔
    ϓϥάΠϯͷΠϯετʔϧ͸෇ଐͷpluginίϚϯυʹͯՄೳ
    ʢ௨ৗ͸ωοτʹ઀ଓ͍ͯ͠Δඞཁ͕͋Δʣ
    ϓϥάΠϯͷҰཡʢhttp://www.elasticsearch.org/guide/reference/modules/
    plugins/ʣ
    13೥8݄30೔༵ۚ೔
    

    View Slide

49. ݕࡧػೳʢPercoratorʣ
    ΫΤϦΛΠϯσοΫεԽͯ͠ొ࿥
    ొ࿥ࡁΈͷΫΤϦʹώοτ͢Δσʔλ͔Ͳ͏͔ΛಛఆͷϦΫΤετʹσʔλ
    Λૹ৴͢Δ͜ͱͰΘ͔Δ࢓૊Έʢར༻ՄೳͳΫΤϦ͸Ұ෦੍ݶ͋Γʣ
    curl -XPUT localhost:9200/_percolator/test/kuku -d '{
    "query" : {
    "term" : {
    "field1" : "value1"
    }
    }
    }
    curl -XGET localhost:9200/test/type1/_percolate -d '{
    "doc" : {
    "field1" : "value1"
    }
    }'
    And the matches are part of the response:
    {"ok":true, "matches":["kuku"]}
    percoratorͷొ࿥
    percoratorͷ֬ೝ
    13೥8݄30೔༵ۚ೔
    

    View Slide

50. ݕࡧػೳʢGeoʣ
    Ң౓ܦ౓σʔλΛར༻ͨ͠ݕࡧ
    Ң౓ܦ౓ɺgeohashɺgeo_shapeͳͲ
    த৺఺͔Βͷڑ཭ɺۣܗɺgeohashʹΑΔߜࠐͳͲ͕Մೳ
    ڑ཭ʹΑΔϑΝηοτ΋
    13೥8݄30೔༵ۚ೔
    

    View Slide

51. ొ࿥ػೳʢόϧΫॲཧʣ
    େྔͷॲཧΛߦ͏ͨΊͷAPIʢ_bulkʣ
    http://www.elasticsearch.org/guide/reference/api/bulk/
    UDP΋ར༻Մೳ
    { "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
    { "field1" : "value1" }
    { "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
    { "create" : { "_index" : "test", "_type" : "type1", "_id" : "3" } }
    { "field1" : "value3" }
    { "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1"} }
    { "doc" : {"field2" : "value2"} }
    όϧΫॲཧσʔλྫ
    13೥8݄30೔༵ۚ೔
    

    View Slide

52. ͦͷଞʢؔ࿈πʔϧͱ͔ʣ
    Kibana3ʴElasticSearchʴ(logstash OR fluentd)
    ElasticSearchΛετϨʔδʹͯ͠ϩάσʔλͷอଘͱՄࢹԽΛߦ͏
    13೥8݄30೔༵ۚ೔
    

    View Slide

53. ࢀߟࢿྉͳͲ
    13೥8݄30೔༵ۚ೔
    

    View Slide

54. ࢀߟϖʔδ
    ຊՈˍΨΠυ
    http://www.elasticsearch.org
    http://www.elasticsearch.org/guide/
    αϙʔτˍτϨʔχϯά
    http://elasticsearch.com
    Solr vs ElasticSearchͷ೔ຊޠ຋༁൛
    https://github.com/minghai/ElasticSearch_VS_Solr
    Sematextͷϒϩά
    http://blog.sematext.com
    13೥8݄30೔༵ۚ೔
    

    View Slide

55. ࢀߟॻ੶ͱ͔
    ElasticSearch ServerʢPackt Publishingɺ2013/02݄ൃചʣ
    http://www.packtpub.com/elasticsearch-server-for-fast-scalable-flexible-
    search-solution/book
    http://elasticsearchserverbook.com
    Mastering ElasticSearchʢPackt Publishingɺ2013/12ൃച༧ఆʣ
    http://www.packtpub.com/mastering-elasticsearch-querying-and-data-
    handling/book
    Elasticsearch in ActionʢManning Publicationsɺ2014/य़ɺMEAPߪೖՄʣ
    http://www.manning.com/hinman/
    13೥8݄30೔༵ۚ೔
    

    View Slide

56. ͓·͚
    13೥8݄30೔༵ۚ೔
    

    View Slide

57. LuceneͱESͷόʔδϣϯ
    ElasticSearch Lucene
    0.90.3 4.4
    0.90.2 4.3.1
    0.90.1 4.3
    0.90.0 4.2.1
    0.90.0rc2 4.2.1
    0.90.0rc1 4.2
    0.20.6 3.6.2
    0.20.5 3.6.2
    0.90.0 beta1 4.1
    13೥8݄30೔༵ۚ೔
    

    View Slide