全文検索エンジン「Elasticsearch」の概要とprismatixでの活用事例

by nakamura-tk

Slide 1

Slide 1 text

QSJTNBUJYࣄۀ෦தଜོߒ શจݕࡧΤϯδϯʮ&MBTUJDTFBSDIʯͷ֓ཁͱ QSJTNBUJYͰͷ׆༻ࣄྫ

Slide 2

Slide 2 text

ࣗݾ঺հ தଜɹོߒ ΫϥεϝιουɹQSJTNBUJYࣄۀ෦   ݕࡧαʔϏε։ൃ୲౰     ओͳۀ຿಺༰ ɾ&MBTUJDTFBSDI   ɾαʔόʔαΠυ +BWB

Slide 3

Slide 3 text

ຊηογϣϯͷ֓ཁ

Slide 4

Slide 4 text

αϚϦʔ w &MBTUJDTFBSDIʹ͍ͭͯ w &MBTUJDTFBSDIͱ͸ w &MBTUJDTFBSDIͷಛ௃ɾϢʔεέʔε w શจݕࡧʹ͍ͭͯ w "NB[PO0QFO4FBSDI4FSWJDFʹ͍ͭͯ w QSJTNBUJYͰͷ׆༻ࣄྫ

Slide 5

Slide 5 text

૝ఆࢹௌऀ w&MBTUJDTFBSDIʹڵຯ͕͋Δํ wશจݕࡧʹ͍ͭͯ஌Γ͍ͨํ w&MBTUJDTFBSDIΛ༻͍ͨαʔϏεఏڙʹڵຯ͕͋Δํ

Slide 6

Slide 6 text

Elasticsearch ͱ͸

Slide 7

Slide 7 text

&MBTUJDTFBSDIͱ͸ &MBTUJD͕ࣾఏڙ͍ͯ͠Δ&MBTUJD4UBDL ʹؚ·ΕΔશจݕࡧٴͼ෼ੳΤϯδϯͰɺ༷ʑ ͳσʔλΛϦΞϧλΠϜʹݕࡧɾ෼ੳɾՄࢹԽ ͢Δ͜ͱ͕ՄೳͰ͢ɻ &MBTUJD4UBDLʹؚ·ΕΔϓϩμΫτ • Elasticsearch ɿ ৺ଁ෦ͱͳΔJSONϕʔεͷݕࡧ/෼ੳΤϯδϯ • Kibana : ݕࡧɺՄࢹԽ༻ͷπʔϧ • Beats ɿ ΞϓϦ΍Πϯϑϥͷ౷ܭ৘ใ౳ͷσʔλऩूπʔϧ • Logstash ɿ σʔλऩूύΠϓϥΠϯ

Slide 8

Slide 8 text

Elasticsearch ͷಛ௃

Slide 9

Slide 9 text

&MBTUJDTFBSDIͷಛ௃ w εέʔϥϏϦςΟ w ෼ࢄݕࡧ w εΩʔϚϨε w ։ൃऀϑϨϯυϦʔ w શจݕࡧ

Slide 10

Slide 10 text

εέʔϥϏϦςΟᶃ &MBTUJDTFBSDI͸୆Ҏ্ͷϊʔυͰΫϥελʔΛߏ੒ ͠ɺͭͷ·ͱ·Γͱͯ͠σʔλΛॲཧ͠·͢ɻ ϊʔυ͸&MBTUJDTFBSDIͷ Πϯελϯε

Slide 11

Slide 11 text

εέʔϥϏϦςΟᶄ .BTUFSϊʔυ %BUBϊʔυ *OHFTUϊʔυ ϊʔυʹ͸༷ʑͳ໾ׂΛׂΓ౰ͯΔ͜ͱ͕ՄೳͰ͢ɻ໾ׂ͝ͱʹϊʔ υ਺Λௐ੔͢Δ͜ͱͰਫฏํ޲ͷεέʔϧ͕Մೳ ར༻ঢ়گʹԠͯ͡ εέʔϧ͕Մೳ γϯϓϧͳϊʔυ ߏ੒

Slide 12

Slide 12 text

෼ࢄݕࡧ σʔλΛ෼ࢄͯ͠อ࣋͢Δ͜ͱͰɺେྔσʔλͰ΋ߴ଎ ͳݕࡧ͕ՄೳɻϊʔυؒͰϨϓϦΧΛอ͍࣋ͯ͠Δͷ Ͱɺো֐࣌ͷ෮چ΋ૣ͍ɻ

Slide 13

Slide 13 text

εΩʔϚϨε ࣄલͷεΩʔϚఆٛແ͠ʹσʔλΛొ࿥ՄೳͰ͢ɻ ొ࿥͞Εͨσʔλ಺༰Λ෼ੳ͠ɺ&MBTUJDTFBSDI͕ࣗಈ Ͱఆٛ͠·͢ɻ

Slide 14

Slide 14 text

։ൃऀϑϨϯυϦʔᶃ w 3&45"1*Ͱ֤छૢ࡞͕Մೳ w $36% w Ϋϥελʔɾϊʔυͷઃఆ w $MJFOU΋༷ʑͳݴޠͰఏڙ w +BWB w 1)1 w 1ZUIPO w 3VCZ w FUD

Slide 15

Slide 15 text

։ൃऀϑϨϯυϦʔᶄ %PDLFSΠϝʔδ͕ఏڙ͞Ε͍ͯͯ؆୯ʹར༻Մೳɻ ϩʔΧϧͰͷ։ൃ΍ݕূ΋%PDLFS্Ͱߦ͑·͢ɻ

Slide 16

Slide 16 text

։ൃऀϑϨϯυϦʔᶅ ݕࡧɾՄࢹԽ༻πʔϧͷ,JCBOBʹ͸ɺ&MBTUJDTFBSDIͷΤϯυϙΠϯτʹ؆୯ʹϦ ΫΤετՄೳͳ%FW5PPMT͕ఏڙ͞Ε͍ͯͯϒϥ΢β͔ΒΞΫηεՄೳ

Slide 17

Slide 17 text

શจݕࡧᶃ ෳ਺ͷจॻʢϑΝΠϧʣ͔ΒಛఆͷจࣈྻΛݕࡧ͢Δ͜ͱɻ ʮϑΝΠϧ໊ݕࡧʯ΍ʮ୯ҰϑΝΠϧ಺ͷจࣈྻݕࡧʯͱҟͳΓɺ ʮෳ਺จॻʹ·͕ͨͬͯɺจॻʹؚ·ΕΔશจΛର৅ͱͨ͠ݕࡧʯ ࢀߟ: https://ja.wikipedia.org/wiki/%E5%85%A8%E6%96%87%E6%A4%9C%E7%B4%A2

Slide 18

Slide 18 text

શจݕࡧᶄ ߴ଎ͳશจݕࡧΛ࣮ݱ͢ΔͨΊɺ&MBTUJDTFBSDIͰ͸ จॻొ࿥࣌ʹసஔΠϯσοΫεΛ࡞੒͠·͢ɻ

Slide 19

Slide 19 text

શจݕࡧᶅ ొ࿥͢ΔจॻΛݴޠॲཧʹֻ͚·͢ɻ ݴޠॲཧͰ͸ෆཁͳจࣈͷআڈ΍খจࣈʹ౷Ұͨ͠Γ͠ɺɹ ୯ޠຖʹ۠੾Γ·͢ɻ

Slide 20

Slide 20 text

શจݕࡧᶆ Ͳͷ୯ޠ͕Ͳͷจॻʹଘࡏ͢Δ͔ͷ ࡧҾ సஔΠϯσοΫε Λ࡞੒͠·͢ɻ

Slide 21

Slide 21 text

શจݕࡧᶇ ݕࡧ࣌ʹ͸ݕࡧϫʔυΛݴޠॲཧʹֻ͚ɺసஔΠϯσοΫε͔Β Ұக͢ΔจॻΛݟ͚ͭ·͢ɻ ୯ޠ୯ҐͰΠϯσοΫεΛ࡞੒͢ΔͨΊɺߴ଎ͳશจݕࡧ͕Մೳ

Slide 22

Slide 22 text

೔ຊޠܗଶૉղੳϓϥάΠϯ

Slide 23

Slide 23 text

೔ຊޠͷݴޠॲཧᶃ ӳޠ͸εϖʔε۠੾ΓͰ୯ޠΛநग़ͨ͠ΓݴޠॲཧΛߦ ͑·͕͢ɺ೔ຊޠͰ͸ҟͳΔରԠ͕ඞཁͰ͢ɻ 5IJTJTBQFOˠ5IJTJTBQFO ͜Ε͸ϖϯͰ͢ˠʁʁʁ

Slide 24

Slide 24 text

೔ຊޠͷݴޠॲཧᶄ ೔ຊޠ༻ͷݴޠղੳϓϥάΠϯ͕ఏڙ͞Εͯ·͢ɻ εϖʔε۠੾ΓͰ͸ͳ͘ɺࣙॻΛར༻ͨ͠ํ๏ʹͳΓ·͢ɻ ར༻ํ๏͸&MBTUJDTFBSDIͷσΟϨΫτϦͰɺίϚϯυ࣮ߦ ͢Δ͜ͱͰಋೖՄೳͰ͢ɻ $sudo bin/elasticsearch-plugin install analysis-kuromoji ࢀߟIUUQTXXXFMBTUJDDPHVJEFFOFMBTUJDTFBSDIQMVHJOTDVSSFOUBOBMZTJTLVSPNPKJIUNM

Slide 25

Slide 25 text

&MBTUJDTFBSDIͷϢʔεέʔε • WebαΠτͷݕࡧػೳ • ϩά΍Πϕϯτσʔλͷอଘɾ෼ੳ • ػցֶशͰϦΞϧλΠϜʹϞσϧΛ࡞੒ • ηΩϡϦςΟ෼ੳ • ஍ཧ৘ใΛར༻ͯ͠ͷۭؒݕࡧ • Ҩ఻ࢠσʔλͷ෼ੳ

Slide 26

Slide 26 text

Amazon OpenSearch Service

Slide 27

Slide 27 text

"NB[PO0QFO4FBSDI4FSWJDF w&MBTUJDTFBSDI͔ΒϑΥʔΫ wϚωʔδυαʔϏε wσϓϩΠ w؅ཧ wෳ਺ͷΫΤϦݴޠαϙʔτ wΠϕϯτϞχλϦϯάɾΞϥʔτ

Slide 28

Slide 28 text

"NB[PO0QFO4FBSDI4FSWJDF Ϛωδϝϯτίϯιʔϧ͔Β؆୯ʹ࡞੒͕ՄೳͰɺ֤छઃ ఆมߋ΍ϞχλϦϯάػೳ΋ఏڙ͞Ε͍ͯ·͢ɻ

Slide 29

Slide 29 text

prismatix ʹ͍ͭͯ

Slide 30

Slide 30 text

QSJTNBUJYͱ͸ w &$$3.ΦϜχνϟωϧ"1*ϓϥοτϑΥʔϜ w ϚΠΫϩαʔϏεΫϥ΢υωΠςΟϒ

Slide 31

Slide 31 text

QSJTNBUJYͱ͸ wൢചνϟωϧͱόοΫΦϑΟεγεςϜͷϋϒ ͱͯ͠Քಇ w&$$3.ʹඞཁͳػೳ܈Λબ୒తʹಋೖՄೳ wϏδωεͷن໛΍αʔϏεɾϑΣʔζʹ߹Θ ͤͯ࠷దͳߏ੒Ͱར༻Մೳ

Slide 32

Slide 32 text

Elasticsearchͷ׆༻ํ๏

Slide 33

Slide 33 text

Ϧιʔε QSJTNBUJYͷ"1*͸Ϧιʔεࢦ޲ΞʔΩςΫνϟ 30" ʹج͍ͮͯઃ ܭ͍ͯ͠·͢ɻ Ϧιʔεͱ͸"1*Λ௨ͯ͡ૢ࡞ՄೳͳॏཁੑΛ࣋ͬͨσʔλͰɺ 30"͸ϦιʔεΛத৺ʹߟ͑ΔΞʔΩςΫνϟͰ͢ɻ w ঎඼αʔϏε w ঎඼ϦιʔεࡏݿϦιʔε w ձһαʔϏε w ձһϦιʔε͓ؾʹೖΓϦιʔε

Slide 34

Slide 34 text

׆༻ํ๏ᶃ αΠτΛߏங͢Δࡍɺඥͮ͘େྔͷϦιʔεΛߜΓࠐΈͭ ͭɺར༻͍߲ͨ͠໨͚ͩΛऔಘ͢Δͷ͸େม w ঎඼໊ɾઆ໌จͰશจݕࡧ͍ͨ͠ w ϨϏϡʔ਺͕ཉ͍͠ w ࡏݿ͕ͳ͍঎඼͸ඇදࣔʹ͍ͨ͠ w ݕࡧʹҰக͢Δ঎඼਺ w ΧςΰϦҰཡ w FUD

Slide 35

Slide 35 text

׆༻ํ๏ᶄ QSJTNBUJYͰ͸֤Ϧιʔεૢ࡞࣌ʹ,JOFTJT%BUB4USFBNΛ ར༻ͯ͠ߴ଎ʹ&MBTUJDTFBSDI΁ू໿

Slide 36

Slide 36 text

׆༻ํ๏ᶅ &MBTUJDTFBSDI΁ͷݕࡧϦΫΤετΛ୅ߦ͢ΔݕࡧαʔϏε Λఏڙ͠ɺू໿ͨ͠ෳ਺ͷϦιʔεΛԣஅͯ͠ݕࡧ͕Մೳ ݕࡧαʔϏε "1*ϦΫΤετ ݕࡧϦΫΤετ Ϩεϙϯε Ϩεϙϯε

Slide 37

Slide 37 text

׆༻ํ๏ᶆ Ϧιʔεຖʹ༷ʑͳ"1*Λఏڙ͓ͯ͠Γɺར༻༻్ʹԠͨ͡ ϦΫΤετύϥϝʔλΛ༻ҙ ݕࡧαʔϏε ݕࡧϖʔδ ళฮϖʔδ ঎඼ϖʔδ ձһϖʔδ FUD FUD "1*ϦΫΤετ Ϩεϙϯε

Slide 38

Slide 38 text

͍͞͝ʹ

Slide 39

Slide 39 text

͍͞͝ʹ w &MBTUJDTFBSDIΛར༻͢Δ͜ͱͰߴ଎ͳશจݕࡧ͕Մೳ w %PDLFSΠϝʔδ͕഑෍͞Ε͍ͯΔͷͰڵຯ͋Δํ͸৮ͬͯΈͯ ͍ͩ͘͞ w IUUQTXXXEPDLFSFMBTUJDDPSFMBTUJDTFBSDI w &MBTUJDTFBSDIͷৄ͍͠ར༻ํ๏ʹ͍ͭͯ͸ެࣜαΠτ΍ฐࣾ ϒϩάΛ֬͝ೝ͍ͩ͘͞ w IUUQTXXXFMBTUJDDPHVJEFJOEFYIUNM w IUUQTEFWDMBTTNFUIPEKQUBHTFMBTUJDTFBSDI

Slide 40

Slide 40 text

No content