Elasticsearch5.0.0で再インデクシングの高速化を探究した話

&MBTUJDTFBSDIͰ ࠶ΠϯσΫγϯάͷߴ଎ԽΛ ୳ڀͨ͠࿩ ೥݄೔ୈճ&MBTUJDTFBSDIษڧձ αΠϘ΢ζɹ౉ล

ࣗݾ঺հ ໊લɿ౉ล੓ٛ ॴଐɿαΠϘ΢ζגࣜձࣾ  ɹɹɹΞϓϦέʔγϣϯج൫νʔϜ ࠓͷؾ࣋ͪɿۓுׂ̔ɺ຾ؾׂɺૣޱʹؾΛ͚ͭΔׂ +BWBͷνϡʔχϯά͔Β$44·Ͱجຊతʹ͸ԿͰ΋΍ΔνʔϜ  DZCP[VDPNͷϓϩμΫτڞ௨ػೳͷ։ൃ  ΞʔΩςΫνϟ࡮৽ϓϩδΣΫτͰ&MBTUJDTFBSDIͷௐࠪͳͲ

αΠϘ΢ζ w ৘ใڞ༗8FCΞϓϦέʔγϣϯ άϧʔϓ΢ΣΞ ͷ։ൃ w DZCP[VDPNͱ͍͏Ϋϥ΢υαʔϏεΛఏڙ w ݕࡧγεςϜ͸ओʹશจݕࡧͱͯ͠ར༻ w
ҰͭͷϓϩμΫτͷଉ͕௕͍ w ݕࡧγεςϜͷόʔδϣϯΞοϓ΍τʔΫφΠβมߋͰ ࠶ΠϯσΫγϯάʹ͕͔͔࣌ؒΔͷ͕ͭΒ͔ͬͨ

&MBTUJDTFBSDIͱͷؔΘΓ w ΞʔΩςΫνϟ࡮৽ϓϩδΣΫτͷҰ؀Ͱௐࠪ w ࣍ظݕࡧج൫ͱͯ͠طଘͷ໰୊఺ΛղফͰ͖Δ͔ w ৐Γ׵͑ํ๏Λݕ౼ͨ͠Γϓϩτ࣮૷ͯ͠Δஈ֊ w ࣮ӡ༻͸·͍ͩͯ͠ͳ͍

࠷ۙϒϩάॻ͖·ͨ͠ w &MBTUJDTFBSDIͷ*OHFTU/PEF༻ϓϥάΠϯ Λॻ͍ͨ࿩ w &MBTUJDTFBSDIͰ࠶ΠϯσΫγϯάͷߴ଎Խ Λ୳ٻ͢Δ

࠷ۙϒϩάॻ͖·ͨ͠ w &MBTUJDTFBSDIͷ*OHFTU/PEF༻ϓϥάΠϯ Λॻ͍ͨ࿩ w &MBTUJDTFBSDIͰ࠶ΠϯσΫγϯάͷߴ଎Խ Λ୳ٻ͢Δ ࠓ೔͸ͬͪ͜ͷ࿩Λ͠·͢

͜ͷεϥΠυͷ໨త ࠶ΠϯσΫγϯάͷߴ଎Խʹ࢖͑Δ  ز͔ͭͷUJQTΛ఻͑Δ͜ͱ

࠶ΠϯσΫγϯά w ࠶ΠϯσοΫεͱ͔3FJOEFYͱ͔ݺͼํ͍Ζ͍Ζ w ͋ΔΠϯσοΫεͷυΩϡϝϯτΛผͷΠϯσοΫ εʹΠϯσΫγϯά͠௚͢͜ͱ w ͔Β3FJOEFY"1*͕Ͱ͖͓ͯखܰʹ࣮ࢪՄೳ w Ϛοϐϯάͷมߋ΍-VDFOFͷόʔδϣϯΞοϓͳ
ͲͰඞཁ

ࠓճͷߴ଎Խͷର৅ w @TPVSDFΛ࢖ͬͯɺ͋ΔΠϯσοΫε TSDJOEFY ͷ  υΩϡϝϯτΛผͷΠϯσοΫε EFTUJOEFY ʹ  ΠϯσΫγϯά͢Δ w
ൈ͖ग़ͨ͠υΩϡϝϯτͷՃ޻͸͠ͳ͍ w ΞφϥΠβͷมߋ΍γϟʔυ਺ͷมߋͳͲΛ૝ఆ

Ξϓϩʔν w 3FJOEFY"1*ͱಉ༷ͷॲཧΛ͢Δ3FJOEFY5PPMΛ࡞ ੒ͯ͠ࢼߦࡨޡ͢Δʢ4DSPMM"1*ͱ#VML"1*Λར༻ʣ w &MBTUJDTFBSDIͷػೳ΍࠶ΠϯσΫγϯά༻ͷπʔ ϧͷ࣮ߦͷฒྻ౓ΛߴΊͯɺΫϥελͷϦιʔεΛ ࢖͍͖ΕΔΑ͏ʹ͢Δ w &MBTUJDTFBSDIͷઃఆύϥϝʔλͷνϡʔχϯάʹ
͸ݴٴ͠·ͤΜ

3FJOEFY5PPM

4DSPMM"1* w ݕࡧ݁ՌͷεφοϓγϣοτΛͱΓɺͦͷઌ಄͔ΒυΩϡ ϝϯτΛ4DSPMM4J[Fຖʹஞ࣍తʹऔಘ͢ΔखஈΛఏڙ͢Δ ʢ3%#ͷΧʔιϧͷΑ͏ͳ΋ͷʣ w ૯υΩϡϝϯτ਺ɺ4DSPMM4J[Fͷ৔߹

#VML"1* w υΩϡϝϯτͷߋ৽΍࡟আΛϦΫΤετͰෳ਺࣮ߦՄೳ w BDUJPO@BOE@NFUB@EBUBͷߦͱPQUJPOBM@TPVSDFͷߦΛ࿈ ଓͯ͠ॻ͘ w ࠓճ͸ෳ਺ߋ৽ͳͷͰɺJOEFYBDUJPOͱTPVSDFΛଓ͚Δ $
curl -XPOST ‘es-watanabe-1:9200/_bulk’ -d ' { "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } } { "field1" : "value1" } { "index" : { "_index" : "test", "_type" : "type1", "_id" : "2" } } { "field1" : "value2" } { "index" : { "_index" : "test", "_type" : "type1", "_id" : "3" } } { "field1" : "value3" } '

Ϋϥελߏ੒ w ҎԼͷεϖοΫͷϚγϯ୆ͷΫϥελ w &MBTUJDTFBSDIͷόʔδϣϯ͸BMQIB ཁૉ ੑೳ උߟ $16
()[ DPSF UISFBET .FNPSZ (# &MBTUJDTFBSDIͷYNY͸(# )%%TFRXSJUF .#T EEͰ؆қܭଌ )%%TFRSFBE .#T IEQBSNͰ؆қܭଌ /FUXPSL (CQT JQFSGͰܭଌ 04 6CVOUV ˞)%%TFRXSJUF͸3"*%ͷޮՌ౳Ͱߴ଎ʹಈ࡞͍ͯ͠ΔΑ͏ʹݟ͑·͕͢ɺ ࣮ࡍ͸ϥϯμϜϥΠτͩͬͨΓͰ͜Μͳ਺ࣈ͸Ͱͳ͍ͷͰɺࢀߟఔ౓ʹߟ͍͑ͯͩ͘͞ɻ

࣮ݧσʔλ w ೔ຊޠXJLJQFEJBهࣄͷΞʔΧΠϒ w TUSFBNFTͰYNMΛΠϯϙʔτ w υΩϡϝϯτ਺ɿ໿ສ݅ w σʔλαΠζɿ(#

NBQQJOHTTFUUJOHT w @BMMEJTBCMF w ϨϓϦΧ਺ w γϟʔυ਺ w SFGSFTI@JOUFSWBM w
JOEFYNFSHFTDIFEVMFSNBY@UISFBE@DPVOU DGIUUQTXXXFMBTUJDDPHVJEFFOFMBTUJDTFBSDIHVJEFDVSSFOUJOEFYJOHQFSGPSNBODFIUNM

NBQQJOHTTFUUJOHT { "settings": { "number_of_replicas": 0, "number_of_shards": 16, "index.refresh_interval":
"-1", "index.merge.scheduler.max_thread_count" : 1 }, "mappings": { "_default_": { "_all": {"enabled": false } } } }

ଌఆํ๏ w ࠶ΠϯσΫγϯά։͔࢝Β׬ྃ·Ͱͷ࣌ؒΛܭଌ w ETUBUͷग़ྗΛ໨ࢹͯ͠ϘτϧωοΫΛ୳Δ w DBUUISFBE@QPPM"1*ͷ݁ՌΛඵִؒͰݟͯɺ &MBTUJDTFBSDIͷฒྻॲཧͷঢ়ଶΛ֬ೝ

ETUBUUDMNESO

DBUUISFBE@QPPM"1* w ϊʔυͷUISFBE@QPPMͷTUBUTΛ֬ೝ͢Δ"1* w UISFBE@QPPMͷछྨΛࢦఆͰ͖Δ  CVML JOEFY TFBSDI SFGSFTI FUDʜ
w UISFBE@QPPMͷঢ়ଶΛ֬ೝͰ͖Δ  BDUJWF RVFVF TJ[F SFKFDUFE

DBUUISFBE@QPPM"1* $ curl 'es-watanabe-1:9200/_cat/thread_pool? v&h=id,host,bt,ba,bq,bs,bc,st,sa,sq,ss,sc' id host bt ba bq
bs bc st sa sq ss sc vM4E es-watanabe-1 fixed 0 0 24 288 fixed 0 0 37 584 btge es-watanabe-2 fixed 0 0 24 13524 fixed 0 0 37 24143 zPof es-watanabe-3 fixed 0 0 24 13497 fixed 0 0 37 24115 3y0A es-watanabe-4 fixed 0 0 24 13609 fixed 0 0 37 24226 IyM- es-watanabe-5 fixed 0 0 24 12979 fixed 0 0 37 24616 iXmJ es-watanabe-6 fixed 0 0 24 13538 fixed 0 0 37 25176 iBLd es-watanabe-7 fixed 0 0 24 13388 fixed 0 0 37 25029 cKjt es-watanabe-8 fixed 0 0 24 13539 fixed 0 0 37 25173 CUCVMLUZQF CBCVMLBDUJWF CRCVMLRVFVF CTCVMLTJ[F CDCVMLDPNQMFUFE TUTFBSDIUZQF TBTFBSDIBDUJWF TRTFBSDIRVFVF TTTFBSDITJ[F TDTFBSDIDPNQMFUFE

DBUUISFBE@QPPM"1* $ curl 'es-watanabe-1:9200/_cat/thread_pool? v&h=id,host,bt,ba,bq,bs,bc,st,sa,sq,ss,sc' id host bt ba bq
bs bc st sa sq ss sc vM4E es-watanabe-1 fixed 0 0 24 288 fixed 0 0 37 584 btge es-watanabe-2 fixed 0 0 24 13524 fixed 0 0 37 24143 zPof es-watanabe-3 fixed 0 0 24 13497 fixed 0 0 37 24115 3y0A es-watanabe-4 fixed 0 0 24 13609 fixed 0 0 37 24226 IyM- es-watanabe-5 fixed 0 0 24 12979 fixed 0 0 37 24616 iXmJ es-watanabe-6 fixed 0 0 24 13538 fixed 0 0 37 25176 iBLd es-watanabe-7 fixed 0 0 24 13388 fixed 0 0 37 25029 cKjt es-watanabe-8 fixed 0 0 24 13539 fixed 0 0 37 25173 CUCVMLUZQF CBCVMLBDUJWF CRCVMLRVFVF CTCVMLTJ[F CDCVMLDPNQMFUFE TUTFBSDIUZQF TBTFBSDIBDUJWF TRTFBSDIRVFVF TTTFBSDITJ[F TDTFBSDIDPNQMFUFE CVML༻ͷεϨου͸࠷େͰ·Ͱಉ࣮࣌ߦՄೳ

୳ڀͷྲྀΕ w 3FJOEFY"1* w 4DSPMM"1*ͱ#VML"1*ΛผεϨουԽ w #VML"1*ͷฒྻԽ w 4DSPMM"1*ͷฒྻԽ w
3FJOEFY5PPMͷ෼ࢄ࣮ߦ w 44%Λ࢖͏

3FJOEFY"1* w 4DSPMM4J[FΛม࣮͑ͯߦ଎౓Λൺֱ w ҎԼͷίϚϯυΛΫϥελ֎ͷϚγϯ্Ͱ࣮ߦ $ curl -X POST
'es-watanabe-1:9200/_reindex' -d '{ "source":{"index":"src", "size": 1000}, "dest" :{"index":"dest"} }'

3FJOEFY"1* w ࠷଎Ͱ໿ඵ w $166TBHF਺ˋ w σΟεΫॻ͖ࠐΈ਺.# w DBUUISFBE@QPPMͷCVMLBDUJWF͸ ɺఔ౓
w ಺෦Ͱ͸4DSPMM"1*ͱ#VML"1*Λ ஞ࣍తʹ࣮ߦ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ

3FJOEFY"1* w ࠷଎Ͱ໿ඵ w $166TBHF਺ˋ w σΟεΫॻ͖ࠐΈ਺.# w DBUUISFBE@QPPMͷCVMLBDUJWF͸ ɺఔ౓
w ಺෦Ͱ͸4DSPMM"1*ͱ#VML"1*Λ ஞ࣍తʹ࣮ߦ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ Ϧιʔε΋ͬͱ࢖͍౗ͤΔ͸ͣ

4DSPMM"1*ͱ#VML"1*ͷผεϨουԽ w 3FJOEFY"1*͸4DSPMM"1*ͱ#VML"1*Λஞ࣮࣍ߦ w 3FJOEFY5PPMΛ࡞੒ͯ͠ɺͦΕΒΛผεϨουͰ ಈ͔ͨ͠Β଎ͦ͏

4DSPMM"1*ͱ#VML"1*ͷผεϨουԽ w υΩϡϝϯτͷΩϡʔΛಋೖ w 4DSPMM"1*εϨου͸ͻͨ͢Β ΩϡʔʹυΩϡϝϯτΛੵΉ w #VML"1*εϨου͸Ωϡʔ͔ ΒυΩϡϝϯτΛऔΓग़ͯ͠Π ϯσΫγϯά͢Δ

4DSPMM"1*ͱ#VML"1*ͷผεϨουԽ w ஗͘ͳͬͨʂ w ࠷଎Ͱ໿ඵ w Ϧιʔε࢖༻ྔ͸3FJOEFY"1*ͱେࠩແ͠ w πʔϧಋೖʹΑΔෛՙ͕ฒྻԽͷϝϦο τΛେ෯ʹ্ճΔ
w ·ͩ߄ͯΔ࣌ؒͰ͸ͳ͍ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ

4DSPMM"1*ͱ#VML"1*ͷผεϨουԽ w ஗͘ͳͬͨʂ w ࠷଎Ͱ໿ඵ w Ϧιʔε࢖༻ྔ͸3FJOEFY"1*ͱେࠩແ͠ w πʔϧಋೖʹΑΔෛՙ͕ฒྻԽͷϝϦο τΛେ෯ʹ্ճΔ
w ·ͩ߄ͯΔ࣌ؒͰ͸ͳ͍ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ &MBTUJDTFBSDI͔Β෼཭ͨ͜͠ͱͰ  "1*ͷฒྻ࣮ߦͳͲ͸͠΍͘͢ͳͬͨ

#VML"1*ͷฒྻԽ w UISFBE@QPPMͰCVMLBDUJWF͸͕."9 w ࣮ࡍʹ͸ ఔ౓͔͠࢖͍͑ͯͳ͍ w #VML"1*ΑΓ͸4DSPMM"1*ͷํ͕ॲཧ͕ૣ͍ɻ  ͭ·ΓɺΩϡʔʹ͸ະॲཧͷυΩϡϝϯτ͕ཷ·Δ w
#VML"1*ͷϦΫΤετͰCVMLBDUJWFͷεϨου͸ɺϊʔυ಺ͷ γϟʔυ਺·Ͱ૿͑ΔՄೳੑ͕͋Δ w TIBSEOPEFͳͷͰɺBDUJWFUISFBETCVMLBQJSFRVFTU w CVMLBDUJWF͸͕."9ͳͷͰɺഒ͸ฒྻʹ࣮ߦͯ͠΋ྑͦ͞͏

#VML"1*ͷฒྻԽ w #VML"1*Λฒྻ࣮ߦ਺ΛࢦఆՄೳ ʢ#VMLฒྻ౓ʣ w ΩϡʔʹੵΉυΩϡϝϯταΠζ ΛࢦఆՄೳʢ#VML4J[Fʣ w #VMLฒྻ౓ #VML4J[F
4DSPMM4J[F ΛมԽͤͯ͞ܭଌ

#VML"1*ͷฒྻԽ w ࠷଎Ͱඵʢ3FJOEFY"1*ͷ໿ഒ଎ʣ w 4DSPMM"1*ͱ#VML"1*Λड͚෇͚ΔϊʔυͰ͸ఔ౓ͷ$16࢖༻཰ w ͦΕҎ֎ͷϊʔυ͸ఔ౓ͷ$16࢖༻཰ w σΟεΫॻ͖ࠐΈ͸ʙ.#T w
UISFBEQPPMͷCVMLBDUJWF͸ʹಧ͔ͳ͍ఔ౓ #VMLฒྻ౓ #VML4J[F 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ ඵ

#VML"1*ͷฒྻԽ w ࠷଎Ͱඵʢ3FJOEFY"1*ͷ໿ഒ଎ʣ w 4DSPMM"1*ͱ#VML"1*Λड͚෇͚ΔϊʔυͰ͸ఔ౓ͷ$16࢖༻཰ w ͦΕҎ֎ͷϊʔυ͸ఔ౓ͷ$16࢖༻཰ w σΟεΫॻ͖ࠐΈ͸ʙ.#T w
UISFBEQPPMͷCVMLBDUJWF͸ʹಧ͔ͳ͍ఔ౓ #VMLฒྻ౓ #VML4J[F 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ ඵ Ϧιʔεతʹ͸  ·ͩ·͍͚ͩͦ͏

4DSPMM"1*ͷฒྻԽ w #VML"1*ΛฒྻԽͨ͠ͳΒ4DSPMM"1*΋ฒྻԽ͍ͨ͠ w ௨ৗɺ4DSPMM"1*͸ஞ࣍తʹΧʔιϧΛճ͢ඞཁ͕͋Δͷ ͰɺฒྻԽͰ͖ͳ͍ w ͔Β͸4DSPMM"1*Λฒྻʹ࣮ߦ͢ΔͨΊͷ4MJDFE 4DSPMMͱ͍͏ػೳ͕͋Δ

4MJDFE4DSPMM w εφοϓγϣοτΛࢦఆͨ͠਺ͷ4MJDFʹ෼ׂ w ݸʑͷ4MJDFΛผʑͷ4DSPMMͱͯ͠ར༻Ͱ͖Δ w 4MJDF਺Λͱͨ͠৔߹ɺͭͷ4DSPMMʹ෼ׂ͞ΕɺͦΕͧΕʹ ͱ͍͏ࣝผࢠ͕༩͑ΒΔ

4MJDFE4DSPMM w εφοϓγϣοτऔಘ࣌ʹTMJDFͷJEΛࢦఆ͢Δ w TMJDF਺͸NBYύϥϝʔλͰࢦఆ͢Δ w TMJDF਺͕ͷ৔߹ɺTMJDFͱTMJDFΛผεϨουͰѻ͏͜ͱͰฒྻԽՄೳ # Thread1
curl -XGET 'localhost:9200/src/_search?scroll=1m' -d '{ "slice": { "id": 0, "max": 2 } }' # Thread2 curl -XGET 'localhost:9200/src/_search?scroll=1m' -d '{ "slice": { "id": 1, "max": 2 } }'

4DSPMM"1*ͷฒྻԽ w 4MJDFE4DSPMMʹΑΓɺ4DSPMM"1* Λฒྻʹ࣮ߦՄೳ w 4MJDF਺͸ #VMLฒྻ౓͸ͱ ͯ͠ #VML4J[F 4DSPMM4J[FΛมԽ
ͤͯ͞ܭଌ w 4MJDF਺͸TIBSE਺ΑΓ૿΍͢ͱ஗ ͘ͳΔͱ͍͏ϦϑΝϨϯεʹै͍ɺ TIBSE਺ͷಉ਺ͷͰݻఆ

4DSPMM"1*ͷฒྻԽ w ࠷଎Ͱඵʢ3FJOEFY"1*ͷ໿ഒ଎ʣ w 4DSPMM"1*ͱ#VML"1*Λड͚෇͚ΔϊʔυͰ͸ఔ౓ͷ$16࢖༻཰ w ͦΕҎ֎ͷϊʔυ͸ఔ౓ͷ$16࢖༻཰ w σΟεΫॻ͖ࠐΈ͸ʙ.#T w
UISFBEQPPMͷCVMLBDUJWF͸ʹಧ͔ͳ͍ఔ౓ #VMLฒྻ౓ #VML4J[F 4MJDF਺ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ

4DSPMM"1*ͷฒྻԽ w ࠷଎Ͱඵʢ3FJOEFY"1*ͷ໿ഒ଎ʣ w 4DSPMM"1*ͱ#VML"1*Λड͚෇͚ΔϊʔυͰ͸ఔ౓ͷ$16࢖༻཰ w ͦΕҎ֎ͷϊʔυ͸ఔ౓ͷ$16࢖༻཰ w σΟεΫॻ͖ࠐΈ͸ʙ.#T w
UISFBEQPPMͷCVMLBDUJWF͸ʹಧ͔ͳ͍ఔ౓ #VMLฒྻ౓ #VML4J[F 4MJDF਺ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ଎͘͸ͳ͕ͬͨ  Ϧιʔεతʹ͸େࠩͳ͠

3FJOEFY5PPMͷ෼ࢄ࣮ߦ w "1*ͷฒྻԽ͸΋͏͓ෲ͍ͬͺ͍ w ΍Γ͍ͨ͜ͱ͸TSDΛEFTUʹΠϯσΫγϯά͢Δ͚ͩ w TSDͱEFTUͷTIBSEJEΛϊʔυ಺Ͱἧ͑ͯɺϊʔυͷϩʔΧ ϧTIBSEͷσʔλΛ࠶ΠϯσΫγϯά͢Ε͹ྑ͍ͷͰ͸ʁ

3FJOEFY5PPMͷ෼ࢄ࣮ߦ w TIBSEJE͸$MVTUFS3FSPVUF"1*Λ࢖ͬͯἧ͑Δ w ֤ϊʔυ͕อ࣋͢ΔTIBSEJE͸4FBSDI4IBSE"1*Ͱऔಘ w औಘͨ͠TIBSEJEΛ4FBSDI"1*ͷQSFGFSFODFύϥϝʔλʹࢦ ఆ͢ΔͱɺϩʔΧϧTIBSEͷυΩϡϝϯτͷΈΛऔಘͰ͖Δ curl
-XGET ‘localhost:9200/src/_search?scroll=1m&preference=_shards:1,2'

3FJOEFY5PPMͷ෼ࢄ࣮ߦ w 3FJOEFY5PPMΛϊʔυ্ͰϩʔΧϧʹ࣮ߦ w 4MJDF਺͸ #VMLฒྻ౓͸ͱͯ͠  #VML4J[F 4DSPMM4J[FΛมԽͤͯ͞ܭଌ

3FJOEFY5PPMͷ෼ࢄ࣮ߦ w ࠷଎Ͱඵʢ3FJOEFY"1*ͷ໿ഒ଎ʣ w ʙఔ౓ͷ$16࢖༻཰ w σΟεΫॻ͖ࠐΈ͸ʙ.#T w UISFBEQPPMͷCVMLBDUJWF͸΄΅Λࣔ͢
#VMLฒྻ౓ #VML4J[F 4MJDF਺ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ

3FJOEFY5PPMͷ෼ࢄ࣮ߦ w ࠷଎Ͱඵʢ3FJOEFY"1*ͷ໿ഒ଎ʣ w ʙఔ౓ͷ$16࢖༻཰ w σΟεΫॻ͖ࠐΈ͸ʙ.#T w UISFBEQPPMͷCVMLBDUJWF͸΄΅Λࣔ͢
#VMLฒྻ౓ #VML4J[F 4MJDF਺ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ଎͘ͳͬͨ͠  Ϧιʔε΋࢖͍౗ͤͯΔʂ

TIBSEJEΛἧ͑ͳ͍৔߹ w TIBSE਺Λมߋ͢ΔϢʔεέʔεΛ૝ఆ w 3FJOEFY5PPM͸֤ϊʔυ্Ͱ࣮ߦ͢ΔͷͰॲཧͷ෼ࢄ͸͞ΕΔ w ϩʔΧϧͰͷ࠶ΠϯσΫγϯάͰ͸ͳ͍ͷͰ௨৴͸ൃੜ͢Δ w 4MJDF਺͸ #VMLฒྻ౓͸ͱͯ͠
#VML4J[F 4DSPMM4J[FΛ มԽͤͯ͞ܭଌ

TIBSEJEΛἧ͑ͳ͍৔߹ w ࠷଎Ͱඵ w एׯ஗͍΋ͷͷϩʔΧϧͷ৔߹ͱେࠩͳ͠ w Ϧιʔεফඅ΋େࠩͳ͠ w QSFGFSFODFࢦఆʹΑΔπʔϧͷ෼ࢄͷޮՌ͕େ͖͘ɺϩʔΧϧͰ ͋Δ͜ͱ͸ͦΕ΄ͲॏཁͰ͸ͳ͍πʔϧͷ࣮૷͕؆୯ʹͳΔ
#VMLฒྻ౓ #VML4J[F 4MJDF਺ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ

44%Λ࢖͏ w /7.F44%ΛੵΜͩϚγϯ͕͋ͬͨͷͰܭଌͯ͠ Έͨɻػࡐͷ౎߹্$16΋εέʔϧΞοϓ w ߏ੒͸࠷଎ͷ3FJOEFY5PPMͷ෼ࢄ࣮ߦͱಉҰ w $16ͱσΟεΫͷεϖοΫ͸ͪ͜Β ཁૉ
ੑೳ උߟ $16 ()[ DPSF UISFBET 44%TFRXSJUF (#T EEͰ؆қܭଌ 44%TFRSFBE .#T IEQBSNͰ؆қܭଌ

44%Λ࢖͏ w ࠷଎Ͱඵ w ଎͘͸ͳ͚ͬͨͲे਺ඵͰऴΘͬͯ͠·͏ͷͰɺπʔϧͷ্ཱͪ͛ ॲཧʢεϨουىಈͳͲʣͷ࣌ؒൺ཰͕ߴͦ͏ w ࠶ΠϯσΫγϯάͷ͕࣌ؒਖ਼͘͠ܭଌͰ͖͍ͯΔ͔ո͍͠ #VMLฒྻ౓
#VML4J[F 4MJDF਺ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ

υΩϡϝϯτΛഒʹෳ੡ w υΩϡϝϯτ਺Λ૿΍ͤ͹࠶ΠϯσΫγϯάࣗମͷॲཧ࣌ؒͷൺ ཰Λ্͛ΒΕͦ͏ w υΩϡϝϯτΛഒʹෳ੡ͯ͠ܭଌͨ͠ w ສυΩϡϝϯτɺ(# w Ϛγϯ͕εέʔϧΞοϓͯ͠ΔͷͰγϟʔυ਺΋มߋͯ͠Έͨ

υΩϡϝϯτΛഒʹෳ੡ w ࠷଎Ͱඵ w ໿.#TʢສઍEPDTTʣఔ౓ͷ଎౓ w (#T΋ݟ͖͑ͯͯເ͕޿͕Δ w )%%ͰͷϚγϯͰͷສυΩϡϝϯτͷ࣮ߦ࣌ؒ͸ඵͩͬ ͨͷͰɺ44%ԽͰഒ͙Β͍଎͘ͳͬͨ
γϟʔυ਺ #VMLฒྻ౓ #VML4J[F 4MJDF਺ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ

࣮ߦ࣌ؒൺֱ ߏ੒ ࣮ߦ଎౓ ଎౓ഒ཰ 3FJOEFY"1* ඵ 4DSPMM"1*ͱ#VML"1*ΛผεϨουԽ ඵ
#VML"1*ΛฒྻԽ ඵ 4DSPMM"1*ΛฒྻԽ ඵ 3FJOEFY5PPMͷ෼ࢄ࣮ߦ ඵ 44%Λ࢖͏ ඵ w #VML"1*ͷฒྻԽɺ3FJOEFY5PPMͷ෼ࢄ࣮ߦͷޮՌ͕ߴ͍ w ϊʔυ૿΍ͤ͹εέʔϧ͢Δ3FJOEFY5PPMͷ෼ࢄ࣮ߦ͕͓͢͢Ί w #VML"1*ͷฒྻԽ͕Ͱ͖Ε͹4DSPMM"1*ͷฒྻԽ΋େͨ͠खؒͰ ͸ͳ͍ͷͰɺ࠷ޙͷ٧ΊʹͲ͏ͧ ˞্͔ΒॱʹྦྷੵͰվળΛద༻͍ͯ͠·͢

஫ҙࣄ߲ w Ϣʔβʔૢ࡞ͷݕࡧϦΫΤετΛःஅͰ͖ͳ͍Α͏ ͳঢ়گͰ͸ɺϦιʔεΛ࢖͍੾Βͳ͍Α͏ʹ͢Δ w #VMLฒྻ౓Λ্͛͗͢Δͱɺ#VML"1*λεΫͷΩϡʔ ͕ᷓΕͯSFKFDU͕ൃੜ͢Δɻ݁ՌɺͦͷλεΫ͕ࣺ ͯΒΕͯυΩϡϝϯτͷίϐʔ͕ෆ׬શʹͳΔ w અ౓ͷ͋ΔϦιʔεར༻Λ৺͕͚·͠ΐ͏

·ͱΊ w ಛఆϢʔεέʔεʹ͓͚Δ࠶ΠϯσΫγϯάͷߴ଎ ԽΛ୳ٻͨ͠ w 4DSPMM"1*ͱ#VML"1*Λ࢖͍౗͢͜ͱͰɺ 3FJOEFY"1*ͱൺֱͯ͠ഒҎ্ߴ଎ʹͳͬͨ w ૣ͘ͷBMQIB͕ͱΕΔͱ͍͍ͳɻนʛТŋ ŧŽŕ

8FBSFIJSJOH w ΞϓϦέʔγϣϯج൫νʔϜʢେମͳΜͰ΋΍Δ΍ͭΒʣ w ϓϩμΫτ։ൃνʔϜ w 43&νʔϜ w ੜ࢈ੑ޲্νʔϜ w
ଞʹ΋৭ʑνʔϜ͋Γ·͢ w ΨπΨπ͸ͯ͠ͳ͍͚Ͳಇ͖΍͍͢ͱࢥ͏ʢࢲݟʣ w ͝ڵຯ͋Ε͹࠾༻ϖʔδݟͯΈ͍ͯͩ͘͞  IUUQDZCP[VDPKQDPNQBOZKPCSFDSVJUNFOUMJTU

͓͠·͍ ͝੩ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ

Elasticsearch5.0.0で再インデクシングの高速化を探究した話

Elasticsearch5.0.0で再インデクシングの高速化を探究した話

More Decks by mwatanabe

Other Decks in Technology

Featured

Transcript