Share
Embed
Transcript
56 slides
-
αΠϘζ w ใڞ༗8FCΞϓϦέʔγϣϯ άϧʔϓΣΞ ͷ։ൃ w DZCP[VDPNͱ͍͏ΫϥυαʔϏεΛఏڙ w ݕࡧγεςϜओʹશจݕࡧͱͯ͠ར༻ w
ҰͭͷϓϩμΫτͷଉ͕͍ w ݕࡧγεςϜͷόʔδϣϯΞοϓτʔΫφΠβมߋͰ ࠶ΠϯσΫγϯάʹ͕͔͔࣌ؒΔͷ͕ͭΒ͔ͬͨ -
ࠓճͷߴԽͷର w @TPVSDFΛͬͯɺ͋ΔΠϯσοΫε TSDJOEFY ͷ υΩϡϝϯτΛผͷΠϯσοΫε EFTUJOEFY ʹ ΠϯσΫγϯά͢Δ w
ൈ͖ग़ͨ͠υΩϡϝϯτͷՃ͠ͳ͍ w ΞφϥΠβͷมߋγϟʔυͷมߋͳͲΛఆ -
#VML"1* w υΩϡϝϯτͷߋ৽আΛϦΫΤετͰෳ࣮ߦՄೳ w BDUJPO@BOE@NFUB@EBUBͷߦͱPQUJPOBM@TPVSDFͷߦΛ࿈ ଓͯ͠ॻ͘ w ࠓճෳߋ৽ͳͷͰɺJOEFYBDUJPOͱTPVSDFΛଓ͚Δ $
curl -XPOST ‘es-watanabe-1:9200/_bulk’ -d ' { "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } } { "field1" : "value1" } { "index" : { "_index" : "test", "_type" : "type1", "_id" : "2" } } { "field1" : "value2" } { "index" : { "_index" : "test", "_type" : "type1", "_id" : "3" } } { "field1" : "value3" } ' -
Ϋϥελߏ w ҎԼͷεϖοΫͷϚγϯͷΫϥελ w &MBTUJDTFBSDIͷόʔδϣϯBMQIB ཁૉ ੑೳ උߟ $16
()[ DPSF UISFBET .FNPSZ (# &MBTUJDTFBSDIͷYNY(# )%%TFRXSJUF .#T EEͰ؆қܭଌ )%%TFRSFBE .#T IEQBSNͰ؆қܭଌ /FUXPSL (CQT JQFSGͰܭଌ 04 6CVOUV ˞)%%TFRXSJUF3"*%ͷޮՌͰߴʹಈ࡞͍ͯ͠ΔΑ͏ʹݟ͑·͕͢ɺ ࣮ࡍϥϯμϜϥΠτͩͬͨΓͰ͜ΜͳࣈͰͳ͍ͷͰɺࢀߟఔʹߟ͍͑ͯͩ͘͞ɻ -
NBQQJOHTTFUUJOHT w @BMMEJTBCMF w ϨϓϦΧ w γϟʔυ w SFGSFTI@JOUFSWBM w
JOEFYNFSHFTDIFEVMFSNBY@UISFBE@DPVOU DGIUUQTXXXFMBTUJDDPHVJEFFOFMBTUJDTFBSDIHVJEFDVSSFOUJOEFYJOHQFSGPSNBODFIUNM -
NBQQJOHTTFUUJOHT { "settings": { "number_of_replicas": 0, "number_of_shards": 16, "index.refresh_interval":
"-1", "index.merge.scheduler.max_thread_count" : 1 }, "mappings": { "_default_": { "_all": {"enabled": false } } } } -
DBUUISFBE@QPPM"1* w ϊʔυͷUISFBE@QPPMͷTUBUTΛ֬ೝ͢Δ"1* w UISFBE@QPPMͷछྨΛࢦఆͰ͖Δ CVML JOEFY TFBSDI SFGSFTI FUDʜ
w UISFBE@QPPMͷঢ়ଶΛ֬ೝͰ͖Δ BDUJWF RVFVF TJ[F SFKFDUFE -
DBUUISFBE@QPPM"1* $ curl 'es-watanabe-1:9200/_cat/thread_pool? v&h=id,host,bt,ba,bq,bs,bc,st,sa,sq,ss,sc' id host bt ba bq
bs bc st sa sq ss sc vM4E es-watanabe-1 fixed 0 0 24 288 fixed 0 0 37 584 btge es-watanabe-2 fixed 0 0 24 13524 fixed 0 0 37 24143 zPof es-watanabe-3 fixed 0 0 24 13497 fixed 0 0 37 24115 3y0A es-watanabe-4 fixed 0 0 24 13609 fixed 0 0 37 24226 IyM- es-watanabe-5 fixed 0 0 24 12979 fixed 0 0 37 24616 iXmJ es-watanabe-6 fixed 0 0 24 13538 fixed 0 0 37 25176 iBLd es-watanabe-7 fixed 0 0 24 13388 fixed 0 0 37 25029 cKjt es-watanabe-8 fixed 0 0 24 13539 fixed 0 0 37 25173 CUCVMLUZQF CBCVMLBDUJWF CRCVMLRVFVF CTCVMLTJ[F CDCVMLDPNQMFUFE TUTFBSDIUZQF TBTFBSDIBDUJWF TRTFBSDIRVFVF TTTFBSDITJ[F TDTFBSDIDPNQMFUFE -
DBUUISFBE@QPPM"1* $ curl 'es-watanabe-1:9200/_cat/thread_pool? v&h=id,host,bt,ba,bq,bs,bc,st,sa,sq,ss,sc' id host bt ba bq
bs bc st sa sq ss sc vM4E es-watanabe-1 fixed 0 0 24 288 fixed 0 0 37 584 btge es-watanabe-2 fixed 0 0 24 13524 fixed 0 0 37 24143 zPof es-watanabe-3 fixed 0 0 24 13497 fixed 0 0 37 24115 3y0A es-watanabe-4 fixed 0 0 24 13609 fixed 0 0 37 24226 IyM- es-watanabe-5 fixed 0 0 24 12979 fixed 0 0 37 24616 iXmJ es-watanabe-6 fixed 0 0 24 13538 fixed 0 0 37 25176 iBLd es-watanabe-7 fixed 0 0 24 13388 fixed 0 0 37 25029 cKjt es-watanabe-8 fixed 0 0 24 13539 fixed 0 0 37 25173 CUCVMLUZQF CBCVMLBDUJWF CRCVMLRVFVF CTCVMLTJ[F CDCVMLDPNQMFUFE TUTFBSDIUZQF TBTFBSDIBDUJWF TRTFBSDIRVFVF TTTFBSDITJ[F TDTFBSDIDPNQMFUFE CVML༻ͷεϨου࠷େͰ·Ͱಉ࣮࣌ߦՄೳ -
୳ڀͷྲྀΕ w 3FJOEFY"1* w 4DSPMM"1*ͱ#VML"1*ΛผεϨουԽ w #VML"1*ͷฒྻԽ w 4DSPMM"1*ͷฒྻԽ w
3FJOEFY5PPMͷࢄ࣮ߦ w 44%Λ͏ -
3FJOEFY"1* w 4DSPMM4J[FΛม࣮͑ͯߦΛൺֱ w ҎԼͷίϚϯυΛΫϥελ֎ͷϚγϯ্Ͱ࣮ߦ $ curl -X POST
'es-watanabe-1:9200/_reindex' -d '{ "source":{"index":"src", "size": 1000}, "dest" :{"index":"dest"} }' -
3FJOEFY"1* w ࠷Ͱඵ w $166TBHFˋ w σΟεΫॻ͖ࠐΈ.# w DBUUISFBE@QPPMͷCVMLBDUJWF ɺఔ
w ෦Ͱ4DSPMM"1*ͱ#VML"1*Λ ஞ࣍తʹ࣮ߦ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ -
3FJOEFY"1* w ࠷Ͱඵ w $166TBHFˋ w σΟεΫॻ͖ࠐΈ.# w DBUUISFBE@QPPMͷCVMLBDUJWF ɺఔ
w ෦Ͱ4DSPMM"1*ͱ#VML"1*Λ ஞ࣍తʹ࣮ߦ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ Ϧιʔεͬͱ͍ͤΔͣ -
4DSPMM"1*ͱ#VML"1*ͷผεϨουԽ w ͘ͳͬͨʂ w ࠷Ͱඵ w Ϧιʔε༻ྔ3FJOEFY"1*ͱେࠩແ͠ w πʔϧಋೖʹΑΔෛՙ͕ฒྻԽͷϝϦο τΛେ෯ʹ্ճΔ
w ·ͩ߄ͯΔ࣌ؒͰͳ͍ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ -
4DSPMM"1*ͱ#VML"1*ͷผεϨουԽ w ͘ͳͬͨʂ w ࠷Ͱඵ w Ϧιʔε༻ྔ3FJOEFY"1*ͱେࠩແ͠ w πʔϧಋೖʹΑΔෛՙ͕ฒྻԽͷϝϦο τΛେ෯ʹ্ճΔ
w ·ͩ߄ͯΔ࣌ؒͰͳ͍ 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ &MBTUJDTFBSDI͔Βͨ͜͠ͱͰ "1*ͷฒྻ࣮ߦͳͲ͘͢͠ͳͬͨ -
#VML"1*ͷฒྻԽ w UISFBE@QPPMͰCVMLBDUJWF͕."9 w ࣮ࡍʹ ఔ͔͍͑ͯ͠ͳ͍ w #VML"1*ΑΓ4DSPMM"1*ͷํ͕ॲཧ͕ૣ͍ɻ ͭ·ΓɺΩϡʔʹະॲཧͷυΩϡϝϯτ͕ཷ·Δ w
#VML"1*ͷϦΫΤετͰCVMLBDUJWFͷεϨουɺϊʔυͷ γϟʔυ·Ͱ૿͑ΔՄೳੑ͕͋Δ w TIBSEOPEFͳͷͰɺBDUJWFUISFBETCVMLBQJSFRVFTU w CVMLBDUJWF͕."9ͳͷͰɺഒฒྻʹ࣮ߦͯ͠ྑͦ͞͏ -
#VML"1*ͷฒྻԽ w ࠷Ͱඵʢ3FJOEFY"1*ͷഒʣ w 4DSPMM"1*ͱ#VML"1*Λड͚͚ΔϊʔυͰఔͷ$16༻ w ͦΕҎ֎ͷϊʔυఔͷ$16༻ w σΟεΫॻ͖ࠐΈʙ.#T w
UISFBEQPPMͷCVMLBDUJWFʹಧ͔ͳ͍ఔ #VMLฒྻ #VML4J[F 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ ඵ -
#VML"1*ͷฒྻԽ w ࠷Ͱඵʢ3FJOEFY"1*ͷഒʣ w 4DSPMM"1*ͱ#VML"1*Λड͚͚ΔϊʔυͰఔͷ$16༻ w ͦΕҎ֎ͷϊʔυఔͷ$16༻ w σΟεΫॻ͖ࠐΈʙ.#T w
UISFBEQPPMͷCVMLBDUJWFʹಧ͔ͳ͍ఔ #VMLฒྻ #VML4J[F 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ඵ ඵ Ϧιʔεతʹ ·ͩ·͍͚ͩͦ͏ -
4MJDFE4DSPMM w εφοϓγϣοτऔಘ࣌ʹTMJDFͷJEΛࢦఆ͢Δ w TMJDFNBYύϥϝʔλͰࢦఆ͢Δ w TMJDF͕ͷ߹ɺTMJDFͱTMJDFΛผεϨουͰѻ͏͜ͱͰฒྻԽՄೳ # Thread1
curl -XGET 'localhost:9200/src/_search?scroll=1m' -d '{ "slice": { "id": 0, "max": 2 } }' # Thread2 curl -XGET 'localhost:9200/src/_search?scroll=1m' -d '{ "slice": { "id": 1, "max": 2 } }' -
4DSPMM"1*ͷฒྻԽ w 4MJDFE4DSPMMʹΑΓɺ4DSPMM"1* Λฒྻʹ࣮ߦՄೳ w 4MJDF #VMLฒྻͱ ͯ͠ #VML4J[F 4DSPMM4J[FΛมԽ
ͤͯ͞ܭଌ w 4MJDFTIBSEΑΓ૿͢ͱ ͘ͳΔͱ͍͏ϦϑΝϨϯεʹै͍ɺ TIBSEͷಉͷͰݻఆ -
4DSPMM"1*ͷฒྻԽ w ࠷Ͱඵʢ3FJOEFY"1*ͷഒʣ w 4DSPMM"1*ͱ#VML"1*Λड͚͚ΔϊʔυͰఔͷ$16༻ w ͦΕҎ֎ͷϊʔυఔͷ$16༻ w σΟεΫॻ͖ࠐΈʙ.#T w
UISFBEQPPMͷCVMLBDUJWFʹಧ͔ͳ͍ఔ #VMLฒྻ #VML4J[F 4MJDF 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ -
4DSPMM"1*ͷฒྻԽ w ࠷Ͱඵʢ3FJOEFY"1*ͷഒʣ w 4DSPMM"1*ͱ#VML"1*Λड͚͚ΔϊʔυͰఔͷ$16༻ w ͦΕҎ֎ͷϊʔυఔͷ$16༻ w σΟεΫॻ͖ࠐΈʙ.#T w
UISFBEQPPMͷCVMLBDUJWFʹಧ͔ͳ͍ఔ #VMLฒྻ #VML4J[F 4MJDF 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ ͘ͳ͕ͬͨ Ϧιʔεతʹେࠩͳ͠ -
3FJOEFY5PPMͷࢄ࣮ߦ w TIBSEJE$MVTUFS3FSPVUF"1*Λͬͯἧ͑Δ w ֤ϊʔυ͕อ࣋͢ΔTIBSEJE4FBSDI4IBSE"1*Ͱऔಘ w औಘͨ͠TIBSEJEΛ4FBSDI"1*ͷQSFGFSFODFύϥϝʔλʹࢦ ఆ͢ΔͱɺϩʔΧϧTIBSEͷυΩϡϝϯτͷΈΛऔಘͰ͖Δ curl
-XGET ‘localhost:9200/src/_search?scroll=1m&preference=_shards:1,2' -
3FJOEFY5PPMͷࢄ࣮ߦ w ࠷Ͱඵʢ3FJOEFY"1*ͷഒʣ w ʙఔͷ$16༻ w σΟεΫॻ͖ࠐΈʙ.#T w UISFBEQPPMͷCVMLBDUJWF΄΅Λࣔ͢
#VMLฒྻ #VML4J[F 4MJDF 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ -
3FJOEFY5PPMͷࢄ࣮ߦ w ࠷Ͱඵʢ3FJOEFY"1*ͷഒʣ w ʙఔͷ$16༻ w σΟεΫॻ͖ࠐΈʙ.#T w UISFBEQPPMͷCVMLBDUJWF΄΅Λࣔ͢
#VMLฒྻ #VML4J[F 4MJDF 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ͘ͳͬͨ͠ Ϧιʔε͍ͤͯΔʂ -
TIBSEJEΛἧ͑ͳ͍߹ w ࠷Ͱඵ w एׯ͍ͷͷϩʔΧϧͷ߹ͱେࠩͳ͠ w Ϧιʔεফඅେࠩͳ͠ w QSFGFSFODFࢦఆʹΑΔπʔϧͷࢄͷޮՌ͕େ͖͘ɺϩʔΧϧͰ ͋Δ͜ͱͦΕ΄ͲॏཁͰͳ͍πʔϧͷ࣮͕؆୯ʹͳΔ
#VMLฒྻ #VML4J[F 4MJDF 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ -
44%Λ͏ w /7.F44%ΛੵΜͩϚγϯ͕͋ͬͨͷͰܭଌͯ͠ Έͨɻػࡐͷ߹্$16εέʔϧΞοϓ w ߏ࠷ͷ3FJOEFY5PPMͷࢄ࣮ߦͱಉҰ w $16ͱσΟεΫͷεϖοΫͪ͜Β ཁૉ
ੑೳ උߟ $16 ()[ DPSF UISFBET 44%TFRXSJUF (#T EEͰ؆қܭଌ 44%TFRSFBE .#T IEQBSNͰ؆қܭଌ -
44%Λ͏ w ࠷Ͱඵ w ͘ͳ͚ͬͨͲेඵͰऴΘͬͯ͠·͏ͷͰɺπʔϧͷ্ཱͪ͛ ॲཧʢεϨουىಈͳͲʣͷ࣌ؒൺ͕ߴͦ͏ w ࠶ΠϯσΫγϯάͷ͕࣌ؒਖ਼͘͠ܭଌͰ͖͍ͯΔ͔ո͍͠ #VMLฒྻ
#VML4J[F 4MJDF 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ -
υΩϡϝϯτΛഒʹෳ w ࠷Ͱඵ w .#TʢສઍEPDTTʣఔͷ w (#Tݟ͖͑ͯͯເ͕͕Δ w )%%ͰͷϚγϯͰͷສυΩϡϝϯτͷ࣮ߦ࣌ؒඵͩͬ ͨͷͰɺ44%ԽͰഒ͙Β͍͘ͳͬͨ
γϟʔυ #VMLฒྻ #VML4J[F 4MJDF 4DSPMM4J[F ࣮ߦ࣌ؒ ඵ ඵ ඵ ඵ -
࣮ߦ࣌ؒൺֱ ߏ ࣮ߦ ഒ 3FJOEFY"1* ඵ 4DSPMM"1*ͱ#VML"1*ΛผεϨουԽ ඵ
#VML"1*ΛฒྻԽ ඵ 4DSPMM"1*ΛฒྻԽ ඵ 3FJOEFY5PPMͷࢄ࣮ߦ ඵ 44%Λ͏ ඵ w #VML"1*ͷฒྻԽɺ3FJOEFY5PPMͷࢄ࣮ߦͷޮՌ͕ߴ͍ w ϊʔυ૿ͤεέʔϧ͢Δ3FJOEFY5PPMͷࢄ࣮ߦ͕͓͢͢Ί w #VML"1*ͷฒྻԽ͕Ͱ͖Ε4DSPMM"1*ͷฒྻԽେͨ͠खؒͰ ͳ͍ͷͰɺ࠷ޙͷ٧ΊʹͲ͏ͧ ˞্͔ΒॱʹྦྷੵͰվળΛద༻͍ͯ͠·͢ -
8FBSFIJSJOH w ΞϓϦέʔγϣϯج൫νʔϜʢେମͳΜͰΔͭΒʣ w ϓϩμΫτ։ൃνʔϜ w 43&νʔϜ w ੜ࢈ੑ্νʔϜ w
ଞʹ৭ʑνʔϜ͋Γ·͢ w ΨπΨπͯ͠ͳ͍͚Ͳಇ͖͍͢ͱࢥ͏ʢࢲݟʣ w ͝ڵຯ͋Ε࠾༻ϖʔδݟͯΈ͍ͯͩ͘͞ IUUQDZCP[VDPKQDPNQBOZKPCSFDSVJUNFOUMJTU