Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Index Compression Using Byte-Aligned ANS Coding...

chie8842
April 15, 2018

Index Compression Using Byte-Aligned ANS Coding and Two-Dimensional Contexts

WSDM 2018 論文読み会の資料
https://atnd.org/events/95510

chie8842

April 15, 2018
Tweet

More Decks by chie8842

Other Decks in Technology

Transcript

  1. "#453"$5 •  #MPDLCBTFEJOWFSUFEJOEFYDPNQSFTTJPOͷ࣮ݧΛߦͬͨɻ •  "TZNNFUSJD/VNFSBM4ZTUFNT "/4 ΤϯτϩϐʔίʔσΟϯ άʹରͯ͠ҎԼͷ֦ுΛߦ͏͜ͱʹΑΔվળ͕ಘΒΕͨɻ –  ͭͷू໿ύϥϝʔλʹΑΔ࣍ݩͷ৚݅ίϯςΩετʹΑͬͯɺγ

    ϯϘϧͷΧςΰϥΠζΛߦ͏ɻ –  #ZUFGSJFOEMZͳϚοϐϯάΛ࢖ͬͯγϯϘϧΛ"/4ͷDPEFXPSE CVDLFUʹΘ͚Δ –  ίϯςΩετϚʔδͷϓϩηεʹΑΓɺෳ਺ͷEJTUSJCVUJPOΛͭͱ Έͳ͞ΕΔɻ •  ͜ͷख๏ʹΑΓɺJOUFSQʹΑΔΠϯσοΫεͷѹॖͷੑೳج ४఺Λ্ճͬͨɻ •  ࣮ݧ͸(#ͷHPWίʔύεͱ৽͍͠χϡʔεهࣄΛ࢖͍ɺ ΫΤϦධՁ͸ผͷCMPDLCBTFEͷख๏ͱൺֱͨ͠ɻ
  2. */530%6$5*0/ •  *OWFSUFE సஔ ΠϯσοΫε͸ςΩετσʔλʹର͢Δ ߴ଎ͳΫΤϦ࣮ߦͷॏཁͳߏ଄Ͱ͋Δɻ term Posting list J1IPOF

    Ϗοάσʔλ Χό ϝϩϯ                     posting ͦΕͧΕͷυΩϡϝϯτ*%ʹରͯ͠୯ޠͷग़ݱස౓΍ϙδ γϣϯͷ৘ใ΋࣋ͭɻ ίϯύΫτͳQPTUJOHMJTUTʹ͸ɺCZUFBMJHOFEDPEFT XPSE BMJHOFEDPEFT CJOBSZQBDLFEΞϓϩʔν͕͋Δɻ
  3. */530%6$5*0/ •  ϋϑϚϯූ߸ԽͷΑ͏ͳΤϯτϩϐʔϕʔεͷख๏΋ఏҊ͞Ε͍ͯ ͕ͨɺҎલ͸͋·ΓސΈΒΕΔ͜ͱ͕ͳ͔ͬͨɻ೥લʹ+FSFL %VEB͕"/4ϝΧχζϜΛ։ൃͨ͜͠ͱʹΑΓɺΤϯτϩϐʔίʔ σΟϯά͕࠶ͼ஫໨͞ΕΔΑ͏ʹͳͬͨɻ •  7#ZUF 4JNQMF 1BDLFEͷख๏Λ༻͍ͯɺͦΕͧΕѹॖޮ཰͕վળ͞

    Εͨɻ •  1BDLFE "/4ΞϓϩʔνͷఏҊ –  ݻఆ௕ϒϩοΫͷγϯϘϧ͕ηϨΫλʹΑͬͯಛఆ͞Εͨ֬཰෼෍ͱؔ࿈ ͯ͠ූ߸Խ͞ΕΔɻ –  1BDLFE "/4Ξϓϩʔνͷ঺հʹ͓͍ͯ͸ɺηϨΫλ͸ϒϩοΫͷ࠷্Ґ ͕ར༻͞ΕΔɻ –  QBDLFEGSBNFPGSFGFSFODF 1'03 ϝΧχζϜͷΑ͏ͳར఺͕͋Δɻ –  ΤϯτϩϐʔίʔσΟϯάͷϝϦοτ •  ΤϯτϩϐʔίʔσΟϯάͷॊೈੑʹΑΓɺࣅ௨ͬͨූ߸ͷ֬཰ٴͼූ ߸௕ͷ໘Ͱѹॖޮ཰্͕͕ͬͨɻ –  ϒϩοΫͷू໿ʹ͓͍ͯ࣍ݩͷηϨΫλ͕ར༻Ͱ͖ͨɻ •  γϯϘϧͷࣝผࢠ͔Β"/4ͷ஋΁ͱϚοϐϯάΛม͑ΔলϝϞϦͰ CZUFGSJFOEMZͳग़ྗ •  σίʔυͷࡍͷϝϞϦ࢖༻ྔΛ࡟ݮ͢ΔͨΊͷ$POUFYU.FSHJOH
  4. ิ଍ɿ7#ZUF 1 0 0 0 0 0 1 0 value

    End marker 1000000110000010 整数自体を符号化するのでなく、逐次差分を符号化する方法
  5. #"$,(306/%ɿQBDLFEDPEFT ˙1BDLFE$PEFT •  ݻఆ௕ͷQPTUJOHTʹର͢ΔQBDLFEDPEFT •  ൺֱత࠷ۙͷٕज़ •  ΩʔΞΠσΞ͸ɺϩʔΧϧͳҰ؏ੑΛར༻͢Δํ๏Ͱ͋ Δɻ • 

    ݻఆ௕ʢେ఍ʣͷϒϩοΫ͝ͱͷ஋ΛͭͷϢχο τͱͯ͠ॲཧ͢Δɻ •  ྫ͑͹ɺ  ͔ΒηϨΫλΛબͿ͜ͱͰɺ͋Δ஋Λ  ͱ͍ͬͨॻ͖ํͰදͤΔɻ
  6. #"$,(306/%ɿ1BUDIFE'SBNFPG3FGFSFODF 2 1 31 1 0 … 2 10 01

    01 01 00 … 10 b=5 b’=2 1exception position3 •  ϒϩοΫຖʹͦͷϒϩοΫ಺ͷ਺ͷ΄ͱΜͲΛද͢͜ͱ͕Ͱ͖ΔϏοτ਺Λ ܾΊͯ Τϯίʔυ͢Δɻ •  খ͍͞Ϗοτ਺ͰදͤΔͨΊσʔλαΠζ͕খ͘͞ͳΔɻ •  1'03ͷมܗͰ͋Δ0QU1'03Ͱ͸ɺ࠷΋શମͷύοέʔδ͕͜Μͺ͘ͱʹͳ ΔΑ͏ʹηϨΫλΛબͿΑ͏ʹͳ͍ͬͯΔɻ b b’ metadata
  7. "/4Λ༻͍ͨΠϯσοΫεѹॖͷྫ •  γϯϘϧ1 2 3͸ͦΕͧΕग़ݱ֬཰ɺɺͱ͢Δ •  ϋϑϚϯූ߸ԽͰ͸ɺ1͕̍Ϗοτɺ2ͱ3͕̎ϏοτͱͳΔɻ •  "/4ූ߸ԽͰ͸ɺ.ݸͷཁૉͷ͕1ɺ͕2ɺ͕3ʹϥϕϦϯά͞Εɺ ͦͷΞϧϑΝϕοτ্ͷه߸ͷγʔέϯε͸ɺରԠ͢ΔϑϨʔϜͷঢ়ଶ஋ͱ͠

    ͯදݱ͞ΕΔɻ •  ଞͷϑϨʔϜΛ࡞Δ৔߹͸ɺଞͷॱྻ΋͘͠͸.ΑΓେ͖͍஋Λ࢖༻͢Δɻ •  ϑϨʔϜͷූ߸Խ͸ɺભҠؔ਺5 ɾ ɾ Λ༻͍ͯߦ͏ɻ5্ͷΧϥϜ͸ɺεςʔ τม਺ͰΠϯσοΫεԽ͞ΕΔɻ •  શମΛූ߸Խ͢Δ৔߹͸ɺεςʔτม਺͕େ͖͘ͳΔɻ •  σίʔυ͢Δͱ͖͸ɺɹɹɹ͕࢖ΘΕΔɻ
  8. "/4$PNQBSFE •  ϋϑϚϯූ߸ٴͼ"SJUINFUJDDPEJOHͱͷൺֱ –  "/4͸ཧ૝తͳූ߸Խ௕͕"SJUINFUSJDDPEJOHͱۙ͘ͳΔɻ –  ߴ͍ग़ݱස౓ͷγϯϘϧΛൃੜ͢ΔͨΊʹɺϏοτͷίʔυ͢ Βൃੜ͠ͳ͍͍ͯ͘৔߹͕͋ΔɻʢϋϑϚϯූ߸ͩͱৗʹϏο τҎ্͕ඞཁɻʣ – 

    "SJUINFUSJDDPEJOHΑΓ΋͸΍͘ɺߴ౓ʹઃܭ͞ΕͨϋϑϚϯූ ߸ͷ࣮૷ʹ͍ۙεϧʔϓοτϨʔτΛग़͢͜ͱ͕Ͱ͖Δɻ –  "/4͸"SJUINFUJDDPEJOHͱҟͳΓ੩తϝΧχζϜͰ͋Γɺҟͳ ΔίϯςΩετͷ΋ͱͰγϯϘϧΛੜ੒͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ ͨͩ͜͠ͷ͜ͱ͸ΠϯσοΫεѹॖʹ͸Өڹ͠ͳ͍ɻ
  9. "/4GPS*OEFY$PNQSFTTJPO •  Ҏલͷݚڀ –  "/4Λ̏ͭͷҟͳΔΠϯσοΫεѹॖํࣜͱ૊Έ߹Θͤͨɻ –  ಛʹ7#ZUFͱ૊Έ߹ΘͤΔΞϓϩʔνΛQPTUJOHMJTUͷѹॖʹ࢖༻ ͨ͠ɻ –  σίʔυͷεϧʔϓοτ͸গ͠௿Լ͢Δ͚ͩͰɺετϨʔδΛ

    ͔ͳΓઅ໿Ͱ͖Δ͜ͱΛ֬ೝͨ͠ɻ •  ࠓճͷख๏ɿʮ1BDLFE "/4ʯϝΧχζϜ –  ʮͭͷύϥϝʔλʯΛ࢖͏ –  গͳ͍ϝϞϦ࢖༻ྔͷίϯύΫτͳ࣮૷ –  ίϯςΩετϚʔδ •  ͜ΕΒͷख๏ͷ༗༻ੑΛ࣮ূ͢ΔͨΊʹɺͭͷେ͖ͳ ςΩετίϨΫγϣϯΛ࢖༻ͯ͠ɺѹॖͷ༗ޮੑͱΫΤ Ϧͷεϧʔϓοτʹؔ͢Δৄࡉͳ࣮ݧ݁ՌΛఏڙ͢Δɻ 
  10. *.1307&%1"$,&%$0.13&44*0/ 1BDLFE "/4 •  1BDLFE "/4ϝΧχζϜͰ͸ɺ͜ΕΒͷ͞·͟·ͳཁૉ͕ີ ઀ʹ݁ͼ͔ͭͳ͍ɻ •  ηϨΫλ͸ϒϩοΫ಺ͷ࠷େ஋ͱ͠ɺ"/4ϑϨʔϜϓʔϧ Λࣝผ͢Δɻ

    •  ֤ϑϨʔϜʹ͸ɺ૊ͷϙεςΟϯάΛॲཧ͢Δ࠷ॳͷύεத ʹ౷ܭΛ஝ੵ͠ɺ֬཰෼෍Λর߹͢ΔͨΊʹߏங͞Εͨҟ ͳΔγϯϘϧ֬཰ηοτΛؚΉɻ •  γϯϘϧͷස౓͸ਐ਺ͷόέοτʹ஝ੵ͞ΕɺͦͷΑ͏ͳ֤ όέοτ಺ͷ͢΂ͯͷγϯϘϧʢͨͱ͑͹ʣʹͳΔɻ •  ྫ͑͹ɺʜ ʜͱʜ͸౳֬཰Ͱ͋ΔͱԾఆ ͞ΕΔɻ •  ΋͕͠γʔέϯεɹͷ൪໨ͷγϯϘϧ͕ϒϩοΫͷηο τͱͯ͠ූ߸Խ͞Ε͍ͯΔͳΒ͹ɺ͸ͱಉ͡ϒϩοΫͷ γϯϘϧʹΘͨΔ࠷େਐ਺͔Βܭࢉ͞ΕͨηϨΫλ஋Ͱ͋Δɻ
  11. 3FEVDJOHUIF'SBNFTJ[F •  1BDLFE "/4Ͱ͸ɺσίʔυʹ࢖༻͞ΕΔςʔϒϧͷ ҡ࣋ίετ͕͔͔Δͱ͍͏σϝϦοτ͕͋Δɻ •  "/4ΛγϯϘϧͷਐ਺ͷൣғͰ࢖༻͠ɺ"/4ετϦʔ ϜͱิॿϏοτετϦʔϜΛฒྻʹॲཧ͢Δ͜ͱͰ஋Λ ࠶ߏ੒͢Δͱ͍͏खஈ͕͋Δɻ • 

    ͜ͷ৔߹௥ՃͷΞϥΠϝϯτʹΑΔίετ͕͔͔Δ ͜ͷͨΊɺ1'03΍0QU1'03Ͱར༻͞ΕΔྫ֎Λఆٛ͠ ͯγϯϘϧ஋ͱ"/4λʔήοτͷؒͷϚοϐϯάΛఆٛ ͢Δɻ •  ͜ΕʹΑΓѹॖͷ༗ޮੑΛߴΊΔ͜ͱ͕Ͱ͖Δɻ
  12. $0/$-64*0/4"/%'6563&803, ˙$PODMVTJPOT •  1BDLFE "/4ϝΧχζϜʹΑΓɺͭͷҟͳΔख๏Λվળͨ͠ɻ –  ࣍ݩίϯςΩετ –  ࠷্ҐόΠτϚοϐϯά – 

    ίϯςΩετϚʔδϯά •  ೥લͷ*OUFSQͷϕϯνϚʔΫʹউͬͨɻ •  1BDLFE "/4ͷ࣮૷͸0QU1'03΍&'PQUʹൺ΂ͯΫΤϦεϐʔ υʹ͓͍ͯ͸ྼ͍ͬͯΔ͕ɺ"/4ʹ͸·ͩվળͷ༨஍͕͋Δɻ •  ͢Ͱʹ΄ͱΜͲͷQPTUJOHT͕খ͍͞αΠζʹූ߸ԽͰ͖Δ͜ͱΛ֬ ೝͨ͠ɻ ˙'VUVSFXPSL •  ෳ਺ͷঢ়ଶΛฒߦͯ͠෮߸͢ΔͨΊʹ֦ு͞Εͨ෮߸Խςʔϒϧ͓ Αͼ4*.%ػߏΛ࢖༻͢Δͱɺ෮߸Խ଎౓ΛഒҎ্ʹ૿Ճͤ͞ΔՄ ೳੑ͕͋Δɻ •  Մม௕ϒϩοΫͷ࢖༻ •  ΫΤϦͷ౷ܭ৘ใΛར༻ͯ͠ҟͳΔѹॖϝΧχζϜΛมߋ͢Δ