仮想的な探索を用いて文脈や時間の経過による番狂わせにも迅速に追従する多腕バンディット手法/wi2_lkf_bandits

ࡾ୐ ༔հ1,2ɺ็ ߃ݑ3
1. Pepabo R&D Institute, GMO Pepabo, Inc., 2. ۝भେֶ େֶӃγεςϜ৘ใՊֶ෎ ৘ใ஌ೳ޻ֶઐ߈
3. ۝भେֶ େֶӃγεςϜ৘ใՊֶݚڀӃ ৘ใ஌ೳ޻ֶ෦໳
2021.12.17 ARG ୈ17ճWebΠϯςϦδΣϯεͱΠϯλϥΫγϣϯݚڀձ
Ծ૝తͳ୳ࡧΛ༻͍ͯ
จ຺΍࣌ؒͷܦաʹΑΔ൪ڰΘͤʹ΋
ਝ଎ʹ௥ै͢Δଟ࿹όϯσΟοτख๏

View Slide

1. ͸͡Ίʹ
2. ଟ࿹όϯσΟοτ໰୊ʹ͓͚Δ൪ڰΘͤͱɺͦͷ՝୊
3. ఏҊख๏
4. ධՁ
5. ͓ΘΓʹ
2
໨࣍

View Slide

1.
͸͡Ίʹ

View Slide

• దԠతͳγεςϜͷ࣮ݱʹ͸ɺγεςϜ͕ར༻ऀͷঢ়گΛΑ͘஌Δ͜ͱ͕ॏཁ
• ECαΠτͷγεςϜͰ͋Ε͹ɺར༻ऀͷᅂ޷Λ೺Ѳ͢Δ͜ͱͰɺ࠷దͳ঎
඼ΛఏҊͰ͖Δ
• ࣮ӡ༻ͷγεςϜʹ͓͍ͯίϛϡχέʔγϣϯʹ͸ίετ͕͔͔Δ
• ʢར༻ऀࣗ਎΋ؚΊͯʣཁٻ΍ᅂ޷͸໌֬Ͱ͸ͳ͘ঃʑʹܗ੒͞Ε͍ͯ͘
• ͦͷظؒதͷෛ୲΍ػձଛࣦ͸୹ظ௕ظͰചΓ্͛ͳͲʹӨڹ͢Δ
• ಛʹɺཁٻ΍ᅂ޷͕มԽ͢Δ؀ڥͰ͸ɺݱ࣌఺ͰՁ஋ͷ௿͍ίϛϡχέʔ
γϣϯ΋ܧଓͯ͠ߦ͏ඞཁ͕͋Δ
4
దԠతͳγεςϜͱίϛϡχέʔγϣϯίετ

View Slide

• ίϛϡχέʔγϣϯΛɺબ୒ࢶͷఏҊͱ൓Ԡͱݶఆ͢Δ͜ͱͰɺ͜ͷίετΛ
࠷దԽ͢Δ໰୊Λʮଟ࿹όϯσΟοτ໰୊ʯͱͯ͠ߟ͑Δ͜ͱ͕Ͱ͖Δ
5
ίϛϡέʔγϣϯίετͷ࠷దԽͱଟ࿹όϯσΟοτ
• ҰํͰɺैདྷͷಉ໰୊ͷํࡦͰ͸ɺબ୒ࢶͷ༗ޮੑ͕ʮٯసʯ͢Δ؀ڥͰ͸ɺ
ૉૣ͘௿ίετͳมԽ௥ै͕Ͱ͖ͳ͍ʢؔ࿈ݚڀͰઆ໌ʣ
• ͜ͷ؀ڥʹ͓͍ͯਝ଎͔ͭޮ཰తͳมԽ௥ै͕ՄೳͳํࡦΛఏҊ͢Δ͜ͱͰɺ
దԠతͳγεςϜͷ࣮༻ԽΛਐΊ͍ͨ

View Slide

• ʮ࿹ʯͱݺ͹ΕΔෳ਺ͷީิ͔ΒಘΒΕΔใुΛ࠷େԽ͢Δ໰୊
• ϓϨΠϠʔ͸Ұ౓ͷࢼߦͰ1ͭͷ࿹Λબ୒͠ɺใुΛಘΔ
• ͦΕͧΕͷ࿹͸͋Δใु෼෍ʹै͍ใुΛੜ੒
• ͨͩ͠ɺϓϨΠϠʔ͸͜ͷใु෼෍Λࢼߦͷ݁Ռ͔Βਪଌ͢Δඞཁ͕͋Δ
6
ଟ࿹όϯσΟοτ໰୊
• ϓϨΠϠʔ͸͋Δ࣌఺ͷ࿹ͷධՁʹج͖ͮʮ׆༻ʯͱʮ୳ࡧʯΛฒߦͯ͠ߦ͏
• ͜ͷτϨʔυΦϑΛղফ͢ΔͨΊʹ༷ʑͳղ๏͕ఏҊ͞Ε͍ͯΔ

View Slide

• ࿹͝ͱͷใु෼෍͸ৗʹಉ͡Ͱ͋Δͱ͍͏Ծఆ
• → ঢ়گ΍ଐੑ͝ͱʹใु෼෍͕ҟͳΔͷͰ͸ͳ͍͔ʁ
• ྫʣ೥୅͝ͱʹਓؾͷ঎඼͕ҧ͏ɺ࠷ۙɺಉ͡ΧςΰϦͷ঎඼Λങͬͨ
• → ࣌ؒͷܦաʹΑͬͯ΋ใु෼෍͕มԽ͢ΔͷͰ͸ͳ͍͔ʁ
• ྫʣ৽͍͠χϡʔεهࣄɺҰ࣌తͳྲྀߦ
7
ଟ࿹όϯσΟοτ໰୊ͷ֦ு
• ʮจ຺෇͖ʯ·ͨ͸ʮඇఆৗʯͳଟ࿹όϯσΟοτ໰୊ͱ֦ͯ͠ு͞Ε͍ͯΔ

View Slide

8
ଟ࿹όϯσΟοτͷ՝୊
• ैདྷͷඇఆৗͳଟ࿹όϯσΟοτͷಘҙͳঢ়گͱෆಘҙͳঢ়گͷྫ
9࣠͸࣌ؒͷܦաΛද͢
:࣠͸֤࿹ͷ༗༻ੑͷଟՉ

View Slide

9
ʮڧ͍ʯ࿹͕ऑ͘ͳΔঢ়گ
ˠਝ଎ʹ࿹Λ੾ସ

View Slide

10
ʮऑ͍ʯ࿹͕ڧ͘ͳΔঢ়گ
ˠ࿹ͷ੾ସ͕஗ΕΔ
ྫᅂ޷৘ใͷ஝ੵʹ൐͏ਪન
ΞϧΰϦζϜͷਫ਼౓޲্ͳͲ
ຊݚڀͰͷ͜ͷঢ়گΛ
ʮ൪ڰΘͤʯͱݺͿ

View Slide

• จ຺ͱ࣌ؒͷܦաʹԠͯ͡ɺධՁ͕௿͍ީิͷਅͷ༗༻ੑ͕޲্ͨ͠ࡍʹ΋ɺ
ਝ଎ʹมԽΛݕग़͠ɺධՁΛߋ৽͢Δɺจ຺෇͖ɺ͔ͭඇఆৗͳଟ࿹όϯ
σΟοτͷํࡦΛఏҊ͢Δ
• ఏҊ͢Δํࡦͷ༗ޮੑΛৄࡉʹධՁ͢ΔͨΊɺมԽݕग़΍ݕग़ޙͷߋ৽ੑೳΛ
ൺֱ͢ΔͨΊͷ৽ͨͳࢦඪΛఏҊ͢Δ
• ͦΕΒͷࢦඪΛ΋ͱʹɺ͜Ε·ͰఏҊ͞Ε͍ͯΔɺจ຺෇͖ɺ͔ͭඇఆৗͳଟ
࿹όϯσΟοτͷํࡦͱൺֱ࣮ݧΛߦ͍ɺఏҊํࡦ͕༗ޮͰ͋Δ͜ͱΛࣔ͢
11
ຊݚڀͷߩݙ

View Slide

2.
ଟ࿹όϯσΟοτ໰୊ʹ͓͚Δ
൪ڰΘͤͱɺͦͷ՝୊

View Slide

• ࿹ͷ༗༻ੑ͸ɺจ຺΍࣌ؒͷܦաʹԠͯ͡มԽ͢Δ[*]
• ଟ࿹όϯσΟοτͷํࡦ͸ɺ͋Δ࣌఺ͰධՁͷߴ͍࿹Λ࠷΋ଟ͘׆༻͢Δ
• ࿹ͷ༗༻ੑͷมԽͷ൑அʹ͸ɺมԽޙͷใु෼෍͔Βͷαϯϓϧ͕Ұఆ਺ඞཁ
13
ଟ࿹όϯσΟοτ໰୊ʹ͓͚Δ൪ڰΘͤͷ՝୊
<>ࡾ୐༔հ ็߃ݑ 4ZOBQTFจ຺ͱ࣌ؒܦաʹԠͯ͡ਪનख๏ͷબ୒Λ࠷దԽ͢ΔϝλਪનγεςϜ ݚڀใࠂ஌ೳγεςϜʢ*$4ʣ 7PM*$4 /P QQ 4FQ

View Slide

14
൪ڰΘͤͰ͸ͳ͍੾ΓସΘΓ
• ࠷΋ଟ͘׆༻͞Ε͍ͯΔͨΊɺมԽޙͷαϯ
ϓϧΛे෼ʹ؍ଌ͠ɺਝ଎ͳ੾ସ͕Մೳɻ
• ͨͩ͠ɺ͜ͷมԽͷ൑அͷͨΊͷػߏͷ࣮ߦ
ʹ͕͔͔࣌ؒΔํࡦ΋͋Δ

View Slide

15
൪ڰΘͤͳ੾ΓସΘΓ
• ݩʑɺ׆༻͞Ε͍ͯͳ͍ͨΊɺαϯϓϧΛಘ
Δ·Ͱʹ͕͔͔࣌ؒΔ
• ݁Ռͱͯ͠ɺ࿹ͷ੾ସ͕஗ΕɺҎલͷ࠷ద࿹
Λ࢖͏͜ͱʹΑΔػձଛࣦʹͭͳ͕Δ

View Slide

16
൪ڰΘͤͳ੾ΓସΘΓ
• ൪ڰΘͤͷݕग़ͷͨΊʹશͯͷ࿹΁ͷ୳ࡧΛ
Ұ཯ʹ૿΍͢[1]ͱมԽ͕ͳ͍ظؒ΍จ຺Ͱͷ
ػձଛࣦʹͭͳ͕Δ
<>$BP : 8FO ; ,WFUPO # 9JF :
/FBSMZ0QUJNBM"EBQUJWF1SPDFEVSFXJUI$IBOHF%FUFDUJPOGPS1JFDFXJTF4UBUJPOBSZ#BOEJU1SPDFFEJOHTPG.BDIJOF-FBSOJOH3FTFBSDI r
1.-3

View Slide

• ͜ͷ໰୊Λਝ଎͔ͭޮ཰తʹղܾ͢Δํࡦ͸ҎԼͷཁ݅Λຬͨ͢
• จ຺Λѻ͏͜ͱ͕Ͱ͖Δ
• ࿹ͷਝ଎ͳධՁߋ৽ͷ࢓૊ΈΛඋ͑Δ
• ධՁͷ௿͍࿹΁ͷޮ཰తͳ୳ࡧͷ࢓૊ΈΛඋ͑Δ
• ͜ΕΒͷ࢓૊Έͷ࣮ߦ͕࣌ؒগͳ͍
17
จ຺ͱɺ൪ڰΘͤΛؚΉඇఆৗੑ΁ͷରॲ

View Slide

18
ैདྷͷଟ࿹όϯσΟοτํࡦͱ൪ڰΘͤ΁ͷରԠ
จ຺ ਝ଎ͳධՁߋ৽ ޮ཰తͳ୳ࡧ ࣮ߦ࣌ؒ
%FDBZ-JO6$#<> P P Y P
"E54<> P P Y Y
E-JO6$#<> P P ˚ P
%FO#BOE<> P P ˚ P
5751<> P P Y Y
"%"*-5$#<> P P ˚ Y
,'."/#<> Y P P P

View Slide

19
"E54<> P P Y Y
E-JO6$#<> P P ˚ P
%FO#BOE<> P P ˚ ˚
5751<> P P Y Y
"%"*-5$#<> P P ˚ Y
,'."/#<> Y P P P
<>$IVORJV;FOH 2JOH8BOH 4IFLPPGFI.PLIUBSJ BOE5BP-J0OMJOFDPOUFYUBXBSFSFDPNNFOEBUJPOXJUIUJNFWBSZJOHNVMUJBSNFECBOEJU*O1SPDFFEJOHTPGUIFOE"$.4*(,%%JOUFSOBUJPOBM
DPOGFSFODFPO,OPXMFEHFEJTDPWFSZBOEEBUBNJOJOH QQr
<>/FHBS)BSJSJ #BNTIBE.PCBTIFS BOE3PCJO#VSLF"EBQUJOHUPVTFSQSFGFSFODFDIBOHFTJOJOUFSBDUJWFSFDPNNFOEBUJPO*O5XFOUZ'PVSUI*OUFSOBUJPOBM+PJOU$POGFSFODFPO"SUJpDJBM*OUFMMJHFODF

<>.VIBNNBE"NNBS)BTTBO/PO4UBUJPOBSZ$POUFYUVBM.VMUJ"SNFE#BOEJUXJUI"QQMJDBUJPOJO0OMJOF3FDPNNFOEBUJPOT1I%UIFTJT 6OJWFSTJUZPG7JSHJOJB
<>2JOHZVO8V /BWFFO*ZFS BOE)POHOJOH8BOH-FBSOJOHDPOUFYUVBMCBOEJUTJOBOPOTUBUJPOBSZFOWJSPONFOU*O5IFTU*OUFSOBUJPOBM"$.4*(*3$POGFSFODFPO3FTFBSDI%FWFMPQNFOUJO
*OGPSNBUJPO3FUSJFWBM QQr
<>2JOHZVO8V )VB[IFOH8BOH :BOFO-J BOE)POHOJOH8BOH%ZOBNJDFOTFNCMFPGDPOUFYUVBMCBOEJUTUPTBUJTGZVTFST`DIBOHJOHJOUFSFTUT*O5IF8PSME8JEF8FC$POGFSFODF QQr
<>:JGBOH$IFO $IVOH8FJ-FF )BJQFOH-VP BOE$IFO:V8FJ"OFXBMHPSJUINGPSOPOTUBUJPOBSZDPOUFYUVBMCBOEJUT&⒏DJFOU PQUJNBMBOEQBSBNFUFSGSFF*O$POGFSFODFPO-FBSOJOH5IFPSZ QQ
r1.-3
<>0MF$ISJTUP⒎FS(SBONPBOE4UJBO#FSH4PMWJOHOPOTUBUJPOBSZCBOEJUQSPCMFNTCZSBOEPNTBNQMJOHGSPNTJCMJOHLBMNBOpMUFST*O*OUFSOBUJPOBM$POGFSFODFPO*OEVTUSJBM &OHJOFFSJOHBOE0UIFS
"QQMJDBUJPOTPG"QQMJFE*OUFMMJHFOU4ZTUFNT QQr4QSJOHFS

View Slide

3.
ఏҊख๏

View Slide

21

View Slide

• → KF-MANBΛআ͘ैདྷͷํࡦ͸ɺબఆͨ͠࿹ͷධՁͷΈΛ༻͍࣮ͯݱ͢Δ
• → બఆ͠ͳ͔ͬͨ࿹ͷධՁ΋ಉ࣌ʹߋ৽Ͱ͖Ε͹ɺΑΓޮՌతͰ͸ͳ͍͔
ɹ ఏҊख๏Ͱ͸͜ͷͨΊʹʮઢܗΧϧϚϯϑΟϧλʯΛ࠾༻͢Δ
22

View Slide

23
"E54<> P P Y Y
E-JO6$#<> P P ˚ P
%FO#BOE<> P P ˚ P
5751<> P P Y Y
"%"*-5$#<> P P ˚ Y
,'."/#<> Y P P P
ఏҊख๏ P P P P

View Slide

24
ํࣜͷྨࣅ͢Δख๏ͱͷࠩҟ
,'."/#<> Y P P P
ఏҊख๏ P P P P
• ਝ଎ͳධՁߋ৽ͱऑ͍࿹΁ͷޮ཰తͳ୳ࡧͷͨΊɺܰྔͳΧϧϚϯϑΟϧλΛར༻͢Δ఺
͕ڞ௨
• ఏҊख๏͸ઢܗΧϧϚϯϑΟϧλΛར༻͢Δ఺͕ҟͳΔ
• ࣌ؒͷܦա͚ͩͰͳ͘จ຺ʹΑͬͯ΋࿹ͷ༗ޮੑ͕ҟͳΔ؀ڥʹద༻Ͱ͖Δ
• τϨϯυ΍पظత੒෼Λߟྀͨ͠ɺΑΓޮ཰తͳ୳ࡧ͕ظ଴Ͱ͖Δ
<>0MF$ISJTUP⒎FS(SBONPBOE4UJBO#FSH4PMWJOHOPOTUBUJPOBSZCBOEJUQSPCMFNTCZSBOEPNTBNQMJOHGSPNTJCMJOHLBMNBOpMUFST*O*OUFSOBUJPOBM$POGFSFODFPO*OEVTUSJBM &OHJOFFSJOHBOE0UIFS
"QQMJDBUJPOTPG"QQMJFE*OUFMMJHFOU4ZTUFNT QQr4QSJOHFS

View Slide

֬཰Ұக๏Λ༻͍ͨ
ଟ࿹όϯσΟοτख๏ͱ
ઢܗΧϧϚϯϑΟϧλͱͷ౷߹

View Slide

• ઢܗΧϧϚϯϑΟϧλʹΑͬͯਪఆͨ͠ঢ়ଶΛ࿹ͷબఆʹར༻
• 1. ઢܗΧϧϚϯϑΟϧλʹΑΔ࿹ͷঢ়ଶਪఆ
• 2. ঢ়ଶਪఆ஋Λ༻͍ͨ֬཰Ұக๏ʹΑΔ࿹ͷબఆ
26
ఏҊख๏ʹ͓͚Δ࿹ͷબఆ

View Slide

• Ұظઌ༧ଌͱ؍ଌ݁ՌʹΑΔϑΟϧλϦϯάΛ܁Γฦͯ͠ঢ়ଶΛܧଓతʹਪఆ
• ঢ়ଶ ͸ฏۉ ͱͦͷ෼ࢄڞ෼ࢄ Ͱදݱ͢Δ͜ͱ͕Ͱ͖Δ
α μ P
27
1. ઢܗΧϧϚϯϑΟϧλʹΑΔ࿹ͷঢ়ଶਪఆ
Ұظઌͷঢ়ଶ༧ଌ
Ұظઌͷ؍ଌ༧ଌ ؍ଌͷޡࠩ
ϑΟϧλϦϯά
ઢܗΧϧϚϯϑΟϧλʹ͓͚Δঢ়ଶߋ৽ͷαΠΫϧ
t = t + 1
AAACinichVHLSsNAFL2Nr1qrrboR3ARLRRDKpAq+EIq6cNmHfUAtJYnTOjQvkmmhFn/AnSvBrhRciB/gB7jxB1z0E8RlBTcuvE0DosV6w2TOnLnnzpm5iqUxhxPS8Qkjo2PjE/7JwFRweiYUnp3LOWbdVmlWNTXTLiiyQzVm0CxnXKMFy6ayrmg0r9T2e/v5BrUdZhpHvGnRki5XDVZhqsyRynNxV+SrUjkcITHihjgIJA9EwIukGX6EYzgBE1Sogw4UDOCINZDBwa8IEhCwkCtBCzkbEXP3KZxDALV1zKKYISNbw38VV0WPNXDdq+m4ahVP0XDYqBQhSl7IPemSZ/JAXsnnn7Vabo2elybOSl9LrXLoYiHz8a9Kx5nD6bdqqGcOFdh0vTL0brlM7xZqX984u+pmttPR1jK5JW/o/4Z0yBPewGi8q3cpmm4P8aOgF3wxbJD0ux2DIBePSWuxeGo9ktjzWuWHRViCFezHBiTgEJKQdetfwjW0haAQF7aEnX6q4PM08/AjhIMvdtyReA==
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
AAAC03ichVE9SyRBEH3O6amrnuuZCCbiogjC0uMdKIIgmhjJfroLji4zc706OF/M9C54c5OIkWBsYKRgIKaCP+CS+wMGmmoihiuYGFg7O9zpyXk19HTVq3rVr7s01zR8wdhVm/ShveNjZ1d3oqe371N/cuDziu/UPJ0Xdcd0vLKm+tw0bF4UhjB52fW4amkmL2lbi818qc4933Dsgth2+ZqlbthG1dBVQVAluaxoVqBYtbASiEk5HJ8rvAB+iFBREpnfqUwLK6wrlio2/WpQCCdz2dyLsJJMsTSLbOStI8dOCrFlnOQFFHyDAx01WOCwIcg3ocKnbxUyGFzC1hAQ5pFnRHmOEAni1qiKU4VK6Bb9NyhajVGb4mZPP2LrdIpJyyPmCMbYJTtlDfaLnbE79vTPXkHUo6llm3atxeVupX9vKP/4X5ZFu8DmH9a7mgWqmIm0GqTdjZDmLfQWv/79oJGfzY0F4+yY3ZP+I3bFftIN7PqDfpLlucN39GikhV6MBiT/PY63zspUWv6Snsp+Tc0vxKPqwjBGMUHzmMY8lpBBkfqf4xo3uJWKUiDtSLutUqkt5gzilUn7z7XisIo=
µt|t
= µt + Kvt
= µt + (PtZTF 1
t
)vt
Pt|t
= Pt KFtKT
AAADBHichVFLS+NQFD6Jr1pf9TEguAkWizJYbqowgyDICCJ0U61V0WhIMrcamhfJbaHG4M7F/IFZuFJwIc5uGGY3m9n4B1z0B7gYZqngxoUnacAX6gnJPfc75/vy3XtUx9A9RkiD41ta29o7Ep3Jru6e3r5U/8CqZ1ddjZY027DddVXxqKFbtMR0ZtB1x6WKqRp0Ta3Mh/W1GnU93bZWWN2hW6ayY+llXVMYQnLqQFJNXzKrss/2WRAImVkhRgKZCR+FfLirYS5Jyczsk9J4QWYb25KpsF2v7K8ECzLb9ifFYOIRpdDUDWWxW5gU8tiVf0SSU2mSJVEILxMxTtIQR8FO/QYJvoINGlTBBAoWMMwNUMDDZxNEIOAgtgU+Yi5melSnEEASuVXsotihIFrB7w7uNmPUwn2o6UVsDf9i4OsiU4AxcknOyDW5IOfkH7l7VcuPNEIvdVzVJpc6ct+34eLtuywTVwa7D6w3PTMow+fIq47enQgJT6E1+bW979fFmeUxP0NOyH/0f0wa5C+ewKrdaKdLdPnoDT8qesEbwwGJz8fxMlnNZcWpbG5pOj33JR5VAkZgFMZxHp9gDhahACXUv+K6uCHuA3/I/+B/8r+arTwXcwbhSfB/7gH5GL88
vt = yt
ˆ
yt
Ft = ZPtZ
T + H
AAACxHichVFNS+NQFD2No1PrV3U2gptiUQSZ8uoIiiCKgnTZqlXROCGJrzaYL5LXQg2dH+AfmMWsFLoQf8D8gNkMzHa68CeISwU3Lrz5gGEU9YbknXvePTfnvau5puELxq5TUs+H3r6P6f7MwODQ8Eh2dGzHdxqezqu6Yzrenqb63DRsXhWGMPme63HV0ky+q52sh/u7Te75hmNvi5bLDy312DZqhq4KopTsiqxZQbOtiNz0ci7ELcKf5boqgjijVJYzG3HBflkR+19lSxV1vxZst2dLSjbPCiyK3EtQTEAeSZSd7E/IOIIDHQ1Y4LAhCJtQ4dNzgCIYXOIOERDnETKifY42MqRtUBWnCpXYE/oeU3aQsDblYU8/Uuv0F5Nej5Q5TLEuu2R37De7Yjfs8dVeQdQj9NKiVYu13FVGzsa3Ht5VWbQK1P+p3vQsUMNi5NUg727EhKfQY33z9Pvd1tLmVDDNLtgt+T9n1+wXncBu3uudCt/88YYfjbzQjdGAis/H8RLszBWKXwpzlfn86loyqjQmMIkZmscCVlFCGVXq38Ef/EVX2pBMyZcacamUSjSf8F9I354ANqupFA==
ˆ
yt = Zµt
AAACoXichVE9S8NQFD3Gr/pddRF0KBbFqbyqoAiC6KJbba2KrZQkvrah+SJ5LdTQxdE/4OCkIChOuvgDXPwDDv0J4qjg4uBNGhAV9YbknXvePTfnvavYuuYKxpptUntHZ1d3pKe3r39gcCg6PLLtWlVH5VnV0i1nV5FdrmsmzwpN6HzXdrhsKDrfUSpr/v5OjTuuZplbom7zfUMumVpRU2VBVCE6kS/LwssrhldvNAoithzb85O8UaWsEI2zBAsi9hMkQxBHGCkreoc8DmBBRRUGOEwIwjpkuPTkkASDTdw+POIcQlqwz9FAL2mrVMWpQia2Qt8SZbmQNSn3e7qBWqW/6PQ6pIxhij2yK/bCHtg1e2Lvv/bygh6+lzqtSkvL7cLQ8Vjm7V+VQatA+VP1p2eBIhYDrxp5twPGP4Xa0tcOT14yS+kpb5qds2fyf8aa7J5OYNZe1YtNnj79w49CXujGaEDJ7+P4CbZnE8m5xOzmfHxlNRxVBOOYxAzNYwErWEcKWep/hEvc4FaKSxtSSkq3SqW2UDOKLyHlPgBnm5vF

View Slide

• ઢܗΧϧϚϯϑΟϧλʹΑΓ࿹͝ͱͷঢ়ଶͷฏۉͱ෼ࢄڞ෼ࢄΛಘΔ
• ಘΒΕͨঢ়ଶͷฏۉͱ෼ࢄڞ෼ࢄΛɺฏۉͱ෼ࢄڞ෼ࢄͱ͢Δଟมྔਖ਼ن෼෍
͔ΒαϯϓϦϯάΛߦ͏
• → ઢܗΧϧϚϯϑΟϧλͷਪఆ஋Λ༻͍ͨ֬཰Ұக๏
28
2. ঢ়ଶਪఆ஋Λ༻͍ͨ֬཰Ұக๏ʹΑΔ࿹ͷબఆ
ଟ࿹όϯσΟοτ໰୊ͱͯ͠ͷ୳ࡧͷ
όϥϯεΛௐ੔͢Δεέʔϧ߲ Λಋೖ
β
খɹɹঢ়ଶɹɹେ
֬཰ີ౓
ଟ͘ͷ৔߹ɺ྘͕બఆ͞Ε
Δ͕ɺ੄໺ͷ޿͍ΦϨϯδ
΋બఆ͞ΕΔՄೳੑ͕͋Δ

View Slide

ܽଛ஋ॲཧʹΑΔԾ૝తͳ୳ࡧ

View Slide

α μ P
30
ઢܗΧϧϚϯϑΟϧλʹΑΔ࿹ͷঢ়ଶਪఆʢ࠶ܝʣ
ϑΟϧλϦϯά
t = t + 1
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
µt|t
= µt + Kvt
= µt + (PtZTF 1
t
)vt
Pt|t
= Pt KFtKT
vt = yt
ˆ
yt
Ft = ZPtZ
T + H
ˆ
yt = Zµt

View Slide

α μ P
31
ઢܗΧϧϚϯϑΟϧλʹΑΔ࿹ͷঢ়ଶਪఆ
ϑΟϧλϦϯά
t = t + 1
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
µt|t
= µt + Kvt
= µt + (PtZTF 1
t
)vt
Pt|t
= Pt KFtKT
vt = yt
ˆ
yt
Ft = ZPtZ
T + H
ˆ
yt = Zµt
෼ࢄڞ෼ࢄΛখ͘͞
෼ࢄڞ෼ࢄΛେ͖͘

View Slide

• ઢܗΧϧϚϯϑΟϧλͰ͸؍ଌ஋͕ಘΒΕͳ͍৔߹΋ܽଛ஋ͱͯ͠ѻ͑Δ
• ͜ͷܽଛ஋ॲཧΛબఆ͞Εͳ͔ͬͨ࿹ʹର͢Δߋ৽ૢ࡞ͱͯ͠औΓೖΕΔ
32
ܽଛ஋ॲཧʹΑΔ୳ࡧͷଅਐ
Ұظઌͷঢ়ଶ༧ଌ ϑΟϧλϦϯά
ઢܗΧϧϚϯϑΟϧλʹ͓͚Δܽଛ஋ॲཧͷαΠΫϧ
t = t + 1
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
µt|t
= µt
Pt|t
= Pt
AAACsnichVFLL8RQFP7UezxmsJHYTEyIjckdJEQiETaW4zEIlUlb12imr7R3JqH6B2wtLKxILMQPEFs2/oDF/ASxJLGxcNqpeAWnae93vnu+0+/eozqG7gnGag1SY1NzS2tbe6Kjs6s7merpXfXsiqvxgmYbtruuKh43dIsXhC4Mvu64XDFVg6+p5flwf63KXU+3rRWx5/AtUylZ+o6uKYKoYmpUVk1fNitB0RcHIhie+ZQHspzIv/MhCIqpDMuyKNI/QS4GGcSRt1NXkLENGxoqMMFhQRA2oMCjZxM5MDjEbcEnziWkR/scARKkrVAVpwqF2DJ9S5RtxqxFedjTi9Qa/cWg1yVlGkPsnl2wJ3bHLtkDe/21lx/1CL3s0arWtdwpJg/7l1/+VZm0Cux+qP70LLCDqcirTt6diAlPodX11f3jp+XppSF/mJ2xR/J/ymrslk5gVZ+180W+dPKHH5W80I3RgHLfx/ETrI5lc+PZscWJzOxcPKo2DGAQIzSPScxiAXkUqP8RrnGDW2lC2pAUSauXSg2xpg9fQjLeAJwJo1o=

View Slide

33
ܽଛ஋ॲཧʹΑΔ୳ࡧͷଅਐ
t = t + 1
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
µt|t
= µt
Pt|t
= Pt
ظ଴͞ΕΔޮՌͦͷ
֬཰Ұக๏ͷ࢓૊ΈʹΑΓબఆػձͷগͳ͍࿹ʹର͢Δ୳ࡧ͕ଅਐ͞ΕΔ

View Slide

34
ܽଛ஋ॲཧʹΑΔঢ়ଶ֬ೝͷলུ
t = t + 1
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
µt|t
= µt
Pt|t
= Pt
τϨϯυ΍पظత੒෼Λߟྀͨ͠༧ଌ
ظ଴͞ΕΔޮՌͦͷ
ঢ়ଶͷਪҠ܏޲ͷ༧ଌʹΑΓɺ੾ସ·ͰͷܦաΛ೺Ѳ͢ΔͨΊͷ୳ࡧΛল͚Δ

View Slide

࣌มߦྻʹΑΔจ຺ʹԠͨ͡
࿹ͷධՁߋ৽

View Slide

α μ P
36
ઢܗΧϧϚϯϑΟϧλʹΑΔ࿹ͷঢ়ଶਪఆʢ࠶ܝʣ
ϑΟϧλϦϯά
t = t + 1
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
µt|t
= µt + Kvt
= µt + (PtZTF 1
t
)vt
Pt|t
= Pt KFtKT
vt = yt
ˆ
yt
Ft = ZPtZ
T + H
ˆ
yt = Zµt

View Slide

• ઢܗΧϧϚϯϑΟϧλʹ͓͚Δߦྻ ΍ ͸ϑΟϧλϦϯά΍Ұظઌ༧ଌͷॲ
ཧʹ͓͍ͯঢ়ଶͷฏۉ΍෼ࢄڞ෼ࢄͷͲͷཁૉΛߋ৽͢Δ͔ܾఆ͍ͯ͠Δ
Z R
37
จ຺ʹԠͨ͡࿹ͷධՁߋ৽
ϑΟϧλϦϯά
t = t + 1
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
µt|t
= µt + Kvt
= µt + (PtZTF 1
t
)vt
Pt|t
= Pt KFtKT
vt = yt
ˆ
yt
Ft = ZPtZ
T + H
ˆ
yt = Zµt
ঢ়ଶਪҠ࣌ͷޡࠩΛ෇༩
؍ଌ࣌ͷޡࠩΛ෇༩

View Slide

• → ଟ༷ͳจ຺Λѻ͏ઃఆͰ͸ৗʹಉ͡ߦྻΛ༻͍ͨߋ৽͸ద͞ͳ͍
Z R
38
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
RQRT =
2
4
2
v
2
v
2
v
2
v
2
v
2
v
2
v
2
v
2
v
3
5 , where
R =
2
4
1
1
1
3
5 , Q =
⇥
2
v
⇤
AAADoHictVE/T9tQED/j0kJKS2gX1C4WEYgBRc+hEqgSUkQXtmJDAIEhsp1L8oT/yX4JDVaGrnwBhk5FQlXVpZ36Abr0C3TgI1QdQWLp0LNjgQgFJs6y7+737nfvdz4rcHgkGDuRBuQHgw8fDQ3nHo88eTqaH3u2Fvmt0MaK7Tt+uGGZETrcw4rgwsGNIETTtRxct3bfJOfrbQwj7nurohPgtms2PF7ntikIqo5JL3RN3zFcUzSjerzaXTAsbHAvtggJ+btuTlEUI+IN16y2d0rK1E2JYdxLpYFe7ULLzF4TQ0z4+v9kqkqv9YXvYyvaHcP131fNF1iRpaZcD9QsKEBmy37+OxhQAx9saIELCB4Iih0wIaJnC1RgEBC2DTFhIUU8PUfoQo64LapCqjAJ3aVvg7KtDPUoT3pGKdumWxx6Q2IqMMl+sc/slP1kX9hv9vfGXnHaI9HSIW/1uBhURw/GV87vZLnkBTQvWbdqFlCH+VQrJ+1BiiRT2D1+e//wdOW1PhlPsSP2h/R/ZCfsB03gtc/sYw31D7fosUgL/TFakNq/juvBWqmozhZL2qtCeTFb1RC8hAmYpn3MQRmWYBkqYEvvpU/SV+mbPCEvyW9lrVc6IGWc53DF5M1/0sTvXA==
ഉଞతͳίϯςΩετͷ࣍ݩʹ΋ޡ͕ࠩ஝ੵͯ͠͠·͏
ίϯςΩετ"Ͱऑ͍࿹͸ɺܽଛ஋ॲཧʹΑΓଞͷίϯςΩετʹରͯ͠΋ޡࠩΛ෇༩͢Δɻ
ίϯςΩετ#Ͱճఔ౓ͷ୳ࡧͰྑ͍΋ͷ͕ճ୳ࡧ͞ΕΔ͔΋͠Εͳ͍

View Slide

• → จ຺ʹԠͨ࣌͡มͳߦྻΛ༻ҙ͢Δ
Z R
39
µt+1 = Tµt|t
Pt+1 = TPt|t
TT + RQRT
ίϯςΩετͷԠͨ͡
Λ༻ҙ͢Δʢ ʹ͍ͭͯ΋ಉ༷ʣ
R R(xt
) Z
RtQRT
t
=
2
4
2
v
0 2
v
0 0 0
2
v
0 2
v
3
5 , where
Rt =
2
4
1
0
1
3
5 , Q =
⇥
2
v
⇤
AAADgXichVHNThRBEK5h/MFFZdWLiZeOGwgmZlOzmEggJkQvHNmFBRIGJtND726H+ctM7wpO9m68wcUDJ0k4GB/AB/DiC3jgEYxHSLxwsOYHiIsuNemp6q/rq/q6i4eujBXisTai37h56/bondLY3Xv3x8sPHq7EQTdyRNMJ3CBa43YsXOmLppLKFWthJGyPu2KVb79Jz1d7Iopl4C+r3VBseHbbly3p2Iogq3zWsFSd1qbp2aoTt5Ll/iuTi7b0E05IJHf6JcaYGcu2Z1u9zRqbZEjrEjDNNAHZheUJmOPDiabwty76PH/bEZFIaaTnXyIMdt4r98YAn9WvkT7Y0SpXsIqZsauBUQQVKGwxKH8FE7YgAAe64IEAHxTFLtgQ07cOBiCEhG1AQlhEkczOBfShRNwuZQnKsAndpn+bdusF6tM+rRlnbIe6uLQiYjKYwB/4GU/wO37Bn3j231pJViPVskue51wRWuMfHi/9vpblkVfQuWQN1aygBTOZVknawwxJb+Hk/N67jydLs42JZBIP8Rfp/4TH+I1u4PdOnaO6aBwM0cNJC70YDcgYHMfVYKVWNaartfqLyvzrYlSj8ASewhTN4yXMwwIsQhMcjWvvtT1tX9f1ZzrqtTx1RCs4j+Av0+f+ALya4JM=
Rt = xt = (1, 0, 1)> (xt
2 {0, 1}m)
AAACunichVFNS9xQFD1Ga/1onVE3QjfBwTKCDDdWUYqCtBuXOjoqGA1J+rSPyRfJm0Eb5g/0D3ThqoKL0h/gWropunbhTyhdKrjponcyKWKl9obknXfuPTfnvetEnkwU0VWX1t3zpPdpX//A4LPnQ4Xi8MhGEjZiV9Tc0AvjLcdOhCcDUVNSeWIrioXtO57YdOpv2/nNpogTGQbr6jASO769H8g96dqKKas4W7XUoun46UHLUvqiXjamaMqY3E1NFUYtUy//yZkyMFNOma1df9IqlqhCWegPgZGDEvJYCYunMPEOIVw04EMggGLswUbCzzYMECLmdpAyFzOSWV6ghQHWNrhKcIXNbJ2/+7zbztmA9+2eSaZ2+S8evzErdUzQJX2ha/pOX+kH/fpnrzTr0fZyyKvT0YrIKnwcW7v9r8rnVeH9nepRzwp7mM+8SvYeZUz7FG5H3/zw6XrtdXUifUnH9JP9f6Yr+sYnCJo37smqqB494sdhL3xjPCDj73E8BBvTFeNVZXp1prT0Jh9VH15gHGWexxyWsIwV1Lj/Ec5wjgttQXM0qdU7pVpXrhnFvdDUbySjo7c=

View Slide

ํࡦͷಛੑΛ෼ੳ͢ΔධՁࢦඪ

View Slide

• ͜Ε·ͰͷࢦඪͰ͋ΔྦྷੵϦάϨοτ͸ɺ֤࣌఺Ͱ࠷ద࿹ͷظ଴஋ͱબఆͨ͠
࿹ͷظ଴஋ͷࠩΛظؒ·Ͱʹ߹ܭͨ͠΋ͷ
• มԽ͢Δ؀ڥʹ͓͍ͯ͸ɺਖ਼͍͠൑அʹجͮ͘୳ࡧͳͷ͔ɺͦ΋ͦ΋௥ै
͕Ͱ͖͍ͯͳ͍ঢ়ଶͰ୳ࡧͱͳ͍ͬͯΔͷ͔͕۠ผͰ͖ͳ͍
• มԽݕग़·Ͱͷੑೳ΍ɺݕग़ޙͷ௥ैੑೳΛൺֱ͕ɺఆੑతͳߟ࡯ʹͱͲ
·Δ͔ɺ෼ੳͷ؍఺͕ෆ౷Ұʹͳͬͯ͠·͏
41
ํࡦͷੑೳൺֱʹ͓͚Δ՝୊
มԽʹର͢Δ֤ํࡦͷڍಈͷಛੑΛఆྔతʹࣔ͢ࢦඪΛఏҊ

View Slide

• ֤࣌఺ͷࢼߦ݁ՌΛɺํࡦͷ൑அʢ׆༻͔୳ࡧ͔ʣͱબఆͨ͠࿹ͷਅͷ༗༻ੑ
ʢ࠷ద࿹͔Ͳ͏͔ʣͱͷ࢛৅ݶʹ෼ྨͨࠞ͠ಉߦྻʢද1ʣ
42
ํࡦͷ࿹ͷબఆʹؔ͢Δࠞಉߦྻ
࠷ద࿹͕ೖΕସΘΔঢ়گ
ʹ͓͚Δޡ൑ఆͷ਺
൪ڰΘͤͰ͸ͳ͍৔߹
ᶅʹ૬౰͢Δࢼߦ͕ɺ࿹ͷධՁͷߋ৽
ͱ੾ΓସΘΓͷݕग़Λ݉ͶΔ
൪ڰΘͤͷ৔߹
ᶄʹ૬౰͢Δࢼߦ͕ɺ৽ͨͳ࠷ద࿹ͷධՁͷ
ߋ৽ʹඞཁɻ৽ͨͳ࠷ద࿹΁ͷ੾Γସ͑ʹ൐
͍ᶅͷ૿Ճ͕ࢭ·Δ
มԽʹڧ͍ํࡦ͸ɺᶃʹର͢Δᶄͱᶅͷ
ൺ཰͕ͦΕͧΕখ͍͞ͱԾఆ

View Slide

• ֤࣌఺ͷࢼߦ݁ՌΛɺํࡦͷ൑அʢ׆༻͔୳ࡧ͔ʣͱબఆͨ͠࿹ͷਅͷ༗༻ੑ
ʢ࠷ద࿹͔Ͳ͏͔ʣͱͷ࢛৅ݶʹ෼ྨͨࠞ͠ಉߦྻʢද1ʣ
43
มԽ΁ͷ௥ैੑͱ୳ࡧޮ཰ͷࢦඪ ( ஋)
F0.5
3FDBMMᶃ ᶃᶄ

൪ڰΘͤͷ৔߹
ᶄʹ૬౰͢Δࢼߦ͕ɺ৽ͨͳ࠷ద࿹ͷධՁͷ
ߋ৽ʹඞཁɻ৽ͨͳ࠷ద࿹΁ͷ੾Γସ͑ʹ൐
͍ᶅͷ૿Ճ͕ࢭ·Δ
൪ڰΘͤͰ͸ͳ͍৔߹
ᶅʹ૬౰͢Δࢼߦ͕ɺ࿹ͷධՁͷߋ৽
ͱ੾ΓସΘΓͷݕग़Λ݉ͶΔ
1SFDJTJPOᶃ ᶃᶅ

F0.5
= 1.25
Precision ⋅ Recall
0.25Precision + Recall
ͳ͓ɺᶄ͸ޡ൑ఆͰ͋Δ΋ͷͷɺ࠷ద࿹Λ
બఆ͓ͯ͠Γ୹ظతͳػձଛࣦ͕গͳ͍ͨ
Ί1SFDJTJPOΛॏࢹ͢Δ

View Slide

4.
ධՁ

View Slide

• จ຺ͱ࣌ؒͷܦաʹԠͯ͡࿹ͷ༗༻ੑ͕มԽ͢Δঢ়گͷγϛϡϨʔγϣϯ
• όϯσΟοτํࡦͰબఆͨ͠࿹͔ΒಘΒΕΔΫϦοΫ਺ΛγϛϡϨʔτ
• ֤࿹ͷύϥϝʔλ ͱίϯςΩετ ͷઢܗ࿨ΛΫϦοΫ཰ͱ͢Δ
• ಛఆͷ࿹ʹ͓͍ͯύϥϝʔλͷ2࣍ݩ໨Λ࣌ؒͷܦաʹԠͯ͡มಈͤ͞Δ
• ࿹͸10ຊ
• ίϯςΩετ͸
θ ∈ R4 x
x ∈ X
45
ධՁํ๏

View Slide

• จ຺ͱ࣌ؒͷܦաʹԠͯ͡࿹ͷ༗༻ੑ͕มԽ͢Δঢ়گͷγϛϡϨʔγϣϯ
• όϯσΟοτํࡦͰબఆͨ͠࿹͔ΒಘΒΕΔΫϦοΫ਺ΛγϛϡϨʔτ
• ֤࿹ͷύϥϝʔλ ͱίϯςΩετ ͷઢܗ࿨ΛΫϦοΫ཰ͱ͢Δ
• ಛఆͷ࿹ʹ͓͍ͯύϥϝʔλͷ2࣍ݩ໨Λ࣌ؒͷܦաʹԠͯ͡มಈͤ͞Δ
• ࿹͸10ຊ
• ίϯςΩετ͸
θ ∈ R4 x
x ∈ X
46
ධՁํ๏
൪ڰΘͤͳ੾ସ
൪ڰΘͤͰ͸ͳ͍੾ସ

View Slide

• ఏҊख๏Ͱ͋ΔLKF͸ঢ়ଶͷ࣍ݩ਺ΛίϯςΩετ ͱἧ͑ͨϞσϧΛར༻
• จ຺෇͖ɺ͔ͭɺඇఆৗͳํࡦͰ͋ΔDecay LinUCBɺAdTSɺdLinUCBɺ
DenBandɺTVTPɺADA-ILTCB+ͱൺֱ
• ͨͩ͠ɺADA-ILTCB+͸ຊઃఆʹ͓͚Δ࣮ߦ࣌ؒͷ౎߹ɺ݁Ռ͔Βআ֎
• ࿹ͷධՁ͕ఆ·ͬͨͷͪͷมԽ΁ͷ௥ैੑΛධՁ͢ΔͨΊ֤࿹ͷ֤ίϯςΩε
τ͝ͱʹ50ճͷࢼߦ݁ՌΛ༧Ί༩͑Δ
• ཚ਺Λ༻͍ͨγϛϡϨʔγϣϯ݁ՌΛฏۉԽ͢ΔͨΊ50ճͷฏۉΛ݁Ռͱ͠
ͯ༻͍Δɻ·֤ͨํࡦͷύϥϝʔλ͸ࣄલͷ༧උධՁʹΑΓྦྷੵΫϦοΫ਺͕
ଟ͍΋ͷΛબఆͨ͠
x
47
ධՁํ๏

View Slide

ྦྷੵϦάϨοτͷਪҠʢ্: શίϯςΩετɺԼ: ʣ
x = (0,1,0,0)
48
ػձଛࣦͷධՁ
• DenBand(94.3)ɺఏҊख๏ͷ
LKF(94.8)ͱ΄΅ಉ౳ͷੑೳ
• ൪ڰΘͤ΁௥ैͨ͜͠ͱͰશମ
తͳϦάϨοτͷ૿ՃΛ཈͑ͨ

View Slide

49
ػձଛࣦͷධՁ
• ํࡦ͝ͱͷ࿹ͷબఆ਺ʹΑΔൺֱ
• ࠷্ஈ͸ཧ૝తͳબఆͰͷਪҠ
• Decay LinUCB͸TVTPͱɺdLinUCB͸DenBandͱಉ͡
ਪҠಛੑͷͨΊলུ
• ͍ͣΕͷख๏΋൪ڰΘͤͰ͸ͳ͍࠷ॳͷ੾ସʹ
͸௥ैʢᒵ৭ͷ૿Ճʣ
• AdTSɺTVTPɺʢDecay LinUCBʣ͸൪ڰΘͤ΁
ͷ௥ै͕஗Ε͍ͯΔʢ྘৭͕؇΍͔ʹ૿Ճʣ
• DenBandɺʢdLinUCBʣɺఏҊख๏ͷLKF͸൪
ڰΘͤʹ΋ਝ଎ʹ௥ैʢ྘৭ͷٸܹʹ૿Ճʣ

View Slide

50
มԽ΁ͷ௥ैੑͱ୳ࡧޮ཰ͷධՁ
Ͱͷ ஋ͷਪҠʢ+: Precision,ɾ:Recallʣ
x = (0,1,0,0) F0.5 • ఏҊख๏ͷLKF͕ߴ͍ਫ४ͰͷPrecisionͱ
RecallΛཱ྆͠ɺ஋͕࠷΋େ͖͍
• = ޡ൑ఆͷظؒͱ৽ͨͳ࿹΁ͷ୳ࡧΛ࠷খݶ
ʹ཈͍͑ͯΔ
• ൪ڰΘͤ΁ͷ௥ै͕஗ΕͨํࡦͰ͸࠷ద࿹΁ͷ
୳ࡧᶄ͕গͳ͍ʢRecall͸େ͖͍ʣ͕ɺҎલͷ
࠷ద࿹΁ͷࢼߦᶅ͕ଟ͍ʢPrecision͕খ͍͞ʣ

View Slide

51
x = (0,1,0,0) F0.5 • AdTSͰ΋൪ڰΘͤ΁ͷ௥ै͕஗ΕͨͨΊLKFΑ
Γ ஋͕௿͍ɻ
• ൪ڰΘͤͷݕग़ͷͨΊຫੑతʹ୳ࡧᶆ͕ଟ͍
͜ͱ͔ΒɺશମͰͷϦάϨοτ͸૿Ճ͍ͯ͠
Δ͕มԽظؒʹ͓͚ΔੑೳΛൺֱ͢Δຊࢦඪ
͔Β͸ಡΈऔΕͳ͍ɻ
→ ຊࢦඪʹ͓͚ΔɺมԽͷͳ͍ظؒͷ୳ࡧޮ
཰ͷ૊ΈࠐΈ͸ࠓޙͷ՝୊ͱ͢Δ
F0.5

View Slide

52
x = (0,1,0,0) F0.5 • dLinUCBͱDenBandͰ͸LKFͱಉ౳ఔ౓ͷϦά
Ϩοτʹ΋ؔΘΒͣLKFΑΓ΋ ஋͕௿͍ɻ
• ࠷ద࿹΁ͷ୳ࡧᶄ͕ۃ୺ʹଟ͍͜ͱʹىҼ͢
Δɻ৽ͨͳ࠷ద࿹΁ͷ୳ࡧΛੵۃతʹߦ͍ͭ
ͭ΋ධՁͷߋ৽͸؇΍͔ͳํࡦͱΘ͔Δ
→ ൪ڰΘ͕ͤ୹ظؒʹߴස౓Ͱൃੜ͢ΔΑ͏
ͳ؀ڥͰͷ՝୊͕͋ΔՄೳੑΛߟྀͨ͠஋ͱ
ΈͳͤΔ
F0.5
ද
ᶃ ᶄ ᶅ ᶆ
%FDBZ-JO6$#
"E54
E-JO6$#
%FO#BOE
5751
-,'

View Slide

• ਝ଎ʹධՁΛߋ৽͢ΔͨΊͷػߏͱ
ͯ͠ɺTVTP͸ཻࢠϑΟϧλɺAdTS͸
ϒʔτετϥοϓΛ࠾༻ͨ͜͠ͱͰ
͜ΕΒͷػߏͷ࣮ߦʹ͕͔͔࣌ؒͬ
͍ͯΔ
• DenBand͸ᮢ஋ͷઃఆʹΑͬͯ͸ࢼ
ߦճ਺ͷ૿Ճʹ൐͍ɺϞσϧ਺ͷ୹
ௐ૿Ճ͕ൃੜ͠ɺ࣮ߦ͕࣌ؒ૿Ճ͠
ଓ͚Δ՝୊ΛؚΜͰ͍Δ
53
࣮ߦ࣌ؒͷධՁʢ௥ՃධՁʣ
ఏҊख๏͸࿹ͷ਺ʹൺྫͯ͠ߋ৽ͷܭ
ࢉྔ͕૿Ճ͢ΔͨΊɺ͜ͷ؍఺Ͱͷൺ
ֱධՁ΋ਐΊ͍ͨ

View Slide

5.
͓ΘΓʹ

View Slide

• ൪ڰΘͤΛؚΉ؀ڥʹ΋ਝ଎͔ͭޮ཰తʹ௥ैՄೳͳɺઢܗΧϧϚϯϑΟϧλ
ͱܽଛ஋ॲཧΛ༻͍ͨଟ࿹όϯσΟοτํࡦΛఏҊͨ͠
• มԽ΁ͷ௥ैੑͱޮ཰ੑΛৄࡉʹ෼ੳ͢ΔͨΊͷࢦඪͱͯ͠ ஋ΛఏҊͨ͠
• ػձଛࣦͷ௿ݮͳΒͼʹ্هࢦඪʹ͓͍ͯɺैདྷͷ࠷ઌ୺ͷํࡦͱൺ΂Ḯ৭͕
ͳ͍͔ɺվળ͢Δ͜ͱΛ֬ೝͨ͠
• ࠓޙ͸τϨϯυ΍पظత੒෼ͷߟྀͷ༗༻ੑΛධՁ͍ͨ͠
• ·ͨɺޡࠩͷ෼ࢄڞ෼ࢄͷஞ࣍తͳਪఆʹΑΔ୳ࡧޮ཰ͷվળͳΒͼʹఏҊࢦ
ඪͷվળΛਐΊ͍ͨ
F0.5
55
͓ΘΓʹ

View Slide

View Slide

仮想的な探索を用いて文脈や時間の経過による番狂わせにも迅速に追従する多腕バンディット手法/wi2_lkf_bandits

仮想的な探索を用いて文脈や時間の経過による番狂わせにも迅速に追従する多腕バンディット手法/wi2_lkf_bandits

monochromegane

More Decks by monochromegane

Other Decks in Research

Featured

Transcript