$30 off During Our Annual Pro Sale. View Details »

Synapse: 文脈と時間経過に応じて推薦手法の選択を最適化するメタ推薦システム/smash21-synapse

monochromegane
September 16, 2021

Synapse: 文脈と時間経過に応じて推薦手法の選択を最適化するメタ推薦システム/smash21-synapse

monochromegane

September 16, 2021
Tweet

More Decks by monochromegane

Other Decks in Research

Transcript

  1. ࡾ୐ ༔հ1,2ɺ็ ߃ݑ3
    1. Pepabo R&D Institute, GMO Pepabo, Inc., 2. ۝भେֶ େֶӃγεςϜ৘ใՊֶ෎ ৘ใ஌ೳ޻ֶઐ߈
    3. ۝भେֶ େֶӃγεςϜ৘ใՊֶݚڀӃ ৘ใ஌ೳ޻ֶ෦໳
    2021.09.16 SMASH21 Summer Symposium
    Synapse: จ຺ͱ࣌ؒܦաʹԠͯ͡
    ਪનख๏ͷબ୒Λ࠷దԽ͢Δ
    ϝλਪનγεςϜ

    View Slide

  2. 1. ͸͡Ίʹ
    2. ؔ࿈ݚڀ
    3. ఏҊख๏
    4. ධՁ
    5. ͓ΘΓʹ
    2
    ໨࣍

    View Slide

  3. 1.
    ͸͡Ίʹ

    View Slide

  4. • ৘ใγεςϜʹ͓͚Δ৘ใաଟ໰୊Λղܾ͢ΔɺਪનγεςϜͷಋೖ
    • ͳΜΒ͔ͷํࡦʢ=ਪનख๏ʣʹج͖ͮଟ਺ͷબ୒ࢶ͔Βར༻ऀ͕ڵຯΛ࣋
    ͭ΋ͷΛఏҊ͢ΔγεςϜ
    4
    എܠ
    6TFST
    3FDPNNFOEBUJPO
    .FUIPE
    1SPEVDUT
    IUUQTJDPOTDPN

    View Slide

  5. • ৘ใγεςϜʹ͓͚Δ৘ใաଟ໰୊Λղܾ͢ΔɺਪનγεςϜͷಋೖ
    • ͳΜΒ͔ͷํࡦʢ=ਪનख๏ʣʹج͖ͮଟ਺ͷબ୒ࢶ͔Βར༻ऀ͕ڵຯΛ࣋
    ͭ΋ͷΛఏҊ͢ΔγεςϜ
    • ਺ଟ͘ͷਪનख๏͕ఏҊ͞Ε͍ͯΔ
    5
    എܠ
    6TFST
    3FDPNNFOEBUJPO
    .FUIPET
    1SPEVDUT
    IUUQTJDPOTDPN

    View Slide

  6. • ৘ใγεςϜʹ͓͚Δ৘ใաଟ໰୊Λղܾ͢ΔɺਪનγεςϜͷಋೖ
    • ͳΜΒ͔ͷํࡦʢ=ਪનख๏ʣʹج͖ͮଟ਺ͷબ୒ࢶ͔Βར༻ऀ͕ڵຯΛ࣋
    ͭ΋ͷΛఏҊ͢ΔγεςϜ
    • ਺ଟ͘ͷਪનख๏͕ఏҊ͞Ε͍ͯΔ → ޮՌతͳʮਪનख๏ͷબఆʯ͕ॏཁ
    6
    എܠ
    6TFST
    #FTU3FDPNNFOEBUJPO
    .FUIPE
    1SPEVDUT
    IUUQTJDPOTDPN

    View Slide

  7. 7
    എܠ
    • ޮՌతͳਪનख๏͸ঢ়گ΍࣌ؒͷܦաʹΑͬͯҟͳΔ
    • ͔͠͠ͳ͕Βɺ࣮؀ڥͰͷܧଓతͳਪનख๏ͷධՁʹ͸ػձଛࣦ͕൐͏
    ӡ༻্ͷ՝୊
    • ৘ใγεςϜʹ͓͚Δ৘ใաଟ໰୊Λղܾ͢ΔɺਪનγεςϜͷಋೖ
    • ͳΜΒ͔ͷํࡦʢ=ਪનख๏ʣʹج͖ͮଟ਺ͷબ୒ࢶ͔Βར༻ऀ͕ڵຯΛ࣋
    ͭ΋ͷΛఏҊ͢ΔγεςϜ
    • ਺ଟ͘ͷਪનख๏͕ఏҊ͞Ε͍ͯΔ → ޮՌతͳʮਪનख๏ͷબఆʯ͕ॏཁ

    View Slide

  8. • ਪનख๏ͷ༏ྼ͸ঢ়گʢ=จ຺ʣͱ࣌ؒͷܦաʹΑͬͯࠨӈ͞ΕΔ
    • ޮՌతͳਪનख๏Λػձଛࣦ͕ͳ͍Α͏ʹจ຺ͱ࣌ؒͷܦաʹԠͯ͡࢖͍෼͚
    ͍ͨ
    • จ຺ͱ࣌ؒͷܦաʹԠͯ͡ਪનख๏ͷબ୒Λࣗಈ͔ͭܧଓతʹ࠷దԽ͢Δ
    ϝλਪનγεςϜ
    • → ࠷ળͳਪનख๏ͷબ୒Λଟ࿹όϯσΟοτ໰୊ͱΈͳͯ͠ղ͘
    8
    ݚڀͷ໨తͱఏҊͷࠎࢠ

    View Slide

  9. 2.
    ؔ࿈ݚڀ

    View Slide

  10. ଟ࿹όϯσΟοτ໰୊

    View Slide

  11. • ʮ࿹ʯͱݺ͹ΕΔෳ਺ͷީิ͔ΒಘΒΕΔใुΛ࠷େԽ͢Δ໰୊
    • ϓϨΠϠʔ͸Ұ౓ͷࢼߦͰ1ͭͷ࿹Λબ୒͠ɺใुΛಘΔ
    • ͦΕͧΕͷ࿹͸͋Δใु෼෍ʹै͍ใुΛੜ੒
    • ͨͩ͠ɺϓϨΠϠʔ͸͜ͷใु෼෍Λࢼߦͷ݁Ռ͔Βਪଌ͢Δඞཁ͕͋Δ
    11
    ଟ࿹όϯσΟοτ໰୊
    • ϓϨΠϠʔ͸͋Δ࣌఺ͷ࿹ͷධՁʹج͖ͮʮ׆༻ʯͱʮ୳ࡧʯΛฒߦͯ͠ߦ͏
    • ͜ͷτϨʔυΦϑΛղফ͢ΔͨΊʹ༷ʑͳղ๏͕ఏҊ͞Ε͍ͯΔ

    View Slide

  12. • ࿹͝ͱͷใु෼෍͸ৗʹಉ͡Ͱ͋Δͱ͍͏Ծఆ
    • → ঢ়گ΍ଐੑ͝ͱʹใु෼෍͕ҟͳΔͷͰ͸ͳ͍͔ʁ
    • ྫʣ೥୅͝ͱʹਓؾͷ঎඼͕ҧ͏ɺ࠷ۙɺಉ͡ΧςΰϦͷ঎඼Λങͬͨ
    • → ࣌ؒͷܦաʹΑͬͯ΋ใु෼෍͕มԽ͢ΔͷͰ͸ͳ͍͔ʁ
    • ྫʣ৽͍͠χϡʔεهࣄɺҰ࣌తͳྲྀߦ
    12
    ଟ࿹όϯσΟοτ໰୊ͷ֦ு
    • ʮจ຺෇͖ʯ·ͨ͸ʮඇఆৗʯͳଟ࿹όϯσΟοτ໰୊ͱ֦ͯ͠ு͞Ε͍ͯΔ

    View Slide

  13. ਪનख๏ͷબఆʹ͓͚Δ
    ؍఺ͷ੔ཧ

    View Slide

  14. • ਪનख๏ͷ૬ରతͳੑೳͷ༏ྼΛॿ௕͢ΔཁҼͷଘࡏ
    14
    ਪનख๏ͷબఆʹ͓͚Δจ຺ͷߟྀͷॏཁੑ
    <>&LTUSBOE .BOE3JFEM +8IFOSFDPNNFOEFSTGBJMQSFEJDUJOHSFDPNNFOEFSGBJMVSFGPSBMHPSJUINTFMFDUJPOBOEDPNCJOBUJPO
    1SPDFFEJOHTPGUIFTJYUI"$.DPOGFSFODFPO3FDPNNFOEFSTZTUFNT QQr

    <>#SBVOIPGFS . $PEJOB 7BOE3JDDJ '4XJUDIJOHIZCSJEGPSDPMETUBSUJOHDPOUFYUBXBSFSFDPNNFOEFSTZTUFNT 1SPDFFEJOHTPGUIF
    UI"$.$POGFSFODFPO3FDPNNFOEFSTZTUFNT QQr

    <>"OEFSTPO " .BZTUSF - "OEFSTPO * .FISPUSB 3BOE-BMNBT ."MHPSJUINJDF⒎FDUTPOUIFEJWFSTJUZPGDPOTVNQUJPOPOTQPUJGZ
    1SPDFFEJOHTPG5IF8FC$POGFSFODF QQr

    ઌߦݚڀ ֓ཁ ਪનख๏ͷબఆʹؔ͢ΔཁҼͷ෼ੳɾར༻
    <>
    ϋΠϒϦουਪનʹ͓͚Δਪનख๏
    ͷબఆࣦഊͷݪҼΛ୳ͬͨݚڀ
    ར༻ऀ͝ͱʹਪનख๏Λ࢖͍෼͚Δ͜ͱͰੑೳ͕վળ͞ΕΔͱใࠂ͠ɼ
    ͦͷཁҼͷ෼ੳ͕ඞཁ
    <>
    ίʔϧυελʔτͷঢ়گʹݶఆͨ͠
    ϋΠϒϦουਪનͷݚڀ
    ਪનख๏ͷબ୒ʹӨڹΛٴ΅͢ཁҼͱͯ͠ɼར༻ऀ΍঎඼ɼίϯςΩε
    τʹର͢ΔධՁͷ஝ੵ۩߹Λར༻
    <>
    ԻָετϦʔϛϯάαʔϏεʹ͓͚
    Δࢹௌ܏޲ͱਪનख๏ͷޮՌͷݚڀ
    ਪનख๏ͷબ୒ʹӨڹΛٴ΅͢ཁҼͱͯ͠ɼࢹௌ܏޲ͷଟ༷ੑʹண໨Ͱ
    ͖Δ͜ͱΛࣔͨ͠

    View Slide

  15. • ར༻ऀͷᅂ޷৘ใͷ஝ੵʹ൐͍ɺਪનਫ਼౓ͷ޲্͕ظ଴Ͱ͖Δਪનख๏ͷଘࡏ
    15
    ਪનख๏ͷબఆʹ͓͚Δ࣌ؒͷܦաͷߟྀͷॏཁੑ
    ಺༰ϕʔεͷਪનख๏͸Ұఆͷਫ਼౓
    ᅂ޷৘ใΛ༻͍Δਪનख๏͸ਫ਼౓͕ঃʑʹ޲্
    ਪનख๏ͷ༗ޮੑ͕ඇఆৗͰ͋Δ͜ͱ
    Λલఏͱͨ͠ղ๏͕ඞཁͱͳΔ

    View Slide

  16. ଟ࿹όϯσΟοτ໰୊ͷղ๏Λ༻͍ͨ
    ਪનख๏ͷબఆ

    View Slide

  17. • ઌߦݚڀʹ͓͚Δจ຺ͱ࣌ؒͷܦաͷߟྀ
    17
    ଟ࿹όϯσΟοτ໰୊ͷղ๏Λ༻͍ͨਪનख๏ͷબఆ
    ઌߦݚڀ จ຺ ࣌ؒͷܦա
    <> º º
    <> <> ˚ º ࿹ͱͳΔਪનख๏͕จ຺Λѻ͏͕ਪનख๏ͷબఆʹ͸จ຺Λߟྀ͠ͳ͍
    <> <> ˓ º <>͸ਪન࣌ͷӾཡதͷ঎඼ಛੑΛจ຺ͱͯ͠ར༻ʢ<>͸ఏࣔͳ͠ʣ
    ˎ͍ͣΕͷख๏΋࣌ؒͷܦաͷߟྀ͕े෼Ͱ͸ͳ͍
    <>'FM
    ’DJP $; 1BJYB
    _P ,7 #BSDFMPT $"BOE1SFVY 1"NVMUJBSNFECBOEJUNPEFMTFMFDUJPOGPSDPMETUBSUVTFSSFDPNNFOEBUJPO 1SPDFFEJOHTPGUIFUI
    $POGFSFODFPO6TFS.PEFMJOH "EBQUBUJPOBOE1FSTPOBMJ[BUJPO QQr

    <>#SPE
    FO # )BNNBS . /JMTTPO #+BOE1BSBTDIBLJT %&OTFNCMFSFDPNNFOEBUJPOTWJBUIPNQTPOTBNQMJOHBOFYQFSJNFOUBMTUVEZXJUIJOF
    DPNNFSDF SEJOUFSOBUJPOBMDPOGFSFODFPOJOUFMMJHFOUVTFSJOUFSGBDFT QQr

    <>$BO
    _BNBSFT 3 3FEPOEP .BOE$BTUFMMT 1.VMUJBSNFESFDPNNFOEFSTZTUFNCBOEJUFOTFNCMFT 1SPDFFEJOHTPGUIFUI"$.$POGFSFODFPO
    3FDPNNFOEFS4ZTUFNT QQr

    <>4BOUBOB .3 .FMP -$ $BNBSHP ') #SBOE
    _BP # 4PBSFT " 0MJWFJSB 3.BOE$BFUBOP 4$POUFYUVBM.FUB#BOEJUGPS3FDPNNFOEFS4ZTUFNT
    4FMFDUJPO 'PVSUFFOUI"$.$POGFSFODFPO3FDPNNFOEFS4ZTUFNT QQr

    <>ࡾ୐༔հɼ็߃ݑ4ZOBQTFจ຺ʹԠͯ͡ܧଓతʹਪનख๏ͷબ୒Λ࠷దԽ͢ΔਪનγεςϜ ిࢠ৘ใ௨৴ֶձ࿦จࢽ% 7PM /P QQr


    View Slide

  18. 3.
    ఏҊख๏

    View Slide

  19. • จ຺ͱ࣌ؒͷܦաʹԠͯ͡ਪનख๏ͷબ୒Λࣗಈ͔ͭܧଓతʹ࠷దԽ͢Δ
    ϝλਪનγεςϜ
    • ػձଛࣦΛ཈͑ͨจ຺ͱ࣌ؒͷܦաʹԠͨ͡࠷ળͳબ୒Λୡ੒͢ΔͨΊɺ
    จ຺෇͖ɺ͔ͭɺඇఆৗͳଟ࿹όϯσΟοτ໰୊ͷղ๏Λ༻͍Δ
    • ਪનख๏ͷ༗ޮੑ͕ٯస͢Δঢ়گʢޙड़ʣʹରॲ͢ΔͨΊɺैདྷͷղ๏Λ
    ֦ு͠ɺੵۃతͳ୳ࡧΛଅ͢
    • બ୒ͨ͠ਪનख๏ͱ͜Εʹର͢Δར༻ऀͷ൓ԠΛه࿥͠ɺબ୒ͷܧଓతͳ
    վળʹ༻͍Δ
    19
    ఏҊγεςϜ (Synapse)

    View Slide

  20. 20
    ఏҊγεςϜ (Synapse)

    View Slide

  21. จ຺ͱ࣌ؒͷܦաΛߟྀͨ͠
    ଟ࿹όϯσΟοτ໰୊ͷղ๏

    View Slide

  22. 22
    Time-varying Thompson Sampling (TVTP) [15]
    Dynamic Context drift Modeling
    จ຺෇͖ɺ͔ͭɺඇఆৗͳใुͷมԽΛ
    ѻ͏ͨΊใुͷมಈΛ૊ΈࠐΜͩϞσϧ
    ͕༻͍ΒΕΔ
    <>;FOH $ 8BOH 2 .PLIUBSJ 4BOE-J 50OMJOFDPOUFYUBXBSFSFDPNNFOEBUJPOXJUIUJNFWBSZJOHNVMUJBSNFECBOEJU 1SPDFFEJOHTPGUIF
    OE"$.4*(,%%JOUFSOBUJPOBMDPOGFSFODFPO,OPXMFEHFEJTDPWFSZBOEEBUBNJOJOH QQr

    <>'JH(SBQIJDBMNPEFMSFQSFTFOUBUJPOGPSCBOEJUQSPCMFN
    yk,t
    ⇠ N(xT
    t
    (cwk
    + ✓k ⌘k,t), 2
    k
    )
    AAAC13ichVG/axRBFP6y8UeMP3LGRrAJHpELhmPuFBSrkDRWklxySSSbDLOTubthf7I7d3pZlnQidlYWVgoW4h+gvY3/gEUaWxXLCDYWvt1bEA3Gt+zO975539tv5jmRpxPD2MGYNX7i5KnTE2cmz547f2GqcnF6PQn7sVRtGXphvOmIRHk6UG2jjac2o1gJ3/HUhuMu5fsbAxUnOgzWzDBS277oBrqjpTBE8UpryFN33mR2on3bF6YnhZfey2oPudkp8qSTrmU1ydMH3M2u26anjOCuHe6Gxs5hoZ6bJ33XFztN7s7xSpXVWREzR0GjBFWUsRxW3sLGLkJI9OFDIYAh7EEgoWcLDTBExG0jJS4mpIt9hQyTpO1TlaIKQaxL3y5lWyUbUJ73TAq1pL949MaknMEs+8hes0P2gb1hX9nPf/ZKix65lyGtzkirIj715PLqj/+qfFoNer9Vx3o26OB24VWT96hg8lPIkX6w9+xw9U5rNr3GXrJv5P8FO2Dv6QTB4Lt8taJaz4/x45AXujEaUOPvcRwF681640a9uXKzurBYjmoCV3AVNZrHLSzgLpbRpv7v8Amf8cW6b+1bj6zHo1JrrNRcwh9hPf0FKiWypw==
    ίϯςΩετ ఆৗ߲ ඇఆৗ߲
    ʢεέʔϧ߲ɺυϦϑτ߲ʣ
    ؍ଌޡࠩ

    View Slide

  23. 23
    Time-varying Thompson Sampling (TVTP) [15]
    ঢ়ଶۭؒϞσϧ
    ใुϞσϧͷύϥϝʔλͷࣄޙ෼෍ͱυ
    Ϧϑτ߲ͷજࡏঢ়ଶͷஞ࣍ਪఆʹཻࢠ
    ϑΟϧλͱΧϧϚϯϑΟϧλ͕༻͍ΒΕΔ
    <>;FOH $ 8BOH 2 .PLIUBSJ 4BOE-J 50OMJOFDPOUFYUBXBSFSFDPNNFOEBUJPOXJUIUJNFWBSZJOHNVMUJBSNFECBOEJU 1SPDFFEJOHTPGUIF
    OE"$.4*(,%%JOUFSOBUJPOBMDPOGFSFODFPO,OPXMFEHFEJTDPWFSZBOEEBUBNJOJOH QQr

    <>'JH(SBQIJDBMNPEFMSFQSFTFOUBUJPOGPSCBOEJUQSPCMFN
    yk,t
    ⇠ N(xT
    t
    (cwk
    + ✓k ⌘k,t), 2
    k
    )
    AAAC13ichVG/axRBFP6y8UeMP3LGRrAJHpELhmPuFBSrkDRWklxySSSbDLOTubthf7I7d3pZlnQidlYWVgoW4h+gvY3/gEUaWxXLCDYWvt1bEA3Gt+zO975539tv5jmRpxPD2MGYNX7i5KnTE2cmz547f2GqcnF6PQn7sVRtGXphvOmIRHk6UG2jjac2o1gJ3/HUhuMu5fsbAxUnOgzWzDBS277oBrqjpTBE8UpryFN33mR2on3bF6YnhZfey2oPudkp8qSTrmU1ydMH3M2u26anjOCuHe6Gxs5hoZ6bJ33XFztN7s7xSpXVWREzR0GjBFWUsRxW3sLGLkJI9OFDIYAh7EEgoWcLDTBExG0jJS4mpIt9hQyTpO1TlaIKQaxL3y5lWyUbUJ73TAq1pL949MaknMEs+8hes0P2gb1hX9nPf/ZKix65lyGtzkirIj715PLqj/+qfFoNer9Vx3o26OB24VWT96hg8lPIkX6w9+xw9U5rNr3GXrJv5P8FO2Dv6QTB4Lt8taJaz4/x45AXujEaUOPvcRwF681640a9uXKzurBYjmoCV3AVNZrHLSzgLpbRpv7v8Amf8cW6b+1bj6zHo1JrrNRcwh9hPf0FKiWypw==
    ΧϧϚϯϑΟϧλ
    ཻࢠϑΟϧλ

    View Slide

  24. 24
    Time-varying Thompson Sampling (TVTP) [15]
    ֬཰Ұக๏
    ֤࿹ͰٻΊͨύϥϝʔλͷࣄޙ෼෍ʹै
    ͍αϯϓϦϯάͨ݁͠ՌΛ࿹ͷબఆʹ༻
    ͍Δ͜ͱͰଟ࿹όϯσΟοτղ๏ͱ౷߹
    <>;FOH $ 8BOH 2 .PLIUBSJ 4BOE-J 50OMJOFDPOUFYUBXBSFSFDPNNFOEBUJPOXJUIUJNFWBSZJOHNVMUJBSNFECBOEJU 1SPDFFEJOHTPGUIF
    OE"$.4*(,%%JOUFSOBUJPOBMDPOGFSFODFPO,OPXMFEHFEJTDPWFSZBOEEBUBNJOJOH QQr

    <>'JH(SBQIJDBMNPEFMSFQSFTFOUBUJPOGPSCBOEJUQSPCMFN
    yk,t
    ⇠ N(xT
    t
    (cwk
    + ✓k ⌘k,t), 2
    k
    )
    AAAC13ichVG/axRBFP6y8UeMP3LGRrAJHpELhmPuFBSrkDRWklxySSSbDLOTubthf7I7d3pZlnQidlYWVgoW4h+gvY3/gEUaWxXLCDYWvt1bEA3Gt+zO975539tv5jmRpxPD2MGYNX7i5KnTE2cmz547f2GqcnF6PQn7sVRtGXphvOmIRHk6UG2jjac2o1gJ3/HUhuMu5fsbAxUnOgzWzDBS277oBrqjpTBE8UpryFN33mR2on3bF6YnhZfey2oPudkp8qSTrmU1ydMH3M2u26anjOCuHe6Gxs5hoZ6bJ33XFztN7s7xSpXVWREzR0GjBFWUsRxW3sLGLkJI9OFDIYAh7EEgoWcLDTBExG0jJS4mpIt9hQyTpO1TlaIKQaxL3y5lWyUbUJ73TAq1pL949MaknMEs+8hes0P2gb1hX9nPf/ZKix65lyGtzkirIj715PLqj/+qfFoNer9Vx3o26OB24VWT96hg8lPIkX6w9+xw9U5rNr3GXrJv5P8FO2Dv6QTB4Lt8taJaz4/x45AXujEaUOPvcRwF681640a9uXKzurBYjmoCV3AVNZrHLSzgLpbRpv7v8Amf8cW6b+1bj6zHo1JrrNRcwh9hPf0FKiWypw==
    a(t) = arg max
    j=1,K
    xT
    t
    ¯
    wk,t 1 ¯
    wk,t 1
    ⇠ Nm(¯
    µwk
    , ¯
    ⌃wk
    )
    AAADNnichVHPa9RAFH6Jv+r6o6seFLwEl8oW1mVSBUVYKAoqCNJ2u22hacNkdnY7biaJyWS3dYh/gP+AB08KHsSTJ7178R/w0Is3EfEiVPDiwZdsQGxpfSGZ9775vjff5HmRLxJFyLZhHjp85OixieOVEydPnZ6snjm7lIRpzHiHhX4Yr3g04b4IeEcJ5fOVKOZUej5f9ga38/3lIY8TEQaLaivia5L2A9ETjCqE3OpbJ+AjFkpJg652aNyXdDPTjqRqI4y0E0sLsSc56PhCCpVkFbqu62o6a5VsVz9s2Y37mbXpqvVCmPT0YuZ4NNajzNWDhrpiZ86jlHat3WAiZKFg1NcPMlfWC4IjU6SM3EHWGNdtgQeV0LRbrZEmKcLam9hlUoMy5sLqO3CgCyEwSEEChwAU5j5QSPBZBRsIRIitgUYsxkwU+xwyqKA2RRZHBkV0gN8+VqslGmCd90wKNcNTfHxjVFowRT6R12SHfCRvyDfye99euuiRe9nC1RtreeROPr3Q/vVflcRVwcZf1YGeFfTgRuFVoPeoQPJbsLF++PjZTvvmwpS+TF6S7+j/BdkmH/AGwfAnezXPF54f4MdDL/jHcED27nHsTZZmmvbV5sz8tdrsrXJUE3ARLkEd53EdZuEezEEHmHHeaBl3jLvme/Oz+cX8OqaaRqk5B/+E+eMP3kjYHA==
    ࣄޙ෼෍ ࣄޙ෼෍

    View Slide

  25. 25
    TVTPͷ՝୊
    a(t) = arg max
    j=1,K
    xT
    t
    ¯
    wk,t 1 ¯
    wk,t 1
    ⇠ Nm(¯
    µwk
    , ¯
    ⌃wk
    )
    AAADNnichVHPa9RAFH6Jv+r6o6seFLwEl8oW1mVSBUVYKAoqCNJ2u22hacNkdnY7biaJyWS3dYh/gP+AB08KHsSTJ7178R/w0Is3EfEiVPDiwZdsQGxpfSGZ9775vjff5HmRLxJFyLZhHjp85OixieOVEydPnZ6snjm7lIRpzHiHhX4Yr3g04b4IeEcJ5fOVKOZUej5f9ga38/3lIY8TEQaLaivia5L2A9ETjCqE3OpbJ+AjFkpJg652aNyXdDPTjqRqI4y0E0sLsSc56PhCCpVkFbqu62o6a5VsVz9s2Y37mbXpqvVCmPT0YuZ4NNajzNWDhrpiZ86jlHat3WAiZKFg1NcPMlfWC4IjU6SM3EHWGNdtgQeV0LRbrZEmKcLam9hlUoMy5sLqO3CgCyEwSEEChwAU5j5QSPBZBRsIRIitgUYsxkwU+xwyqKA2RRZHBkV0gN8+VqslGmCd90wKNcNTfHxjVFowRT6R12SHfCRvyDfye99euuiRe9nC1RtreeROPr3Q/vVflcRVwcZf1YGeFfTgRuFVoPeoQPJbsLF++PjZTvvmwpS+TF6S7+j/BdkmH/AGwfAnezXPF54f4MdDL/jHcED27nHsTZZmmvbV5sz8tdrsrXJUE3ARLkEd53EdZuEezEEHmHHeaBl3jLvme/Oz+cX8OqaaRqk5B/+E+eMP3kjYHA==
    ¯
    ⌃wk
    =
    1
    p2
    p
    X
    i=1
    2(i)
    k
    ⌃(i)
    wk
    , where p is number of particles
    AAAC+3ichVHLahRBFL3dvmISzUQ3gpvGIRJBhuqJkBAIBN24TDJOEkhnmuqyZlJMP4qq6smj6B/wB1y4UnHhYyt+gBt/wEXAHxCXEdy48HZPg2gw3qa7zj11z+1TdSMZC20IOXbcc+cvXLw0cXlyavrK1ZnG7LVNneWK8S7L4kxtR1TzWKS8a4SJ+bZUnCZRzLei4YNyf2vElRZZ+sgcSr6b0EEq+oJRg1TYGAURVTboiEFCi9Duh8NiJegryqxfWNlrF4HOk54MrVjxEZdlPdueF3eKcDhW9WyVVdLA8AM0Ye96+3tccU96QntpnkRceVnfk1QZwWKui7DRJC1ShXca+DVoQh1rWeMDBPAYMmCQQwIcUjCIY6Cg8dkBHwhI5HbBIqcQiWqfQwGTqM2ximMFRXaI3wFmOzWbYl721JWa4V9ifBUqPZgjn8lrckI+kbfkK/n5z1626lF6OcQ1Gmu5DGee3Oj8+K8qwdXA3m/VmZ4N9GGp8irQu6yY8hRsrB8dPT3pLG/M2dvkBfmG/p+TY/IRT5COvrNX63zj2Rl+IvSCN4YD8v8ex2mw2W75C632+r3m6v16VBNwE27BPM5jEVbhIaxBF/t/cVxnypl2C/el+8Z9Ny51nVpzHf4I9/0vB86+jg==
    • ࢼߦճ਺ͷ૿Ճʹ൐͏ਪનख๏ͷબఆͷภΓ
    • ͋Δ࣌఺ͰධՁͷ௿͍ਪનख๏Λ୳ࡧ͢Δػձ͕ۃ୺ʹ௿Լ
    • ༗ޮੑ͕ٯస͢ΔΑ͏ͳঢ়گ΁ͷ௥ै͕஗ΕΔ
    ࢼߦճ਺ͷ૿Ճʹ൐͏ٸܹͳݮগʹΑΓ͋Δ
    ࣌఺ͷධՁʹج͍ͮͨબ୒ʹݻఆ͞ΕΔ

    View Slide

  26. 26
    ఏҊํࣜ: Aggressive Exploration TVTP (AE-TVTP)
    a(t) = arg max
    j=1,K
    xT
    t
    ¯
    wk,t 1 ¯
    wk,t 1
    ⇠ Nm(¯
    µwk
    , ¯
    ⌃wk
    )
    AAADNnichVHPa9RAFH6Jv+r6o6seFLwEl8oW1mVSBUVYKAoqCNJ2u22hacNkdnY7biaJyWS3dYh/gP+AB08KHsSTJ7178R/w0Is3EfEiVPDiwZdsQGxpfSGZ9775vjff5HmRLxJFyLZhHjp85OixieOVEydPnZ6snjm7lIRpzHiHhX4Yr3g04b4IeEcJ5fOVKOZUej5f9ga38/3lIY8TEQaLaivia5L2A9ETjCqE3OpbJ+AjFkpJg652aNyXdDPTjqRqI4y0E0sLsSc56PhCCpVkFbqu62o6a5VsVz9s2Y37mbXpqvVCmPT0YuZ4NNajzNWDhrpiZ86jlHat3WAiZKFg1NcPMlfWC4IjU6SM3EHWGNdtgQeV0LRbrZEmKcLam9hlUoMy5sLqO3CgCyEwSEEChwAU5j5QSPBZBRsIRIitgUYsxkwU+xwyqKA2RRZHBkV0gN8+VqslGmCd90wKNcNTfHxjVFowRT6R12SHfCRvyDfye99euuiRe9nC1RtreeROPr3Q/vVflcRVwcZf1YGeFfTgRuFVoPeoQPJbsLF++PjZTvvmwpS+TF6S7+j/BdkmH/AGwfAnezXPF54f4MdDL/jHcED27nHsTZZmmvbV5sz8tdrsrXJUE3ARLkEd53EdZuEezEEHmHHeaBl3jLvme/Oz+cX8OqaaRqk5B/+E+eMP3kjYHA==
    ¯
    ⌃wk
    =
    1
    p2
    p
    X
    i=1
    2(i)
    k
    ⌃(i)
    wk
    , where p is number of particles
    AAAC+3ichVHLahRBFL3dvmISzUQ3gpvGIRJBhuqJkBAIBN24TDJOEkhnmuqyZlJMP4qq6smj6B/wB1y4UnHhYyt+gBt/wEXAHxCXEdy48HZPg2gw3qa7zj11z+1TdSMZC20IOXbcc+cvXLw0cXlyavrK1ZnG7LVNneWK8S7L4kxtR1TzWKS8a4SJ+bZUnCZRzLei4YNyf2vElRZZ+sgcSr6b0EEq+oJRg1TYGAURVTboiEFCi9Duh8NiJegryqxfWNlrF4HOk54MrVjxEZdlPdueF3eKcDhW9WyVVdLA8AM0Ye96+3tccU96QntpnkRceVnfk1QZwWKui7DRJC1ShXca+DVoQh1rWeMDBPAYMmCQQwIcUjCIY6Cg8dkBHwhI5HbBIqcQiWqfQwGTqM2ximMFRXaI3wFmOzWbYl721JWa4V9ifBUqPZgjn8lrckI+kbfkK/n5z1626lF6OcQ1Gmu5DGee3Oj8+K8qwdXA3m/VmZ4N9GGp8irQu6yY8hRsrB8dPT3pLG/M2dvkBfmG/p+TY/IRT5COvrNX63zj2Rl+IvSCN4YD8v8ex2mw2W75C632+r3m6v16VBNwE27BPM5jEVbhIaxBF/t/cVxnypl2C/el+8Z9Ny51nVpzHf4I9/0vB86+jg==
    • ࢼߦճ਺ͷ૿Ճʹ൐͏ਪનख๏ͷબఆͷภΓΛղফ
    • ͋Δ࣌఺ͰධՁͷ௿͍ਪનख๏Λੵۃతʹ୳ࡧ͢ΔػձΛઃ͚Δ
    • ࿹ͷ༗༻ੑ͕ٯస͢Δ؀ڥʹ͓͍ͯɺ௥ैੑͷ޲্ΛਤΔ
    ཻࢠͷฏۉ ཻ֤ࢠͰͷ৐ࢉͷΈ

    View Slide

  27. 4.
    ධՁ

    View Slide

  28. • ࣮ࡍͷECαΠτ͔Β࠾औͨ͠4ͭͷਪનख๏ͷ঎඼ΧςΰϦ͝ͱͷΫϦοΫ཰
    ͷਪҠ࣮੷σʔλΛ༻͍ͯఏҊγεςϜͷ༗ޮੑΛධՁ͢Δ
    • 2019/6/20ʙ8/4·Ͱͷ໿225ສճͷਪનσʔλ͔Βࢉग़
    • 4ͭͷਪનख๏ͱ18ͷ঎඼ΧςΰϦ͝ͱʹɺ1࣌ؒ୯ҐͰूܭ
    28
    ධՁσʔλͱਪનख๏

    View Slide

  29. • બ୒ͨ͠ਪનख๏͔ΒಘΒΕΔΫϦοΫ਺ͷγϛϡϨʔγϣϯ
    • ਪનख๏ͷબ୒ʹ͸ɺఏҊํࣜΛؚΉଟ࿹όϯσΟοτͷղ๏Λ༻͍Δ
    • બ୒ͨ͠ਪનख๏͸࣮੷σʔλͷΫϦοΫ཰Λύϥϝʔλͱ͢ΔϕϧψʔΠ
    ෼෍ʹै͍ɺਪન݁Ռ͕ΫϦοΫ͞ΕΔ΋ͷͱ͢Δ
    • ཚ਺Λ༻͍ͨγϛϡϨʔγϣϯΛฏۉԽ͢ΔͨΊ50ճͷγϛϡϨʔγϣϯ
    ݁ՌͷฏۉΛ݁Ռͱͯ͠༻͍Δ
    • ࣮ࡍͷਪનγεςϜͷڍಈͱ߹ΘͤΔͨΊɺใु͸1࣌ؒ͝ͱʹ·ͱΊͯ
    ϑΟʔυόοΫ͞ΕΔ΋ͷͱ͢Δ
    29
    ධՁํ๏(1/2)

    View Slide

  30. • จ຺ͱ࣌ؒͷܦաͷߟྀͷͦΕͧΕͷد༩౓Λ໌Β͔ʹ͢Δ4άϧʔϓͷγ
    ϛϡϨʔγϣϯΛ࣮ࢪ
    30
    ධՁํ๏(2/2)
    ࣌ؒͷܦա
    º ˓
    จ຺
    º
    "ىटͷ࠷ળͳਪનख๏ΛશظؒҰ؏
    ͯ͠༻͍Δ
    $࣌఺͝ͱʹධՁͷߴ͍ਪનख๏Λόϯ
    σΟοτΛ༻͍ͯબఆ
    ˓
    #จ຺͝ͱʹ࠷ળͳਪનख๏Λશظؒ
    Ұ؏ͯ͠༻͍Δ
    %จ຺͝ͱ࣌఺͝ͱʹධՁͷߴ͍ਪનख
    ๏ΛόϯσΟοτΛ༻͍ͯબఆ
    • จ຺ʹ͸ɺਪન࣌ʹӾཡதͷ঎඼ΧςΰϦΛ༻͍Δ
    • ଟ࿹όϯσΟοτղ๏͸ɺLTS(จ຺)
    ɺTVTP(จ຺/࣌ؒͷܦա)
    ɺAE-TVTP(จ຺/࣌ؒͷܦա)

    View Slide

  31. • Bάϧʔϓʢจ຺ʣ͸ظट࣌఺ʹ͓
    ͍ͯAάϧʔϓͱจ຺ʹΑΔࠩҟ͕
    ΄΅ͳ͍ͨΊ݁Ռ΋ࠩҟͳ͠
    • Cάϧʔϓʢ࣌ؒͷܦաʣ͸ਪનख
    ๏ͷ༗ޮੑͷมԽʹ௥ैͨ͜͠ͱ
    Ͱվળ͕ݟΒΕΔ
    • Dάϧʔϓʢจ຺ͱ࣌ؒͷܦաʣ͸
    ঎඼ΧςΰϦ͝ͱͷมԽʹ௥ै͠
    ͨ͜ͱͰߋͳΔվળ͕ݟΒΕΔ
    31
    ධՁ݁Ռ: AάϧʔϓΛج४ͱͨ͠ྦྷੵใुͷࠩͷൺֱ
    จ຺ͱ࣌ؒͷܦաͷߟྀͳΒͼʹɺٯస؀ڥͷ௥
    ैੑΛߴΊͨఏҊํࣜʹΑͬͯ໿૿Ճ
    จ຺ͷΈ
    ࣌ؒͷܦաͷΈ

    View Slide

  32. • Dάϧʔϓͷจ຺͝ͱͷվળ݁ՌΛ
    ෼ੳ͢Δͱɺਪનख๏ͷ༗ޮੑͷ
    มಈͷগͳ͍จ຺Ͱ͸ɺશͯͷղ
    ๏ʹ͓͍ͯ୳ࡧͷίετΛճऩͰ
    ͖͍ͯͳ͍
    • มಈͷେ͖͍จ຺Ͱ͸ੵۃతͳ୳
    ࡧʹΑΓAE-TVTP͕େ͖͘վળ͠
    ͨɻTVTP͸ਪનख๏͕ݻఆ͞Εվ
    ળʹࢸΒͳ͔ͬͨ
    32
    ධՁ݁Ռ: ਪનख๏ͷ༗ޮੑͷมಈ౓߹͍ͱվળ཰

    View Slide

  33. 33
    ߟ࡯
    • ਪનख๏ͷ༗ޮੑ͕ٯస͢Δࠨྻ
    ʹ͓͍ͯ͸ఏҊख๏͕༗ޮ
    • ӈྻʹ͓͍ͯ͸ɺఏҊख๏ͷੵۃ
    తͳ୳ࡧʹىҼͯ͠ɺظؒதܧଓత
    ʹྦྷੵϦάϨοτ͕૿Ճ͢Δɻ3ׂ
    ఔ౓Ͱಉ༷ͷࣄ৅Λ֬ೝɻ
    ਪનख๏ͷ༗ޮੑʹٯస͕͋Δ঎඼ΧςΰϦ ࠨ
    ͱɺ
    ͳ͍঎඼ΧςΰϦ ӈ
    ʹ͓͚ΔྦྷੵϦάϨοτͷਪҠ
    ˎྦྷੵϦάϨοτ͸ਪનख๏ͷ͏ͪ࠷େͷظ଴஋ͱબ୒ͨ͠ਪનख
    ๏ͷظ଴஋ͷࠩΛظؒ·Ͱʹ߹ܭͨ͠΋ͷ
    มԽͷͳ͍ظؒʹ͓͍ͯ΋ػձଛࣦΛ
    ௿ݮ͢ΔదԠతͳ୳ࡧख๏ͷݚڀ΁

    View Slide

  34. 5.
    ͓ΘΓʹ

    View Slide

  35. • ଟ࿹όϯσΟοτΛ༻͍ͯɺจ຺ͱ࣌ؒͷܦաʹԠͯ͡ਪનख๏ͷબ୒Λࣗಈ
    త͔ͭܧଓతʹ࠷దԽ͢ΔϝλਪનγεςϜΛఏҊ͠ɺͦͷ༗ޮੑΛࣔͨ͠
    • จ຺ͱ࣌ؒͷܦաͷߟྀͳΒͼʹٯస؀ڥͷ௥ैੑΛߴΊͨఏҊํࣜʹΑͬ
    ͯɺߟྀ͠ͳ͍৔߹ͱൺֱͯ͠ྦྷੵΫϦοΫ਺ͷ޲্ʹܨ͕Δ͜ͱ͕Θ͔ͬͨ
    • ਪનख๏ͷ༏ྼʹมԽͷͳ͍ظؒͰ͸݁Ռ͕ٯస͢ΔՄೳੑ͕֬ೝ͞Εͨ͜ͱ
    ͰɺػձଛࣦΛ௿ݮ͢Δղ๏͕ॏཁͰ͋Δ͜ͱ΋ࣔࠦ͞Εͨ
    • ࠓޙͷ՝୊ͱͯ͠ɺਪનख๏ͷબ୒ʹӨڹΛٴ΅͢ޮՌͷߴ͍จ຺ͷൃݟɺͳ
    Βͼʹಛఆͷจ຺ʹ͓͍ͯධՁͷߴ͍ਪનख๏ͷཱ֬ʹΑΔจ຺͝ͱͷޮՌ޲
    ্ͷ࣮ݱ͕ڍ͛ΒΕΔ
    35
    ͓ΘΓʹ

    View Slide

  36. View Slide