Montezuma’s Revenge Solved by Go-Explore, a New Algorithm for Hard-Exploration Problems 

39c6dc454be13be8e4a2593c55352f45?s=47 kzmssk
December 10, 2018
Technology
0
62

Montezuma’s Revenge Solved by Go-Explore, a New Algorithm for Hard-Exploration Problems 

Dwango Media Villageのエンジニア　佐々木です。
ATARIの難問題「Montezuma’s Revenge」のSOTAを大幅に更新した強化学習の探索手法Go-Exploreを解説します！

39c6dc454be13be8e4a2593c55352f45?s=128

kzmssk

December 10, 2018
Tweet

Want more?

39c6dc454be13be8e4a2593c55352f45?s=48 kzmssk
0
210
39c6dc454be13be8e4a2593c55352f45?s=48 kzmssk
0
840
39c6dc454be13be8e4a2593c55352f45?s=48 kzmssk
0
77
7cbd3f5f922d798cc312eaf596de7ae6?s=48 iamctodd
4
810
3ab1249be442027903e1180025340b3f?s=48 reverentgeek
15
900
A9a491b0fcbe0fbce3d64063a37add99?s=48 rmw
6
490
766b9746b59e6f09e280cd33cf4ed419?s=48 skipperchong
0
320
61857dafbd287b3027c4dcea9008ad3c?s=48 carmenhchung
8
650
06f8b41980eb4c577fa40c41d5030c19?s=48 keathley
9
330
C0390e8b11079f8761c0cd3274ed61cb?s=48 productmarketing
3
370
C35e01544a0dd94a2cf1619ee8a42ebb?s=48 clairelew
5
930
6bb82b13cda86d3b821fa44100335ddf?s=48 thoeni
1
240

Transcript

  1. 1.

    .POUF[VNB`T3FWFOHF4PMWFECZ(P&YQMPSF  B/FX"MHPSJUINGPS)BSE&YQMPSBUJPO1SPCMFNT "ESJFO&DP⒎FU +PPTU)VJ[JOHB +PFM-FINBO ,FOOFUI04UBOMFZ BOE+F⒎$MVOF IUUQFOHVCFSDPNHPFYQMPSF

  2. 2.

    ڧԽֶशͷֶशΞϧΰϦζϜ "5"3*ͷήʔϜͷதͰ΋೉͍͠໰୊ͭͰ405" .POUF[VNB`T3FWFOHF 1JUGBMM ใु͕εύʔεͳ؀ڥͰޮ཰తʹ୳ࡧΛߦ͏ํ๏ ΛఏҊ աڈͷ୳ࡧཤྺΛอଘ Α͔ͬͨ΋ͷΛબΜͰ͔ͦ͜ΒϥϯμϜͰ୳ࡧ ୳ࡧཤྺΛߋ৽ ୳ࡧཤྺͷ໛฿ֶशΛ͔ͭͬͯϞσϧʹ஌ࣝΛ

    ఆணͤ͞Δ ֓ཁ  
  3. 3.

    l%FDFQUJWFzͳใु ୳ࡧ͕༰қͳখ͞ͳใुΛಘΔΑ͏ʹֶश͍ͯ͘͠ͱେہతʹ େ͖ͳใु͕ಘΒΕͳ͘ͳͬͯ͠·͏ ྫɿ1JGBMM ΞΫγϣϯήʔϜʣใुΛಘΔͨΊʹඞཁͳߦಈ ΛͱΔͱଈ࠲ʹখ͞ͳෛͷใु͕༩͑ΒΕΔ খ͞ͳใु͕ಘΒΕΔྖҬ͔Β୳ࡧ͕ൈ͚ग़ͤ͞ͳ͍ ྫɿͳʹ΋ߦಈΛͱΒͳ͘ͳͬͯ͠·͏ .POUF[VNB`T3FWFOHF1JUGBMM͸ͳͥ೉͍͔͠ 

     IUUQTHZNPQFOBJDPNFOWT.POUF[VNB3FWFOHFSBNW
  4. 4.

    l*OUSJOTJDNPUJWBUJPO *. zͷಋೖ ୳ࡧதʹ৽͍͠ঢ়ଶΛಘΔ͜ͱʹใुΛ༩͑Δ ͦΕͰ΋΄ͱΜͲΫϦΞͰ͖͍ͯͳ͔ͬͨ *.ͷ໰୊఺ɿl%FUBDINFOUzʢ཭୤ʣ աڈʹ୳ࡧʹΑͬͯಘͨใु͕΋Β͑ΔྖҬΛΤʔ δΣϯτ͕๨Εͯ͠·͏ʢഁ໓త๨٫ʣ ໎࿏ͷྫɿ ྘ͷྖҬ͸ະ౸ୡͳͷͰ୳ࡧ͢Δͱใु͕΋

    Β͑Δ Կ͔͠ΒͷݪҼͰ୳ࡧ͍ͯͨ͠ྖҬ͔Β཭Ε Δ ͢Ͱʹ୳ࡧͨ͠ྖҬΛ௨Βͳ͍ͱ৽͍͠ܦݧ ͕ಘΒΕͳ͍ طଘΞϓϩʔνͷ໰୊఺  
  5. 5.

    ୳ࡧͨ݁͠Ռʢঢ়ଶʣͷ͏ͪʮΑ͍ʯ΋ͷʢʹ$FMMʣΛอଘ͓͖ͯ͠ɺաڈͷ$FMM͔Β୳ࡧΛ։࢝͢Δ ΞϧΰϦζϜ1IBTFɿ&YQMPSFVOUJMTPMWFE աڈʹอଘ͍ͯͨ͠$FMM܈͔Β֬཰తʹͭΛબͿ ։࢝ޙ͔ΒબΜͩ$FMMʹ౸ୡ͢Δ ୳ࡧʢྫɿϥϯμϜʣ ୳ࡧʹΑͬͯಘΒΕͨঢ়ଶ͕ࠓͷ$FMMΑΓΑ͚Ε͹$FMMΛߋ৽ ΞϧΰϦζϜ1IBTFɿ3PCVTUJGZ χϡʔϥϧωοτͷϞσϧΛ࢖ͬͯݱࡏ΋࠷΋ྑ͍ཤྺΛ࢖ͬͯ໛฿ֶश ఏҊख๏ɿ୳ࡧཤྺʢ$FMMʣͷಋೖ 

    
  6. 6.

    %PNBJOͷ஌ࣝΛೖΕͳ͍৔߹ ؍ଌ৘ใΛѹॖͨ͠΋ͷ "5"3*ͷ৔߹͸ը૾Λμ΢ϯαϯϓϦϯάͨ͠΋ͷ %PNBJOͷ஌ࣝΛೖΕΔ৔߹ ؀ڥ͔ΒऔಘͰ͖Δ༷ʑͳ৘ใ ΩϟϥΫλʔͷҐஔ είΞͳͲ $FMMͷอଘͱ࠶ੜ  

    ؀ڥ͕ϦηοτՄೳͳ৔߹ ͦͷ··$FMMͷ΋ͭঢ়ଶΛ࢖ͬͯ౸ୡՄೳʢࠓճ͸͜Εʣ ؀ڥ͕ϦηοτෆՄೳ͕ܾͩఆ࿦తͳ৔߹ $FMMʹ౸Δ·ͰͷߦಈཤྺΛ࠶ੜ͢Δ ؀ڥ͕ϦηοτෆՄೳͰ͔ͭ֬཰తͳ৔߹ (PBMDPOEJUJPOFE1PMJDZ Λ࢖͏ ΰʔϧͱͳΔঢ়ଶΛϞσϧʹೖྗ͢Δ อଘ ࠶ੜ 4DIBVM5 )PSHBO% (SFHPS, BOE4JMWFS%  6OJWFSTBMWBMVFGVODUJPOBQQSPYJNBUPST*O *OUFSOBUJPOBM$POGFSFODFPO.BDIJOF-FBSOJOH QQ 
  7. 7.

    ୳ࡧʹΑͬͯ৽͍͠෦԰Λଟ͘ݟ͚ͭͨ ໛฿ֶशͨ͠ϞσϧͰഒҎ্405"Λߋ৽ ݁Ռ.POUF[VNB`T3FWFOHF  

  8. 8.

    %PNBJOͷ஌ࣝΛ$FMMʹಋೖͨ͠৔߹ɺ͞Βʹଟ͘ͷ෦԰Λݟ͚ͭΒΕͨ ΤʔδΣϯτͷ࠲ඪɺݱࡏͷ෦԰ɾϨϕϧɾอ͍࣋ͯ͠Δ伴 ݁Ռɿ.POUF[VNB`T3FWFOHF  

  9. 9.

    %PNBJO஌ࣝͳ͠ʢը૾ͷΈʣͩͱใुΛಘΔ͜ͱ͕Ͱ͖ͳ͔ͬͨ ը૾ͱͯ͠͸ಉ͕ͩ͡ҟͳΔঢ়ଶ͕ଟ͋ͬͨͨ͘Ί ஌ࣝ͋ΓͰ405" ݁Ռɿ1JUGBMM  

  10. 10.

    (P&YQMPSF͸ਐԽܭࢉ ஶऀͨͪ΋ਐԽܭࢉΞϧΰϦζϜ͔ΒώϯτΛಘͨΒ͍͠ ϦηοτෆՄೳͰ͔ͭ֬཰࿦తʹৼΔ෣͏؀ڥͰ͜ͷख๏͕༗ޮͳͷ͔͸ٙ໰ "5"3*ͷήʔϜͳΒ୳ࡧͷΈͰڧԽֶश͍Βͳ͍ͷͰ͸ʁ ॴײ