Upgrade to Pro — share decks privately, control downloads, hide ads and more …

これからの強化学習2.7

moyomot
May 19, 2017
97

 これからの強化学習2.7

moyomot

May 19, 2017
Tweet

Transcript

  1. ͜Ε͔ΒͷڧԽֶश
    2.7 ෳརܕڧԽֶश
    GUNOSY σʔλϚΠχϯάݚڀձ #121

    View Slide

  2. INTRODUCTION
    ͓࿩͢Δ͜ͱ
    ▸ རӹ཰ͷෳརޮՌΛ۩ମྫΛ௨ͯ͠ཧղ͠
    ▸ ෳརΛQֶशͷ࢓૊ΈʹఆࣜԽ͢Δ

    View Slide

  3. INTRODUCTION
    ໨࣍
    ▸ 2.7.1 རӹ཰ͷෳརޮՌͱ౤ࢿൺ཰
    ▸ زԿฏۉΛ࢖༻ͨ͠ෳརޮՌͷ۩ମྫ
    ▸ 2.7.2 ෳརܕڧԽֶशͷ࿮૊Έ
    ▸ ߦಈՁ஋ؔ਺QͷఆࣜԽ
    ▸ 2.7.3 ෳརܕڧԽֶशΞϧΰϦζϜ
    ▸ ෳརܕQֶश
    ▸ ෳརܕOnPSʢOnline Profit Sharingʣ
    ▸ 2.7.4 ౤ࢿൺ཰ͷ࠷దԽ
    ▸ 2.7.5 ϑΝΠφϯε΁ͷԠ༻ɿࠃ࠴໏ฑબ୒

    View Slide

  4. 2.7.1 རӹ཰ͷෳརޮՌͱ౤ࢿൺ཰
    3ຊ࿹όϯσΟοτ໰୊
    ▸ ͲͷϚγϯ͕͓ಘ͔ʁ

    View Slide

  5. 2.7.1 རӹ཰ͷෳརޮՌͱ౤ࢿൺ཰
    ͲͷϚγϯ͕͓ಘ͔ʁ
    ▸ ໰୊ͷຊ࣭͸ࢉज़ฏۉ͔زԿฏۉ͔
    ▸ ֫ಘͨ͠རӹ΋ؚΊͯશֹ͔͚ଓ͚Δͱ͖͸زԿฏۉΛߟྀ͢Δ
    ඞཁ͕͋ΔʢAͷબ୒͕ྑ͍ʣ
    ▸ زԿฏۉ͸ෳརͷΑ͏ͳൺ཰ͰมԽ͢Δͱ͖ʹ࢖༻͢Δ
    ▸ ʢຖճ1υϧBET͢Δ৔߹͸ࢉज़ฏۉͷ΄͏͕ྑ͍݁ՌʹͳΔ͸
    ͣʣ
    ▸ https://www.jstage.jst.go.jp/article/tjsai/26/2/26_2_330/_pdf

    View Slide

  6. 2.7.1 རӹ཰ͷෳརޮՌͱ౤ࢿൺ཰
    ෳརޮՌΛ࠷େԽ͢ΔͨΊʹ͸έϦʔج४
    ▸ ΫϩʔυɾγϟϊϯΒͱڞʹɺ৘ใڞ༗Λར༻͠ɺΪϟϯϒϧͰ࠷΋ޮ཰ͷΑ
    ͍Ṍ͚ํΛݚڀͨ͠ɻͨͩ͠ɺࣗΒṌ͚Δ͜ͱ͸͠ͳ͔ͬͨɻʢwikipediaʣ

    View Slide

  7. 2.7.2 ෳརܕڧԽֶशͷ࿮૊Έ
    ऩӹׂҾ࿨ͷൺֱ
    ऩӹͷׂҾ࿨
    ׂҾෳརརӹ཰
    R: རӹ཰, γ: ׂҾ཰, f: ౤ࢿൺ཰
    r: ใु
    ໨త: ߦಈՁ஋ؔ਺QΛ࠶ؼతͳܗͰఆࣜԽ͠Q-
    learningʹ͍͖͍࣋ͬͯͨ

    View Slide

  8. 2.7.2 ෳརܕڧԽֶशͷ࿮૊Έ
    ঢ়ଶՁ஋؍਺ͷఆࣜԽ

    View Slide

  9. 2.7.2 ෳརܕڧԽֶशͷ࿮૊Έ
    ߦಈՁ஋؍਺ͷఆࣜԽ
    ͋ͱ͸QΛ࠷େԽ͢ΔํࡦπΛֶश͢Δ

    View Slide

  10. 2.7.3 ෳརܕڧԽֶशΞϧΰϦζϜ
    ෳརܕQֶश
    ▸ ҰൠతͳQֶशͱߟ͑ํ͸ಉ͡
    ▸ ใुΛརӹ཰ͷର਺Ͱஔ͖׵͑ͨ

    View Slide

  11. 2.7.3 ෳརܕڧԽֶशΞϧΰϦζϜ
    ෳརܕQֶशͷΞϧΰϦζϜ

    View Slide

  12. 2.7.3 ෳརܕڧԽֶशΞϧΰϦζϜ
    ෳརܕOnPS(ONLINE PROFIT SHARING)
    ▸ Profit Sharing
    ▸ Qֶश͸ϚϧίϑੑɺProfit Sharing͸ඇϚϧίϑੑOK
    ▸ ঢ়ଶs, ߦಈaͷ༏ઌ౓ΛPΛஔ͘, F͸৴༻ׂ౰ؔ਺ʢڧԽؔ
    ਺ʣ
    ▸ ใु֫ಘʹෆඞཁͳߦಈΛଟ࣮͘ߦ͢Δඇ߹ཧͳํࡦΛֶ
    श͢Δ՝୊͋Γ
    ▸ https://www.jstage.jst.go.jp/article/fss/27/0/27_0_304/
    _pdf

    View Slide

  13. 2.7.3 ෳརܕڧԽֶशΞϧΰϦζϜ
    ෳརܕOnPSͷΞϧΰϦζϜ

    View Slide

  14. 2.7.4 ౤ࢿൺ཰ͷ࠷దԽ
    ౤ࢿൺ཰GͬͯͲ͏΍ͬͯબͿͷʁ
    ▸ ΦϯϥΠϯޯ഑๏ͰfΛߋ৽͢Ε͹OK

    View Slide

  15. Ͳͷࠃͷࠃ࠴Λߪೖ͢Ε͹Α͍͔
    ▸ ෳརܕQֶशͱैདྷͷQֶशΛൺֱ
    ▸ ෳརܕQֶश͸زԿฏۉͰརӹ཰ͷ
    ෳརޮՌΛେ͖͘͢Δ͜ͱ͕Ͱ͖ͯ
    ͍Δ
    2.7.5 ϑΝΠφϯε΁ͷԠ༻ྫɿࠃ࠴໏ฑબ୒

    View Slide