Introduction for Reading "Reinforcement Learning"

Transcript

1. ʰڧԽֶशʱྠಡձ Introduction @mocobt @icebee__ ओ࠵ @lystahi

2. Agenda ։࠵֓ཁ (10min) by @mocobt ࣗݾ঺հ (10min) ୈ1ষղઆ (60min) by

   @lystahi ୲౰ܾΊ (10min) ୈ2ষղઆ (as possible) by @icebee__ ఫऩ (5min)

3. ։࠵֓ཁ ໨త: ڧԽֶशͷཧ࿦ΛʮԿͱͳ͘ʯཧղ͢Δ (෮शਪ঑) ର৅ • ڧԽֶशͷཧ࿦Λֶͼ͍ͨਓ • ॳֶऀͱަྲྀ͢Δ͜ͱͰ৽ͨͳؾ͖ͮΛಘ͍ͨਓ (׻ܴ)

   ৔ॴ:ौ୩͔৽॓ (Ԡ૬ஊ) ࢀՃඅ༻: ݪଇ(৔ॴ୅/ਓ਺)ԁɽֶͨͩ͠ੜແྉ

4. ର৅ॻ੶ • ػցֶशϓϩϑΣογϣφϧγϦʔζʮڧԽֶशʯ • ৿ଜ఩࿠ ஶ • ߨஊࣾαΠΤϯςΟϑΟΫ ग़൛ •

   ཧ࿦த৺Ͱ࣮૷ܥͷ࿩͸ແ͠ • https://www.kspub.co.jp/book/detail/5155912.html

5. ਐΊํ • 1ճʹ͖ͭݪଇ1ষΛਐΊ͍ͯ͘ - ૣ͘ਐΜͩ৔߹͸લ౗͠ʹ͢ΔͨΊɼ࣍ि୲౰ऀ΋ࢿྉ࡞੒͓ئ͍͠·͢ • ൃද୲౰ऀͷTODO • ॏཁͱࢥͬͨՕॴΛϐοΫΞοϓͯ͠εϥΠυΛ࡞Δ -

   Θ͔Βͳ͍͜ͱ͸ʮΘ͔Γ·ͤΜͰͨ͠ʯͰOK - ΋ͪΖΜ׬ᘳʹ௥͍͚ͬͯͨͩΔͱେมॿ͔Γ·͢ʂ - Α΄Ͳͷࣄ৘͕ͳ͍ݶΓɼυλΩϟϯ͸͓߇͍͑ͩ͘͞ • ௌߨऀͷTODO: ܰ͘ಡΜͰ͘Δ

6. εέδϡʔϧ ೔ఔ Chapter ಺༰ ୲౰ऀ 11/05 (Ր) 1 ४උ @lystahi

   11/19 (Ր) 2 ϓϥϯχϯά @icebee__ 12/03 (Ր) 3 ୳ࡧͱ׆༻ͷτϨʔυΦϑ • 12/03 (Ր) 4 ϞσϧϑϦʔܕͷڧԽֶश 12/17 (Ր) 5 ϞσϧϕʔεܕͷڧԽֶश 01/14 (Ր) 6 ؔ਺ۙࣅΛ༻͍ͨڧԽֶश 01/28 (Ր) 7 ෦෼؍ଌϚϧίϑܾఆաఔ 02/11 (Ր) 8 ࠷ۙͷ࿩୊ ※ ୈ3ষͱୈ4ষ͸ಉ͡୲౰ऀͱ͢Δ ※ ೔ఔ͸ࢀՃऀͷ༧ఆʹԠͯ͡దٓมߋͷՄೳੑ͋Γ

7. (͕࣌ؒ༨ͬͨ࣌༻) ͪΐͬͱࡶஊ: ڧԽֶश΍ͬͯԿ͕͏Ε͍͠ͷʁ

8. ڧԽֶशͷࡶͳཧղ https://deepage.net/machine_learning/2017/08/10/reinforcement-learning.html 1. ΤʔδΣϯτ͕ߦಈΛߦ͏͜ͱͰใुΛಘΔ 2. ใुʹԠͯ͡ΤʔδΣϯτͷঢ়ଶ͕ભҠ͢Δ 3. ֫ಘใु͕࠷େͱͳΔΑ͏ͳߦಈΛֶश͢Δ ऩଋ͢Δ·Ͱ ܁Γฦ͢

9. ڧԽֶशͷԠ༻ྫ ΩϟϥΫλʔΞχϝʔγϣϯੜ੒ ٯڧԽֶशʹΑΔNN Architecture୳ࡧ ը૾෮ݩ Ray Tracing

10. RL-Restore [Yu++. CVPR 2018] ྼԽͷछྨʹґΒͣɼ౷Ұతʹը૾෮ݩ͢Δख๏ͷݚڀ طଘݚڀͷ՝୊ λεΫಛԽͷArchitecture͸൚༻ੑ͕௿͍ (ྫ͑͹ɼDeblur͢Δ͚ͩ…) ৽نੑ ෳ਺ͷCNNΛ༻͍ͯɼྼԽͷछྨʹԠͯ͡

    ಈతʹArchitectureΛมߋ͢ΔRLख๏ Deblurring Denoising

11. IRLAS [Guo++. CVPR 2019] “ڧ͍” NN ArchitectureΛ୳ࡧ͢ΔͨΊͷख๏ͷݚڀ ٯڧԽֶश (Inverse RL)

    ࠷దͳߦಈ͔ΒใुΛਪఆ͢Δख๏ (ใुΛఆٛͮ͠Β͍৔߹ʹศར) ৽نੑ Inverse RLΛ༻͍ͨNeural Architecture Search (ࠨ: ResNeXt, த: NASNet, ӈ: ఏҊख๏ʹΑΔNN) https://qiita.com/neka-nat@github/items/aaab6184aea7d285b103

12. DeepMimic [Peng++. TOG 2018] λεΫΛ͜ͳ্͢Ͱ࠷΋Β͍͠ϞʔγϣϯΛੜ੒͢Δݚڀ طଘݚڀͷ՝୊ Mocap΍RL͚ͩΛ༻͍ͨಈ͖͸ΩϞ͍ (ϊΠζͷӨڹ or λεΫΛ͜ͳ͚ͩ͢)

    ৽نੑ Reference MotionΛ༻͍ͭͭɼ RLͰಛఆλεΫΛ͜ͳ͢Ϟʔγϣϯੜ੒

13. Learning Light Transport the Reinforced Way [Dahm++. SIGGRAPH 2017 Talks]

    ୭͔ͷ࣮૷: https://github.com/PalashBansal96/CUDAPathTracerRL Ray TracingͰRLΛ༻͍ͯޮ཰తʹRayΛඈ͹͢ݚڀ طଘݚڀͷ՝୊ RayΛඈ͹ͯ͠΋தʑऩଋͤͣɼNoisy (a,b,c: ैདྷख๏, d: ఏҊख๏ͰಉҰ਺ͷRay) ৽نੑ Path TracingͷSamplingʹRLΛಋೖͨ͠఺

14. References • cousera ڧԽֶशઐ໳ߨ࠲: ·ͱ΋ͳڭ͑Λड͚͍ͨਓ޲͚ • Richard S. SuttonઌੜͷHP: ڧԽֶशͷࢀߟॻଟ਺

    • https://github.com/openai/gym: ࣮૷͍ͨ͠ਓ޲͚ • AWS DeepRacer: ڧԽֶशΛ༻͍ͨࣗಈӡసίϯϖɽಆ૪ΛٻΊΔਓ޲͚ • https://github.com/aikorea/awesome-rl: awesome repository (survey) • Mathpix Snipping Tool: εϥΠυʹ਺ࣜΛೖΕ͍ͨͱ͖ʹศར (ͨͩ͠΄΅༗ྉ੍ʹ)

15. ࣗݾ঺հλΠϜ

16. • CGͷR&DΤϯδχΞ - LightingͱModelingؔ࿈ • ػցֶश͸झຯ & Kaggle Expert •

    ڧԽֶशૉਓ - ׂͱςΩτʔͳ͜ͱݴ͍·͢ About me @mocobt