Reinforcement Learning from classic to DQN

ъച೟ण੄੉ۿҗपઁ ੿䤯ࢳ

ъച೟ण੄੉ۿҗपणਸ߽೯ ੋҕ૑מगಌܻ݃য়ܳझझ۽ٜ݅ӝਤೠ Ѣ੄ݽٚѪ੉׸ѹ੓णפ׮

ࣗѐ ੿ਗࢳোҳਗ City University of New York -Baruch College Data
Science ੹ҕ ConnexionAIোҳਗ Freelancer Data Scientist ݽف੄োҳࣗъച೟णোҳਗ Github: https://github.com/wonseokjung Facebook: https://www.facebook.com/ws.jung.798 Blog: https://wonseokjung.github.io/

1. Dynamic Programming a. Policy iteration b. Value iteration 2.
Monte Carlo method 3. Temporal-Difference Learning a. Sarsa b. Q-learning ٩۞׬೐ۨ੐ਕ௼ாۄझࣗѐ߂गಌܻ݃য়ജ҃ҳ୷ %2/ਸ੉ਊೠੋҕ૑מगಌܻ݃য়ٜ݅ӝ ࣽࢲ

Monte Carlo method 3. Temporal-Difference Learning a. Sarsa b. Q-learning गಌܻ݃য়ജ҃ҳ୷߂٩۞׬೐ۨ੐ਕ௼ாۄझࣗѐ %2/ਸ੉ਊೠੋҕ૑מगಌܻ݃য়ٜ݅ӝ .PEFMGSFF .PEFMCBTFE %FFQMFBSOJOH    3- पण

Monte Carlo method 3. Temporal-Difference Learning a. Sarsa b. Q-learning गಌܻ݃য়ജ҃ҳ୷߂٩۞׬೐ۨ੐ਕ௼ாۄझࣗѐ %2/ਸ੉ਊೠੋҕ૑מगಌܻ݃য়ٜ݅ӝ (SJEXPSME पण

#FGPSF%FFQMFBSOJOH "GUFS%FFQMFBSOJOH 5BCVMBS *NBHF UFYU WPJDFj ജ҃ࢶఖ੄੉ਬ

$MBTTJD3- %FFQ-FBSOJOH Ҋ੹ъച೟ण੉઺ਃೠ੉ਬ

2MFBSOJOH $//%2/ %2/

п-FWFM੄4UBUFо׮ܰӝٸޙী(FOFSBMBHFOUܳ  ٜ݅ӝоয۵׮ ಽܻ૑ঋ਷ޙઁٜ

गಌܻ݃য়֤ޙ University of California, Berkeley ICML 2017 Curiosity-driven Exploration by
Self-supervised Prediction

IUUQTHJUIVCDPNXPOTFPLKVOH,*14@3FJOGPSDFNFOU पण੗ܐח $PEF +VQZUFS/PUFCPPL઱ࢳ ࢸ஖ߨݽفઁҕ ҾӘೠ੼੉੓ਵदݶѐੋ੸ਵ۽োۅ઱ࣁਃ

Markov Decision Process

3FUVSOPG&QJTPEF &QJTPEFزউ3FUVSOػ3FXBSE੄೤ 5PUBM3FXBSE

%JTDPVOUFE3FUVSO %JTDPVOUFEGBDUPSо੸ਊػ3FXBSE੄೤ 5PUBM3FXBSEXJUI%JTDPVOUFE

.%1ীࢲ੄Y(SJEXPSME Grid World Environment

.%1ীࢲ੄Y(SJEXPSME 4UBUFӒܻ٘੄ઝ಴ "DUJPO࢚ ೞ ઝ ਋ 3FXBSEೣ੿ ݾ಴  5SBOTJUJPO1SPCBCJMJUZ %JTDPVOUGBDUPS
3FXBSE 3FXBSE 4UBUF "DUJPO Grid World Environment

4UBUFWBMVFGVODUJPO 1PMJDZܳٮܲTUBUFWBMVFGVODUJPO 4UBUFWBMVF

"DUJPO7BMVFGVODUJPO 1PMJDZܳٮܲBDUJPOWBMVFGVODUJPO 4UBUFBDUJPOWBMVF

#FMMNBOFRVBUJPO "'VOEBNFOUBMQSPQFSUZPGWBMVFGVODUJPO

0QUJNBM1PMJDZܳ଺੗TUBUF 7BMVFܳ୭؀۽ 0QUJNBMTUBUFWBMVFGVODUJPO

0QUJNBM1PMJDZܳ଺੗TUBUFBDUJPO 7BMVFܳ୭؀۽ 0QUJNBMTUBUFBDUJPOWBMVFGVODUJPO

#FMMNBOFRVBUJPO 0QUJNBMJUZ #FMMNBOPQUJNBMJUZFRVBUJPOW

#FMMNBOFRVBUJPO 0QUJNBMJUZ #FMMNBOPQUJNBMJUZFRVBUJPOR

൒ܴ .%1 3FUVSO&QJTPEF 3FUVSO&QTJTPEF EJTDPVOU 4UBUFWBMVFGVODUJPO "DUJPOWBMVFGVODUJPO 0QUJNBM1PMJDZ #FMMNBO&RVBUJPO
#FMMNBOPQUJNBMFRVBUJPO #FMMNBO&RVBUJPO 0QUJNBM1PMJDZ

Dynamic Programming

ઑѤ State, Reward, Action

ઑѤ Transition Probability ژೠ઱য઎׮

Dynamic Programming  7BMVFGVODUJPOਸࢎਊೞৈࠁ׮ա਷1PMJDZ ܳ଺ӝਤ೧ҳઑചदఃҊ੿ܻೡࣻ੓׮ Dynamic programming੄Key idea!

Dynamic programming

Y(SJEXPSMEীࢲ%ZOBNJD1SPHSBNNJOH Grid World Environment

Y(SJEXPSME 4UBUFӒܻ٘੄ઝ಴ "DUJPO࢚ ೞ ઝ ਋ 3FXBSEೣ੿ ݾ಴  5SBOTJUJPO1SPCBCJMJUZ %JTDPVOUGBDUPS
3FXBSE 3FXBSE അ੤TUBUF "DUJPO Grid World Environment ׮਺TUBUF ׮਺TUBUF

6QEBUF3VMF  #FMMNBOFRVBUJPOਸࢎਊೞৈসؘ੉౟ೠ׮ 4UBUF

فઙܨ੄0QUJNBM7BMVFGVODUJPOT 4UBUF7BMVF #FMMNBOPQUJNBMJUZFRVBUJPOT୽઒

فઙܨ੄0QUJNBM7BMVFGVODUJPOT "DUJPO7BMVF #FMMNBOPQUJNBMJUZFRVBUJPOT୽઒

Dynamic Programming  فઙܨ੄୭੸7BMVFGVODUJPO State-action Value function

1PMJDZ*UFSBUJPO 7BMVF*UFSBUJPO Dynamic Programming

1PMJDZJUFSBUJPO 1.Policyܳٮۄ state-valueܳ҅࢑ೞ੗ Policy Evaluation ؊જ਷Policyܳ଺੗ Policy Improvement ୭੸੄1PMJDZܳ ଺ӝਤೠفо૑
җ੿

Policy iteration- Policy Evaluation 6QEBUF3VMFਸࢎਊೞৈ&WBMVBUJPOਸೠ׮ 7BMVFVQEBUF 1PMJDZ 5SBOTJUJPO 1SPCBCJMJUZ 3FXBSE
/FYU4UBUF  FTUJNBUFEWBMVF

ݽٚTUBUFܳ7 T ਵ۽ୡӝചदఅ׮ пTUBUFܳ6QEBUF3VMFਸࢎਊೞৈ7 T ܳসؘ੉౟ೠ׮ Policy iteration- Policy
Evaluation সؘ੉౟ೞݴ7 T ੄߸ച۝੉ݒ਋੘ਸٸসؘ੉౟ܳݥ୸׮ Policyܳٮۄ state-valueܳ҅࢑ೞ੗

Policy iteration- Improvement 1PMJDZܳٮۄ7BMVFGVODUJPOਸ҅࢑ೠ੉ਬח؊ա਷ 1PMJDZܳ଺ӝਤ೧ࢲ੉׮ (SFFEZ1PMJDZ

Policy iteration- Improvement (SFFEZ1PMJDZ੸ਊ

1PMJDZJUFSBUJPO 1PMJDZJUFSBUJPO਷0QUJNBMQPMJDZܳ଺ਸٸө૑ 1PMJDZ&WBMVBUJPOҗ1PMJDZ*NQSPWFNFOUܳ߈ࠂೠ׮

(SJE8PSME&OWJSPONFOU Y(SJEXPSME

Y(SJEXPSME 4UBUFӒܻ٘੄ઝ಴ "DUJPO࢚ ೞ ઝ ਋ 3FXBSEೠ஢਑૒ੌٸ݃׮ 5SBOTJUJPO1SPCBCJMJUZ %JTDPVOUGBDUPS (SJE8PSME&OWJSPONFOU1PMJDZJUFSBUJPO
3FXBSE (PBM "DUJPO (PBM 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE 3FXBSE

(SJE8PSME&OWJSPONFOU1PMJDZJUFSBUJPO Lੌٸ ୡӝച 7L (SFFE1PMJDZ

7L (SFFE1PMJDZ L
(SJE8PSME&OWJSPONFOU1PMJDZJUFSBUJPO

7L (SFFE1PMJDZ LJOG
(SJE8PSME&OWJSPONFOU1PMJDZJUFSBUJPO

1PMJDZJUFSBUJPO दো

1PMJDZ*UFSBUJPO 7BMVF*UFSBUJPO Dynamic Programming

7L (SFFE1PMJDZ Lࣻ۴ೞݶ
(SJE8PSME&OWJSPONFOU7BMVFJUFSBUJPO

7BMVF*UFSBUJPO ߈ࠂೞ૑ঋח׮ 4UBUF "DUJPO

7BMVFJUFSBUJPO दো

.PEFM੉হ׮ݶ पઁ۽҃೷ਸ೧ࠁݴജ҃җ࢚ഐ੘ਊਸ೧ঠೠ׮

Monte Carlo method

.POUF$BSMPNFUIPEח%ZOBNJDQSPHSBNJOH୊ۢ ݽٚ੿ࠁܳঌҊद੘ೞחѪ੉ইצ पઁ۽҃೷ਸೞݴജ҃җ࢚ഐ੘ਊਸೠ׮ .POUF$BSMP

पઁ۽҃೷ਸೞݴߓ਋חߑߨ੉જ਷੼਷FOWJSPONFOU੄ ੿ࠁоহযبपઁ۽҃೷ਸೞݴPQUJNBMCFIBWJPSਸ੉ܖӝ ⮚ٸޙ .POUF$BSMP

.POUF$BSMP .POUF$BSMPחFQJTPEFCZFQJTPEF۽সؘ੉౟ೠ׮ ীೖࣗ٘੄݃૑݄झప੉౟UFSNJOBMTUBUFө૑ оࢲসؘ੉౟ೠ׮ .POUF$BSMPח҃೷ਸೞݴSFUVSOػTBNQMFਸ੉ਊೞৈ TUBUFBDUJPOWBMVFܳಣӐೞৈসؘ੉౟ೠ׮

(PBM .POUF$BSMP(SJE8PSME ՘ө૑оࠄٍ6QEBUF 4UBSU

.POUF$BSMP दো

Temporal-Difference Learning

݅ডъച೟णਸ؀಴ೡࣻ੓חই੉٣যо੓׮ݶӒ Ѫ਷5% UFNQPSBMEJGGFSFODF MFBSOJOHੌѪ੉׮ 4VUUPO 5FNQPSBM%JGGFSFODF-FBSOJOH

5FNQPSBM%JGGFSFODF-FBSOJOH .POUF$BSMP %ZOBNJDQSPHSBNNJOH .POUF$BSMP୊ۢݽ؛হ੉҃೷ਸాೞৈWBMVFܳஏ੿ೞݴ %1୊ۢ՘ө૑о૑ঋইب઺р઺рWBMVFܳFTUJNBUFೞחѪ੉оמ

5FNQPSBM%JGGFSFODF-FBSOJOH അ੤TUBUFীࢲBDUJPOਸࢶఖೞݴ߉ਸ3FXBSEҗ׮਺4UBUFীEJTDPVOUGBDUPSо੸ਊػ TUBUFWBMVFܳFTUJNBUFೞݴVQEBUFೠ׮ .POUF$BSPMPীࢲ੄(Uܳঌইঠসؘ੉౟оמ

5%੄੢੼ .POUF$BSMP੄ъ੼ੋജ҃੄ݽ؛ਸঌ૑ޅ೧بࢎਊоמ %ZOBNJDQSPHSBNNJOHীࢲ୊ۢ0OMJOF೟ण੉׮ ՘ө૑ӝ׮ܻ૑ঋইب ઺р઺рVQEBUFооמೞӝীFQJTPEFо ݆੉ӡѢա഑਷DPOUJOVFೠNPEFMীࢲࢎਊೞӝજ׮

5%੄ই੉٣য 5FNQPSBM%JGGSFOFDF-FBSOJOH 4BSTB 2MFBSOJOH 5FNQPSBM%JGGSFOFDF-FBSOJOH੉4BSTB৬2MFBSOJOH੄߄ఔই੉٣যоغ঻׮ 0OQPMJDZ 0GGQPMJDZ

4BSTB 2MFBSOJOH Temporal-Diffrenece Learning

4BSTB POQPMJDZߑߨਸࢎਊೞח4BSTB TUBUFWBMVFGVODUJPO؀नBDUJPOWBMVFGVODUJPOਸ೟ण

4BSTB ׮਺UJNFTUFQীࢲTUBUF৬BDUJPOܳل׮ࢎਊೞৈBDUJPOWBMVFܳFTUJNBUFೠ׮

4BSTBQTFVEPDPEF

4BSTBQTFVEPDPEF 0OQPMJDZ

4BSTBHSJEXPSME (PBM 4UBSU "U 4U

4BSTBHSJEXPSME
҃೷ೞ૑ঋ਷झప੉౟੄੿ࠁоহ׮

4BSTBHSJEXPSME
҃೷ਸৈ۞ߣ೧ࠁݴBDUJPOWBMVFܳসؘ੉౟ೠ׮ 1PMJDZח0OQPMJDZ

4BSTB दো

4BSTB 2MFBSOJOH Temporal-Diffrenece Learning

2MFBSOJOH 2MFBSOJOH੉ۄҊܻࠛחPGGQPMJDZ5%DPOUSPMੋ೧ъച೟ण੉ߊ੹ೞח҅ӝоغ঻׮ 8BULJOT FYQMPSBUJPOҗFYQMPJUBUJPOਸэ੉ೠ׮

2MFBSOJOHQTFVEPDPEF 0GGQPMJDZ

RMFBSOJOHHSJEXPSME (PBM 4UBSU "SHNBY 4U

2MFBSOJOHHSJEXPSME
҃೷ೞ૑ঋ਷झప੉౟੄੿ࠁоহ׮

2MFBSOJOHHSJEXPSME
҃೷ਸৈ۞ߣ೧ࠁݴBDUJPOWBMVFܳসؘ੉౟ೠ׮ 1PMJDZח0GGQPMJDZ

2MFBSOJOH दো

٩۞׬೐ۨ੐ਕ௼ாۄझࣗѐगಌܻ݃য়ജ҃ҳ୷

%FFQMFBSOJOH https://goo.gl/images/VA89CC

%FFQMFBSOJOHਵ۽ੋ೧ https://chaosmail.github.io/deeplearning/2016/10/22/intro-to-deep-learning-for-computer-vision/ ࢎ૓ਸJOQVUਵ۽߉חѪ੉оמ೧૗

%FFQ3FJOGPSDFNFOU-FBSOJOH %FFQMFBSOJOH 3FJOGPSDFNFOU-FBSOJOH https://goo.gl/images/oNu5Gr

%FFQNJOE %2/ https://www.youtube.com/watch?v=V1eYniJ0Rnk %FFQMFBSOJOHਸъച೟णী੸ਊೞৈ ࢎۈࠁ׮೒ۨ੉ܳੜೞחੋҕ૑מਸ݅ٞ

%2/ ,FSBT CSFBLPVU ௏٘ࢸݺ .BJO MJCSBSZ 'VODUJPO

%2/ ,FSBT CSFBLPVU ௏٘ࢸݺ .BJO MJCSBSZ %2/

ജ҃ਸࠛ۞ৡ׮ BHFOUܳࢤࢿೠ׮ TDPSF FQJTPEF HMPCBM@TUFQܳ੿೧ળ׮ ੿೧ળীೖࣗ٘݅ఀ೟णਸद੘ ݾऀ׮ࢽѐо઱য૓׮ ജ҃੄ୡӝчਸоઉৡ׮ ୊਺ੌ੿ҳрزউ߄оBDUJPOਸ੐੄۽ࢶఖೡࣻ ੓ѱೠ׮
ਤ੄ജ҃ୡӝчਸ੉޷૑੹୊ܻ೧ળ׮ ੹୊ܻ೧ળ੉޷૑۽֎ѐ੄൤झషܻܳ݅ٚ׮ .BJO

ѱ੐੉՘զٸө૑҅ࣘجѱೞחXIJMFޙਸ݅ٚ ׮ SFOEFS੄ਬޖܳഛੋೠ׮ SFOEFSਸਬޖীٮ ۄ೟णࣘبо׳ۄ૓׮ Ӗ۽ߥझచਸೞաঀט۰ળ׮ झచਸೞաঀט۰ળ׮ ֎ѐ੄TUBUF IJTUPSZ ܳ੉ਊೞৈBDUJPOਸ
ࢶఖೠ׮ .BJO

.BJO ࢶఖೠBDUJPOਵ۽ജ҃җ࢚ഐ੘ਊೞݴജ҃ীࢲ׮਺झప੉ ౟ SFXBSE EPOF JOGP੄чਸ߉ח׮ ߉਷׮਺झప੉౟ܳ׮द੹୊ܻ೧ળ׮ IJTUPSZীࢲখ੄ࣁѐ৬ߑӘ߉ইৡTUBUFܳ೤୛ࢲ OFYU@IJTUPSZ۽ࢶ঱ R@NBY੄ಣӐਸ҅࢑ೞӝਤ೧ࢲഅ੤੄NPEFM۽ࠗఠաৡ2
ч੄NBYܳBHFOUBWH@R@NBYী؊ೠ׮ ݅ডEFBEੋ҃਋EFBEܳ5SVF۽߄ԲҊ TUBSU@MJGFܳೞա ઴ৈળ׮

.BJO ੌ੿दр݃׮UBSHFUNPEFMਸVQEBUFೠ׮ ݅ডীલਵݶEFBEGBMTF۽߄ԲҊইפݶ OFYUIJTUPSZчਸIJTUPSZо߉ח׮ ݅ডীEPOF੉ݶীೖࣗ٘੄೟ण੿ࠁܳӝ۾ ೞৈ୹۱ೠ׮ ੌ੿ীೖࣗ٘݃׮ݽ؛ਸ੷੢ೠ׮

.BJO ׮ܲইఋܻѱ੐ীࢲب੸ਊೡࣻ੓ب۾ܻਕ٘੄ߧਤܳ_ ۽ೠ׮ T B S Tܻܳ೒ۨ੉ݫݽܻী੷੢ೠ׮ ܻ೒ۨ੉ݫݽܻоद੘੼ࠁ׮֫ই૑ݶ೟णਸद੘ೠ׮

.BJO ੌ੿दр݃׮UBSHFUNPEFMਸVQEBUFೠ׮ ݅ডীલਵݶEFBEGBMTF۽߄ԲҊইפݶ OFYUIJTUPSZчਸIJTUPSZо߉ח׮ ੌ੿ীೖࣗ٘݃׮ݽ؛ਸ੷੢ೠ׮

*NQPSU ೙ਃೠۄ੉࠳۞ܻܳࠛ۞ৡ׮ B,FSBT $//MBZFS %FOTFMBZFS PQUJNJ[FS ாۄझীࢲ੄٩۞׬ݽ؛

*NQPSU C੉޷૑੹୊ܻ JOQVUਵ۽ٜযয়ח੉޷૑௼ӝઑ੺ 3(#ܳ(SBZ۽݅٘חۄ੉࠳۞ܻ SFQMBZNFNPSZ݅٘ח D5FOTPSGMPX UFOTPSGMPXCBDLFOE UFOTPSGMPX Eӝఋ
OVNQZ SBOEPN HZN PT

%2/ ↟ SFOEFS੄ਬޖ ↟ NPEFM੄MPBEਬޖ ↟ TUBUFࢎ੉ૉ
↟ BDUJPOࢎ੉ૉ ↟ FQTJMPOч ↟ FQTJMPO੄द੘੼җ՘੼ EFDBZܳਤ ೧ ↟ FQTJMPO੄EFDBZTUFQ੿੄ ↟ ୡӝച

%2/ ↟ ܻ೒ۨ੉ݫݽܻীࢲࡳਸߓ஖ࢎ੉ૉ੿੄ ↟ ೟णਸद੘ೡӝળ੿੄ ↟ ੿׹ݽ؛۽সؘ੉౟઱ӝ੿੄
↟ EJTDPVOUGBDUPS ↟ ܻ೒ۨ੉ݫݽܻ୭؀௼ӝ੿੄ ↟ झఋ౟ೡٸBDUJPOਸ੐੄۽੿೧઱חࢸ੿ ↟ %FFQMFBSOJOHNPEFM ↟ 5BSHFUNPEFM ↟ VQEBUFUBSHFUNPEFM ↟ ୡӝച

%2/ ↟ PQUJNJ[FS ↟ 5FOTPSCPBSE ↟ ୡӝച
4BWFػݽ؛੄ਝ੉౟ܳоઉৢٸࢎਊ

%2/ ,FSBT۽٩۞׬ݽ؛ٜ݅ӝ $//-BZFST %FOTF-BZFS

%2/ BDUJPOਸࢶఖೞחೣࣻ QPMJDZ ৈ ӝࢲח&QTJMPOHSFFEZ അ੤NPEFM੄XFJHIUܳоઉ৬ࢲUBSHFU NPEFM੄ਝ੉౟۽সؘ੉౟ೞחೣࣻ

%2/ TUBUF BDUJPO SFXBSE OFYUTUBUFܻܳ೒ۨ੉ݫݽܻী੷੢೧઱חೣࣻ 3FQMBZ.FNPSZ

%2/ ܻ೒ۨ੉ݫݽܻীࢲࡳইৡߓ஖۽ݽ؛ਸ೟णೞחೣࣻ 3FQMBZNFNPSZ

%2/ ↟ PQUJNJ[FS ↟ 5FOTPSCPBSE пীೖࣗ٘׼೟ण੿ࠁܳӝ۾ೞ חೣࣻ ୡӝച

%2/ ↟੉޷૑੹୊ܻܳਤೠೣࣻ ↟ ୡӝച

%2/ 0QUJNJ[FSೣࣻ ৈӝࢲח)VCFS-PTTࢎਊ https://goo.gl/images/XGsfYx

%2/ ,FSBT CSFBLPVU दো

गಌܻ݃য়ജ҃ҳ୷

)VNBO "* %FFQMFBSOJOH 3FJOGPSDFNFOU-FBSOJOH ׮ܲبݫੋীࢲب੸ਊ੉оמೞ׮

ৈ۞ജ҃ীࢎਊ ೞ૑݅ജ҃ীٮܲ੸੺ೠъച೟णঌҊ્ܻਸ੸ਊ೧ঠೠ׮

৘޹ೠъച೟ण 4UBUF௼ӝ BDUJPO੉׮ܰ׮    э਷ঌҊ્ܻਸࢎਊೞ؊ۄب %FFQMFBSOJOHNPEFM IZQFSQBSBNFUFSਸ ੸੺ೞѱࢎਊ೧ঠೠ׮

&NVMBUPS &OWJSPONFOU Algorithm 1SPHSBNNJOH-BOHVBHF गಌܻ݃য়ࢸ஖ী೙ਃೠ֎о૑

IUUQTXXXQZUIPOPSHEPXOMPBETWFSTJPO IUUQTXXXBOBDPOEBDPNEPXOMPBE"OBDPOEB IUUQTXXXUFOTPSGMPXPSHJOTUBMM5FOTPS'MPX IUUQTLFSBTJPJOTUBMMBUJPO,FSBT 1SPHSBNNJOH-BOHVBHF1ZUIPO

&NVMBUPS IUUQXXXGDFVYDPNXFCIPNFIUNM 6CVOUV TVEPBQUHFUVQEBUF TVEPBQUHFUJOTUBMMGDFVY ."$ IUUQTCSFXTIIPNFCSFXXFCTJUF 5FSNJOBMPQFOCSFXJOTUBMMGDFVY TVEPBQUHFUJOTUBMMGDFVY &NVMBUPS'$69

Environment 0QFO"*@(ZN IUUQTHJUIVCDPNPQFOBJHZN QJQJOTUBMMHZN HJUDMPOFIUUQTHJUIVCDPNPQFOBJHZNHJU  DEHZN  QJQJOTUBMMF 0QFO"*@(ZN 0QFO"*੄(ZNਸࢎਊೞݶࠁ׮औѱъച೟णप೷੉оמೞ׮

Environment #BTFMJOFT IUUQTHJUIVCDPNPQFOBJCBTFMJOFT QJQJOTUBMMCBTFMJOFT HJUDMPOFIUUQTHJUIVCDPNPQFOBJCBTFMJOFTHJU  DECBTFMJOFT  QJQJOTUBMMF 0QFO"*@#BTFMJOFT

Environment 1IJMJQ1BRVFUUF IUUQTHJUIVCDPNQQBRVFUUFHZNTVQFSNBSJP QJQJOTUBMMHZNQVMM JNQPSUHZN JNQPSUHZN@QVMM HZN@QVMMQVMM HJUIVCDPNQQBRVFUUFHZNTVQFSNBSJP FOWHZNNBLF
QQBRVFUUF4VQFS.BSJP#SPTW 4VQFS.BSJP

Algorithm DEEP Q-NETWORK "MHPSJUIN%2/

ࢸ஖ীޙઁоࢤӟ׮ݶ IUUQTHJUIVCDPNXPOTFPLKVOH,*14@3FJOGPSDFNFOUUSFF NBTUFS%2/    য়טъച೟णपणъ੄੹ਊHJUIVCী੗ࣁೠࢸ஖ߨৢ۰֬ওणפ׮

%2/ਸ੉ਊೠੋҕ૑מगಌܻ݃য়ٜ݅ӝ

Ӓܻ٘ਘ٘৬যڌѱ׮ܳө 4UBUFӒܻ٘੄ઝ಴ "DUJPO࢚ ೞ ઝ ਋ 3FXBSEೣ੿ ݾ಴  5SBOTJUJPO1SPCBCJMJUZ %JTDPVOUGBDUPS
3FXBSE 3FXBSE 4UBUF "DUJPO Ӓܻ٘ਘ٘৬गಌܻ݃য়ജ҃

(PBM੄࠺Ү (PBM 4UBSU गಌܻ݃য়חӥߊਸ੟ח੉ݾ಴ Ӓܻ٘ਘ٘੄ݾ಴חHPBMTUBUF۽оחѪ

गಌܻ݃য়ীࢲ੄ജ҃ 4UBUFചݶ "DUJPO࢚ ೞ ઝ ਋ ੼೐ ׳ܻӝ BDUJPO੄ઑ೤ 3FXBSEখਵ۽੹૓ೡٸ3FXBSE
ٍ۽оݶ  5SBOTJUJPO1SPCBCJMJUZ %JTDPVOUGBDUPS 4UBUF "DUJPO ب଱૑ੋӥߊীоө੉тࣻ۾֫਷SFXBSEܳ߉ח׮

҅ࣘغחपಁj

੉ग ܻ݃য়оখਵ۽੹૓ೞ૑ঋਵ۰Ҋೞחഅ࢚ 4UBUFоCSFBLPVUࠁ׮؊ࠂ੟ೞҊBDUJPO੉݆׮

3FXBSEࢸ੿ ݾ಴׳ࢿೞ૑ޅೞݶ दр੉૑զٸ݃׮ ӥߊীࢲݣয૑ݶ ӥߊীоөਕ૑ݶ ݾ಴ীب଱ೞݶ 1FOBMUZ #POVTSFXBSE୶о

%FFQMFBSOJOHNPEFM 7((NPEFMBOESFHVMBS࠺Ү https://goo.gl/images/eoXooC https://goo.gl/images/s8XrCK ؊Өѱऺইࠁ੗

ъച೟ण SFJOGPSDFNFOUMFBSOJOH ӝୡࢸݺ߂6OJUZNMBHFOUܳ੉ਊೞৈ݅ٚജ҃ীъച೟णঌҊ્ܻ੸ਊ (JUIVC IUUQTHJUIVCDPNXPOTFPLKVOH 'BDFCPPL IUUQTXXXGBDFCPPLDPNXTKVOH #MPH IUUQTXPOTFPLKVOHHJUIVCJPࢿҕ

%2/ਸ੉ਊೠੋҕ૑מगಌܻ݃য়ٜ݅ӝ दো

ۨ߰ਸ௿ܻযೞחܻ݃য়ח݅ٚറ ׮ܲۨ߰ীࢲࢿמ੉ݒ਋ڄয૓׮0WFSGJUUJOH੉ইקө https://goo.gl/images/6uDmqH

ъച೟ण੄োҳחഝߊ൤૓೯઺ 3FXBSE &YQMPSBUJPO "MHPSJUIN

хࢎ೤פ׮ (JUIVC IUUQTHJUIVCDPNXPOTFPLKVOH 'BDFCPPL IUUQTXXXGBDFCPPLDPNXTKVOH #MPH IUUQTXPOTFPLKVOHHJUIVCJP

3FGFSFODFT 3FJOGPSDFNFOU-FBSOJOH"O*OUSPEVDUJPO3JDIBSE44VUUPOBOE"OESFX(#BSUP4FDPOE&EJUJPO JOQSPHSFTT.*51SFTT $BNCSJEHF ." IUUQTHJUIVCDPNSMDPEFSFJOGPSDFNFOUMFBSOJOHLS

Reinforcement Learning from classic to DQN

Reinforcement Learning from classic to DQN

More Decks by Wonseok Jung

Other Decks in Research

Featured

Transcript