Faster Reinforcement Learning via Transfer

'BTUFS3FJOGPSDFNFOU-FBSOJOH 8POTFPL+VOH WJB5SBOTGFS +PIO4DIVMNBO

FASTER REINFORCEMENT LEARNING VIA TRANSFER

OVERVIEW 1. Policy Gradients  Success Stories   Limitations 2. Meta
Reinforcement Learning 3. Gym retro OVERVIEW

TERMINOLOGY 3FJOGPSDFNFOU-FBOJOH    5SJBMBOEFSSPSਸೞݴ3FXBSEܳ୭؀ചೠ׮  %FFQ3-    /FVSBMOFUXPSLܳࢎਊೞৈ3-BMHPSJUINਸSFQSFTFOUೠѪ  .FUB-FBSOJOH    t-FBSOJOHIPXUP-FBSOuযځೠ-FBSOJOHীҙৈೞח5BTLܳ.BTUFS
REINFORCEMENT LEARNING

TERMINOLOGY REINFORCEMENT LEARNING 3FJOGPSDFNFOU-FBOJOH    5SJBMBOEFSSPSਸೞݴ3FXBSEܳ୭؀ചೠ׮  %FFQ3-    /FVSBMOFUXPSLܳࢎਊೞৈ3-BMHPSJUINਸSFQSFTFOUೠѪ  .FUB-FBSOJOH 
  t-FBSOJOHIPXUP-FBSOuযځೠ-FBSOJOHীҙৈೞח5BTLܳ.BTUFS

MARKOV DECISION PROCESS "DUJPO "HFOU &OWJSPONFOU 3FXBSE At Rt 4UBUF
St Rt+1 St+1 REINFORCEMENT LEARNING

AGENT "DUJPO "HFOU &OWJSPONFOU 3FXBSE At Rt 4UBUF St Rt+1
St+1 REINFORCEMENT LEARNING

ACTION "DUJPO "HFOU &OWJSPONFOU 3FXBSE At Rt 4UBUF St Rt+1
St+1 REINFORCEMENT LEARNING

OBSERVATION, REWARD "DUJPO "HFOU &OWJSPONFOU 3FXBSE At Rt 4UBUF St
Rt+1 St+1 REINFORCEMENT LEARNING

TRAJECTORY REINFORCEMENT LEARNING (St , At , Rt+1 , St+1
) (St+1 , At+1 , Rt+2 , St+2 ) (St+2 , At+2 , Rt+3 , St+3 )

RETURN 4UBUFWBMVF REINFORCEMENT LEARNING 4UBUF"DUJPOWBMVF

1. POLICY GRADIENTS

POLICY 1PMJDZ   0CTFSWBUJPOী੄೧"DUJPOਸࢶఖೞחೣࣻ REINFORCEMENT LEARNING

REINFORCEMENT LEARNING POLICY GRADIENTS https://wonseokjung.github.io//reinforcementlearning/update/RL-PG_RE/ 1PMJDZHSBEJFOUTী؀೧ખ؊੗ࣁ൤ঌҊर׮ݶ 1PMJDZ(SBEJFOUTNFUIPE ؊જ਷1PMJDZܳ଺ӝਤ೧ࢲ1PMJDZ੗୓ܳ୭੸ചೞחъച೟णঌҊ્ܻ

PSEUDO CODE REINFORCEMENT LEARNING

POLICY GRADIENTS - HISTORY REINFORCEMENT LEARNING

POLICY GRADIENTS - HISTORY REINFORCEMENT LEARNING https://wonseokjung.github.io//reinforcementlearning/update/RL-PG_RE_AC/ 1PMJDZ(SBEJFOUT 3&*/'03$& "$503$3*5*$֤ޙܻ࠭

POLICY GRADIENTS - PPO REINFORCEMENT LEARNING

ALPHAGO, DOTA REINFORCEMENT LEARNING

ROBOTIC MANIPULATION REINFORCEMENT LEARNING

RL REQUIRES A LOT OF TRAINING TIME REINFORCEMENT LEARNING

PRIOR KNOWLEDGE REINFORCEMENT LEARNING

HOW WE CAN ALLOW OUT A.I SYSTEM MAKE TO USE
PRIOR KNOWLEDGE? REINFORCEMENT LEARNING https://ubisafe.org/explore/demeanure-clipart-prior-knowledge/

META REINFORCEMENT LEARNING REINFORCEMENT LEARNING https://ubisafe.org/explore/demeanure-clipart-prior-knowledge/ Meta Reinforcement Learning

META REINFORCEMENT LEARNING REINFORCEMENT LEARNING https://ubisafe.org/explore/demeanure-clipart-prior-knowledge/ 4JOH5BTL

SINGLE R.L : MAZE NAVIGATION REINFORCEMENT LEARNING 4JOH5BTL    -FBSOUPOBWJHBUFGSPNTUBSUUPHPBMBTGBTUBTQPTTJCMFJOB
TJOHMFNB[F

META R.L : MAZE NAVIGATION REINFORCEMENT LEARNING

META RL IS A SPECIAL CASE OF NORMAL RL REINFORCEMENT
LEARNING

REINFORCEMENT LEARNING RL2

REINFORCEMENT LEARNING META-RL IN MAZE

REINFORCEMENT LEARNING LEARNING DEXTEROUS IN HAND MANIPULATION

REINFORCEMENT LEARNING META-RL IN ROBOT

REINFORCEMENT LEARNING META RL : LIMITATION OF APPROACHES DESCRIBE PREVIOUSLY

REINFORCEMENT LEARNING META RL : CHANGES TO PROBLEM FORMULATION 'JOJUFTFUPGUSBJOJOHUBTLTBOEBUFTUTFUPGUBTLT

REINFORCEMENT LEARNING GYM RETRO

REINFORCEMENT LEARNING ALGORITHMS

REINFORCEMENT LEARNING PPO(JOINT) + FINE TUNING

REINFORCEMENT LEARNING RETRO CONTEST

REINFORCEMENT LEARNING IMPROVEMENT

REINFORCEMENT LEARNING MODULABS CTRL

(JUIVC IUUQTHJUIVCDPNXPOTFPLKVOH 'BDFCPPL IUUQTXXXGBDFCPPLDPNXTKVOH #MPH IUUQTXPOTFPLKVOHHJUIVCJP хࢎ೤פ׮   5IBOLZPV

Faster Reinforcement Learning via Transfer

Faster Reinforcement Learning via Transfer

More Decks by Wonseok Jung

Other Decks in Research

Featured

Transcript