Metalearning shared Hierarchy

Metalearning shared Hierarchy

논문 review

Transcript

1. .FUB-FBOJOH
   TIBSFE
   )JFSBSDIZ 8POTFPL
   +VOH 3FJOGPSDFNFOU
   -FBSOJOH
2. ੿ਗࢳ
 8POTFPL
   +VOH
   $JUZ
   6OJWFSTJUZ
   PG
   /FX
   :PSL
   
   #BSVDI
   $PMMFHF
   %BUB
   4DJFODF
   .BKPS
   $POOFYJPO"*
   "*
   3FTFBSDIFS
   %FFQ-FBSOJOH$PMMFHF
   3FJOGPSDFNFOU
   -FBSOJOH
   3FTFBSDIFS
   .PEVMBCT
   $53-
   -FBEFS
   3FJOGPSDFNFOU
   -FBSOJOH
   0CKFDU
   %FUFDUJPO
   $IBUCPU
   (JUIVC
   IUUQTHJUIVCDPNXPOTFPLKVOH
   'BDFCPPL
   IUUQTXXXGBDFCPPLDPNXTKVOH
   #MPH
   IUUQTXPOTFPLKVOHHJUIVCJP
   
3. ݾର 1. Introduction 2. Problem Statement 3. Algorithm 4. Experiments META LEARNING SHARED HIERARCHIES
4. 1.INTRODUCTION
5. 1. UTILIZE PRIOR KNOWLEDGE META LEARNING SHARED HIERARCHIES 6UJMJ[F
   QSJPS
   LOPXMFEHF .BTUFS
   OFX
   UBTL
6. 1.1 BUT REINFORCEMENT… META LEARNING SHARED HIERARCHIES How about Reinforcement Learning?
7. 1.2 SOLVE EACH TASK INDEPENDENTLY AND FROM SCRATCH SUPERMARIO WITH R.L https://www.youtube.com/watch?v=IjvbhwuCaF0
8. 1.3 ISSUES META LEARNING SHARED HIERARCHIES Sharing information Task1 Task2 Task3 θ1 θ2 θ3
9. 1.4 MASTER POLICY META LEARNING SHARED HIERARCHIES Master Policy Sub1 Sub2 Sub3 θ1 θ2 θ3
10. 1.5 MLSH META LEARNING SHARED HIERARCHIES Metalearning shared hierarchies
11. 2.PROBLEM STATEMENT
12. 2.1 NOTATION Time step Action Transition Function Reward Set of states Set of actions Start state Discount factor t a P(s′, r ∣ s, a) r A S S0 γ Set of reward
 
 Policy Reward State R π r REINFORCEMENT LEARNING s
13. 2.2 NOTATION META LEARNING SHARED HIERARCHIES
    
    EJTUSJCVUJPO
    PWFS
    .%1T
    
    
    
    
    "HFOUח
    QBSBNFUFS
    WFDUPS
    
    
    ܳ
    ઱ӝ੸ਵ۽
    VQEBUFೠ׮
    
    
    
    
    పझ௼ٜ
    ՙܻ
    ҕਬೞח
    ౵ۄ޷ఠ੄
    ૘೤
    
    
    п
    పझ௼
    ౵ۄ޷ఠ੄
    ૘೤ 

    
    
    
    BHFOUо
    അ੤
    పझ௼
    .ਸ
    ߓ਋ݴ
    সؘ੉౟
    ೞח
    ౵ۄ޷ఠ
    
    PM πθ,ϕ(a∣s) ϕ θ
14. "DUJPO "HFOU &OWJSPONFOU 3FXBSE At Rt 4UBUF St Rt+1 St+1 REINFORCEMENT LEARNING 2.3 OBJECTIVE MDP
15. REINFORCEMENT LEARNING 2.4 NEW MDP &OWJSPONFOU 3FXBSE At Rt St Rt+1 St+1 5BQ
    UIF
    CBMM 1PTJUJWF
    3FXBSE
 New MDP
16. SUPERMARIO WITH R.L 2.5 NEW MDP-2 "DUJPO "HFOU &OWJSPONFOU 3FXBSE At Rt 4UBUF St Rt+1 St+1 3FXBSE
    
    
    1FOBMUZ
    
     Another New MDP
17. 2.6 FIND SHARING PARAMETER META LEARNING SHARED HIERARCHIES maximizeϕ EM∼PM , t = 0...T − 1[R]
18. 2.7 STRUCTURE META LEARNING SHARED HIERARCHIES
19. 3.ALGORITHM
20. 3.1 MLSH ALGORITHM META LEARNING SHARED HIERARCHIES
21. 3.2 MLSH ALGORITHM META LEARNING SHARED HIERARCHIES Two main components
22. 3.3 MLSH ALGORITHM META LEARNING SHARED HIERARCHIES Joint update period Warmup period
23. 3.4 MLSH ALGORITHM META LEARNING SHARED HIERARCHIES Joint update period Warmup period
24. 3.5 MLSH ALGORITHM META LEARNING SHARED HIERARCHIES Joint update period Warmup period θ θ, ϕ update
25. 3.6 MLSH ALGORITHM-2 META LEARNING SHARED HIERARCHIES Joint update period Warmup period θ θ, ϕ update
26. 3.7 MLSH ALGORITHM-WARMUP META LEARNING SHARED HIERARCHIES update
27. 3.8 MLSH ALGORITHM- JOINT UPDATE PERIOD META LEARNING SHARED HIERARCHIES update
28. 3.8 MLSH ALGORITHM META LEARNING SHARED HIERARCHIES update
29. 4. EXPERIMENTS
30. 4.1 2D MOVING BANDITS TASK META LEARNING SHARED HIERARCHIES
31. 4.2 RESULT(2D BALL) META LEARNING SHARED HIERARCHIES
32. 4.3 WALKING, CRAWLING META LEARNING SHARED HIERARCHIES
33. 4.4 WALKING, CRAWLING META LEARNING SHARED HIERARCHIES