DeepRacer for learning RL

Transcript

1. 4 D 29 26 1 . 0 I 1

2. & .-*  (2  ,0'/4"# 51 
   83;7 +)

   !&%$9( 6:  Attention

3. 3 
    #cmdevio2019

4. 4 os t m ( L @S g E b

   i _d L rMI D @ E ( ( ( ) ( e a n k AWS E

5. 5  
    DeepRacer 

6. 6  
    D

7. 7    
    ) (

8. 8 …

9. 9 DeepRacer
   
   
   
   4 D 26 9 01 .

10. 10 DeepRacer
    
    
    
    A A A

11. 11
    
       
       

    1 2 3

12. 12 DeepRacer
    

13. 13  
     

14. 14 DeepRacer 1/18
    
    
    
        

     3D
     
    
    AWS DeepRacer League

15. 15 DeepRacer
     https://aws.amazon.com/jp/deepracer/

16. 16 DeepRacer
     ! &%$ +)*2 1 '/
    *2   

     (#-, 0.  "

17. 17 3D
    
    
    AWS RoboMaker Robot Operating System (ROS) Gazebo rqt

18. 18 AWS DeepRacer League ⁻ 0 1 : 9 A

    ⁻ 9 2 R ⁻  
     ⁻ D I ⁻ 1 2 ⁻   https://aws.amazon.com/jp/deepracer/league/

19. 19 
    

20. 20   (Artificial Intelligence, AI)  (Machine Learning, ML)

    
       NeuralNetwork DeepLearning   

21. 21 
    

22. 22 = 1    (   

      ) ( (

23. 23 L N - ) ( - D Q

24. 24 DeepRacer     Cliped PPO PPO (Proximal

    Policy Optimization) OpenAI2017 

25. 25 
         ( ( )

    )

26. 26 
     1

27. 27 
     ) () (

28. 28 DeepRacer
      

29. 29 DeepRacer
     + + +

30. 30 DeepRacer
    

31. 31 DeepRacer 
    …

32. 32 DeepRacer 
     
    

33. 33 orz

34. 34 DeepRacer 
    + + +

35. 35 DeepRacer 
    D A D

36. 36 $
    
      ' + (# 
      &!

    %"

37. 37  
     ( ) ) https://docs.aws.amazon.com/ja_jp /deepracer/latest/developerguide/ deepracer-train-models-define- reward-function.html

38. 38 
      
     

39. 39   
        ⁻ 10 ⁻

    :

40. 40 SageMaker RL + RoboMaker

41. 41 SageMeker RLRoboMakerGA

42. 42 SageMaker “RL”  ⁻ ⁻ M ⁻ M M

    ⁻ M  
    ⁻ ⁻ ⁻ J M S

43. 43 DeepRacer ) D ( ) ) ( )

44. 44 SageMaker 
          
    

      https://dev.classmethod.jp/machine -learning/sagemaker-robomaker- deepracer-sample/

45. 45
         $#   "

    
        ! https://github.com/awslabs/amazon-sagemaker-examples

46. 46 Jupyter
       !
      

47. 47 
     ( ( ) 

48. 48   

49. 49  2   1  
    

50. 50  2   2  
     

       
    

51. 51   (. ( )( 
      

52. 52 $2# " /1(+ $2#,- !  https://docs.aws.amazon.com/ja_jp/deepracer/latest/developerguide/deepracer -iteratively-enhance-reward-functions.html 
    *

     $2%) '.0& )

53. 53 
     
     Best Practices when training with PPO

    (Unity Technologies) https://github.com/Unity-Technologies/ml-agents/blob/master/docs/Training-PPO.md   
    
        

54. 54 DeepRacer "% 
     
      !  
    "%

     
     
     
     $#  
    
     
     !  
    
    !

55. 55 DeepRacer
    

56. 56 DeepRacer
     
       

57. 57 DeepRacer
     
    

58. 58 DeepRacer
        
    

59. 59 DeepRacer
       
      

60. 60 
    

61. 61 • g • + + • M D c

    • R S D a k • b • LL e

62. 62 DeepRacer 

63. None