AWS DeepRacer

Chainer Meetup #10 【強化学習】にて
2019/08/31（Sat） 13:30〜

https://chainer.connpass.com/

Transcript

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer Amazon Web Services Japan K.K. Machine Learning Solutions Architect Shoko Utsunomiya $ I B J O F S
   . F F U V Q
     
   
   3 F J O G P S D F N F O U
   - F B S O J O H
2. h
   
   "NB[PO
   8FC
   4FSWJDFT
   *OD
   PS
   JUT
   BGGJMJBUFT
   "MM
   SJHIUT
   SFTFSWFE
      • eS • eS • W M • O • a • AE • •
3. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
4. h
   
   "NB[PO
   8FC
   4FSWJDFT
   *OD
   PS
   JUT
   BGGJMJBUFT
   "MM
   SJHIUT
   SFTFSWFE AWS DeepRacer     
   
5. h
   
   "NB[PO
   8FC
   4FSWJDFT
   *OD
   PS
   JUT
   BGGJMJBUFT
   "MM
   SJHIUT
   SFTFSWFE AWS DeepRacer 1 / 8
6. h
   
   "NB[PO
   8FC
   4FSWJDFT
   *OD
   PS
   JUT
   BGGJMJBUFT
   "MM
   SJHIUT
   SFTFSWFE %FFQ3BDFS ׾饥׵ׇ׷׋׭ח • . • ) (
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 
   
8. h
   
   "NB[PO
   8FC
   4FSWJDFT
   *OD
   PS
   JUT
   BGGJMJBUFT
   "MM
   SJHIUT
   SFTFSWFE "84
   %FFQ3BDFS חֶֽ׷䓼⻉㷕统ך銲稆 
       
    
9. h
   
   "NB[PO
   8FC
   4FSWJDFT
   *OD
   PS
   JUT
   BGGJMJBUFT
   "MM
   SJHIUT
   SFTFSWFE ءىُٖ٦ةד㷕统 
    (reward fn)
10. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE S G = 2
    
     
11. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE إٝة٦ٓ؎ٝ׾饥׷״ֲח؎ٝإٝذ؍ـ׾♷ִ׷ 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 S 2 2 2 2 2 2 G = 2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9 S 10.4 9.4 8.2 6.9 5.4 3.8 G = 2 8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9 
    
12. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE "84
    %FFQ3BDFS ךصُ٦ٕٓطحزٙ٦ؙ圓鸡  –  (
    )  – 
13. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE 
     (BEHAVIORAL CLONING) • •   ( ( %FFQ3BDFS חֶֽ׷䓼⻉㷕统 WT
    ׉׸⟃㢩ך،ٕ؞ٔؤي  • • )   ( (
14. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE AWS Summit Tokyo 2019 DeepRacer League
15. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
16. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark 䓼⻉㷕统ך圓䧭 ~ Amazon SageMaker, RoboMaker ׾⢪׏׋ DeepRacer ~  湡涸ꟼ侧鰵ָٖ٦ٝ׾כ׫⳿ׁ׆؞٦ٕׅ׷  橆㞮"84
    3PCP.BLFS דمأزׁ׸גְ׷%س ٓ؎ؽؚٝءىُٖ٦ة  朐䡾鮦ךقحسًؕٓד乆䕦׃׋麊鯄⚥ך 107
    歗⫷  ،ؙءّٝأذ،ؚٔٝ錬䏝ծأٗحزٕ  㜠ꂹ<鏣㹀鏣㹀ך⢽>
    ⚥䗰简ח鵚בֻֿהדفٓأ חزٓحַؙ׵㢩׸׷ֿהח״׶㣐ֹזلشٕذ؍ָ 铬ׁ׸׷⢽ִלأذ،ؚٔٝلشٕذ؍׾鷄⸇ׅ׷ֿ הזו鑫稢ך鏣㹀׮〳腉
17. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE AWS Cloud AWS DeepRacer NAT gateway VPC AWS DeepRacer      
     "84
    %FFQ3BDFS ءىُٖ٦ءّٝ،٦ؗذؙثٍ
18. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE AWS DeepRacer 
    
19. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE 遤⹛瑞꟦ך鏣㹀 • أؾ٦سה أذ،ؚٔٝ ך穈さׇד㹀纏 • 稢ְַ锃侭׾遤 ֲ׋׭ח磛䏝׾ 鏣㹀〳腉
20. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE 㜠ꂹꟼ侧ך㹋鄲
21. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE 
     (waypoints)      
      
22. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE
23. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE 
      
      • ,6(! #4 
    $  )* ,6(! • 0. 2"  /  • 0.&3)* + • 0.  %51' •   7-
24. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer
    
25. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE AWS DeepRacer 
    CAR:18th scale 4WD with monster truck chassis CPU: Intel Atom Processor MEMORY: 4 GB RAM STORAGE: 32 GB (expandable) WI-FI: 802.11ac CAMERA: 4 MP camera with MJPEG DRIVE BATTERY: 1000 mAh lithium polymer COMPUTE BATTERY: 13600 mAh USB-C SENSORS: Integrated accelerometer and gyroscope PORTS: 4x USB-A, 1x USB-C, 1x Micro-USB, 1x HDMI SOFTWARE: Ubuntu OS 16.04.3 LTS, Intel OpenVINO toolkit, ROS Kinetic
26. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE Stored file ROS nodes Video M-JPEG Web
     ! %     
      "84
    %FFQ3BDFS ،٦ؗذؙثٍ   % $"   $"  
    
    # # Web
     publisher &
    
    &
    
27. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    DeepRacer   
28. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE "NB[PO
    4BHF.BLFS
    3FJOGPSDFNFOU
    -FBSOJOH • ؜٦يװٗنحزךءىُٖ٦ءّٝ橆㞮ה窟さ׃׋ ♳ך䓼⻉㷕统 • 䓼⻉㷕统خ٦ٕؗحز ה׃ג - ה ׾؟ه٦ز • AWS RoboMaker זוךءىُٖ٦ة׾ ؎ٝة٦ؿؑ٦أ穗 歋דⵃ欽〳腉ⴓ侔㷕统הءىُٖ٦ءّٝך⚛⴨⻉ָ〳腉 Redis 
     ,    
      Container for Agent Container for Agent Container for environment Container for environment OpenAI gym, simulator… 
     AWS RoboMaker  Coach, RLLib
29. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Amazon Confidential and Trademark Amazon SageMaker Reinforcement Learning Robotics Industrial Control HVAC Autonomous Vehicles Operators Finance Games NLP End-to-end examples for classic RL and real-world RL application RL Environment to model real-world problems Amazon Sumerian Amazon RoboMaker AWS Simulation Environment DQN PPO RL-Coach EnergyPlus RoboSchool Open Source Environment PyBullet … Bring Your Own MATLAB& Simulink Commercial Simulators Custom Environments Open AI Gym RL-Ray RLLib HER Rainbow … APEX ES IMPALA A3C … Open AI Baselines TRPO GAIL … … TensorFlow SageMaker Deep Learning Frameworks Single Machine / Distributed Training Options Local / Remote simulation CPU/GPU Hardware MXNet PyTorch Chainer RL Toolkits that provide RL agent algorithm implementations SageMaker supported Customer BYO
30. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE https://github.com/aws-samples/aws-deepracer- workshops/blob/master/Workshops/2019- AWSSummits-AWSDeepRacerService/Lab1/Readme- Japanese.md http://bit.ly/dr-ja AWS DeepRacer Workshop Labs AWS DeepRacer !
31. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE 4JNVMBUJPO
    3VO
    -PH
    "OBMZTJT
    BOE
    7JTVBMJ[BUJPO
    GPS
    "84
    %FFQ3BDFS Reward distribution for all actions Path taken in a particular Iteration Historgram for action distribution https://github.com/aws-samples/aws-deepracer- workshops/blob/master/log- analysis/DeepRacer%20Log%20Analysis.ipynb
32. h
    
    "NB[PO
    8FC
    4FSWJDFT
    *OD
    PS
    JUT
    BGGJMJBUFT
    "MM
    SJHIUT
    SFTFSWFE https://ml-loft.connpass.com/ ML@Loft
33. Thank you! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.