AWS DeepRacer

AWS DeepRacer

Chainer Meetup #10 【強化学習】にて
2019/08/31(Sat) 13:30〜

https://chainer.connpass.com/

4e10f0bc53073da7f55420c1a2021c8b?s=128

Shoko Utsunomiya

August 31, 2019
Tweet

Transcript

  1. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. AWS DeepRacer Amazon Web Services Japan K.K. Machine Learning Solutions Architect Shoko Utsunomiya $ I B J O F S  . F F U V Q        3 F J O G P S D F N F O U  - F B S O J O H
  2. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE    • eS • eS

    • W M • O • a • AE • •
  3. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved.
  4. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE AWS DeepRacer     

    
  5. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE AWS DeepRacer 1 / 8

  6. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE %FFQ3BDFS ׾饥׵ׇ׷׋׭ח • . • ) (

  7. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 
  8. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE "84%FFQ3BDFS חֶֽ׷䓼⻉㷕统ך銲稆     

    
  9. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE ءىُٖ٦ةד㷕统   (reward fn)

  10. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE S G = 2   

    
  11. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE إٝة٦ٓ؎ٝ׾饥׷״ֲח؎ٝإٝذ؍ـ׾♷ִ׷ 0.1 0.1 0.1 0.1 0.1 0.1

    0.1 0.1 S 2 2 2 2 2 2 G = 2 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9 S 10.4 9.4 8.2 6.9 5.4 3.8 G = 2 8.6 9.5 8.5 7.5 6.3 5.0 3.5 1.9 
  12. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE "84%FFQ3BDFS ךصُ٦ٕٓطحزٙ٦ؙ圓鸡  –  () 

    – 
  13. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE   (BEHAVIORAL CLONING) • • 

     ( ( %FFQ3BDFS חֶֽ׷䓼⻉㷕统 WT׉׸⟃㢩ך،ٕ؞ٔؤي  • • )   ( (
  14. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE AWS Summit Tokyo 2019 DeepRacer League

  15. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved.
  16. © 2018, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark 䓼⻉㷕统ך圓䧭 ~ Amazon SageMaker, RoboMaker ׾⢪׏׋ DeepRacer ~  湡涸ꟼ侧鰵ָٖ٦ٝ׾כ׫⳿ׁ׆؞٦ٕׅ׷  橆㞮"843PCP.BLFS דمأزׁ׸גְ׷%س ٓ؎ؽؚٝءىُٖ٦ة  朐䡾鮦ךقحسًؕٓד乆䕦׃׋麊鯄⚥ך 107 歗⫷  ،ؙءّٝأذ،ؚٔٝ錬䏝ծأٗحزٕ  㜠ꂹ<鏣㹀鏣㹀ך⢽>⚥䗰简ח鵚בֻֿהדفٓأ חزٓحַؙ׵㢩׸׷ֿהח״׶㣐ֹזلشٕذ؍ָ 铬ׁ׸׷⢽ִלأذ،ؚٔٝلشٕذ؍׾鷄⸇ׅ׷ֿ הזו鑫稢ך鏣㹀׮〳腉
  17. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE AWS Cloud AWS DeepRacer NAT gateway VPC

    AWS DeepRacer         "84%FFQ3BDFS ءىُٖ٦ءّٝ،٦ؗذؙثٍ
  18. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE AWS DeepRacer 

  19. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE 遤⹛瑞꟦ך鏣㹀 • أؾ٦سה أذ،ؚٔٝ ך穈さׇד㹀纏 • 稢ְַ锃侭׾遤

    ֲ׋׭ח磛䏝׾ 鏣㹀〳腉
  20. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE 㜠ꂹꟼ侧ך㹋鄲

  21. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE  (waypoints)     

        
  22. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE

  23. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE       •

    ,6(! #4 $  )* ,6(! • 0. 2"  /  • 0.&3)* + • 0.  %51' •   7-
  24. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved. AWS DeepRacer 
  25. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE AWS DeepRacer  CAR:18th scale 4WD with

    monster truck chassis CPU: Intel Atom Processor MEMORY: 4 GB RAM STORAGE: 32 GB (expandable) WI-FI: 802.11ac CAMERA: 4 MP camera with MJPEG DRIVE BATTERY: 1000 mAh lithium polymer COMPUTE BATTERY: 13600 mAh USB-C SENSORS: Integrated accelerometer and gyroscope PORTS: 4x USB-A, 1x USB-C, 1x Micro-USB, 1x HDMI SOFTWARE: Ubuntu OS 16.04.3 LTS, Intel OpenVINO toolkit, ROS Kinetic
  26. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE Stored file ROS nodes Video M-JPEG Web

    ! %        "84%FFQ3BDFS ،٦ؗذؙثٍ   % $"   $"   # # Web publisher & &
  27. S U M M I T © 2019, Amazon Web

    Services, Inc. or its affiliates. All rights reserved.  DeepRacer   
  28. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE "NB[PO4BHF.BLFS3FJOGPSDFNFOU-FBSOJOH • ؜٦يװٗنحزךءىُٖ٦ءّٝ橆㞮ה窟さ׃׋ ♳ך䓼⻉㷕统 • 䓼⻉㷕统خ٦ٕؗحز ה׃ג

    - ה ׾؟ه٦ز • AWS RoboMaker זוךءىُٖ٦ة׾ ؎ٝة٦ؿؑ٦أ穗 歋דⵃ欽〳腉ⴓ侔㷕统הءىُٖ٦ءّٝך⚛⴨⻉ָ〳腉 Redis  ,      Container for Agent Container for Agent Container for environment Container for environment OpenAI gym, simulator…   AWS RoboMaker  Coach, RLLib
  29. © 2018, Amazon Web Services, Inc. or its Affiliates. All

    rights reserved. Amazon Confidential and Trademark Amazon SageMaker Reinforcement Learning Robotics Industrial Control HVAC Autonomous Vehicles Operators Finance Games NLP End-to-end examples for classic RL and real-world RL application RL Environment to model real-world problems Amazon Sumerian Amazon RoboMaker AWS Simulation Environment DQN PPO RL-Coach EnergyPlus RoboSchool Open Source Environment PyBullet … Bring Your Own MATLAB& Simulink Commercial Simulators Custom Environments Open AI Gym RL-Ray RLLib HER Rainbow … APEX ES IMPALA A3C … Open AI Baselines TRPO GAIL … … TensorFlow SageMaker Deep Learning Frameworks Single Machine / Distributed Training Options Local / Remote simulation CPU/GPU Hardware MXNet PyTorch Chainer RL Toolkits that provide RL agent algorithm implementations SageMaker supported Customer BYO
  30. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE https://github.com/aws-samples/aws-deepracer- workshops/blob/master/Workshops/2019- AWSSummits-AWSDeepRacerService/Lab1/Readme- Japanese.md http://bit.ly/dr-ja AWS DeepRacer

    Workshop Labs AWS DeepRacer !
  31. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE 4JNVMBUJPO3VO-PH"OBMZTJTBOE7JTVBMJ[BUJPOGPS"84%FFQ3BDFS Reward distribution for all actions Path

    taken in a particular Iteration Historgram for action distribution https://github.com/aws-samples/aws-deepracer- workshops/blob/master/log- analysis/DeepRacer%20Log%20Analysis.ipynb
  32. h "NB[PO8FC4FSWJDFT *ODPSJUTBGGJMJBUFT"MMSJHIUTSFTFSWFE https://ml-loft.connpass.com/ ML@Loft

  33. Thank you! S U M M I T © 2019,

    Amazon Web Services, Inc. or its affiliates. All rights reserved.