DeepRacer DeepDive

1. Naoki Jogan 1 Classmethod, Inc. AWS Business Unit Consulting Dev. 2018/12/5 %FFQ3BDFS
   %FFQ %JWF re:Growth 2018
2. ࣗݾ঺հ  • ιϦϡʔγϣϯΞʔΩςΫτ
   • ೥݄ೖࣾ
   —
   ೥ؒ4*FSͰ"1ΤϯδχΞ
   • ࠷ۙ͞Θ͍ͬͯΔαʔϏε
   —
   &,4
   
   4VNFSJBO ৓؛
   ௚رʢδϣ΢Ψϯ
   φΦΩʣ "84ࣄۀຊ෦
   ίϯαϧςΟϯά෦
3. re:Invent 2018 KeyNote 
4. ࠓ೔࿩͢͜ͱ  w%FFQ3BDFSͰԿ͕Ͱ͖Δͷ͔ɹʙ
   NJO
   w%FFQ3BDFSΛͲ͏΍ͬͯಈ͔͢ͷ͔ɹʙ
   NJO
   w%FFQ3BDFS͸Ͳ͏΍ֶͬͯΜͰ͍Δͷ͔ɹʙ
   NJO
5. %FFQ3BDFSͰԿ͕Ͱ͖Δͷʁ 5
6. %FFQ3BDFSͰԿ͕Ͱ͖Δͷʁ  ڧԽֶशΛֶ΂Δ
7. %FFQ3BDFS஀ੜͷഎܠ  How can we put Reinforcement Learning in the hands of all developers?
8. %FFQ3BDFS͸ڧԽֶशΛֶͿͨΊͷखஈ  wڧԽֶशΛ։ൃऀͷखʹ౉͢ ΤΩαΠςΟϯάͳखஈ
   wڧԽֶशͷϋϯζΦϯΛఏڙ
9.  ڧԽֶशͱ͸ʁ
10. ػցֶशͷछྨ  UCL Course on RL Lecture 1: ntroduction to Reinforcement Learning wڭࢣ͋Γֶश
    wڭࢣͳֶ͠श
    wڧԽֶशʢ%FFQ3BDFSʣ
11. ࣗಈӡస 
12. ғޟকع 
13.  ڧԽֶशͬͯͲ͏΍Δͷʁ
14. ڧԽֶशʢྫʣ  ใु RLΞϧΰϦζϜ wΤαΛͨ͘͞Μ৯΂ͯ
    wఢ͕͍ͳ͍ͱ͜ΖʹਐΜͰ wγϛϡϨʔγϣϯ
    wใुΛ࠷େԽ
15. %FFQ3BDFSΛಈ͔ͯ͠ΈΔ 15
16. %FFQ3BDFSΛಈ͔͢·Ͱ  wεςοϓ̍ɿ3FXBSEؔ਺࡞੒
    wεςοϓ̎ɿγϛϡϨʔγϣϯ
    wεςοϓ̏ɿ෺ཧͰ૸ΒͤΔ
17. %FFQ3BDFSΛಈ͔͢·Ͱ  wεςοϓ̍ɿ3FXBSEؔ਺࡞੒
    wεςοϓ̎ɿγϛϡϨʔγϣϯ
    wεςοϓ̏ɿ෺ཧͰ૸ΒͤΔ
18. εςοϓ̍ɿ3FXBSEؔ਺  ใुؔ਺ʢPythonʣ ɾEJTUBODF@GSPN@DFOUFS
    ɹ
    ηϯλʔϥΠϯ͔Βͷڑ཭
    ɾPO@USBDL
    ɹ
    
    ं྆ͷલ෦͕നઢͷ֎ଆʹ͋Δ͔Ͳ͏͔
    ɾUISPUUMF
    ɹ
    
    ंͷ଎౓ɹ͸ఀࢭΛࣔ͠ɺ͸࠷ߴ଎౓
    ɾUSBDL@XJEUI
    ɹ
    
    τϥοΫ෯
    ɹ
    ɹͳͲͳͲ Πϯϓοτ Ξ΢τϓοτ
19. εςοϓ̍ɿ3FXBSEؔ਺ 
    
    
    
    NBSLFS@
    
    
    
    USBDL@XJEUI
    
    
    
    
    NBSLFS@
    
    
    
    USBDL@XJEUI
    
    
    
    
    NBSLFS@
    
    
    
    USBDL@XJEUI
    
    
    
    
    SFXBSE
    
    F
    
    
    
    
    JG
    EJTUBODF@GSPN@DFOUFS
    
    
    BOE
    EJTUBODF@GSPN@DFOUFS
    
    NBSLFS@
    
    
    
    
    
    
    
    
    SFXBSE
    
    
    
    
    
    
    FMJG
    EJTUBODF@GSPN@DFOUFS
    
    NBSLFS@
    
    
    
    
    
    
    
    
    SFXBSE
    
    
    
    
    
    
    FMJG
    EJTUBODF@GSPN@DFOUFS
    
    NBSLFS@
    
    
    
    
    
    
    
    
    SFXBSE
    
    
    
    
    
    
    FMTF
    
    
    
    
    
    
    
    
    SFXBSE
    
    F
    
    
    MJLFMZ
    DSBTIFE
    DMPTF
    UP
    PGG
    USBDL
20. %FFQ3BDFSΛಈ͔͢·Ͱ  wεςοϓ̍ɿ3FXBSEؔ਺࡞੒
    wεςοϓ̎ɿγϛϡϨʔγϣϯ
    wεςοϓ̏ɿ෺ཧͰ૸ΒͤΔ
21. εςοϓ̎ɿϞσϧ࡞੒ 
22. εςοϓ̎ɿϞσϧ࡞੒ 
23. %FFQ3BDFSΛಈ͔͢·Ͱ  wεςοϓ̍ɿ3FXBSEؔ਺࡞੒
    wεςοϓ̎ɿγϛϡϨʔγϣϯ
    wεςοϓ̏ɿ෺ཧͰ૸ΒͤΔ
24. εςοϓ̏ɿ෺ཧͰ૸ΒͤΔ  ֶशϞσϧ
25. εςοϓ̏ɿ෺ཧͰ૸ΒͤΔ 
26. "84্ͷΞʔΩςΫνϟ  SageMaker RoboMaker S3 kinesis video streams CloudWatch Logs Client
27. %FFQ3BDFS͸Ͳ͏΍ֶͬͯΜͰ͍Δͷ͔ 27
28. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  https://www.slideshare.net/AmazonWebServices/robocar-rally-2018-aim206r20-aws-reinvent-2018 ɾ45"5&ɿঢ়ଶ
    ɹ
    
    %FFQ3BDFSͷϑϩϯτΧϝϥ
    ɾ3&8"3%ɿใु
    ɹ
    
    ηϯλʔϥΠϯʹ͚ۙΕ͹(PPE
    ɹ
    
    εϐʔυ͕ग़͍ͯΕ͹(PPE
    ɾ"$5*0/ɿߦಈ
    ɹ
    
    εϐʔυΛ্͛Δ
    ɹ
    
    ӈʹۂ͕Δɺࠨʹۂ͕Δ
29. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ t=0 S0
30. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ t=0 S0
31. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  t=1 S "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ 1
32. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  t=1 S "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ R :ಓ֎ΕͨͷͰใुͳ͠ 1
33. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ t=0 S0
34. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ t=0 S0
35. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  t=1 S "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ 1
36. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  t=1 S "$5*0/
    ɾ
    εϐʔυΛ্͛Δ
    ɾ
    ௚ਐ͢Δ
    ɾ
    ӈʹۂ͕Δ
    ɾ
    ࠨʹۂ͕Δ R :ঢ়گʹԠͨ͡ใु 1
37. τϨʔχϯά͸ͲͷΑ͏ʹͯ͠ߦΘΕΔ͔ʁ  https://www.slideshare.net/AmazonWebServices/robocar-rally-2018-aim206r20-aws-reinvent-2018
38. ·ͱΊ 38
39. ·ͱΊ  w%FFQ3BDFS͸ڧԽֶशΛֶͿͨΊͷखஈ
    w%FFQ3BDFS͸ڧԽֶशͱ͔ؔ܎ͳ͘୯७ʹָ͍͠
    wϨʔεେձ΋͋ΔͷͰΈΜͳࢀՃ͠Α͏ʂʂΞϝϦΧͰʁ
40. 40 ԶͰڧԽֶशΛ ֶΜͰ͘Εʂʂ ࠓͳΒˈ249
41.  Let's start reinforcement learning with DeepRacer