mxnet-on-aws-batch

JAWS-UG HPC * AI @ 2017.03.031

Transcript

1. MXNet on AWS Batch JAWS-UG: HPC #9 & AI #5 ߹ಉษڧձ @ 2017.03.31
2. @pottava SUPINF Inc.
3. • AWS Batch ͷಛ௃ • MXNet on AWS Batch σϞ • ߏ੒ɺϙΠϯτ • ϋϯζΦϯͷ͝঺հ ͓୊
4. AWS Batch ͷಛ௃
5. Պֶٕज़ܭࢉɾϋΠύϑΥʔϚϯείϯϐϡʔςΟϯά ༻్ͰਅՁΛൃش͢Δɺେن໛ͳεέʔϧɺδϣϒͷґ ଘఆ͕ٛՄೳͳϚωʔδυฒྻ෼ࢄॲཧج൫ɻ AWS Batch
6. ͢Ͱʹ Black Belt ͷࢿྉ͕ެ։͞Ε͍ͯ·͢ɻ AWS Batch http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws-batch.html
7. ࢲ΋Ϣʔβࢹ఺Ͱݱঢ়Λ·ͱΊ·ͨ͠ɻ AWS Batch http://qiita.com/pottava/items/d9886b2e8835c5c0d30f
8. MXNet on AWS Batch
9. σϞ
10. σΟʔϓϥʔχϯάΛར༻ʢLSTM with MXNetʣ AWS നॻɺPDF 27 ϑΝΠϧΛֶशσʔλʹར༻ ग़ͩ͠ͷ਺୯ޠ͔ΒɺͦΕΒ͍͠ޙଓͷจষΛࣗಈੜ੒ AWS ϗϫΠτϖʔύʔ͘Μ
11. 1. ·ͣҰ౓ʢֶशෆे෼ͳʣਪ࿦͕ಈ͘͜ͱΛ֬ೝ 2. ͦͷޙվΊͯɺσʔλͷऔಘɾՃ޻ 3. ֶश 4. ৽൛ਪ࿦αʔϏεσϓϩΠ σϞͷྲྀΕ
12. ʮAmazon EC2 isʯΛ༩͑ͨ࣌ͷɺֶशෆे෼ͳਪ࿦ྫ ਪ࿦ͷ༷ࢠ ৽͍͠ݴޠ͕ੜ·Ε·ͨ͠ɻ
13. ʮAmazon EC2 isʯͰɺͦΕͳΓʹֶशͨ͠ޙͷਪ࿦ྫ ਪ࿦ͷ༷ࢠ ݴ͍͍ͨ͜ͱ͸Θ͔Γ·ͤΜ͕ɺ୯ޠ΍จ๏͸ਵ෼੒௕͠·ͨ͠ɻ
14. γεςϜߏ੒
15. γεςϜߏ੒ AWS Batch S3 2.ֶशδϣϒ౤ೖ ΤϯδχΞ SpotFleet DeepLearning AMI v2 1.ֶशσʔλ౤ೖ 3.δϣϒεέδϡʔϧ 4.σʔλऔಘ & ֶश 5.݁ՌϞσϧΛอଘ ҰൠϢʔβ AWS Lambda 7.Ϟσϧऔಘ & ਪ࿦ APIGateway 6.ਪ࿦ϦΫΤετ 8.݁ՌԠ౴ ECS EC2 g2.2xlarge EC2 g2.2xlarge
16. ϙΠϯτ
17. GPU ར༻ AMI Λ࢖͏ • Unmanaged ؀ڥͰ΋ SpotFleet Ͱ҆͘ʂ • CloudFormation Ͱ؀ڥͷల։Λ༰қʹ
18. NVIDIA-docker, awslogs • NVIDIA-docker ͸ඞਢͰ͸ͳ͍΋ͷͷೖΕΔͱศར • Unmanaged ؀ڥͰ΋ log ͸ CloudWatch Logs ʹ
19. privileged ϞʔυͰ job Λఆٛ
20. git ϦϙδτϦ
21. git clone on your machine! https://github.com/pottava/mxnet-char-lstm
22. ϋϯζΦϯ΋͋Γ·͢
23. COBOL on AWS Batch http://qiita.com/pottava/items/435c65b1fa72cb643f6e
24. JAWS-UG AI ࢧ෦
25. ίϯςϯπ • AWS Ͱ AI αʔϏεΛ࣮૷ɾӡ༻͢ΔͨΊͷ ɹҰൠతͳٕज़৘ใɺ஌ݟɺࣄྫڞ༗ͷ৔ • ͢Ͱʹ׆༻͍ͯ͠Δํ • ಋೖΛݕ౼͍ͯ͠Δํ • ԿͦΕ͓͍͍͠ͷʁͳํʢ։࠵͝ͱʹ೉қ౓͕ଟগҧ͍·͢ʣ
26. ӡӦϝϯόʔ
27. ࢀߟจݙ ࢀߟจݙ: • AWS Batch – ؆୯ʹ࢖͑ͯޮ཰తͳόονίϯϐϡʔςΟϯάػೳ – AWS https://aws.amazon.com/jp/batch/ • AWS Black Belt Online SeminarʮAWS Batchʯͷࢿྉ͓ΑͼQAެ։ http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws- batch.html#QCPzBdn.twitter_tweet_count_m • re:Invent 2016: AWS Big Data & Machine Learning Sessionsɻ https://aws.amazon.com/blogs/big-data/reinvent-2016-aws-big-data- machine-learning-sessions/