mxnet-on-aws-batch - Speaker Deck

mxnet-on-aws-batch

by ryo nakamaru

Slide 1

Slide 1 text

MXNet on AWS Batch JAWS-UG: HPC #9 & AI #5 ߹ಉษڧձ @ 2017.03.31

Slide 2

Slide 2 text

@pottava SUPINF Inc.

Slide 3

Slide 3 text

• AWS Batch ͷಛ௃ • MXNet on AWS Batch σϞ • ߏ੒ɺϙΠϯτ • ϋϯζΦϯͷ͝঺հ ͓୊

Slide 4

Slide 4 text

AWS Batch ͷಛ௃

Slide 5

Slide 5 text

Պֶٕज़ܭࢉɾϋΠύϑΥʔϚϯείϯϐϡʔςΟϯά ༻్ͰਅՁΛൃش͢Δɺେن໛ͳεέʔϧɺδϣϒͷґ ଘఆ͕ٛՄೳͳϚωʔδυฒྻ෼ࢄॲཧج൫ɻ AWS Batch

Slide 6

Slide 6 text

͢Ͱʹ Black Belt ͷࢿྉ͕ެ։͞Ε͍ͯ·͢ɻ AWS Batch http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws-batch.html

Slide 7

Slide 7 text

ࢲ΋Ϣʔβࢹ఺Ͱݱঢ়Λ·ͱΊ·ͨ͠ɻ AWS Batch http://qiita.com/pottava/items/d9886b2e8835c5c0d30f

Slide 8

Slide 8 text

MXNet on AWS Batch

Slide 9

Slide 9 text

σϞ

Slide 10

Slide 10 text

σΟʔϓϥʔχϯάΛར༻ʢLSTM with MXNetʣ AWS നॻɺPDF 27 ϑΝΠϧΛֶशσʔλʹར༻ ग़ͩ͠ͷ਺୯ޠ͔ΒɺͦΕΒ͍͠ޙଓͷจষΛࣗಈੜ੒ AWS ϗϫΠτϖʔύʔ͘Μ

Slide 11

Slide 11 text

1. ·ͣҰ౓ʢֶशෆे෼ͳʣਪ࿦͕ಈ͘͜ͱΛ֬ೝ 2. ͦͷޙվΊͯɺσʔλͷऔಘɾՃ޻ 3. ֶश 4. ৽൛ਪ࿦αʔϏεσϓϩΠ σϞͷྲྀΕ

Slide 12

Slide 12 text

ʮAmazon EC2 isʯΛ༩͑ͨ࣌ͷɺֶशෆे෼ͳਪ࿦ྫ ਪ࿦ͷ༷ࢠ ৽͍͠ݴޠ͕ੜ·Ε·ͨ͠ɻ

Slide 13

Slide 13 text

ʮAmazon EC2 isʯͰɺͦΕͳΓʹֶशͨ͠ޙͷਪ࿦ྫ ਪ࿦ͷ༷ࢠ ݴ͍͍ͨ͜ͱ͸Θ͔Γ·ͤΜ͕ɺ୯ޠ΍จ๏͸ਵ෼੒௕͠·ͨ͠ɻ

Slide 14

Slide 14 text

γεςϜߏ੒

Slide 15

Slide 15 text

γεςϜߏ੒ AWS Batch S3 2.ֶशδϣϒ౤ೖ ΤϯδχΞ SpotFleet DeepLearning AMI v2 1.ֶशσʔλ౤ೖ 3.δϣϒεέδϡʔϧ 4.σʔλऔಘ & ֶश 5.݁ՌϞσϧΛอଘ ҰൠϢʔβ AWS Lambda 7.Ϟσϧऔಘ & ਪ࿦ APIGateway 6.ਪ࿦ϦΫΤετ 8.݁ՌԠ౴ ECS EC2 g2.2xlarge EC2 g2.2xlarge

Slide 16

Slide 16 text

ϙΠϯτ

Slide 17

Slide 17 text

GPU ར༻ AMI Λ࢖͏ • Unmanaged ؀ڥͰ΋ SpotFleet Ͱ҆͘ʂ • CloudFormation Ͱ؀ڥͷల։Λ༰қʹ

Slide 18

Slide 18 text

NVIDIA-docker, awslogs • NVIDIA-docker ͸ඞਢͰ͸ͳ͍΋ͷͷೖΕΔͱศར • Unmanaged ؀ڥͰ΋ log ͸ CloudWatch Logs ʹ

Slide 19

Slide 19 text

privileged ϞʔυͰ job Λఆٛ

Slide 20

Slide 20 text

git ϦϙδτϦ

Slide 21

Slide 21 text

git clone on your machine! https://github.com/pottava/mxnet-char-lstm

Slide 22

Slide 22 text

ϋϯζΦϯ΋͋Γ·͢

Slide 23

Slide 23 text

COBOL on AWS Batch http://qiita.com/pottava/items/435c65b1fa72cb643f6e

Slide 24

Slide 24 text

JAWS-UG AI ࢧ෦

Slide 25

Slide 25 text

ίϯςϯπ • AWS Ͱ AI αʔϏεΛ࣮૷ɾӡ༻͢ΔͨΊͷ ɹҰൠతͳٕज़৘ใɺ஌ݟɺࣄྫڞ༗ͷ৔ • ͢Ͱʹ׆༻͍ͯ͠Δํ • ಋೖΛݕ౼͍ͯ͠Δํ • ԿͦΕ͓͍͍͠ͷʁͳํʢ։࠵͝ͱʹ೉қ౓͕ଟগҧ͍·͢ʣ

Slide 26

Slide 26 text

ӡӦϝϯόʔ

Slide 27

Slide 27 text

ࢀߟจݙ ࢀߟจݙ: • AWS Batch – ؆୯ʹ࢖͑ͯޮ཰తͳόονίϯϐϡʔςΟϯάػೳ – AWS https://aws.amazon.com/jp/batch/ • AWS Black Belt Online SeminarʮAWS Batchʯͷࢿྉ͓ΑͼQAެ։ http://aws.typepad.com/sajp/2017/02/aws-black-belt-online-seminar-aws- batch.html#QCPzBdn.twitter_tweet_count_m • re:Invent 2016: AWS Big Data & Machine Learning Sessionsɻ https://aws.amazon.com/blogs/big-data/reinvent-2016-aws-big-data- machine-learning-sessions/