Slide 1

Slide 1 text

DeepLearningͷຊ൪؀ڥʹ SageMakerΛར༻ͯ͠Δ࿩ ota42y (FiNC Technologies) JAWS DAYS 2019

Slide 2

Slide 2 text

ࠓ೔ͷ಺༰ • TensorFlowͷϞσϧͷຊ൪ӡ༻ͷ࿩ • SageMakerʹࡌͤସ͑ΔͱָͩΑ • एׯϋϚͬͨͷͰͦͷղܾํ๏

Slide 3

Slide 3 text

ࣗݾ঺հ • @ota42y • αʔόαΠυΤϯδχΞ • RubyͷਓͰ͕͢ࠓ೔͸΄΅PythonͷΈ • Amazon Web ServicesαʔόʔϨεϨγϐͬͯຊΛॻ͖·ͨ͠ (AWS Serverless Application Modelʹ͍ͭͯͷ࿩)

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

DeepLearningɺͯ͠·͔͢

Slide 6

Slide 6 text

FiNCΞϓϦͰDeepLearningͯ͠·͢

Slide 7

Slide 7 text

৯ࣄͷه࿥ • ೔ʑͷ৯ࣄΛอଘ͢ΔػೳΛఏڙ • ඼໨Λొ࿥͢Δͱͦͷ೔ͷΧϩϦʔ΍PFCόϥϯε͕Θ͔Δ

Slide 8

Slide 8 text

৯ࣄͷࣸਅղੳ • ৯ࣄͷࣸਅ͔Β඼໨ΛࣗಈͰೝࣝ͢ΔػೳΛఏڙ • ඼໨Λొ࿥ͤͣͱ΋؆୯ʹͦͷ৯ࣄͷӫཆ͕Θ͔Δ

Slide 9

Slide 9 text

ࣸਅͷೝࣝ͸Faster R-CNN • Կ͕ࣸͬͯΔ͔൑ఆ͢ΔλεΫ(Object Detection) • TensorFlowͰֶशͤͨ͞Faster R-CNNϞσϧ

Slide 10

Slide 10 text

RubyͷձࣾͳͷͰϞσϧͷӡ༻͕೉͍͠ • FiNCͷαʔό͸΄ͱΜͲ͕Ruby on Rails • ৯ࣄͷࣸਅΛอଘ͢Δαʔό΋Ruby on RailsͰͰ͖͍ͯΔ • ͦͷ··Ͱ͸TensorFlowͷϞσϧ͸࢖͑ͳ͍

Slide 11

Slide 11 text

RubyͱTensorFlowΛͲ͏૊Έ߹ΘͤΔ͔ • Ruby͔ΒPyCallͰPythonΛݺͿ • GPUαʔόʹPythonͱRubyͷ྆ํͷ؀ڥΛ΋ͨͤΔͷେม • PythonͷTensorFlowΛॲཧ͢ΔΞϓϦαʔόΛཱͯΔ • ϝϯς͕େม • TensorFlow ServingΛ࢖͏ • ͜Ε͕ྑͦ͞͏

Slide 12

Slide 12 text

TensorFlow ServingͰAPI࿈ܞ • TensorFlowͷϞσϧΛӡ༻͢ΔެࣜαʔόΞϓϦ • ෳ਺Ϟσϧͷ؅ཧɾόʔδϣχϯάՄೳ • ֤Ϟσϧʹରͯ͠REST/gRPCͷAPI͕ఏڙ͞ΕΔ • αʔόΞϓϦέʔγϣϯͷίʔυΛॻ͔ͳͯ͘ྑ͍ • Docker image΋͋ΔͷͰ؆୯ʹཱͯΒΕΔ

Slide 13

Slide 13 text

ӡ༻͸ΘΓͱେม… • ϞσϧΛӡ༻͢ΔαʔόΛॻ͔ͳ͍͍ͯ͘ͷ͸ศར • ΞϓϦαʔόΛ࡞Δ͜ͱʹूதͰ͖Δ • GPUαʔόͷ؅ཧɺεέʔϧ͸ࣗ෼ͨͪͰ΍Δඞཁ͕͋Δ • ϞσϧϑΝΠϧͷߋ৽ϑϩʔߏங • DockerԽͯ͠ΔͱΑΓ΍΍͘͜͠… • gRPCͱALBͷ૬ੑ͕ѱͯ͘…

Slide 14

Slide 14 text

ͦ͜Ͱ AWS SageMaker ࠓ5෼͙Β͍ͳΒ΋͏গ͠ৄ͘͠࿩͍͖ͯ͠·͢

Slide 15

Slide 15 text

AWS SageMakerͱ͸ • ϑϧϚωʔδυͳػցֶशαʔϏε • TensorFlow౳ͷ؀ڥ͕ߏங͞ΕͨJupyter Notebook • ֶश༻GPUαʔό΍αʔόΛ؅ཧ͢ΔSDK(෼ࢄֶश΋ରԠ) • ֶशͨ͠ϞσϧΛӡ༻͢Δproductionαʔό

Slide 16

Slide 16 text

SageMakerͩͱ Ϟσϧͷ։ൃɾֶशɾӡ༻ શ෦Ͱ͖Δ

Slide 17

Slide 17 text

طଘϞσϧͷӡ༻΋Ͱ͖Δ • ΋ͪΖΜ։ൃɾֶशɾӡ༻ͷҰ෦͚ͩ࢖͏͜ͱ΋Ͱ͖Δ • ࠓճ͸ࣗࣾGPUαʔόͰ࡞ͬͨTensorFlowͷϞσϧΛɹɹɹɹ SageMakerͰӡ༻ • ӡ༻͚ͩར༻Ͱ΋σϓϩΠ͕؆୯ʹͳΓɺ
 GPUαʔόͷ؅ཧ͔Βղ์͞ΕΔ

Slide 18

Slide 18 text

جຊతͳػೳ΋੔ͬͯΔ • Φʔτεέʔϧػೳ΋ఏڙͯ͠Δ • ৽͍͠Ϟσϧ͸ΠϛϡʔλϒϧσϓϩΠϝϯτ͞ΕΔ • ಛʹઃఆͱ͔͸ෆཁ • deploy͢Δ͚ͩͰ৽͍͠ͷ্ཱ͕͕͔ͪͬͯΒަ׵ • ෳ਺ϞσϧͷॏΈ෇͖ӡ༻΋Ͱ͖Δ

Slide 19

Slide 19 text

TensorFlowͷϞσϧΛಈ͔͢ • (࣮࣭)ϑϧϚωʔδυͳTensorFlow Serving͕࢖͑Δ • ϞσϧΛ༻ҙ͢Ε͹Φʔτεέʔϧ͢ΔREST API͕ੜ͑Δ • S3ʹϞσϧϑΝΠϧΛஔ͍࣮࣭ͯ3ߦॻ͚ͩ͘ from sagemaker.tensorflow.serving import Model model = Model(model_data='s3://xxx/model.tar.gz', role=……) model.deploy(initial_instance_count=1, instance_type=‘ml.c5.xlarge')

Slide 20

Slide 20 text

Ruby͔ΒݺͿ • AWSͷSDKʹJSONͰσʔλΛ౉͚ͩ͢ client = Aws::SageMakerRuntime::Client.new(access_key_id: ....) res = client.invoke_endpoint({ endpoint_name: "endpoint", body: data.to_json, content_type: ...})

Slide 21

Slide 21 text

Before After S3 ECS্ͷ Ruby on Rails ECS্ͷ TensorFlow Serving ࣸਅΛS3͔Βऔಘ ࣸਅΛ൑ผͤ͞Δ (gPRC) S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ ࣸਅΛ൑ผͤ͞Δ (JSON)

Slide 22

Slide 22 text

௒؆୯

Slide 23

Slide 23 text

Ͱ͸ͳ͔ͬͨ

Slide 24

Slide 24 text

• S3ʹอଘͨࣸ͠ਅΛαʔό͕ಡΈࠐΈɺTensorFlowʹૹΔ • SageMakerʹૹΔσʔλ͸5MB·Ͱͷ੍ݶ͕͋Δ • ը૾ͷ୹ล͕700ʙ800pxͰ5MBҎ্ʹͳͬͯࢮ๢ • ʘ(^o^)ʗ SageMakerͷ੍ݶʹҾ͔͔ͬͬͯ͠·ͬͨ

Slide 25

Slide 25 text

• ݩʑ͸ECS্ͷTensorFlow ServingʹgRPCͰૹͬͯͨ • ըૉͷ഑ྻ͸JSONʹ͢Δͱ༰ྔ͕๲ΕΔ • (n*n) pxͷRGB஋͔ΒͳΔ഑ྻΛJSONʹ͢Δͱ໿2ʙ3ഒʹ
 ؒͷεϖʔεແ͠([1,2] rubyͱ͔) ؒͷεϖʔε͋Γ([1, 2] pythonͱ͔) • ࠓ·Ͱ͸໰୊ͳ͔͚ͬͨͲJSONͩͱ…ʘ(^o^)ʗ Կ͕໰୊ʹͳ͔ͬͨ 8n2 + 2n + 1 3n2 11n2 + 2n 3n2

Slide 26

Slide 26 text

S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ ࣸਅΛ൑ผͤ͞Δ (JSON) ͕͜͜༰ྔΦʔόʔ

Slide 27

Slide 27 text

• TensorFlow Serving͕ड͚Δͷ͸σʔλͷ഑ྻ • શ෦ࠐΈͰͷΞϓϦͳͷͰ೚ҙͷॲཧΛೖΕΒΕΔ෦෼͸ແ͍ • ͱ͍͏͔ެࣜυΩϡϝϯτʹॻ͍ͯ͋Δ • ଧͭखͳ͔͠…ʘ(^o^)ʗ TensorFlow Servingతʹແཧͦ͏ 5FOTPS'MPX4FSWJOHCBTFETFSWFSUPQSPWJEFBTVQFSTFUPGUIF5FOTPS'MPX4FSWJOH 3&45"1*5IJTPQUJPOEPFTOPUSFRVJSF PSBMMPX BDVTUPNQZUIPOTDSJQU

Slide 28

Slide 28 text

ࠓ11෼͙Β͍ͳΒ΋͏গ͠ৄ͘͠࿩͍͖ͯ͠·͢

Slide 29

Slide 29 text

No content

Slide 30

Slide 30 text

SageMaker্Ͱ΋લॲཧΛ࣮ߦͰ͖Δ • TensorFlow ServingͰ͸ͳ͍σϓϩΠ΋αϙʔτͯ͠Δ
 ɹ(Python-based Endpoints) • ͦͷ৔߹͸લॲཧɾޙॲཧΛ࣮ߦͰ͖Δ

Slide 31

Slide 31 text

SageMaker্Ͱ΋લॲཧΛ࣮ߦͰ͖Δ • σϓϩΠ࣌ʹ࣮ߦ͢ΔϑΝΠϧΛؚΊΔ͚ͩ
 (class͕ҧ͏ͷͰ஫ҙ) • requirements.txtͰґଘύοέʔδ΋ೖΕΒΕΔ from sagemaker.tensorflow import TensorFlowModel model = TensorFlowModel(model_data = 's3://xxx/model.tar.gz', ɹɹɹɹɹɹɹɹɹɹɹɹɹɹ entry_point='pred.py', role=...) model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge')

Slide 32

Slide 32 text

entry_point.pyͷྫ import numpy as np def input_fn(serialized_input, content_type): """An input_fn that loads a pickled object""" if request_content_type == "application/python-pickle": deserialized_input = pickle.loads(serialized_input) return deserialized_input else: # Handle other content-types here or raise an Exception # if the content type is not supported. pass def output_fn(prediction_result, accepts): """An output_fn that dumps a pickled object as response""" if request_content_type == "application/python-pickle": return np.dumps(prediction_result) else: # Handle other content-types here or raise an Exception # if the content type is not supported. pass

Slide 33

Slide 33 text

εΫϦϓτ͸͜Μͳײ͡ import boto3 import json import numpy as np from PIL import Image import io AWS_S3_BUCKET_NAME = 'NAME' s3 = boto3.resource('s3') def input_fn(serialized_input, content_type): request_json = json.loads(serialized_input.decode()) s3_key = request_json['key'] response = s3.Object(AWS_S3_BUCKET_NAME, s3_key).get() image_bytes = response['Body'].read() img = Image.open(io.BytesIO(image_bytes)) bmp_data = np.asarray(img) return bmp_data[np.newaxis, :, :] ೖྗ͸JSONͰ
 {“key”: “/path/to/a.jpg”}
 S3͔Βը૾ΛऔΓ
 RGBʹม׵ 
 ͜͜ʹ༰ྔ੍ݶ͸ͳ͍

Slide 34

Slide 34 text

Before After S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛ൑ผͤ͞Δ (JSON) S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ S3ͷkey͚ͩૹΔ (༰ྔখ) ࣸਅΛS3͔Βऔಘ ͕͜͜໰୊ͩͬͨ

Slide 35

Slide 35 text

Before After S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛ൑ผͤ͞Δ (JSON) S3 ECS্ͷ Ruby on Rails SageMaker ࣸਅΛS3͔Βऔಘ S3ͷkey͚ͩૹΔ (༰ྔখ) ࣸਅΛS3͔Βऔಘ ௨৴ྔ͕ݮͬͯϨεϙϯε΋ૣ͘

Slide 36

Slide 36 text

༏উ

Slide 37

Slide 37 text

Ώ͘Ώ͘͸։ൃ΍ֶश΋SageMakerʹ • AIͷݚڀऀʹ։ൃɾֶश༻αʔόͷӡ༻Λߟ͑ͤͨ͘͞ͳ͍ • ṷ͸ṷ԰ɺݚڀऀʹ͸ݚڀऀʹ͔͠Ͱ͖ͳ͍͜ͱΛ • શ෦ΤϯδχΞ͕ೖΔͷ΋ίϛϡχέʔγϣϯͷແବ͕…ɹɹɹ • SageMaker͸αΫοͱͰ͖ΔͷͰɺ͜Ε͙Β͍ͳΒ͍͚ͦ͏

Slide 38

Slide 38 text

·ͱΊ • AWS SageMaker͸։ൃɾֶशɾӡ༻શ෦ग़དྷΔ • Ϟσϧͷӡ༻͚ͩར༻͢Δͷ΋͘͢͝ศར • લॲཧɾޙॲཧ͕Ͱ͖ΔͷͰ΄ͱΜͲͷ༻్Ͱ࢖͑Δ • SageMakerͰ৭ʑ͍ͨ͠ਓ͸we’re hiringͯ͠·͢(ƅ㱼ƅ)oኯʄ

Slide 39

Slide 39 text

TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc. TensorFlow Logo Copyright © 2019 The TensorFlow authors Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.