DeepLearningの本番環境にSageMakerを利用してる話

JAWS DAYS 2019でのランチセッションの発表です

Transcript

1. DeepLearningͷຊ൪؀ڥʹ
   SageMakerΛར༻ͯ͠Δ࿩
   ota42y (FiNC Technologies)
   JAWS DAYS 2019
   

   View Slide

2. ࠓ೔ͷ಺༰
   • TensorFlowͷϞσϧͷຊ൪ӡ༻ͷ࿩
   • SageMakerʹࡌͤସ͑ΔͱָͩΑ
   • एׯϋϚͬͨͷͰͦͷղܾํ๏
   

   View Slide

3. ࣗݾ঺հ
   • @ota42y
   • αʔόαΠυΤϯδχΞ
   • RubyͷਓͰ͕͢ࠓ೔͸΄΅PythonͷΈ
   • Amazon Web ServicesαʔόʔϨεϨγϐͬͯຊΛॻ͖·ͨ͠
   (AWS Serverless Application Modelʹ͍ͭͯͷ࿩)
   

   View Slide

4. View Slide

5. DeepLearningɺͯ͠·͔͢
   

   View Slide

6. FiNCΞϓϦͰDeepLearningͯ͠·͢
   

   View Slide

7. ৯ࣄͷه࿥
   • ೔ʑͷ৯ࣄΛอଘ͢ΔػೳΛఏڙ
   • ඼໨Λొ࿥͢Δͱͦͷ೔ͷΧϩϦʔ΍PFCόϥϯε͕Θ͔Δ
   

   View Slide

8. ৯ࣄͷࣸਅղੳ
   • ৯ࣄͷࣸਅ͔Β඼໨ΛࣗಈͰೝࣝ͢ΔػೳΛఏڙ
   • ඼໨Λొ࿥ͤͣͱ΋؆୯ʹͦͷ৯ࣄͷӫཆ͕Θ͔Δ
   

   View Slide

9. ࣸਅͷೝࣝ͸Faster R-CNN
   • Կ͕ࣸͬͯΔ͔൑ఆ͢ΔλεΫ(Object Detection)
   • TensorFlowͰֶशͤͨ͞Faster R-CNNϞσϧ
   

   View Slide

10. RubyͷձࣾͳͷͰϞσϧͷӡ༻͕೉͍͠
    • FiNCͷαʔό͸΄ͱΜͲ͕Ruby on Rails
    • ৯ࣄͷࣸਅΛอଘ͢Δαʔό΋Ruby on RailsͰͰ͖͍ͯΔ
    • ͦͷ··Ͱ͸TensorFlowͷϞσϧ͸࢖͑ͳ͍
    

    View Slide

11. RubyͱTensorFlowΛͲ͏૊Έ߹ΘͤΔ͔
    • Ruby͔ΒPyCallͰPythonΛݺͿ
    • GPUαʔόʹPythonͱRubyͷ྆ํͷ؀ڥΛ΋ͨͤΔͷେม
    • PythonͷTensorFlowΛॲཧ͢ΔΞϓϦαʔόΛཱͯΔ
    • ϝϯς͕େม
    • TensorFlow ServingΛ࢖͏
    • ͜Ε͕ྑͦ͞͏
    

    View Slide

12. TensorFlow ServingͰAPI࿈ܞ
    • TensorFlowͷϞσϧΛӡ༻͢ΔެࣜαʔόΞϓϦ
    • ෳ਺Ϟσϧͷ؅ཧɾόʔδϣχϯάՄೳ
    • ֤Ϟσϧʹରͯ͠REST/gRPCͷAPI͕ఏڙ͞ΕΔ
    • αʔόΞϓϦέʔγϣϯͷίʔυΛॻ͔ͳͯ͘ྑ͍
    • Docker image΋͋ΔͷͰ؆୯ʹཱͯΒΕΔ
    

    View Slide

13. ӡ༻͸ΘΓͱେม…
    • ϞσϧΛӡ༻͢ΔαʔόΛॻ͔ͳ͍͍ͯ͘ͷ͸ศར
    • ΞϓϦαʔόΛ࡞Δ͜ͱʹूதͰ͖Δ
    • GPUαʔόͷ؅ཧɺεέʔϧ͸ࣗ෼ͨͪͰ΍Δඞཁ͕͋Δ
    • ϞσϧϑΝΠϧͷߋ৽ϑϩʔߏங
    • DockerԽͯ͠ΔͱΑΓ΍΍͘͜͠…
    • gRPCͱALBͷ૬ੑ͕ѱͯ͘…
    

    View Slide

14. ͦ͜Ͱ AWS SageMaker
    ࠓ5෼͙Β͍ͳΒ΋͏গ͠ৄ͘͠࿩͍͖ͯ͠·͢
    

    View Slide

15. AWS SageMakerͱ͸
    • ϑϧϚωʔδυͳػցֶशαʔϏε
    • TensorFlow౳ͷ؀ڥ͕ߏங͞ΕͨJupyter Notebook
    • ֶश༻GPUαʔό΍αʔόΛ؅ཧ͢ΔSDK(෼ࢄֶश΋ରԠ)
    • ֶशͨ͠ϞσϧΛӡ༻͢Δproductionαʔό
    

    View Slide

16. SageMakerͩͱ
    Ϟσϧͷ։ൃɾֶशɾӡ༻
    શ෦Ͱ͖Δ
    

    View Slide

17. طଘϞσϧͷӡ༻΋Ͱ͖Δ
    • ΋ͪΖΜ։ൃɾֶशɾӡ༻ͷҰ෦͚ͩ࢖͏͜ͱ΋Ͱ͖Δ
    • ࠓճ͸ࣗࣾGPUαʔόͰ࡞ͬͨTensorFlowͷϞσϧΛɹɹɹɹ
    SageMakerͰӡ༻
    • ӡ༻͚ͩར༻Ͱ΋σϓϩΠ͕؆୯ʹͳΓɺ

    GPUαʔόͷ؅ཧ͔Βղ์͞ΕΔ
    

    View Slide

18. جຊతͳػೳ΋੔ͬͯΔ
    • Φʔτεέʔϧػೳ΋ఏڙͯ͠Δ
    • ৽͍͠Ϟσϧ͸ΠϛϡʔλϒϧσϓϩΠϝϯτ͞ΕΔ
    • ಛʹઃఆͱ͔͸ෆཁ
    • deploy͢Δ͚ͩͰ৽͍͠ͷ্ཱ͕͕͔ͪͬͯΒަ׵
    • ෳ਺ϞσϧͷॏΈ෇͖ӡ༻΋Ͱ͖Δ
    

    View Slide

19. TensorFlowͷϞσϧΛಈ͔͢
    • (࣮࣭)ϑϧϚωʔδυͳTensorFlow Serving͕࢖͑Δ
    • ϞσϧΛ༻ҙ͢Ε͹Φʔτεέʔϧ͢ΔREST API͕ੜ͑Δ
    • S3ʹϞσϧϑΝΠϧΛஔ͍࣮࣭ͯ3ߦॻ͚ͩ͘
    from sagemaker.tensorflow.serving import Model
    model = Model(model_data='s3://xxx/model.tar.gz', role=……)
    model.deploy(initial_instance_count=1, instance_type=‘ml.c5.xlarge')
    

    View Slide

20. Ruby͔ΒݺͿ
    • AWSͷSDKʹJSONͰσʔλΛ౉͚ͩ͢
    client = Aws::SageMakerRuntime::Client.new(access_key_id: ....)
    res = client.invoke_endpoint({ endpoint_name: "endpoint",
    body: data.to_json,
    content_type: ...})
    

    View Slide

21. Before After
    S3
    ECS্ͷ
    Ruby on Rails
    ECS্ͷ
    TensorFlow Serving
    ࣸਅΛS3͔Βऔಘ
    ࣸਅΛ൑ผͤ͞Δ
    (gPRC)
    S3
    ECS্ͷ
    Ruby on Rails
    SageMaker
    ࣸਅΛS3͔Βऔಘ
    ࣸਅΛ൑ผͤ͞Δ
    (JSON)
    

    View Slide

22. ௒؆୯
    

    View Slide

23. Ͱ͸ͳ͔ͬͨ
    

    View Slide

24. • S3ʹอଘͨࣸ͠ਅΛαʔό͕ಡΈࠐΈɺTensorFlowʹૹΔ
    • SageMakerʹૹΔσʔλ͸5MB·Ͱͷ੍ݶ͕͋Δ
    • ը૾ͷ୹ล͕700ʙ800pxͰ5MBҎ্ʹͳͬͯࢮ๢
    • ʘ(^o^)ʗ
    SageMakerͷ੍ݶʹҾ͔͔ͬͬͯ͠·ͬͨ
    

    View Slide

25. • ݩʑ͸ECS্ͷTensorFlow ServingʹgRPCͰૹͬͯͨ
    • ըૉͷ഑ྻ͸JSONʹ͢Δͱ༰ྔ͕๲ΕΔ
    • (n*n) pxͷRGB஋͔ΒͳΔ഑ྻΛJSONʹ͢Δͱ໿2ʙ3ഒʹ

    ؒͷεϖʔεແ͠([1,2] rubyͱ͔) ؒͷεϖʔε͋Γ([1, 2] pythonͱ͔)
    • ࠓ·Ͱ͸໰୊ͳ͔͚ͬͨͲJSONͩͱ…ʘ(^o^)ʗ
    Կ͕໰୊ʹͳ͔ͬͨ
    8n2 + 2n + 1
    3n2
    11n2 + 2n
    3n2
    

    View Slide

26. S3
    ECS্ͷ
    Ruby on Rails
    SageMaker
    ࣸਅΛS3͔Βऔಘ
    ࣸਅΛ൑ผͤ͞Δ
    (JSON)
    ͕͜͜༰ྔΦʔόʔ
    

    View Slide

27. • TensorFlow Serving͕ड͚Δͷ͸σʔλͷ഑ྻ
    • શ෦ࠐΈͰͷΞϓϦͳͷͰ೚ҙͷॲཧΛೖΕΒΕΔ෦෼͸ແ͍
    • ͱ͍͏͔ެࣜυΩϡϝϯτʹॻ͍ͯ͋Δ
    • ଧͭखͳ͔͠…ʘ(^o^)ʗ
    TensorFlow Servingతʹແཧͦ͏
    5FOTPS'MPX4FSWJOHCBTFETFSWFSUPQSPWJEFBTVQFSTFUPGUIF5FOTPS'MPX4FSWJOH
    3&45"1*5IJTPQUJPOEPFTOPUSFRVJSF PSBMMPX
    BDVTUPNQZUIPOTDSJQU
    

    View Slide

28. ࠓ11෼͙Β͍ͳΒ΋͏গ͠ৄ͘͠࿩͍͖ͯ͠·͢
    

    View Slide

29. View Slide

30. SageMaker্Ͱ΋લॲཧΛ࣮ߦͰ͖Δ
    • TensorFlow ServingͰ͸ͳ͍σϓϩΠ΋αϙʔτͯ͠Δ

    ɹ(Python-based Endpoints)
    • ͦͷ৔߹͸લॲཧɾޙॲཧΛ࣮ߦͰ͖Δ
    

    View Slide

31. SageMaker্Ͱ΋લॲཧΛ࣮ߦͰ͖Δ
    • σϓϩΠ࣌ʹ࣮ߦ͢ΔϑΝΠϧΛؚΊΔ͚ͩ

    (class͕ҧ͏ͷͰ஫ҙ)
    • requirements.txtͰґଘύοέʔδ΋ೖΕΒΕΔ
    from sagemaker.tensorflow import TensorFlowModel
    model = TensorFlowModel(model_data = 's3://xxx/model.tar.gz',
    ɹɹɹɹɹɹɹɹɹɹɹɹɹɹ entry_point='pred.py', role=...)
    model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge')
    

    View Slide

32. entry_point.pyͷྫ
    import numpy as np
    def input_fn(serialized_input, content_type):
    """An input_fn that loads a pickled object"""
    if request_content_type == "application/python-pickle":
    deserialized_input = pickle.loads(serialized_input)
    return deserialized_input
    else:
    # Handle other content-types here or raise an Exception
    # if the content type is not supported.
    pass
    def output_fn(prediction_result, accepts):
    """An output_fn that dumps a pickled object as response"""
    if request_content_type == "application/python-pickle":
    return np.dumps(prediction_result)
    else:
    # Handle other content-types here or raise an Exception
    # if the content type is not supported.
    pass
    

    View Slide

33. εΫϦϓτ͸͜Μͳײ͡
    import boto3
    import json
    import numpy as np
    from PIL import Image
    import io
    AWS_S3_BUCKET_NAME = 'NAME'
    s3 = boto3.resource('s3')
    def input_fn(serialized_input, content_type):
    request_json = json.loads(serialized_input.decode())
    s3_key = request_json['key']
    response = s3.Object(AWS_S3_BUCKET_NAME, s3_key).get()
    image_bytes = response['Body'].read()
    img = Image.open(io.BytesIO(image_bytes))
    bmp_data = np.asarray(img)
    return bmp_data[np.newaxis, :, :]
    ೖྗ͸JSONͰ

    {“key”: “/path/to/a.jpg”}

    S3͔Βը૾ΛऔΓ

    RGBʹม׵
    

    ͜͜ʹ༰ྔ੍ݶ͸ͳ͍
    

    View Slide

34. Before After
    S3
    ECS্ͷ
    Ruby on Rails
    SageMaker
    ࣸਅΛ൑ผͤ͞Δ
    (JSON)
    S3
    ECS্ͷ
    Ruby on Rails
    SageMaker
    ࣸਅΛS3͔Βऔಘ
    S3ͷkey͚ͩૹΔ
    (༰ྔখ)
    ࣸਅΛS3͔Βऔಘ
    ͕͜͜໰୊ͩͬͨ
    

    View Slide

35. Before After
    S3
    ECS্ͷ
    Ruby on Rails
    SageMaker
    ࣸਅΛ൑ผͤ͞Δ
    (JSON)
    S3
    ECS্ͷ
    Ruby on Rails
    SageMaker
    ࣸਅΛS3͔Βऔಘ
    S3ͷkey͚ͩૹΔ
    (༰ྔখ)
    ࣸਅΛS3͔Βऔಘ
    ௨৴ྔ͕ݮͬͯϨεϙϯε΋ૣ͘
    

    View Slide

36. ༏উ
    

    View Slide

37. Ώ͘Ώ͘͸։ൃ΍ֶश΋SageMakerʹ
    • AIͷݚڀऀʹ։ൃɾֶश༻αʔόͷӡ༻Λߟ͑ͤͨ͘͞ͳ͍
    • ṷ͸ṷ԰ɺݚڀऀʹ͸ݚڀऀʹ͔͠Ͱ͖ͳ͍͜ͱΛ
    • શ෦ΤϯδχΞ͕ೖΔͷ΋ίϛϡχέʔγϣϯͷແବ͕…ɹɹɹ
    • SageMaker͸αΫοͱͰ͖ΔͷͰɺ͜Ε͙Β͍ͳΒ͍͚ͦ͏
    

    View Slide

38. ·ͱΊ
    • AWS SageMaker͸։ൃɾֶशɾӡ༻શ෦ग़དྷΔ
    • Ϟσϧͷӡ༻͚ͩར༻͢Δͷ΋͘͢͝ศར
    • લॲཧɾޙॲཧ͕Ͱ͖ΔͷͰ΄ͱΜͲͷ༻్Ͱ࢖͑Δ
    • SageMakerͰ৭ʑ͍ͨ͠ਓ͸we’re hiringͯ͠·͢(ƅ㱼ƅ)oኯʄ
    

    View Slide

39. TensorFlow, the TensorFlow logo and any related marks are trademarks of Google Inc.
    TensorFlow Logo
    Copyright © 2019 The TensorFlow authors Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the
    License at https://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
    

    View Slide