Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Amazon SageMaker Deep Dive

Amazon SageMaker Deep Dive

5c772b62f1974e9da3a88fbb4ef02696?s=128

Yoshitaka Haribara

June 13, 2019
Tweet

Transcript

  1. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T T O KYO
  2. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker Deep Dive Yoshitaka Haribara Solutions Architect Amazon Web Services Japan K.K. A 2 - 0 8
  3. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T 3bdU • P> f\ (  ) • Startup Solutions Architect @ AWS •  $ x <e;_ [X1F O? • T # Amazon SageMaker • ca • 2018034 `L (GC8H;) • !(*+" - (EKM5I:];) QYZ  R6S;VJN9 EK< AW'(,). D2=^7 (./=/ ) @B %.&
  4. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T    • ' • Amazon SageMaker "+!) & %(  •  • *$ "+!) #
  5. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Agenda • Amazon SageMaker  •      •  Deep Dive
  6. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.      Our mission at AWS
  7. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Customer-focused 90%=- ML (" JG D  Multi-framework 9:'#) ! Pace of innovation H+200=- ML 25 /79:8BF> Breadth and depth IA AI/ML  (%*C0 Security and analytics KE$&QL4 28BS;.P?N Embedded R&D OJ,@  ( state-of-the-art 3< AWS 8R6M 1 (
  8. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T AWS     ( ) SyntheticGestalt
  9. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T AI ML AMAZON SAGEMAKER A M A Z O N E C 2 C 5 I n s t a n c e s A M A Z O N E C 2 P 3 I n s t a n c e s F P G A s Frameworks AWS & A m a z o n R e k o g n i t i o n I m a g e / V i d e o A m a z o n P o l l y A m a z o n T r a n s c r i b e A m a z o n T r a n s l a t e A m a z o n C o m p r e h e n d A m a z o n L e x Chatbots A m a z o n F o r e c a s t Forecasting A m a z o n T e x t r a c t A m a z o n P e r s o n a l i z e Recommendations Vision Speech Language E l a s t i c I n f e r e n c e Infrastructure Interfaces
  10. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  11. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T       Amazon SageMaker
  12. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   07)=(96:(9 1;!#" Amazon SageMaker !#  # !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'# 
  13. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   07)=(96:(9 1;!#" !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon SageMaker Ground Truth !#  # Amazon SageMaker
  14. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   @J3W2OIP2O!% BS&(' &( FD C?:1 )EFD ' 2O <>=? ML %$  /U4N0 1 2 3 )EFD :1$(  Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning &( #  "( • k-means #$( • Factorization Machines (& () • DeepAR (,KR7Q) • BlazingText (Word2Vec) • XGBoost • ;T+L9.8* • Seq2Seq • LDA / Neural Topic Modelling (!%) • 56++V • AG2OH (-M / +L) Amazon SageMaker
  15. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   07)=(96:(9 1;!#" !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning !#  # Amazon SageMaker
  16. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T !#  #   07)=(96:(9 1;!#" !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning Amazon SageMaker
  17. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T !#  #   07)=(96:(9 1;!#" !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning Amazon SageMaker Neo Amazon SageMaker
  18. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T !#  #   07)=(96:(9 1;!#" !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth Amazon Elastic Inference AWS Marketplace for Machine Learning Amazon SageMaker Neo Amazon SageMaker
  19. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  20. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T D M • ( 3 ( )( ( 3 3 3 3 • 3, A K
  21. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker
  22. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker from sagemaker.mxnet import MXNet #   Estimator  estimator = MXNet("train.py", #     role=sagemaker.get_execution_role(), train_instance_count=1, train_instance_type="ml.p3.2xlarge", framework_version="1.4.0")
  23. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker from sagemaker.mxnet import MXNet #   Estimator  estimator = MXNet("train.py", #     role=sagemaker.get_execution_role(), train_instance_count=1, train_instance_type="ml.p3.2xlarge", framework_version="1.4.0") estimator.fit("s3://mybucket/data/train") # fit  
  24. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker from sagemaker.mxnet import MXNet # # $ Estimator  estimator = MXNet("train.py", #    !  role=sagemaker.get_execution_role(), train_instance_count=1, train_instance_type="ml.p3.2xlarge", framework_version="1.4.0") estimator.fit("s3://mybucket/data/train") # fit  predictor = estimator.deploy(initial_instance_count=1, instance_type="ml.m4.xlarge") # deploy   "
  25. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T    CUDA, cuDNN   train.py Deep Learning Framework
  26. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.  (Script Mode) train.py import argparse if __name__ == '__main__’: parser = argparse.ArgumentParser() # hyperparameters parser.add_argument('--epochs', type=int, default=10) # input data and model directories parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN']) parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TEST']) parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR']) args, _ = parser.parse_known_args() /opt/ml/input/data/train /opt/ml/input/data/test /opt/ml/model
  27. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.  (Script Mode) train.py import argparse if __name__ == '__main__’: parser = argparse.ArgumentParser() # hyperparameters parser.add_argument('--epochs', type=int, default=10) # input data and model directories parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN']) parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TEST']) parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR']) args, _ = parser.parse_known_args() /opt/ml/input/data/train /opt/ml/input/data/test /opt/ml/model
  28. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office. 
  29. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker  Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
  30. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker  Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office. Amazon Elastic Container Registry (ECR)
  31. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker  Jupyter Notebook/Lab Amazon S3  Amazon EC2 P3 Instances Amazon ECR The Jupyter Trademark is registered with the U.S. Patent & Trademark Office. /opt/ml/input/data/train /opt/ml/input/data/test /opt/ml/model
  32. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker   Amazon EC2 P3 Instances Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
  33. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker   Amazon EC2 P3 Instances Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
  34. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker    Amazon EC2 P3 Instances Jupyter Notebook/Lab Endpoint/ Batch transform Amazon S3 Amazon ECR The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
  35. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker   Amazon EC2 P3 Instances Endpoint/ Batch transform Amazon S3 Amazon API Gateway AWS Lambda User The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
  36. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  37. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T 0./   "#     $- !   , )*(  +%&'
  38. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T mf rip a • W c H a F a L h • - , - • mf WL l • S cWW neip • A h h h A o h • / - ,
  39. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T AWS Step Functions • JSON L A • A • CloudWatch Event Start End Train Deploy Fetch data AWS Lambda Amazon SageMaker AWS Lambda (Amazon SageMaker) Amazon CloudWatch Events (Schedule / event trigger)
  40. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T AWS Step Functions workflow    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) Amazon Elastic Container Registry (ECR) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push
  41. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T AWS Step Functions workflow    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) Amazon Elastic Container Registry (ECR) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push
  42. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T AWS Step Functions workflow    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push Amazon Elastic Container Registry (ECR)
  43. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T AWS Step Functions workflow    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) Amazon Elastic Container Registry (ECR) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push
  44. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Apache Airflow • ) 2 a DPM S CMR • 2 + ( D A • + G E Raw data Cleaned data Train data Test data Amazon SageMaker Training / HPO Model artifact Amazon SageMaker Batch transform Airflow DAG Filter long-tailed data sparse data format → RecordIO protobuf Analyze model performance based on test data Operator PythonOperator PythonOperator SageMakerTrainOperator/ SageMakerTransformOperator PythonOperator SageMakerTuningOperator Blog: https://aws.amazon.com/jp/blogs/news/build-end-to-end-machine-learning-workflows-with-amazon-sagemaker-and-apache-airflow/ Prediction results
  45. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T ( • ( 3 )  • • • ( • • I S( • ( • U • • • ( S( • S •  • R • S( • ( • S (
  46. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T :;#( • ; 84@  • 7?> ."%>0/  • +52  ' &*A1=3 -),$!%9<6
  47. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  48. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker Ground Truth • 7 o • il7 b 4 • 0 c 7 a • % ) (% n d 0 e n
  49. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T +
  50. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T 1( $ *0) %'  " - !  *0 #+ ! ,/( . "& ! !#+!  !
  51. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   )23  !1 $%   &0 #  /" ,-+  .'(*
  52. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  53. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T  () 90% 10% 
  54. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T  #!   REST API &%    "$
  55. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T  Endpoint Estimator.deploy   
  56. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T    Model aws sagemaker create-model --model-name model1 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model1.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me
  57. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T    Model Endpoint configuration aws sagemaker create-model --model-name model1 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model1.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model1-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model1”, “VariantName”: “AllTraffic”}’
  58. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T    Model Endpoint configuration Endpoint aws sagemaker create-model --model-name model1 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model1.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model1-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model1”, “VariantName”: “AllTraffic”}’ aws sagemaker create-endpoint --endpoint-name my-endpoint --endpoint-config-name model1-config
  59. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T ) / (
  60. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   Model v2 aws sagemaker create-model --model-name model2 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model2.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me
  61. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   Model v2  endpoint configuration aws sagemaker create-model --model-name model2 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model2.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model2-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model2”, “VariantName”: “AllTraffic”}’
  62. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T  Model v2 endpoint configuration  endpoint aws sagemaker create-model --model-name model2 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model2.tar.gz”} --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model2-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model2”, “VariantName”: “AllTraffic”}’ aws sagemaker update-endpoint --endpoint-name my-endpoint --endpoint-config-name model2-config
  63. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T  0( "!   1'   &/% +2 ),  $  -.#*
  64. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T        endpoint configuration aws sagemaker create-endpoint-config --endpoint-config-name both-models-config --production-variants ‘[{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 95, “ModelName”: “model1”, “VariantName”: “model1-traffic”}, {“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 5, “ModelName”: “model2”, “VariantName”: “model2-traffic”}]’
  65. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T       endpoint configuration  endpoint aws sagemaker create-endpoint-config --endpoint-config-name both-models-config --production-variants ‘[{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 95, “ModelName”: “model1”, “VariantName”: “model1-traffic”}, {“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 5, “ModelName”: “model2”, “VariantName”: “model2-traffic”}]’ aws sagemaker update-endpoint --endpoint-name my-endpoint --endpoint-config-name both-models-config
  66. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T       endpoint configuration  endpoint  aws sagemaker create-endpoint-config --endpoint-config-name both-models-config --production-variants ‘[{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 95, “ModelName”: “model1”, “VariantName”: “model1-traffic”}, {“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 5, “ModelName”: “model2”, “VariantName”: “model2-traffic”}]’ aws sagemaker update-endpoint --endpoint-name my-endpoint --endpoint-config-name both-models-config aws sagemaker update-endpoint-weights-and-capacities --endpoint-name my-endpoint --desired-weights-and-capacities ‘{“VariantName”: ”model1”, “DesiredWeight”: 5}’
  67. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T    CPU / GPU /  Amazon CloudWatch  
  68. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   SageMaker  • Min / max    • Target   • invocations per instance • Cool down time
  69. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T  
  70. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T  
  71. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T CPU  automatic scaling policy  aws application-autoscaling register-scalable-target --service-namespace sagemaker --resource-id endpoint/my-endpoint/variant/model2 --scalable-dimension sagemaker:variant:DesiredInstanceCount --min-capacity 2 --max-capacity 5
  72. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T CPU   automatic scaling policy    aws application-autoscaling register-scalable-target --service-namespace sagemaker --resource-id endpoint/my-endpoint/variant/model2 --scalable-dimension sagemaker:variant:DesiredInstanceCount --min-capacity 2 --max-capacity 5 aws application-autoscaling put-scaling-policy --policy-name model2-scaling --service-namespace sagemaker --resource-id endpoint/my-endpoint/variant/model2 --scalable-dimension sagemaker:variant:DesiredInstanceCount --policy-type TargetTrackingScaling --target-tracking-scaling-policy-configuration ‘{"TargetValue": 50, "CustomizedMetricSpecification": {"MetricName": "CPUUtilization", "Namespace": "/aws/sagemaker/Endpoints", "Dimensions": [{"Name": "EndpointName", "Value": "my-endpoint"}, {"Name": "VariantName","Value": ”model2"}], "Statistic": "Average", "Unit": "Percent”}}’
  73. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  74. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T 
  75. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T +&-  "./ 1. Batch Transform Job • %$ !'*  2. Amazon Elastic Inference • +& %$ 3. Amazon SageMaker Neo • +&     ,(#)#
  76. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T   
  77. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T    Amazon S3
  78. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T 0 50 100 150 200 1 2 4 6 8 10   
  79. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon Elastic Inference      
  80. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
  81. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker Neo      K E Y F E A T U R E S Neo-AI    (Apache license 2.0)   DL  1/10  https://github.com/neo-ai/
  82. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Amazon SageMaker Neo
  83. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Parse Model Optimize Tensors Generate Code Optimize Graph TensorFlow, MXNet, PyTorch, XGBoost ")C %05 ML " (NN) & %$ @B1?+:  8D 6E/ (-% shape  %$4* .;>!, 2 %= '#"$  =97A<3 Neo  (TVM / treelite) Pruning Operator fusion Nested loop tiling Vectorization / Tensorization Data layout transform
  84. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T WP • D , T GcrzG • FmyGo dFlszaT • , , P e kn • GcrzG ul P T • , , A B T GcrzG G • D , A A S pkGf • T T fnwbS M • w gatI ihIE K J IS MN D D ,
  85. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T Related breakouts | l k hu WML G 32 Ot dg a 32 2 C : A 5 : 5 ry e ,4D 4E 248 04 mik SW cens 2D :5. A 4 -1 ,4D 4E 248 04 ho z O c k 32 2 C : A 5 : 5 2C : 04 D4 ,4D
  86. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T ! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yoshitaka Haribara hariby@amazon.co.jp @_hariby
  87. © 2019, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. S U M M I T References • ML@Loft [Blog#1] • Amazon SageMaker   [Web#1, Blog#2, #3, #4, #5] • AWS Black Belt Online Seminar • Basic [Movie, Slides] • Advanced [Movie, Slides] • AWS : Keras [Blog], Apache Airflow [Blog], Kubeflow [Blog], • HPO: SageMaker default [Blog], Optuna [Blog] •   GPU  [Blog] • SageMaker Containers [GitHub] • Jupyter  /IDE    [SageMaker Python SDK]   API