Amazon SageMaker Deep Dive

Transcript

1. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T T O KYO
2. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker Deep Dive Yoshitaka Haribara Solutions Architect Amazon Web Services Japan K.K. A 2 - 0 8
3. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 3bdU • P> f\ (  ) • Startup Solutions Architect @ AWS •  $ x <e;_ [X1F O? • T # Amazon SageMaker • ca • 2018034 `L (GC8H;) • !(*+" - (EKM5I:];) QYZ
   R6S;VJN9 EK< AW'(,). D2=^7 (./=/ ) @B %.&
4. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T    • ' • Amazon SageMaker "+!) & 
   %(  •  • *$ "+!) #
5. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Agenda • Amazon SageMaker  •   
   
   •  Deep Dive
6. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.  
       Our mission at AWS
7. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Customer-focused 90%=- ML (" JG D  Multi-framework 9:'#) ! Pace of innovation H+200=- ML 25 /7
   9:8BF> Breadth and depth IA AI/ML  (%*C0 Security and analytics KE$&
   QL4 28BS;.P?N Embedded R&D OJ,@  ( state-of-the-art 3< AWS 8R6M 1 (
8. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS    
    ( ) SyntheticGestalt
9. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AI ML AMAZON SAGEMAKER A M A Z O N E C 2 C 5 I n s t a n c e s A M A Z O N E C 2 P 3 I n s t a n c e s F P G A s Frameworks AWS & A m a z o n R e k o g n i t i o n I m a g e / V i d e o A m a z o n P o l l y A m a z o n T r a n s c r i b e A m a z o n T r a n s l a t e A m a z o n C o m p r e h e n d A m a z o n L e x Chatbots A m a z o n F o r e c a s t Forecasting A m a z o n T e x t r a c t A m a z o n P e r s o n a l i z e Recommendations Vision Speech Language E l a s t i c I n f e r e n c e Infrastructure Interfaces
10. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
11. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
     
     
     Amazon SageMaker
12. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T   07)=(9
    6:(9 1;
    !#
    " Amazon SageMaker !#  # !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'# 
13. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T   07)=(9
    6:(9 1;
    !#
    " !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon SageMaker Ground Truth !#  # Amazon SageMaker
14. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T   @J3W2O
    IP2O!% BS
    &(
    ' &( FD C?:1 )EFD ' 2O <>=? ML %$  /U4N0 1 2 3 )EFD :1$(  Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning &( #  "( • k-means #$( • Factorization Machines (& () • DeepAR (,KR7Q) • BlazingText (Word2Vec) • XGBoost • ;T+L
    9.8* • Seq2Seq • LDA / Neural Topic Modelling (!%) • 56++V • AG2OH (-M / +L) Amazon SageMaker
15. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T   07)=(9
    6:(9 1;
    !#
    " !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning !#  # Amazon SageMaker
16. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T !#  #   07)=(9
    6:(9 1;
    !#
    " !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning Amazon SageMaker
17. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T !#  #   07)=(9
    6:(9 1;
    !#
    " !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth AWS Marketplace for Machine Learning Amazon SageMaker Neo Amazon SageMaker
18. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T !#  #   07)=(9
    6:(9 1;
    !#
    " !# 53 2/+' $453 " (9 ,.-/ ML   %<*8& 1 2 3 $453 +'#  Amazon EC2 P3 Instances Amazon SageMaker RL Amazon SageMaker Ground Truth Amazon Elastic Inference AWS Marketplace for Machine Learning Amazon SageMaker Neo Amazon SageMaker
19. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
20. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T D M • ( 3 ( )( ( 3 3 3 3 • 3, A K
21. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker
22. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker from sagemaker.mxnet import MXNet # 
    
     Estimator  estimator = MXNet("train.py", # 
       role=sagemaker.get_execution_role(), train_instance_count=1, train_instance_type="ml.p3.2xlarge", framework_version="1.4.0")
23. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker from sagemaker.mxnet import MXNet # 
    
     Estimator  estimator = MXNet("train.py", # 
        role=sagemaker.get_execution_role(), train_instance_count=1, train_instance_type="ml.p3.2xlarge", framework_version="1.4.0") estimator.fit("s3://mybucket/data/train") # fit  
    
24. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. SageMaker Python SDK import sagemaker from sagemaker.mxnet import MXNet # #
    
    $ Estimator  estimator = MXNet("train.py", # 
       !  role=sagemaker.get_execution_role(), train_instance_count=1, train_instance_type="ml.p3.2xlarge", framework_version="1.4.0") estimator.fit("s3://mybucket/data/train") # fit 
     predictor = estimator.deploy(initial_instance_count=1, instance_type="ml.m4.xlarge") # deploy   "
25. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T    CUDA, cuDNN 
      train.py Deep Learning Framework
26. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 
     (Script Mode) train.py import argparse if __name__ == '__main__’: parser = argparse.ArgumentParser() # hyperparameters parser.add_argument('--epochs', type=int, default=10) # input data and model directories parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN']) parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TEST']) parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR']) args, _ = parser.parse_known_args() /opt/ml/input/data/train /opt/ml/input/data/test /opt/ml/model
27. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. 
     (Script Mode) train.py import argparse if __name__ == '__main__’: parser = argparse.ArgumentParser() # hyperparameters parser.add_argument('--epochs', type=int, default=10) # input data and model directories parser.add_argument('--train', type=str, default=os.environ['SM_CHANNEL_TRAIN']) parser.add_argument('--test', type=str, default=os.environ['SM_CHANNEL_TEST']) parser.add_argument('--model-dir', type=str, default=os.environ['SM_MODEL_DIR']) args, _ = parser.parse_known_args() /opt/ml/input/data/train /opt/ml/input/data/test /opt/ml/model
28. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office. 
    
29. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker 
    Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
30. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker 
    Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office. Amazon Elastic Container Registry (ECR)
31. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker  Jupyter Notebook/Lab Amazon S3
     Amazon EC2 P3 Instances Amazon ECR The Jupyter Trademark is registered with the U.S. Patent & Trademark Office. /opt/ml/input/data/train /opt/ml/input/data/test /opt/ml/model
32. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker 
     Amazon EC2 P3 Instances Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
33. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker 
     Amazon EC2 P3 Instances Jupyter Notebook/Lab Amazon S3 The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
34. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker 
      Amazon EC2 P3 Instances Jupyter Notebook/Lab Endpoint/ Batch transform Amazon S3 Amazon ECR The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
35. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker
      Amazon EC2 P3 Instances Endpoint/ Batch transform Amazon S3 Amazon API Gateway AWS Lambda User The Jupyter Trademark is registered with the U.S. Patent & Trademark Office.
36. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
37. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 0./   "#     $-
    !   , )*(  +%
    &'
38. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T mf rip a • W c H a F a L h • - , - • mf WL l • S cWW neip • A h h h A o h • / - ,
39. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Step Functions • JSON L A • A • CloudWatch Event Start End Train Deploy Fetch data AWS Lambda Amazon SageMaker AWS Lambda (Amazon SageMaker) Amazon CloudWatch Events (Schedule / event trigger)
40. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Step Functions workflow   
    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) Amazon Elastic Container Registry (ECR) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push
41. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Step Functions workflow   
    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) Amazon Elastic Container Registry (ECR) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push
42. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Step Functions workflow   
    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push Amazon Elastic Container Registry (ECR)
43. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T AWS Step Functions workflow   
    Test data Train data Data Scientists/ Developers Git webhook docker push AWS Glue Amazon S3 (data) Amazon SageMaker Training Job / Batch Transform AWS CodeCommit or 3rd party Git repository Amazon S3 (raw data) Amazon Elastic Container Registry (ECR) AWS CodeBuild Endpoint https://aws.amazon.com/blogs/machine-learning/automated-and-continuous-deployment-of-amazon-sagemaker-models-with-aws-step-functions/ AWS Lambda SageMaker Endpoint deploy Amazon S3 (trained model) git push
44. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Apache Airflow • ) 2 a DPM S CMR • 2 + ( D A • + G E Raw data Cleaned data Train data Test data Amazon SageMaker Training / HPO Model artifact Amazon SageMaker Batch transform Airflow DAG Filter long-tailed data sparse data format → RecordIO protobuf Analyze model performance based on test data Operator PythonOperator PythonOperator SageMakerTrainOperator/ SageMakerTransformOperator PythonOperator SageMakerTuningOperator Blog: https://aws.amazon.com/jp/blogs/news/build-end-to-end-machine-learning-workflows-with-amazon-sagemaker-and-apache-airflow/ Prediction results
45. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T ( • ( 3 )  • • • ( • • I S( • ( • U • • • ( S( • S •
     • R • S( • ( • S (
46. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T :;#( • ; 84@  • 7?>
    ."%>0/  • +52  ' &*A1=3 -),$!%9<6
47. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
48. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker Ground Truth • 7 o • il7 b 4 • 0 c 7 a • % ) (% n d 0 e n
49. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T +
50. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 1( $ *0)
     %' 
     " -
     
    !  *0 #+ 
    ! ,/( .
     "& 
    ! 
    !#+
    !  
    !
51. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T   )2
    3  !1 $%   &0 #  /" ,-+  .'(*
52. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
53. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T  () 90% 10%
    
54. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T  #!   REST API &%  
    
    "$
55. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T
     Endpoint Estimator.deploy   
    
56. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
     Model aws sagemaker create-model --model-name model1 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model1.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me
57. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
     Model Endpoint configuration aws sagemaker create-model --model-name model1 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model1.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model1-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model1”, “VariantName”: “AllTraffic”}’
58. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
     Model Endpoint configuration Endpoint aws sagemaker create-model --model-name model1 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model1.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model1-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model1”, “VariantName”: “AllTraffic”}’ aws sagemaker create-endpoint --endpoint-name my-endpoint --endpoint-config-name model1-config
59. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T ) / (
60. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
     Model v2 aws sagemaker create-model --model-name model2 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model2.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me
61. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
     Model v2  endpoint configuration aws sagemaker create-model --model-name model2 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model2.tar.gz”}’ --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model2-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model2”, “VariantName”: “AllTraffic”}’
62. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
    Model v2 endpoint configuration  endpoint aws sagemaker create-model --model-name model2 --primary-container ‘{“Image”: “123.dkr.ecr.amazonaws.com/algo”, “ModelDataUrl”: “s3://bkt/model2.tar.gz”} --execution-role-arn arn:aws:iam::123:role/me aws sagemaker create-endpoint-config --endpoint-config-name model2-config --production-variants ‘{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 1, “ModelName”: “model2”, “VariantName”: “AllTraffic”}’ aws sagemaker update-endpoint --endpoint-name my-endpoint --endpoint-config-name model2-config
63. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T  0( "!   1'   &
    /% +2 ),  $  -.#*
64. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T  
     
      endpoint configuration aws sagemaker create-endpoint-config --endpoint-config-name both-models-config --production-variants ‘[{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 95, “ModelName”: “model1”, “VariantName”: “model1-traffic”}, {“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 5, “ModelName”: “model2”, “VariantName”: “model2-traffic”}]’
65. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T  
     
      endpoint configuration  endpoint aws sagemaker create-endpoint-config --endpoint-config-name both-models-config --production-variants ‘[{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 95, “ModelName”: “model1”, “VariantName”: “model1-traffic”}, {“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 5, “ModelName”: “model2”, “VariantName”: “model2-traffic”}]’ aws sagemaker update-endpoint --endpoint-name my-endpoint --endpoint-config-name both-models-config
66. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T       endpoint configuration  endpoint 
    aws sagemaker create-endpoint-config --endpoint-config-name both-models-config --production-variants ‘[{“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 95, “ModelName”: “model1”, “VariantName”: “model1-traffic”}, {“InitialInstanceCount”: 2, “InstanceType”: “ml.m4.xlarge”, “InitialVariantWeight”: 5, “ModelName”: “model2”, “VariantName”: “model2-traffic”}]’ aws sagemaker update-endpoint --endpoint-name my-endpoint --endpoint-config-name both-models-config aws sagemaker update-endpoint-weights-and-capacities --endpoint-name my-endpoint --desired-weights-and-capacities ‘{“VariantName”: ”model1”, “DesiredWeight”: 5}’
67. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
    
     CPU / GPU /  Amazon CloudWatch  
68. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T   SageMaker 
    • Min / max    • Target   • invocations per instance • Cool down time
69. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
    
     
70. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
    
     
71. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T CPU  automatic scaling policy
     aws application-autoscaling register-scalable-target --service-namespace sagemaker --resource-id endpoint/my-endpoint/variant/model2 --scalable-dimension sagemaker:variant:DesiredInstanceCount --min-capacity 2 --max-capacity 5
72. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T CPU   automatic scaling policy  
     aws application-autoscaling register-scalable-target --service-namespace sagemaker --resource-id endpoint/my-endpoint/variant/model2 --scalable-dimension sagemaker:variant:DesiredInstanceCount --min-capacity 2 --max-capacity 5 aws application-autoscaling put-scaling-policy --policy-name model2-scaling --service-namespace sagemaker --resource-id endpoint/my-endpoint/variant/model2 --scalable-dimension sagemaker:variant:DesiredInstanceCount --policy-type TargetTrackingScaling --target-tracking-scaling-policy-configuration ‘{"TargetValue": 50, "CustomizedMetricSpecification": {"MetricName": "CPUUtilization", "Namespace": "/aws/sagemaker/Endpoints", "Dimensions": [{"Name": "EndpointName", "Value": "my-endpoint"}, {"Name": "VariantName","Value": ”model2"}], "Statistic": "Average", "Unit": "Percent”}}’
73. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
74. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 
    
75. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T +&-  "./ 1. Batch Transform Job • %$ !'*
      2. Amazon Elastic Inference • +& %$
     3. Amazon SageMaker Neo • +&     ,(#)#
76. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T   
    
77. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T  
    Amazon S3
78. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T 0 50 100 150 200 1 2 4 6 8 10  
    
79. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon Elastic Inference   
       
80. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
81. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker Neo     
     K E Y F E A T U R E S Neo-AI    (Apache license 2.0)   DL 
    1/10
     https://github.com/neo-ai/
82. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Amazon SageMaker Neo
83. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Parse Model Optimize Tensors Generate Code Optimize Graph TensorFlow, MXNet, PyTorch, XGBoost ")C %05 ML " (NN) & %$ @B
    1?+:  8D 6E/ (-% shape  %$4*
    .;>!, 2 %= '#"$  =97A<3 Neo 
     (TVM / treelite) Pruning Operator fusion Nested loop tiling Vectorization / Tensorization Data layout transform
84. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T WP • D , T GcrzG • FmyGo dFlszaT • , , P e kn • GcrzG ul P T • , , A B T GcrzG G • D , A A S pkGf • T T fnwbS M • w gatI ihIE K J IS MN D D ,
85. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T Related breakouts | l k hu WML G 32 Ot dg a 32 2 C : A 5 : 5 ry e ,4D 4E 248 04 mik SW cens 2D :5. A 4 -1 ,4D 4E 248 04 ho z O c k 32 2 C : A 5 : 5 2C : 04 D4 ,4D
86. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T ! S U M M I T © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. Yoshitaka Haribara hariby@amazon.co.jp @_hariby
87. © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved. S U M M I T References • ML@Loft [Blog#1] • Amazon SageMaker   [Web#1, Blog#2, #3, #4, #5] • AWS Black Belt Online Seminar • Basic [Movie, Slides] • Advanced [Movie, Slides] • AWS : Keras [Blog], Apache Airflow [Blog], Kubeflow [Blog], • HPO: SageMaker default [Blog], Optuna [Blog] •   GPU 
     [Blog] • SageMaker Containers [GitHub] • Jupyter  /IDE 
      [SageMaker Python SDK]   API