Amazon SageMaker

Transcript

1. Julien Simon Global Evangelist, AI & Machine Learning @julsimon From Notebook to Production with Amazon SageMaker
2. Put Machine Learning in the hands of every developer and data scientist Our mission
3. Application Services Platform Services Frameworks & Infrastructure API-driven services: Vision, Language & Speech Services, Chatbots AWS ML Stack h t t p s : / / m l . a w s h t t p s : / / m e d i u m . c o m / @ j u l s i m o n / a - m a p - f o r - m a c h i n e - l e a r n i n g - o n - a w s - a 2 8 5 f c d 8 d 9 3 2 Deploy machine learning models with high-performance machine learning algorithms, broad framework support, and one-click training, tuning, and inference. Develop sophisticated models with any framework, create managed, auto-scaling clusters of GPUs for large scale training, or run prediction on trained models.
4. Application Services Platform Services Frameworks & Infrastructure API-driven services: Vision, Language & Speech Services, Chatbots Deploy machine learning models with high-performance machine learning algorithms, broad framework support, and one-click training, tuning, and inference. Develop sophisticated models with any framework, create managed, auto-scaling clusters of GPUs for large scale training, or run prediction on trained models. AWS ML Stack h t t p s : / / m l . a w s h t t p s : / / m e d i u m . c o m / @ j u l s i m o n / a - m a p - f o r - m a c h i n e - l e a r n i n g - o n - a w s - a 2 8 5 f c d 8 d 9 3 2
5. Data Visualization & Analysis Business Problem ML problem framing Data Collection Data Integration Data Preparation & Cleaning Feature Engineering Model Training & Parameter Tuning Model Evaluation Are Business Goals met? Model Deployment Monitoring & Debugging Yes No Data Augmentation Feature Augmentation The Machine Learning Process Re-training Predictions
6. Amazon SageMaker Pre-built notebooks for common problems K-Means Clustering Principal Component Analysis Neural Topic Modelling Factorization Machines Linear Learner XGBoost Latent Dirichlet Allocation Image Classification Seq2Seq, And more! ALGORITHMS Apache MXNet, Chainer TensorFlow, PyTorch, scikit-learn FRAMEWORKS S e t u p a n d m a n a g e e n v i r o n m e n t s f o r t r a i n i n g T r a i n a n d t u n e m o d e l ( t r i a l a n d e r r o r ) D e p l o y m o d e l i n p r o d u c t i o n S c a l e a n d m a n a g e t h e p r o d u c t i o n e n v i r o n m e n t Built-in, high- performance algorithms Build Git integration Elastic inference
7. Git integration
8. Elastic Inference https://aws.amazon.com/blogs/aws/amazon-elastic-inference-gpu-powered-deep-learning-inference-acceleration/
9. Amazon SageMaker Pre-built notebooks for common problems K-Means Clustering Principal Component Analysis Neural Topic Modelling Factorization Machines Linear Learner XGBoost Latent Dirichlet Allocation Image Classification Seq2Seq, And more! ALGORITHMS Apache MXNet, Chainer TensorFlow, PyTorch, scikit-learn FRAMEWORKS S e t u p a n d m a n a g e e n v i r o n m e n t s f o r t r a i n i n g T r a i n a n d t u n e m o d e l ( t r i a l a n d e r r o r ) D e p l o y m o d e l i n p r o d u c t i o n S c a l e a n d m a n a g e t h e p r o d u c t i o n e n v i r o n m e n t Built-in, high- performance algorithms Build New built-in algorithms scikit-learn environment Model marketplace Search
10. Search training jobs
11. Machine Learning Marketplace
12. Amazon SageMaker Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter optimization Train Deploy model in production Scale and manage the production environment P3DN, C5N TensorFlow on 256 GPUs Resume HPO tuning job Build
13. Amazon SageMaker Fully managed hosting with auto- scaling One-click deployment Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter optimization Deploy Model compilation Elastic inference Inference pipelines Train Build
14. Amazon SageMaker Fully managed hosting with auto- scaling One-click deployment Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter optimization Deploy Model compilation Elastic inference Inference pipelines Train Build P3DN, C5N TensorFlow on 256 GPUs Resume HPO tuning job New built-in algorithms scikit-learn environment Model marketplace Search Git integration Elastic inference
15. Working with Amazon SageMaker
16. The Amazon SageMaker API • Python SDK orchestrating all Amazon SageMaker activity • High-level objects for algorithm selection, training, deploying, automatic model tuning, etc. • Spark SDK (Python & Scala) • AWS CLI: ‘aws sagemaker’ • AWS SDK: boto3, etc.
17. Model Training (on EC2) Model Hosting (on EC2) Training data Model artifacts Training code Helper code Helper code Inference code Ground Truth Client application Inference code Training code Inference request Inference response Inference Endpoint
18. Training code Factorization Machines Linear Learner Principal Component Analysis K-Means Clustering XGBoost And more Built-in Algorithms Bring Your Own Container Bring Your Own Script Model options
19. Built-in algorithms orange: supervised, yellow: unsupervised Linear Learner: regression, classification Image Classification: Deep Learning (ResNet) Factorization Machines: regression, classification, recommendation Object Detection (SSD): Deep Learning (VGG or ResNet) K-Nearest Neighbors: non-parametric regression and classification Neural Topic Model: topic modeling XGBoost: regression, classification, ranking https://github.com/dmlc/xgboost Latent Dirichlet Allocation: topic modeling (mostly) K-Means: clustering Blazing Text: GPU-based Word2Vec, and text classification Principal Component Analysis: dimensionality reduction Sequence to Sequence: machine translation, speech to text and more Random Cut Forest: anomaly detection DeepAR: time-series forecasting (RNN) Object2Vec: general-purpose embedding IP Insights: usage patterns for IP addresses Semantic Segmentation: Deep Learning
20. XGBoost • Open Source project • Popular tree-based algorithm for regression, classification and ranking • Builds a collection of trees. • Handles missing values and sparse data • Supports distributed training • Can work with data sets larger than RAM https://github.com/dmlc/xgboost https://xgboost.readthedocs.io/en/latest/ https://arxiv.org/abs/1603.02754
21. Demo: XGBoost AWS re:Invent 2018 workshop https://gitlab.com/juliensimon/ent321
22. Optimizing TensorFlow https://aws.amazon.com/blogs/machine-learning/faster-training- with-optimized-tensorflow-1-6-on-amazon-ec2-c5-and-p3- instances/ (March 2018) Training a ResNet-50 benchmark with the synthetic ImageNet dataset using our optimized build of TensorFlow 1.11 on a c5.18xlarge instance type is 11x faster than training on the stock binaries. https://aws.amazon.com/about-aws/whats- new/2018/10/chainer4-4_theano_1-0- 2_launch_deep_learning_ami/ (October 2018)
23. Demo: Elastic Inference with TensorFlow https://github.com/awslabs/amazon-sagemaker- examples/blob/master/sagemaker-python- sdk/tensorflow_iris_dnn_classifier_using_estimators/
24. Automatic Model Tuning
25. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Automatic Model Tuning Finding the optimal set of hyper parameters 1. Manual Search (”I know what I’m doing”) 2. Grid Search (“X marks the spot”) • Typically training hundreds of models • Slow and expensive 3. Random Search (“Spray and pray”) • Works better and faster than Grid Search • But… but… but… it’s random! 4. HPO: use Machine Learning • Training fewer models • Gaussian Process Regression and Bayesian Optimization • You can now resume from a previous tuning job
26. Demo: XGBoost + HPO AWS re:Invent 2018 workshop https://gitlab.com/juliensimon/ent321
27. Model Compilation
28. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Optimizing for the underlying hardware https://aws.amazon.com/blogs/aws/amazon-sagemaker-neo-train-your-machine-learning-models-once-run-them-anywhere/ • Train once, run anywhere • Frameworks and algorithms • TensorFlow, Apache MXNet, PyTorch, ONNX, and XGBoost • Hardware architectures • ARM, Intel, and NVIDIA starting today • Cadence, Qualcomm, and Xilinx hardware coming soon • Amazon SageMaker Neo will be released as open source enabling hardware vendors to customize it for their processors and devices.
29. Demo: Compiling ResNet-50 for the Raspberry Pi Configure the compilation job { "RoleArn":$ROLE_ARN, "InputConfig": { "S3Uri":"s3://jsimon-neo/model.tar.gz", "DataInputConfig": "{\"data\": [1, 3, 224, 224]}", "Framework": "MXNET" }, "OutputConfig": { "S3OutputLocation": "s3://jsimon-neo/", "TargetDevice": "rasp3b" }, "StoppingCondition": { "MaxRuntimeInSeconds": 300 } } Compile the model $ aws sagemaker create-compilation-job --cli-input-json file://config.json --compilation-job-name resnet50-mxnet-pi $ aws s3 cp s3://jsimon-neo/model- rasp3b.tar.gz . $ gtar tfz model-rasp3b.tar.gz compiled.params compiled_model.json compiled.so Predict with the compiled model from dlr import DLRModel model = DLRModel('resnet50', input_shape, output_shape, device) out = model.run(input_data)
30. Inference Pipelines
31. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Inference Pipelines • Linear sequence of 2-5 containers that process inference requests • Feature engineering with scikit-learn or SparkML (on AWS Glue or Amazon EMR) • Predict with built-in or custom containers • The sequence is deployed as a a single model • Useful to preprocess, predict, and post-process • Available for real-time prediction and batch transform
32. Demo: Inference with scikit-learn and linear learner https://github.com/awslabs/amazon-sagemaker-examples/blob/master/sagemaker- python-sdk/scikit_learn_inference_pipeline/
33. Amazon SageMaker Fully managed hosting with auto- scaling One-click deployment Pre-built notebooks for common problems Built-in, high- performance algorithms One-click training Hyperparameter optimization Build Train Deploy FREE TIER
34. Selected Amazon SageMaker customers
35. © 2018, Amazon Web Services, Inc. or Its Affiliates. All rights reserved. Getting started http://aws.amazon.com/free https://ml.aws https://aws.amazon.com/sagemaker https://github.com/aws/sagemaker-python-sdk https://github.com/aws/sagemaker-spark https://github.com/awslabs/amazon-sagemaker-examples https://gitlab.com/juliensimon/ent321 https://medium.com/@julsimon https://gitlab.com/juliensimon/dlnotebooks
36. Thank you! Julien Simon Global Evangelist, AI and Machine Learning @julsimon