Build and deploy PyTorch models with Azure Machine Learning

With machine learning becoming more and more an engineering problem the need to track, work together and easily deploy ML experiments with integrated CI/CD tooling is becoming more relevant then ever.

In this session we take a deep-dive into Azure Machine Learning, a cloud service that you can use to track as you build, train, deploy, and manage models. We use the Azure Machine Learning Python SDK to manage the complete life cycle of a PyTorch model, from managing the data, to train the model and finally run it into a production Kubernetes cluster.

At the end of this session you have a good grasp of the technological building blocks of Azure machine learning services and train a PyTorch model on scale.

Transcript

1. Build and deploy production ready PyTorch models Henk Boelman Senior Cloud Advocate @ Microsoft
2. Henk Boelman Senior Cloud Advocate @ Microsoft @hboelman Github.com/hnky https://www.henkboelman.com Aka.ms/ABitOfAI
3. What are we building? Endpoint { “homer”: 0.9 } Lego Dataset PyTorch Classification Model
4. Before we start
5. Machine Learning Ability to learn without being explicitly programmed.
6. Programming Algorithm Data Answers
7. Machine Learning Algorithm Data Answers
8. Machine Learning Model Data Answers
9. Machine Learning Model Data Answers
10. Machine Learning Predictions Data Model Data Answers
11. What is Deep Learning
13. What is Transfer Learning Transfer learning, used in machine learning, is the reuse of a pre- trained model on a new problem. In transfer learning, a machine exploits the knowledge gained from a previous task to improve generalization about another. For example, in training a classifier to predict whether an image contains food, you could use the knowledge it gained during training to recognize drinks.
14. What is Computer Vision Computer vision is a field of artificial intelligence that trains computers to interpret and understand the visual world. Using digital images and deep learning, machines can accurately detect and classify objects.
15. What is PyTorch PyTorch is an open source machine learning framework based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook's AI Research lab (FAIR). It is free and open-source software released under the Modified BSD license.
16. What are we building? Endpoint { “homer”: 0.9 } Simpson Dataset PyTorch Classification Model
17. How can Azure ML help bring a model to production
18. Distributed training Challenges of distributed training Dependencies and containers Schedule jobs Distribute data Scale resources Provision clusters of VMS Gather results Secure Access Handling failures
19. What is Azure Machine Learning? Set of Azure Cloud Services Python SDK AML Studio or Interface of your choice That enables you to:  Prepare Data  Manage Models  Build Models  Track Experiments  Train Models  Deploy Models
20. Azure ML service Key Elements Workspace Models Environments Experiments Deployment Pipelines Datastores Compute Target Data Labelling
21. Interface
22. Train Data Deploy and Run
23. Compute Run AML Instance Local Compute Linked Compute AML Cluster AutoML Visual Designer SDK Pipeline Training Script Deployment Scoring Script Deployment configuration Dataset Datastore Data labeling Experiment Artifacts Model Logs & Metrics Model Management Environments Curated Custom AKS ACI Online Endpoint Batch Endpoint
24. Training Options 3 ways to train your model in Azure ML 1. All-code: Run configuration 2. No-code: Automated machine learning (AutoML) 3. Some-code: Machine learning pipeline using the Azure ML designer
25. What is Automated Machine Learning? Automated machine learning (automated ML) automates feature engineering, algorithm and hyperparameter selection to find the ‘best model’ for your data. User inputs Featurization & Feature engineering Algorithm selection Hyperparameter tuning Model Leaderboard Dataset Configuration & Constraints 76% 34% 82% 41% 88% 72% 81% 54% 73% 88% 90% 91% 95% 68% 56% 89% 89% 79% Rank Model Score 1 95% …
26. Azure ML Designer What is the Azure ML Designer? AML Designer is an UI interface which enables you to build machine learning pipelines with drag-n-drop experience and simplify the publishment and deployment of your pipelines. Key advantages: Connect to your own data with ease. Hundred of pre-build components help you to build and train models without writing code. Automate model validation, evaluation and interpretation in your pipeline. Deploy models and publish endpoints with a few clicks
27. All Code: Run Configuration Run configuration is an all-code approach for training. Submit jobs with the follow specifications: The training script to execute The compute target to execute the run on The environment to execute the run in Any other parameters such as data, distributed configuration, custom variable, etc Then use Azure ML Experiments UI to track the results of your run
28. Bring our Lego model to production
29. DevOps is the union of people, process, and products to enable continuous delivery of value to your end users. “ ”
30. Start change in the pipeline
31. MLOps Workflow Model management & monitoring Model performance analysis Train model Validate model Deploy model Monitor model Retrain model Build app Collaborate Test app Release app Monitor app App developer using Azure DevOps Data scientist using Azure Machine Learning Model reproducibility Model validation Model deployment Model retraining
32. ML Life Cycle Data Preparation Upload data to Blob Create a dataset Model Building Use PyTorch and Python in GitHub Codespaces Model Training Code first pipeline Using a PyTorch training Environment Run Azure Machine Learning Cluster Using a GitHub Action Model Registration Pipeline registers the model in Model Management Model Deployment Deploy model in an Online Endpoint using a GitHub Action
33. Compute Run AML Instance Local Compute Linked Compute AML Cluster AutoML Visual Designer SDK Pipeline Training Script Deployment Scoring Script Deployment configuration Dataset Datastore Data labeling Experiment Artifacts Model Logs & Metrics Model Management Environments Curated Custom AKS ACI Online Endpoint Batch Endpoint
34. Model Creation
35. ML Pipelines An Azure Machine Learning pipeline is an independently executable workflow of a complete machine learning task. Subtasks are encapsulated as a series of steps within the pipeline. An Azure Machine Learning pipeline can be as simple as one that calls a Python script, so may do just about anything. Workflows of steps that can use Data Sources, Datasets and run on Compute targets
36. Create a step / component Input Output Runs a script on a Compute Target in a Docker container. Parameters
38. Create a pipeline Dataset of Simpsons Images Train the Model with a PyTorch model Register the model Blob Storage Account Model Management
40. Model Deployment
41. Choosing an inferencing target First time deploying? Yes: Local deployment No: Are you prioritizing low latency or high throughput? Low latency: are you prioritizing high availability or minimal costs? Yes: what kind of workload? Traditional ML, low throughput: Azure Kubernetes Service on CPUs Deep learning, high throughput: Azure Kubernetes Service on NVIDIA GPUs Low costs: Azure Container Instances (ACI) High throughput: batch inference on AmlCompute High availability: Do you need to manage your own cluster? No: Azure Machine Learning Online Endpoint
42. Online Endpoint (preview)
43. Event Grid Azure Event Grid is a complete event routing service.
44. Logic App
45. Recap GitHub Action Training Pipeline Create Compute Start deployment to Online Point Creates & Starts A pipeline Model Management Adds event in event grid Adds Message in Queue Starts Logic App workflow GitHub Action Deploy Pipeline Create Environment Online Endpoint { “homer”: 0.9 } User Simpson Dataset
46. Thank you Aka.ms/henk/lego