Build and deploy PyTorch models with Azure Machine Learning

With machine learning becoming more and more an engineering problem the need to track, work together and easily deploy ML experiments with integrated CI/CD tooling is becoming more relevant then ever.

In this session we take a deep-dive into Azure Machine Learning service, a cloud service that you can use to track as you build, train, deploy, and manage models. We zoom into the building blocks provided and show, through some demos, how to use them.

At the end of this session you have a good grasp of the technological building blocks of Azure machine learning services. Just waiting to be used in your own projects afterwards.

Transcript

1. Henk Boelman Cloud Advocate @ Microsoft Build and deploy PyTorch models with Azure Machine Learning HenkBoelman.com @hboelman
2. Agenda Machine Learning on Azure Data science process Simpson recognition using Azure Machine Learning
3. Machine Learning Ability to learn without being explicitly programmed.
4. Programming Algorithm Data Answers
5. Machine Learning Algorithm Data Answers
6. Machine Learning Model Data Answers
7. Machine Learning Model Data Answers
8. Machine Learning Predictions Data Model Data Answers
9. Sophisticated pretrained models To simplify solution development Azure Databricks Machine Learning VMs Popular frameworks To build advanced deep learning solutions TensorFlow Keras Pytorch Onnx Azure Machine Learning Language Speech … Azure Search Vision On-premises Cloud Edge Productive services To empower data science and development teams Powerful infrastructure To accelerate deep learning Flexible deployment To deploy and manage models on intelligent cloud and edge Machine Learning on Azure Cognitive Services
10. Vision Speech Language Knowledge Cognitive Services: Pre-Trained models in the cloud and on the edge
12. …
13. Ask a sharp question Collect the data Prepare the data Select the algorithm Train the model Use the answer The data science process
14. Ask a sharp question
15. Ask a sharp question How much / how many? Is it this or that? Is it weird? Which group? Which action?
16. What is the price of a second hand car?
17. Collect the data
18. Collect the data
19. Collect the data
20. Prepare the data
21. Prepare the data Every column in your dataset has to be: Relevant Independent Simple Clean
22. Prepare the data: Relevant
23. Prepare the data: Independent
24. Prepare the data: Simple
25. Prepare the data: Clean
26. Prepare the data: Clean
27. Select the algorithm
28. Select the algorithm
29. Train the Model
32. Use the answer
34. Azure Machine Learning studio A fully-managed cloud service that enables you to easily build, deploy, and share predictive analytics solutions.
35. Is it Marge or Homer?
36. Simpson Lego dataset https://github.com/hnky/dataset-lego-figures
37. Sophisticated pretrained models To simplify solution development Azure Databricks Machine Learning VMs Popular frameworks To build advanced deep learning solutions TensorFlow Keras Pytorch Onnx Azure Machine Learning Language Speech … Azure Search Vision On-premises Cloud Edge Productive services To empower data science and development teams Powerful infrastructure To accelerate deep learning Flexible deployment To deploy and manage models on intelligent cloud and edge Machine Learning on Azure Cognitive Services
39. Prepare your environment Experiment with your model & data Deploy Your model into production
40. Step 1: Prepare your environment
41. Setup your environment VS Code Azure Notebooks Azure Portal
42. Azure Notebook / Jupyter Notebook
43. Create a workspace ws = Workspace.create( name='<NAME>', subscription_id='<SUBSCRIPTION ID>', resource_group='<RESOURCE GROUP>', location='westeurope') ws.write_config() ws = Workspace.from_config() Create a workspace
45. Datasets – registered, known data sets Experiments – Training runs Models – Registered, versioned models Endpoints: Real-time Endpoints – Deployed model endpoints Pipeline Endpoints – Training workflows Compute – Managed compute Datastores – Connections to data Azure Machine Learning Service
46. Create Compute cfg = AmlCompute.provisioning_configuration( vm_size='STANDARD_NC6', min_nodes=1, max_nodes=6) cc = ComputeTarget.create(ws, '<NAME>', cfg) Create a workspace Create compute
48. Demo: Setup your workspace Create a workspace Create compute Setup storage
49. Step 1 Prepare your environment Create a workspace Create compute Setup storage
50. Step 2: Experiment with your model & data
51. Create an experiment exp = Experiment(workspace=ws, name=“<ExperimentName>”) Create an Experiment
52. Create a training file Create an Experiment Create a training file
53. Create an estimator params = {'--data-folder': ws.get_default_datastore().as_mount()} estimator = TensorFlow( source_directory = script_folder, script_params = params, compute_target = computeCluster, entry_script = 'train.py’, use_gpu = True, conda_packages = ['scikit-learn','keras','opencv’], framework_version='1.10') Create an Experiment Create a training file Create an estimator
54. Submit the experiment to the cluster run = exp.submit(estimator) RunDetails(run).show() Create an Experiment Create a training file Submit to the AI cluster Create an estimator
55. Create an Experiment Create a training file Submit to the AI cluster Create an estimator Demo: Creating and run an experiment
56. Azure Notebook Compute Target Experiment Docker Image Data store 1. Snapshot folder and send to experiment 2. create docker image 3. Deploy docker and snapshot to compute 4. Mount datastore to compute 6. Stream stdout, logs, metrics 5. Launch the script 7. Copy over outputs
57. Register the model model = run.register_model( model_name='SimpsonsAI', model_path='outputs') Create an Experiment Create a training file Submit to the AI cluster Create an estimator Register the model
58. Create an Experiment Create a training file Submit to the AI cluster Create an estimator Register the model Demo: Register and test the model
59. Step 2 Experiment with your model & data
60. Step 3: Deploy your model into production
61. AMLS to deploy The Model Score.py Environment file Docker Image
62. Score.py %%writefile score.py from azureml.core.model import Model def init(): model_root = Model.get_model_path('MyModel’) loaded_model = model_from_json(loaded_model_json) loaded_model.load_weights(model_file_h5) def run(raw_data): url = json.loads(raw_data)['url’] image_data = cv2.resize(image_data,(96,96)) predicted_labels = loaded_model.predict(data1) return json.dumps(predicted_labels)
63. Environment File from azureml.core.runconfig import CondaDependencies cd = CondaDependencies.create() cd.add_conda_package('keras==2.2.2') cd.add_conda_package('opencv') cd.add_tensorflow_conda_package() cd.save_to_file(base_directory='./', conda_file_path='myenv.yml')
64. Inference config inference_config = InferenceConfig( runtime= "python", entry_script="score.py", conda_file="myenv.yml" )
65. Deployment using AMLS
66. Deploy to ACI aciconfig = AciWebservice.deploy_configuration( cpu_cores = 1, memory_gb = 2) service = Model.deploy(workspace=ws, name='simpsons-aci', models=[model], inference_config=inference_config, deployment_config=aciconfig)
67. Deploy to AKS aks_target = AksCompute(ws,"AI-AKS-DEMO") deployment_config = AksWebservice.deploy_configuration( cpu_cores = 1, memory_gb = 1) service = Model.deploy(workspace=ws, name="simpsons-ailive", models=[model], inference_config=inference_config, deployment_config=deployment_config, deployment_target=aks_target) service.wait_for_deployment(show_output = True)
68. Demo: Deploy to ACI
70. Pipelines Azure ML Service Pipelines Azure Pipelines
71. Azure Machine Learning Pipelines Workflows of steps that can use Data Sources, Datasets and Compute targets Unattended runs Reusability Tracking and versioning
72. Azure Pipelines Orchestration for Continuous Integration and Continuous Delivery Gates, tasks and processes for quality Integration with other services Trigger on code and non-code events
73. Create a pipeline step Input Output Runs a script on a Compute Target in a Docker container. Parameters
74. Create a pipeline Dataset of Simpsons Images Prepare data Train the Model with a PyTorch estimator Processed dataset model Register the model Blob Storage Account Model Management
75. Continuous Integration
76. Demo: Setup Azure Pipeline for AMLS Pipeline
77. Complete Pipeline
78. “The future we invent is a choice we make, not something that just happens” Satya Nadella
79. @hboelman github.com/hnky henkboelman.com Thank you! Read more on: henkboelman.com