Data Services Office Hour (Ep 10): Tackling AI/ML Workloads using Red Hat OpenShift & OpenVino

As AI/ML workloads become more popular, one of the challenges is getting models to scale in production within intelligent applications. Working together, Red Hat and Intel have integrated Intel OpenVINO into Red Hat OpenShift Data Science, a newly introduced cloud service that gives data scientists and developers a powerful AI/ML platform for building intelligent applications. See it in action – after a brief intro, Audrey will demonstrate how model inferencing performance can be improved for use cases like AI at the edge.

YouTube: https://youtu.be/7pLymUEB9TA

Transcript

1. Presentation title should not exceed two lines 1 Tackling AI/ML Workloads using Red Hat OpenShift & OpenVINO Audrey Reznik Red Hat Data Scientist Ryan Loney Intel Product Manager
2. Agenda What we’ll discuss today ▸ Overview of Red Hat OpenShift Data Science, a managed cloud service ▸ Overview of OpenVINO ▸ Demo ･ Part 1 - High level overview of the Red Hat OpenShift Data Science platform and it’s Managed Services. ･ Part 2 - Model Development, Training and Deployment with Tensorflow and OpenVino
3. Open source and cloud-based software to power AI initiatives 3 of enterprises use a mix of open source and cloud-based software to power AI initiatives. 69% Source: Data collected from Dec. 16, 2020 - Jan. 22, 2021 | Respondents: 100 enterprise IT and data leaders The need for open hybrid cloud and an open AI platform
4. 4 Overview of Red Hat OpenShift Data Science Key features of Red Hat OpenShift Data Science Combines Red Hat components, open source software, and ISV certified software available on Red Hat Marketplace Increased capabilities/collaboration Model outputs are hosted on the Red Hat OpenShift managed service or exported for integration into an intelligent application Rapid experimentation use cases Available on Red Hat OpenShift Dedicated (AWS) and Red Hat OpenShift Service on AWS Cloud Service Provides data scientists and intelligent application developers the ability to build, train, and deploy ML models Core data science workflow Addressing AI/ML experimentation and integration use cases on a managed platform
5. Gather and prepare data Integrate models in app dev Model monitoring and management Develop model 5 Overview of Red Hat OpenShift Data Science Red Hat Managed Cloud Services Open hybrid cloud platform with self service capabilities Accelerators Cloud infrastructure Source to image Red Hat OpenShift Service on Amazon Web Services Retrain models ISV managed cloud services Red Hat managed cloud services Red Hat managed cloud platform Customer managed ISV software
6. Typical Data Science Challenges Typical Data Science Challenges that benefit from Software Acceleration 6 Exploration and Development time By using a pre-trained and optimized model a Data Scientist can save significant exploration and development time. Exploration, Development, Optimization, Quantization, Deployment… Optimize Model Performance Optimization and Quantization of your models Deploy Efficient Pipelines Deploy your model as a service using efficient pipelines.
7. OpenVINO™ Toolkit 7
8. 8 OPENVINO™ TOOLKIT INFERENCE 220+ PRE-TRAINED DEEP LEARNING MODELS AVAILABLE, INCLUDING: Object recognition Semantic segmentation Instance segmentation Human pose estimation Image processing Text detection Question answering Text spotting Machine translation Image retrieval Depth estimation Text recognition Action recognition
9. 9 SCALE DEPLOYMENTS WITH OPENSHIFT CLIENT APPLICATION Inference ?????? CLIENT APPLICATION Inference ?????? CLIENT APPLICATION Inference ?????? CLIENT APPLICATION Inference ?????? Load Balancer MODEL SERVER MODEL SERVER MODEL SERVER MODEL SERVER
10. 10 ECOSYSTEM ADOPTION
11. Red Hat OpenShift & OpenVino Demo Demo Part 1 - High level overview of the Red Hat OpenShift Data Science managed cloud service Part 2 - Model Development, Training to Deployment with Tensorflow and OpenVINO ◦ Example demonstrates how to train, convert and deploy an image classification model with TensorFlow and OpenVINO. ◦ This particular notebook shows the process where we perform the inference step on the freshly trained model The training code is based on the official TensorFlow Image Classification Tutorial: (https://www.tensorflow.org/tutorials/images/classification).
12. linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you 12