Data Services Office Hour (Ep 10): Tackling AI/ML Workloads using Red Hat OpenShift & OpenVino

As AI/ML workloads become more popular, one of the challenges is getting models to scale in production within intelligent applications. Working together, Red Hat and Intel have integrated Intel OpenVINO into Red Hat OpenShift Data Science, a newly introduced cloud service that gives data scientists and developers a powerful AI/ML platform for building intelligent applications. See it in action – after a brief intro, Audrey will demonstrate how model inferencing performance can be improved for use cases like AI at the edge.

YouTube: https://youtu.be/7pLymUEB9TA

Transcript

1. Presentation title should not
   exceed two lines
   1
   Tackling AI/ML Workloads using
   Red Hat OpenShift & OpenVINO
   Audrey Reznik
   Red Hat Data Scientist
   Ryan Loney
   Intel Product Manager
   

   View Slide

2. Agenda
   What we’ll
   discuss today ▸ Overview of Red Hat OpenShift Data Science, a managed cloud service
   ▸ Overview of OpenVINO
   ▸ Demo
   ･ Part 1 - High level overview of the Red Hat OpenShift Data Science
   platform and it’s Managed Services.
   ･ Part 2 - Model Development, Training and Deployment with
   Tensorflow and OpenVino
   

   View Slide

3. Open source and cloud-based software to power AI initiatives
   3
   of enterprises use a mix of open
   source and cloud-based software
   to power AI initiatives.
   69%
   Source:
   Data collected from Dec. 16, 2020 - Jan. 22, 2021 | Respondents: 100 enterprise IT and data leaders
   The need for open hybrid cloud and an open AI platform
   

   View Slide

4. 4
   Overview of Red Hat OpenShift Data Science
   Key features of Red Hat OpenShift Data Science
   Combines Red Hat components, open source
   software, and ISV certified software available
   on Red Hat Marketplace
   Increased capabilities/collaboration
   Model outputs are hosted on the Red Hat
   OpenShift managed service or exported for
   integration into an intelligent application
   Rapid experimentation use cases
   Available on Red Hat OpenShift Dedicated
   (AWS) and Red Hat OpenShift Service on
   AWS
   Cloud Service
   Provides data scientists and intelligent
   application developers the ability to build,
   train, and deploy ML models
   Core data science workflow
   Addressing AI/ML experimentation and integration use cases on a managed platform
   

   View Slide

5. Gather and prepare data Integrate models in app dev
   Model monitoring
   and management
   Develop model
   5
   Overview of Red Hat OpenShift Data Science
   Red Hat Managed Cloud Services
   Open hybrid cloud platform with self service capabilities
   Accelerators
   Cloud infrastructure
   Source to image
   Red Hat OpenShift Service on
   Amazon Web Services
   Retrain models
   ISV managed cloud
   services
   Red Hat managed
   cloud services
   Red Hat managed
   cloud platform
   Customer managed
   ISV software
   

   View Slide

6. Typical Data Science Challenges
   Typical Data Science Challenges that benefit from Software Acceleration
   6
   Exploration and Development time
   By using a pre-trained and optimized model a Data Scientist can save significant exploration and
   development time.
   Exploration, Development, Optimization, Quantization, Deployment…
   Optimize Model Performance
   Optimization and Quantization of your models
   Deploy Efficient Pipelines
   Deploy your model as a service using efficient pipelines.
   

   View Slide

7. OpenVINO™
   Toolkit
   7
   

   View Slide

8. 8
   OPENVINO™ TOOLKIT INFERENCE
   220+ PRE-TRAINED DEEP LEARNING MODELS AVAILABLE, INCLUDING:
   Object
   recognition
   Semantic
   segmentation
   Instance
   segmentation
   Human pose
   estimation
   Image
   processing
   Text
   detection
   Question
   answering
   Text
   spotting
   Machine
   translation
   Image
   retrieval
   Depth
   estimation
   Text recognition
   Action
   recognition
   

   View Slide

9. 9
   SCALE DEPLOYMENTS
   WITH OPENSHIFT
   CLIENT APPLICATION
   Inference
   ??????
   CLIENT APPLICATION
   Inference
   ??????
   CLIENT APPLICATION
   Inference
   ??????
   CLIENT APPLICATION
   Inference
   ??????
   Load
   Balancer
   MODEL
   SERVER
   MODEL
   SERVER
   MODEL
   SERVER
   MODEL
   SERVER
   

   View Slide

10. 10
    ECOSYSTEM ADOPTION
    

    View Slide

11. Red Hat OpenShift & OpenVino Demo
    Demo
    Part 1 - High level overview of the Red Hat OpenShift Data Science managed cloud service
    Part 2 - Model Development, Training to Deployment with Tensorflow and OpenVINO
    ○ Example demonstrates how to train, convert and deploy an image classification model with
    TensorFlow and OpenVINO.
    ○ This particular notebook shows the process where we perform the inference step on
    the freshly trained model
    The training code is based on the official TensorFlow Image Classification Tutorial:
    (https://www.tensorflow.org/tutorials/images/classification).
    

    View Slide

12. linkedin.com/company/red-hat
    youtube.com/user/RedHatVideos
    facebook.com/redhatinc
    twitter.com/RedHat
    Red Hat is the world’s leading
    provider of enterprise open source
    software solutions. Award-winning
    support, training, and consulting
    services make
    Red Hat a trusted adviser to the
    Fortune 500.
    Thank you
    12
    

    View Slide