APAC Hybrid Cloud KOPI Hour (E6) - AI

We are joined by Red Hat APAC CTO Vincent Caldeira to dive into all things artificial! We’ll learn about what makes AI work, how businesses in APAC are using it today, and how Red Hat’s strategy is all in for AI!

Transcript

1. Presentation title should not exceed two lines 1 10 August 2023 APAC Hybrid Cloud Kopi Hour: GenAI & Foundation Models 101 Vincent Caldeira CTO APAC, Red Hat vincent.caldeira@redhat.com
2. Primer on Generative AI & Foundation Models 2
3. 3 AI and Machine Learning Overview Contextual introduction of the basics of AI/ML and where Generative AI fits Sources: What’s Generative AI? Explore Underlying Layers of Machine Learning and Deep Learning (Amol Wagh on Medium, March 26, 2023) Artificial Intelligence: Capability of a computer system to mimic human cognitive functions such as learning problem solving, leveraging vast amounts of data and mathematics. Machine learning: It is a subset and an application of AI that refers to systems that can learn by themselves, with direct instructions. ML models take in data and fit the data to an algorithm, studying patterns to make predictions and perform tasks on its own. Deep learning: It is a subset of machine learning that’s based on artificial neural networks, mimicking the brain structure. It involves the use of complex algorithms across multiple layers of representation, allowing to model highly non-linear relationships on large amounts of data. Generative AI: It is a subset of deep learning models that can produce new content based on what is described in the input. The collection of generative AI models that can produce language, code, and images.
4. 4 How Foundation Models fit into the AI Landscape Foundation Models powering Generative AI Sources: What Are Foundation Models? (Amol Wagh on Medium, March 26, 2023) Foundation Models 101: A step-by-step guide for beginners (scribbleData) A foundation model is an AI neural network — trained on mountains of raw data, generally with unsupervised learning — that can be adapted to accomplish a broad range of tasks.
5. 5 Implementing Foundation Models Example of Architecture for Large Language Model Applications Sources: Emerging Architectures for LLM Applications (Matt Bornstein and Rajko Radovanovic, Andreesen Horowitz)
6. Available on https://github.com/caldeirav/foundation-models-demo 6 Prepare a dataset from Wikipedia articles, build embeddings and store them in a local file (for a start) Question Answering Prepare Embeddings Use model inputs via embeddings to help GPT answer questions on unfamiliar topics through Search-Ask method Model Input & Prompt Engineering Build a simple pet name generator application and understand interaction with Completion API Demo: Leveraging OpenAI API, Prompting & Embeddings
7. 7 Generative AI in the Real World Benefits, Challenges and most promising industry use-cases Generative AI Use-Cases • Automotive: 3D models for simulation and car development. Synthetic data for training autonomous vehicles. • Natural Sciences: Develop new protein sequences to aid in drug discovery. Climate risk / simulation. • Entertainment: Content creation for video games, films, animation and virtual reality. Sources: What is Generative AI? (NVIDIA) Benefits Challenges New content Synthetic Data Identify Patterns Automate Tasks Scale Sampling Speed Data Quality Data Licenses
8. Red Hat OpenShift AI Strategy 8
9. 9 Applications which don't use data to build continuous learning systems to scale and adapt autonomously are going to be obsolete. “” Mark Little Chief Data Strategist & Head of Engineering at Anonos Fischer, Lukas, Lisa Ehrlinger, Verena Geist, Rudolf Ramler, Florian Sobiezky, Werner Zellinger, David Brunner, Mohit Kumar, and Bernhard Moser. 2021. "AI System Engineering—Key Challenges and Lessons Learned" Machine Learning and Knowledge Extraction 3, no. 1: 56-83
10. 10 Red Hat OpenShift AI Accelerates Generative AI Adoption Enterprise grade AI and MLOps platform for the Open Hybrid Cloud Model development Conduct exploratory data science in JupyterLab with access to core AI / ML libraries and frameworks including TensorFlow and PyTorch using our notebook images or your own. Model serving & monitoring Deploy models across any cloud, fully managed, and self-managed OpenShift footprint and centrally monitor their performance. Lifecycle management Create repeatable data science pipelines for model training and validation and integrate them with devops pipelines for delivery of models across your enterprise. Increased capabilities / collaboration Create projects and share them across teams. Combine Red Hat components, open source software, and ISV certified software. Now available as fully managed cloud service or traditional software product on-prem or in the cloud!
11. 11 What sets our approach apart Ease of Access Red Hat provides the “glue” to to the integration of open source tooling and tracks changes and fixes to open source AI/ML projects in order to enable customer access to upstream innovation Open Source Collaborate on a common platform to bring IT, data science and application development teams together with a unified platform for continuous delivery of Intelligent Applications Collaboration Simple enterprise cluster configuration and deployment on a secure and proven platform, that are provided on a self-service model, with autoscaling and FinOps capabilities Train, deploy and manage models in containerized format for intelligent apps consistently either on-premise or in cloud Hybrid Cloud Standardized platform for creating production AI/ML models, as well as running the resulting applications
12. 12 Unified Continuous Delivery for Intelligent Applications ML applications are a collaboration between Data Scientists, DevOps Engineers, ML Engineers, and Application Developers MLOps Foundational Capabilities for MLOps Models are built and deployed using CI/CD pipelines ensuring that they are Repeatable and Automated Solutions are GitOps Managed making git the primary source of truth to represent how the model is trained and deployed Metrics are captured across the entire ML lifecycle and made Observable to allow teams to make better decisions and understand the state of their applications ML Applications present new challenges and require teams to consider how to Secure their application from the start Repeatable and Automated GitOps Managed Collaboration Secure Observable
13. data source data pipeline training pipeline Intelligent Application = ML Code + App Code 13 Machine Learning Lifecycle An end-to-end lifecycle from data ingestion to performance monitoring explore data preprocess data engineer features train/tune models validate models validate infrastructure push models preprocess data (INPUT) serve models prediction (OUTPUT) log, monitor, alert condition (OUTPUT) feature store model hub metadata store Physical* Virtual Private cloud Public cloud Edge server edge web / API inferencing (production) training (development) stream batch ingest data heuristics
14. Define and Schedule ML Work on Demand Define Workloads Kubernetes/ OpenShift Queue Workloads Scale Up Resources Run Workloads Foundation Models need a lot more data and training So how do we distribute training? This is what an ideal workflow could look like Data Scientists need simple ways of defining, submitting and monitoring distributed workloads without worrying (too much) about cloud infrastructure Cluster Administrators need a way to enable this access to infrastructure while keeping access and cloud spend in check Challenges
15. 15 Foundation Models Architecture for Model Training and Fine-tuning The CodeFlare stack solves a number challenges faced by data scientists and administrators Sources: Red Hat blog “A guide to fine-tuning and serving AI/ML foundation models” June 2023 Red Hat blog “AI/ML Models Batch Training at Scale with Open Data Hub” May 2023 Data Scientist • Ease of Use • No code rewrites • Scalability with Ray framework • Safety via resource guarantees IT Administrator • Simple deployment • Resource management • Cost management
16. linkedin.com/company/red-hat youtube.com/user/RedHatVideos facebook.com/redhatinc twitter.com/RedHat Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you 16