$30 off During Our Annual Pro Sale. View Details »

APAC Hybrid Cloud KOPI Hour (E6) - AI

APAC Hybrid Cloud KOPI Hour (E6) - AI

We are joined by Red Hat APAC CTO Vincent Caldeira to dive into all things artificial! We’ll learn about what makes AI work, how businesses in APAC are using it today, and how Red Hat’s strategy is all in for AI!

Red Hat Livestreaming

September 21, 2023
Tweet

More Decks by Red Hat Livestreaming

Other Decks in Technology

Transcript

  1. Presentation title should not
    exceed two lines
    1
    10 August 2023
    APAC Hybrid Cloud Kopi Hour:
    GenAI & Foundation Models 101
    Vincent Caldeira
    CTO APAC, Red Hat
    [email protected]

    View Slide

  2. Primer on Generative
    AI & Foundation Models
    2

    View Slide

  3. 3
    AI and Machine Learning Overview
    Contextual introduction of the basics of AI/ML and where Generative AI fits
    Sources:
    What’s Generative AI? Explore Underlying Layers of Machine Learning and Deep Learning (Amol Wagh on Medium, March 26, 2023)
    Artificial Intelligence: Capability of a computer system to mimic
    human cognitive functions such as learning problem solving, leveraging
    vast amounts of data and mathematics.
    Machine learning: It is a subset and an application of AI that refers to
    systems that can learn by themselves, with direct instructions. ML
    models take in data and fit the data to an algorithm, studying patterns to
    make predictions and perform tasks on its own.
    Deep learning: It is a subset of machine learning that’s based on
    artificial neural networks, mimicking the brain structure. It involves the
    use of complex algorithms across multiple layers of representation,
    allowing to model highly non-linear relationships on large amounts of
    data.
    Generative AI: It is a subset of deep learning models that can produce
    new content based on what is described in the input. The collection of
    generative AI models that can produce language, code, and images.

    View Slide

  4. 4
    How Foundation Models fit into the AI Landscape
    Foundation Models powering Generative AI
    Sources:
    What Are Foundation Models? (Amol Wagh on Medium, March 26, 2023)
    Foundation Models 101: A step-by-step guide for beginners (scribbleData)
    A foundation model is an AI neural network — trained on
    mountains of raw data, generally with unsupervised
    learning — that can be adapted to accomplish a broad
    range of tasks.

    View Slide

  5. 5
    Implementing Foundation Models
    Example of Architecture for Large Language Model Applications
    Sources:
    Emerging Architectures for LLM Applications (Matt Bornstein and Rajko Radovanovic, Andreesen Horowitz)

    View Slide

  6. Available on https://github.com/caldeirav/foundation-models-demo
    6
    Prepare a dataset from Wikipedia articles,
    build embeddings and store them in a
    local file (for a start)
    Question Answering
    Prepare Embeddings
    Use model inputs via embeddings to help
    GPT answer questions on unfamiliar
    topics through Search-Ask method
    Model Input & Prompt Engineering
    Build a simple pet name generator
    application and understand interaction
    with Completion API
    Demo: Leveraging OpenAI API, Prompting & Embeddings

    View Slide

  7. 7
    Generative AI in the Real World
    Benefits, Challenges and most promising industry use-cases
    Generative AI Use-Cases
    ● Automotive: 3D models for simulation and car
    development. Synthetic data for training autonomous
    vehicles.
    ● Natural Sciences: Develop new protein sequences to aid
    in drug discovery. Climate risk / simulation.
    ● Entertainment: Content creation for video games, films,
    animation and virtual reality.
    Sources:
    What is Generative AI? (NVIDIA)
    Benefits
    Challenges
    New content Synthetic Data
    Identify Patterns Automate Tasks
    Scale Sampling Speed
    Data Quality Data Licenses

    View Slide

  8. Red Hat OpenShift
    AI Strategy
    8

    View Slide

  9. 9
    Applications which don't use data to build
    continuous learning systems to scale and
    adapt autonomously are going to be
    obsolete.
    “”
    Mark Little
    Chief Data Strategist & Head of Engineering at Anonos
    Fischer, Lukas, Lisa Ehrlinger, Verena Geist, Rudolf Ramler, Florian Sobiezky, Werner Zellinger, David Brunner, Mohit Kumar, and Bernhard Moser. 2021. "AI
    System Engineering—Key Challenges and Lessons Learned" Machine Learning and Knowledge Extraction 3, no. 1: 56-83

    View Slide

  10. 10
    Red Hat OpenShift AI Accelerates Generative AI Adoption
    Enterprise grade AI and MLOps platform for the Open Hybrid Cloud
    Model development
    Conduct exploratory data science in JupyterLab with access to core
    AI / ML libraries and frameworks including TensorFlow and PyTorch
    using our notebook images or your own.
    Model serving & monitoring
    Deploy models across any cloud, fully managed, and self-managed
    OpenShift footprint and centrally monitor their performance.
    Lifecycle management
    Create repeatable data science pipelines for model training and
    validation and integrate them with devops pipelines for delivery of
    models across your enterprise.
    Increased capabilities / collaboration
    Create projects and share them across teams. Combine Red Hat
    components, open source software, and ISV certified software.
    Now available as fully managed cloud
    service or traditional software product
    on-prem or in the cloud!

    View Slide

  11. 11
    What sets our approach apart
    Ease of Access
    Red Hat provides the “glue” to to
    the integration of open source
    tooling and tracks changes and
    fixes to open source AI/ML
    projects in order to enable
    customer access to upstream
    innovation
    Open Source
    Collaborate on a common
    platform to bring IT, data
    science and application
    development teams together
    with a unified platform for
    continuous delivery of
    Intelligent Applications
    Collaboration
    Simple enterprise cluster
    configuration and deployment
    on a secure and proven
    platform, that are provided on
    a self-service model, with
    autoscaling and FinOps
    capabilities
    Train, deploy and manage
    models in containerized
    format for intelligent apps
    consistently either
    on-premise or in cloud
    Hybrid Cloud
    Standardized platform for creating production AI/ML models, as well as running the resulting
    applications

    View Slide

  12. 12
    Unified Continuous Delivery for Intelligent Applications
    ML applications are a collaboration between Data
    Scientists, DevOps Engineers, ML Engineers, and
    Application Developers
    MLOps
    Foundational Capabilities for MLOps
    Models are built and deployed using CI/CD pipelines
    ensuring that they are Repeatable and Automated
    Solutions are GitOps Managed making git the primary
    source of truth to represent how the model is trained
    and deployed
    Metrics are captured across the entire ML lifecycle and
    made Observable to allow teams to make better
    decisions and understand the state of their applications
    ML Applications present new challenges and require
    teams to consider how to Secure their application from
    the start
    Repeatable and Automated
    GitOps Managed
    Collaboration
    Secure Observable

    View Slide

  13. data source
    data pipeline training pipeline
    Intelligent Application = ML Code + App Code
    13
    Machine Learning Lifecycle
    An end-to-end lifecycle from data ingestion to performance monitoring
    explore
    data
    preprocess
    data
    engineer
    features
    train/tune
    models
    validate
    models
    validate
    infrastructure
    push
    models
    preprocess
    data
    (INPUT)
    serve
    models
    prediction
    (OUTPUT)
    log,
    monitor,
    alert
    condition
    (OUTPUT)
    feature
    store
    model hub
    metadata
    store
    Physical* Virtual Private cloud Public cloud Edge
    server edge
    web / API
    inferencing
    (production)
    training
    (development)
    stream
    batch
    ingest
    data
    heuristics

    View Slide

  14. Define and Schedule ML Work on Demand
    Define Workloads
    Kubernetes/ OpenShift
    Queue Workloads Scale Up Resources Run Workloads
    Foundation Models need a lot more data and training
    So how do we distribute training? This is what an ideal workflow could look like
    Data Scientists need simple ways of
    defining, submitting and monitoring
    distributed workloads without worrying
    (too much) about cloud infrastructure
    Cluster Administrators need a way to
    enable this access to infrastructure while
    keeping access and cloud spend in check
    Challenges

    View Slide

  15. 15
    Foundation Models Architecture for Model Training and Fine-tuning
    The CodeFlare stack solves a number challenges faced by data scientists and administrators
    Sources:
    Red Hat blog “A guide to fine-tuning and serving AI/ML foundation models” June 2023
    Red Hat blog “AI/ML Models Batch Training at Scale with Open Data Hub” May 2023
    Data Scientist
    ● Ease of Use
    ● No code rewrites
    ● Scalability with Ray framework
    ● Safety via resource guarantees
    IT Administrator
    ● Simple deployment
    ● Resource management
    ● Cost management

    View Slide

  16. linkedin.com/company/red-hat
    youtube.com/user/RedHatVideos
    facebook.com/redhatinc
    twitter.com/RedHat
    Red Hat is the world’s leading provider of enterprise
    open source software solutions. Award-winning
    support, training, and consulting services make
    Red Hat a trusted adviser to the Fortune 500.
    Thank you
    16

    View Slide