Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Azure OpenAI & private Large Language Models sicher deployen mit Terraform

Azure OpenAI & private Large Language Models sicher deployen mit Terraform

Mit Azure OpenAI Service bietet Microsoft die Möglichkeit, trainierte Large Language Models (LLMs) – wie beispielsweise GPT-4 – in Anwendungen und Business-Prozesse zu integrieren. Doch jenseits der programmatischen Integration, und dem Anreichern mit eigenen Daten stellen sich wichtige und grundlegende Fragen, die es zu betrachten bzw. zu beantworten gilt: Wie deploye ich Azure OpenAI und die gewünschten LLMs deklarativ mit Tools wie HashiCorp Terraform? Kann ich Azure OpenAI in meine virtuelle, private Netzwerk-Infrastruktur integrieren? Kann ich Zugriffe auf das LLM limitieren und kontrollieren? Diese und weitere Fragen beantwortet Thorsten Hans in diesem Webinar anhand eines praktischen Beispiels, wodurch Sie sich in die Lage versetzen, LLMs mit Azure OpenAI sicher in bestehende oder neue Cloud-Infrastrukturen zu integrieren.

Thorsten Hans

October 18, 2023
Tweet

More Decks by Thorsten Hans

Other Decks in Technology

Transcript

  1. Azure OpenAI & private LLMs
    sicher deployen mit Terraform
    Thorsten Hans
    @ThorstenHans
    Consultant

    View full-size slide

  2. Consultant @ Thinktecture
    #Azure #Kubernetes
    #CloudNative #Docker
    [email protected]
    thinktecture.com
    thorsten-hans.com
    @ThorstenHans
    Microsoft MVP | Docker Captain
    Thorsten Hans

    View full-size slide

  3. What we won’t cover
    • Terraform basics
    • You should know what Terraform is, how it works and how to describe
    infrastructure aspects using Terraform Configuration Language (previously
    known as HashiCorp Configuration Language)
    • Azure basics
    • You should know about essential building blocks e.g., Azure Virtual Network
    infrastructure, Azure Private Endpoints, …
    Non-Scope

    View full-size slide

  4. What we won’t cover
    Non-Scope

    View full-size slide

  5. • Introduction
    • Demo
    • Conclusion
    Agenda

    View full-size slide

  6. • Introduction
    • Demo
    • Conclusion
    Agenda

    View full-size slide

  7. Azure OpenAI Service is a platform-as-a-service (PaaS) offering
    allowing us – as customers- to run and interact with different
    large language models (LLMs)
    Introduction

    View full-size slide

  8. We can deploy multiple large language models to a single instance of
    Azure OpenAI Service
    (some models are in private preview)
    Introduction

    View full-size slide

  9. Azure OpenAI Service is built on top of Azure Resource Manager (ARM),
    which allows us to use tools like
    Terraform, Pulumi, or Project Bicep
    to deploy and mutate the service and model deployments
    Introduction

    View full-size slide

  10. Terraform AzureRM Provide (azurerm) provides resource and data types
    for describing Azure OpenAI services and deployments:
    • azurerm_cognitive_account: Represents the PaaS service
    • azurerm_cognitive_deployment: Represents a LLM deployment
    Introduction

    View full-size slide

  11. • Introduction
    • Demo
    • Conclusion
    Agenda

    View full-size slide

  12. Overall Architecture

    View full-size slide

  13. • Underlying Network Infrastructure (vNet)
    • Network Security Group (NSG)
    Demo

    View full-size slide

  14. • Azure OpenAI Service and Model Deployment
    • Private DNS
    • Private Endpoint
    Demo

    View full-size slide

  15. • Introduction
    • Demo
    • Conclusion
    Agenda

    View full-size slide

  16. • Azure OpenAI Service can be integrated with new/existing Azure Virtual Network Infrastructure
    • With Azure Private Link, we can restrict access to Azure OpenAI Service to individual compute units also
    deployed into a Azure Virtual Network
    • Deployment and Mutation of Azure OpenAI Service and its model deployments works seamlessly in
    Project Bicep, Pulumi, and Terraform
    Conclusion

    View full-size slide

  17. Thanks for your attention
    @ThorstenHans

    View full-size slide