Azure OpenAI & private Large Language Models sicher deployen mit Terraform

Mit Azure OpenAI Service bietet Microsoft die Möglichkeit, trainierte Large Language Models (LLMs) – wie beispielsweise GPT-4 – in Anwendungen und Business-Prozesse zu integrieren. Doch jenseits der programmatischen Integration, und dem Anreichern mit eigenen Daten stellen sich wichtige und grundlegende Fragen, die es zu betrachten bzw. zu beantworten gilt: Wie deploye ich Azure OpenAI und die gewünschten LLMs deklarativ mit Tools wie HashiCorp Terraform? Kann ich Azure OpenAI in meine virtuelle, private Netzwerk-Infrastruktur integrieren? Kann ich Zugriffe auf das LLM limitieren und kontrollieren? Diese und weitere Fragen beantwortet Thorsten Hans in diesem Webinar anhand eines praktischen Beispiels, wodurch Sie sich in die Lage versetzen, LLMs mit Azure OpenAI sicher in bestehende oder neue Cloud-Infrastrukturen zu integrieren.

Transcript

1. Azure OpenAI & private LLMs sicher deployen mit Terraform Thorsten Hans @ThorstenHans Consultant
2. Consultant @ Thinktecture #Azure #Kubernetes #CloudNative #Docker thorsten.hans@thinktecture.com thinktecture.com thorsten-hans.com @ThorstenHans Microsoft MVP | Docker Captain Thorsten Hans
3. What we won’t cover • Terraform basics • You should know what Terraform is, how it works and how to describe infrastructure aspects using Terraform Configuration Language (previously known as HashiCorp Configuration Language) • Azure basics • You should know about essential building blocks e.g., Azure Virtual Network infrastructure, Azure Private Endpoints, … Non-Scope
4. What we won’t cover Non-Scope
5. • Introduction • Demo • Conclusion Agenda
6. • Introduction • Demo • Conclusion Agenda
7. Azure OpenAI Service is a platform-as-a-service (PaaS) offering allowing us – as customers- to run and interact with different large language models (LLMs) Introduction
8. We can deploy multiple large language models to a single instance of Azure OpenAI Service (some models are in private preview) Introduction
9. Azure OpenAI Service is built on top of Azure Resource Manager (ARM), which allows us to use tools like Terraform, Pulumi, or Project Bicep to deploy and mutate the service and model deployments Introduction
10. Terraform AzureRM Provide (azurerm) provides resource and data types for describing Azure OpenAI services and deployments: • azurerm_cognitive_account: Represents the PaaS service • azurerm_cognitive_deployment: Represents a LLM deployment Introduction
11. • Introduction • Demo • Conclusion Agenda
12. Overall Architecture
13. • Underlying Network Infrastructure (vNet) • Network Security Group (NSG) Demo
14. • Azure OpenAI Service and Model Deployment • Private DNS • Private Endpoint Demo
15. • Introduction • Demo • Conclusion Agenda
16. • Azure OpenAI Service can be integrated with new/existing Azure Virtual Network Infrastructure • With Azure Private Link, we can restrict access to Azure OpenAI Service to individual compute units also deployed into a Azure Virtual Network • Deployment and Mutation of Azure OpenAI Service and its model deployments works seamlessly in Project Bicep, Pulumi, and Terraform Conclusion
17. Thanks for your attention @ThorstenHans