Azure OpenAI & private LLMs sicher deployen mit Terraform Thorsten Hans @ThorstenHans Consultant

Consultant @ Thinktecture #Azure #Kubernetes #CloudNative #Docker [email protected] thinktecture.com thorsten-hans.com @ThorstenHans Microsoft MVP | Docker Captain Thorsten Hans

What we won’t cover • Terraform basics • You should know what Terraform is, how it works and how to describe infrastructure aspects using Terraform Configuration Language (previously known as HashiCorp Configuration Language) • Azure basics • You should know about essential building blocks e.g., Azure Virtual Network infrastructure, Azure Private Endpoints, … Non-Scope

What we won’t cover Non-Scope

• Introduction • Demo • Conclusion Agenda

• Introduction • Demo • Conclusion Agenda

Azure OpenAI Service is a platform-as-a-service (PaaS) offering allowing us – as customers- to run and interact with different large language models (LLMs) Introduction

We can deploy multiple large language models to a single instance of Azure OpenAI Service (some models are in private preview) Introduction

Azure OpenAI Service is built on top of Azure Resource Manager (ARM), which allows us to use tools like Terraform, Pulumi, or Project Bicep to deploy and mutate the service and model deployments Introduction

Terraform AzureRM Provide (azurerm) provides resource and data types for describing Azure OpenAI services and deployments: • azurerm_cognitive_account: Represents the PaaS service • azurerm_cognitive_deployment: Represents a LLM deployment Introduction

• Introduction • Demo • Conclusion Agenda

Overall Architecture

• Underlying Network Infrastructure (vNet) • Network Security Group (NSG) Demo

• Azure OpenAI Service and Model Deployment • Private DNS • Private Endpoint Demo

• Introduction • Demo • Conclusion Agenda

• Azure OpenAI Service can be integrated with new/existing Azure Virtual Network Infrastructure • With Azure Private Link, we can restrict access to Azure OpenAI Service to individual compute units also deployed into a Azure Virtual Network • Deployment and Mutation of Azure OpenAI Service and its model deployments works seamlessly in Project Bicep, Pulumi, and Terraform Conclusion

Thanks for your attention @ThorstenHans