Slide 1

Slide 1 text

Azure OpenAI & private LLMs sicher deployen mit Terraform Thorsten Hans @ThorstenHans Consultant

Slide 2

Slide 2 text

Consultant @ Thinktecture #Azure #Kubernetes #CloudNative #Docker [email protected] @ThorstenHans Microsoft MVP | Docker Captain Thorsten Hans

Slide 3

Slide 3 text

What we won’t cover • Terraform basics • You should know what Terraform is, how it works and how to describe infrastructure aspects using Terraform Configuration Language (previously known as HashiCorp Configuration Language) • Azure basics • You should know about essential building blocks e.g., Azure Virtual Network infrastructure, Azure Private Endpoints, … Non-Scope

Slide 4

Slide 4 text

What we won’t cover Non-Scope

Slide 5

Slide 5 text

• Introduction • Demo • Conclusion Agenda

Slide 6

Slide 6 text

• Introduction • Demo • Conclusion Agenda

Slide 7

Slide 7 text

Azure OpenAI Service is a platform-as-a-service (PaaS) offering allowing us – as customers- to run and interact with different large language models (LLMs) Introduction

Slide 8

Slide 8 text

We can deploy multiple large language models to a single instance of Azure OpenAI Service (some models are in private preview) Introduction

Slide 9

Slide 9 text

Azure OpenAI Service is built on top of Azure Resource Manager (ARM), which allows us to use tools like Terraform, Pulumi, or Project Bicep to deploy and mutate the service and model deployments Introduction

Slide 10

Slide 10 text

Terraform AzureRM Provide (azurerm) provides resource and data types for describing Azure OpenAI services and deployments: • azurerm_cognitive_account: Represents the PaaS service • azurerm_cognitive_deployment: Represents a LLM deployment Introduction

Slide 11

Slide 11 text

• Introduction • Demo • Conclusion Agenda

Slide 12

Slide 12 text

Overall Architecture

Slide 13

Slide 13 text

• Underlying Network Infrastructure (vNet) • Network Security Group (NSG) Demo

Slide 14

Slide 14 text

• Azure OpenAI Service and Model Deployment • Private DNS • Private Endpoint Demo

Slide 15

Slide 15 text

• Introduction • Demo • Conclusion Agenda

Slide 16

Slide 16 text

• Azure OpenAI Service can be integrated with new/existing Azure Virtual Network Infrastructure • With Azure Private Link, we can restrict access to Azure OpenAI Service to individual compute units also deployed into a Azure Virtual Network • Deployment and Mutation of Azure OpenAI Service and its model deployments works seamlessly in Project Bicep, Pulumi, and Terraform Conclusion

Slide 17

Slide 17 text

Thanks for your attention @ThorstenHans