Slide 1

Slide 1 text

Securing AI Apps on Azure: Using Keyless Auth with Azure AI Services Pamela Fox Python Cloud Advocacy @pamelafox Marlene Mhangami Python Cloud Advocacy @marlene_zw

Slide 2

Slide 2 text

Securing AI Apps on Azure Date Topic Speakers July 2 5-6PM UTC Using Keyless Auth with Azure AI Services Marlene Mhangami Pamela Fox July 8 5-6PM UTC Add User Login to AI Apps using Built-in Auth James Casey Pamela Fox July 9 7-8PM UTC Add User Login to AI Apps using MSAL SDK Ray Luo Pamela Fox July 10 7-8PM UTC Handling User Auth for a SPA App on Azure Matt Gotteiner July 17 7-8PM UTC Data Access Control for AI RAG Apps on Azure Matt Gotteiner Pamela Fox July 25 11PM-12PM Deploying an AI App to a Private Network on Azure Matt Gotteiner Anthony Shaw https://aka.ms/S-1355

Slide 3

Slide 3 text

Key-based authentication

Slide 4

Slide 4 text

Key-based authentication for OpenAI All services support key based authentication. The OpenAI SDK defaults to key-based, since that's all that OpenAI.com supports: openai_client = openai.OpenAI( api_key=os.getenv("AZURE_OPENAI_KEY") ) azure_client = openai.AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), api_key=os.getenv("AZURE_OPENAI_KEY") )

Slide 5

Slide 5 text

Key-based authentication for Azure services The Azure Python SDK also allows key-based credentials, using the KeyCredential class. from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient search_client = SearchClient( endpoint=os.environ["SEARCH_ENDPOINT"], index_name=os.environ["SEARCH_INDEX"], credential=AzureKeyCredential(os.environ["SEARCH_KEY"]) ) Example with AI Search client:

Slide 6

Slide 6 text

The risks of API keys • API keys can be easily leaked • API keys can be passed around a company (unintentionally) • API keys can be painful to rotate

Slide 7

Slide 7 text

Keyless authentication for local development

Slide 8

Slide 8 text

Keyless auth is based on OAuth2 Instead of a key, an OAuth2 token proves we can use a resource.

Slide 9

Slide 9 text

Steps for keyless authentication during local dev 1. Create the Azure service 2. Give your user permissions to use that service 3. Log in to Azure locally 4. Use the Azure Identity SDK to generate a token/provider and pass that along to the service SDK You Azure service Azure SDK

Slide 10

Slide 10 text

Keyless auth with Azure OpenAI You Azure OpenAI OpenAI SDK Example project: aka.ms/azai/keyless 1. Create the Azure OpenAI service 2. Give your user permissions to use that OpenAI service 3. Log in to Azure locally with the Azure Developer CLI 4. Use the Azure Identity SDK to generate a token provider and pass that along to the OpenAI SDK

Slide 11

Slide 11 text

Give permissions to Azure OpenAI in Azure Portal 1 2 3

Slide 12

Slide 12 text

Give permissions to Azure OpenAI with Azure CLI az role assignment create \ --role "5e0bd9bd-7b93-4f28-af87-19fc36ad61bd" \ --assignee-object-id "$PRINCIPAL_ID" \ --assignee-principal-type User \ --scope /subscriptions/"$SUBSCRIPTION_ID"/resourceGroups/"$RESOURCE_GROUP" az ad signed-in-user show --query id -o tsv Get your principal ID: Assign "Cognitive Services OpenAI User" to role for target resource group:

Slide 13

Slide 13 text

Give permissions to Azure OpenAI with Bicep module openAiRoleUser 'core/security/role.bicep' = { scope: openAiResourceGroup name: 'openai-role-user' params: { principalId: principalId roleDefinitionId: '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd' principalType: 'User' } } If you're using the Azure Developer CLI, add this to main.parameters.json: "principalId": { "value": "${AZURE_PRINCIPAL_ID}" }

Slide 14

Slide 14 text

Use Azure Identity with the OpenAI SDK • Use the azure-identity SDK to get a token provider for your identity • Pass the token provider to the OpenAI SDK • Token providers take care of token refresh for you azure_credential = DefaultAzureCredential() token_provider = get_bearer_token_provider(azure_credential, "https://cognitiveservices.azure.com/.default") client = AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_ad_token_provider=token_provider )

Slide 15

Slide 15 text

Keyless authentication in hosted Azure platforms

Slide 16

Slide 16 text

Keyless auth with Azure OpenAI in Azure Container Apps App Identity Azure OpenAI OpenAI SDK Example project: aka.ms/keyless-azure-containerapps 1. Create the Azure OpenAI service 2. Create the Azure Container App 3. Create an identity for the App to use 4. Give your App identity permissions to use the OpenAI service 5. Use the Azure Identity SDK (specifying app identity ID if needed) to generate a token provider and pass that along to the OpenAI SDK

Slide 17

Slide 17 text

App identity options: System vs. User-assigned Azure Container App System identity Azure OpenAI Azure Container App User-assigned identity Azure OpenAI OPTION 1 OPTION 2

Slide 18

Slide 18 text

Assign an identity to Container App in Azure Portal System identity User-assigned identity

Slide 19

Slide 19 text

Assign an identity to Container App with Bicep resource appIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { name: '${prefix}-id-aca' location: location } resource app 'Microsoft.App/containerApps@2022-03-01' = { name: name location: location identity: { type: 'UserAssigned' userAssignedIdentities: { '${appIdentity.id}': {} } } ... }

Slide 20

Slide 20 text

Give permissions to OpenAI with Bicep roleDefinitionId = '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd' resource role 'Microsoft.Authorization/roleAssignments@2022-04-01' = { name: guid(subscription().id, resourceGroup().id, principalId, roleDefinitionId) properties: { principalId: appIdentity.properties.principalId principalType: 'ServicePrincipal' roleDefinitionId: resourceId( 'Microsoft.Authorization/roleDefinitions', roleDefinitionId) } } Assign "Cognitive Services OpenAI User" role to the app identity:

Slide 21

Slide 21 text

Use Azure Identity with the OpenAI SDK azure_credential = DefaultAzureCredential( managed_identity_client_id=os.getenv("APP_IDENTITY_ID")) token_provider = get_bearer_token_provider(azure_credential "https://cognitiveservices.azure.com/.default") client = AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_ad_token_provider=token_provider ) If using a user-assigned identity, you either need to set AZURE_CLIENT_ID in environment or pass in the ID to DefaultAzureCredential:

Slide 22

Slide 22 text

Keyless authentication with Langchain

Slide 23

Slide 23 text

Using Langchain with keyless authentication from langchain_openai import AzureChatOpenAI token_provider = azure.identity.get_bearer_token_provider( azure.identity.DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default" ) llm = AzureChatOpenAI( azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_deployment=os.getenv("AZURE_OPENAI_DEPLOYMENT"), openai_api_version=os.getenv("AZURE_OPENAI_VERSION"), azure_ad_token_provider=token_provider, ) Example project: aka.ms/keyless-azure-langchain

Slide 24

Slide 24 text

Next steps

Slide 25

Slide 25 text

Start using keyless auth with Azure OpenAI! Sample Azure platform Language aka.ms/keyless-azure-containerapps Container Apps Python aka.ms/keyless-azure-langchain TBD Python aka.ms/ragchat App Service Python aka.ms/azai/js/code Container Apps JavaScript/TypeScript aka.ms/azai/net/code Container Apps C# https://aka.ms/azure-openai-keyless-guide Learn more (in your favorite language): Start from a working template:

Slide 26

Slide 26 text

Securing AI Apps on Azure Date Topic Speakers July 2 5-6PM UTC Using Keyless Auth with Azure AI Services Marlene Mhangami Pamela Fox July 8 5-6PM UTC Add User Login to AI Apps using Built-in Auth James Casey Pamela Fox July 9 7-8PM UTC Add User Login to AI Apps using MSAL SDK Ray Luo Pamela Fox July 10 7-8PM UTC Handling User Auth for a SPA App on Azure Matt Gotteiner July 17 7-8PM UTC Data Access Control for AI RAG Apps on Azure Matt Gotteiner Pamela Fox July 25 11PM-12PM Deploying an AI App to a Private Network on Azure Matt Gotteiner Anthony Shaw https://aka.ms/S-1355