Securing AI Apps on Azure: Using Keyless Auth with Azure AI Services Pamela Fox Python Cloud Advocacy @pamelafox Marlene Mhangami Python Cloud Advocacy @marlene_zw

Securing AI Apps on Azure Date Topic Speakers July 2 5-6PM UTC Using Keyless Auth with Azure AI Services Marlene Mhangami Pamela Fox July 8 5-6PM UTC Add User Login to AI Apps using Built-in Auth James Casey Pamela Fox July 9 7-8PM UTC Add User Login to AI Apps using MSAL SDK Ray Luo Pamela Fox July 10 7-8PM UTC Handling User Auth for a SPA App on Azure Matt Gotteiner July 17 7-8PM UTC Data Access Control for AI RAG Apps on Azure Matt Gotteiner Pamela Fox July 25 11PM-12PM Deploying an AI App to a Private Network on Azure Matt Gotteiner Anthony Shaw https://aka.ms/S-1355

Key-based authentication

Key-based authentication for OpenAI All services support key based authentication. The OpenAI SDK defaults to key-based, since that's all that OpenAI.com supports: openai_client = openai.OpenAI( api_key=os.getenv("AZURE_OPENAI_KEY") ) azure_client = openai.AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), api_key=os.getenv("AZURE_OPENAI_KEY") )

Key-based authentication for Azure services The Azure Python SDK also allows key-based credentials, using the KeyCredential class. from azure.core.credentials import AzureKeyCredential from azure.search.documents import SearchClient search_client = SearchClient( endpoint=os.environ["SEARCH_ENDPOINT"], index_name=os.environ["SEARCH_INDEX"], credential=AzureKeyCredential(os.environ["SEARCH_KEY"]) ) Example with AI Search client:

The risks of API keys • API keys can be easily leaked • API keys can be passed around a company (unintentionally) • API keys can be painful to rotate

Keyless authentication for local development

Keyless auth is based on OAuth2 Instead of a key, an OAuth2 token proves we can use a resource.

Steps for keyless authentication during local dev 1. Create the Azure service 2. Give your user permissions to use that service 3. Log in to Azure locally 4. Use the Azure Identity SDK to generate a token/provider and pass that along to the service SDK You Azure service Azure SDK

Keyless auth with Azure OpenAI You Azure OpenAI OpenAI SDK Example project: aka.ms/azai/keyless 1. Create the Azure OpenAI service 2. Give your user permissions to use that OpenAI service 3. Log in to Azure locally with the Azure Developer CLI 4. Use the Azure Identity SDK to generate a token provider and pass that along to the OpenAI SDK

Give permissions to Azure OpenAI in Azure Portal 1 2 3

Give permissions to Azure OpenAI with Azure CLI az role assignment create \ --role "5e0bd9bd-7b93-4f28-af87-19fc36ad61bd" \ --assignee-object-id "$PRINCIPAL_ID" \ --assignee-principal-type User \ --scope /subscriptions/"$SUBSCRIPTION_ID"/resourceGroups/"$RESOURCE_GROUP" az ad signed-in-user show --query id -o tsv Get your principal ID: Assign "Cognitive Services OpenAI User" to role for target resource group:

Give permissions to Azure OpenAI with Bicep module openAiRoleUser 'core/security/role.bicep' = { scope: openAiResourceGroup name: 'openai-role-user' params: { principalId: principalId roleDefinitionId: '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd' principalType: 'User' } } If you're using the Azure Developer CLI, add this to main.parameters.json: "principalId": { "value": "${AZURE_PRINCIPAL_ID}" }

Use Azure Identity with the OpenAI SDK • Use the azure-identity SDK to get a token provider for your identity • Pass the token provider to the OpenAI SDK • Token providers take care of token refresh for you azure_credential = DefaultAzureCredential() token_provider = get_bearer_token_provider(azure_credential, "https://cognitiveservices.azure.com/.default") client = AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_ad_token_provider=token_provider )

Keyless authentication in hosted Azure platforms

Keyless auth with Azure OpenAI in Azure Container Apps App Identity Azure OpenAI OpenAI SDK Example project: aka.ms/keyless-azure-containerapps 1. Create the Azure OpenAI service 2. Create the Azure Container App 3. Create an identity for the App to use 4. Give your App identity permissions to use the OpenAI service 5. Use the Azure Identity SDK (specifying app identity ID if needed) to generate a token provider and pass that along to the OpenAI SDK

App identity options: System vs. User-assigned Azure Container App System identity Azure OpenAI Azure Container App User-assigned identity Azure OpenAI OPTION 1 OPTION 2

Assign an identity to Container App in Azure Portal System identity User-assigned identity

Assign an identity to Container App with Bicep resource appIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { name: '${prefix}-id-aca' location: location } resource app 'Microsoft.App/containerApps@2022-03-01' = { name: name location: location identity: { type: 'UserAssigned' userAssignedIdentities: { '${appIdentity.id}': {} } } ... }

Give permissions to OpenAI with Bicep roleDefinitionId = '5e0bd9bd-7b93-4f28-af87-19fc36ad61bd' resource role 'Microsoft.Authorization/roleAssignments@2022-04-01' = { name: guid(subscription().id, resourceGroup().id, principalId, roleDefinitionId) properties: { principalId: appIdentity.properties.principalId principalType: 'ServicePrincipal' roleDefinitionId: resourceId( 'Microsoft.Authorization/roleDefinitions', roleDefinitionId) } } Assign "Cognitive Services OpenAI User" role to the app identity:

Use Azure Identity with the OpenAI SDK azure_credential = DefaultAzureCredential( managed_identity_client_id=os.getenv("APP_IDENTITY_ID")) token_provider = get_bearer_token_provider(azure_credential "https://cognitiveservices.azure.com/.default") client = AzureOpenAI( api_version="2024-02-15-preview", azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_ad_token_provider=token_provider ) If using a user-assigned identity, you either need to set AZURE_CLIENT_ID in environment or pass in the ID to DefaultAzureCredential:

Keyless authentication with Langchain

Using Langchain with keyless authentication from langchain_openai import AzureChatOpenAI token_provider = azure.identity.get_bearer_token_provider( azure.identity.DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default" ) llm = AzureChatOpenAI( azure_endpoint=os.getenv("AZURE_OPENAI_ENDPOINT"), azure_deployment=os.getenv("AZURE_OPENAI_DEPLOYMENT"), openai_api_version=os.getenv("AZURE_OPENAI_VERSION"), azure_ad_token_provider=token_provider, ) Example project: aka.ms/keyless-azure-langchain

Next steps

Start using keyless auth with Azure OpenAI! Sample Azure platform Language aka.ms/keyless-azure-containerapps Container Apps Python aka.ms/keyless-azure-langchain TBD Python aka.ms/ragchat App Service Python aka.ms/azai/js/code Container Apps JavaScript/TypeScript aka.ms/azai/net/code Container Apps C# https://aka.ms/azure-openai-keyless-guide Learn more (in your favorite language): Start from a working template:

Securing AI Apps on Azure Date Topic Speakers July 2 5-6PM UTC Using Keyless Auth with Azure AI Services Marlene Mhangami Pamela Fox July 8 5-6PM UTC Add User Login to AI Apps using Built-in Auth James Casey Pamela Fox July 9 7-8PM UTC Add User Login to AI Apps using MSAL SDK Ray Luo Pamela Fox July 10 7-8PM UTC Handling User Auth for a SPA App on Azure Matt Gotteiner July 17 7-8PM UTC Data Access Control for AI RAG Apps on Azure Matt Gotteiner Pamela Fox July 25 11PM-12PM Deploying an AI App to a Private Network on Azure Matt Gotteiner Anthony Shaw https://aka.ms/S-1355