A Journey into Open- Source Hugging Face Models on Azure AI Nitya Narasimhan, Senior AI Advocate, Microsoft Thasmika Gokal, Product Manager (AI), Microsoft

Visual Studio Copilot Studio GitHub Azure AI Foundry SDK Azure AI Foundry Model Catalog Open-source models Foundational models Task models Industry models Azure AI Content Safety Azure AI Search Azure AI Agent Service Azure OpenAI Service Observability Customization Evaluations Governance Monitoring Announcing

Models are at the heart of AI applications!

ADVANCE PROJECT REVERT PROJECT Capacity and cost trade offs Monitoring Security and data privacy Operationalize Content Filtering ADVANCE PROJECT Model selection Techniqu es - Prompt Engineering, RAG, Fin e-tuning Optimize REVERT PROJECT Pick a flagship model Define use cases Build prototypes Establish feasibility Prototype Can I scale for real-world workloads? Can AI solve my use case? How do I select the “right” model for my use case? Model Selection Requires a Structured Approach

Azure AI Model Catalog Offers The Best Selection Flagship LLMs • GPT-4 • Mistral Large • Llama3 70b • Llama 405b • Command R+ Small language models • Phi3 • Mistral OSS models • Llama3 8b • Ministral LLMs and SLMs Multi-modal • GPT-4o, Phi3-vision Image generation • DALL·E 3, Stability AI Embedding models • Ada, Cohere • Function calling & JSON support Modalities, tasks, and tools • Core42 JAIS Arabic language LLM • Mistral Large is focused on European languages • Nixtla TimeGEN-1 Timeseries forecasting Regional and domain specific Premium models first on Azure: OpenAI, Mistral Large, Cohere Command R+ 1.6K+ open models from Hugging Face Open models from Meta, Databricks, Snowflake and NVIDIA Open and proprietary Microsoft Phi Azure OpenAI Mistral AI Meta AI Cohere Hugging Face Deci AI Databricks NVIDIA Nixtla G42 Snowflake

And it meets key enterprise requirements for usage Data privacy Security & Compliance Responsible AI & content safety Microsoft Phi Azure OpenAI Mistral AI Meta AI Cohere Hugging Face Deci AI Databricks NVIDIA Nixtla G42 Snowflake

© Copyright Microsoft Corporation. All rights reserved. Can I SOLVE my use case with AI? I want to - use an open-source model from HF - use a model suitable for healthcare - use a model to help me understand SQL - use a biomed model for text generation - use an SLM (didn’t HF build one?) - use a model for an inference task “X” This is a DISCOVERY problem. Can I find a model that can do the task required by my application

© Copyright Microsoft Corporation. All rights reserved. Can I SELECT the right model for fit? I want to - Understand the model capabilities - Know where the model ranks against others - Deploy the model to Azure - Validate the model with a simple test question This is a DEVELOPMENT problem. Once I narrow down my selection, how do I check if that model works for my app requirements?

© Copyright Microsoft Corporation. All rights reserved. Can I SCALE the model in production? I want to - Deploy and access the model from code - Integrate the model within rich architectures - Build, evaluate, and deploy, rich apps with the model! This is a DELIVERY problem of going from catalog – to code – to cloud. How do I build complex architectures with diverse models?

© Copyright Microsoft Corporation. All rights reserved. Can I SCALE the model in production? I want to - Build an interactive retail chat app - I want to develop this in Python - And deploy with Azure Container Apps - And use open-source models Start from an existing template Deconstruct it – then remix it! This is also a DEVELOPER TOOLS story. Can I remix reference samples?

© Copyright Microsoft Corporation. All rights reserved.