Slide 1

Slide 1 text

From Catalog To Cloud: An AI Engineer’s Journey Nitya Narasimhan, PhD Senior AI Advocate, Microsoft Follow Me For AI Updates At: https://linkedin.com/in/nityan 2024 Community Days Keynote

Slide 2

Slide 2 text

Let me give you the Big Picture of AI Engineering! ž Motivation – The Rise of the AI Engineer ž Catalog – Understand Model Choice ž Code – Understand Development Workflow ž Cloud – From Prototype To Production ž Copilot – Learn By Deconstructing Samples

Slide 3

Slide 3 text

The Rise Of The AI Engineer ML Engineers know how to build language models Software Engineers know how to build production apps There is a knowledge and skills gap between these two roles

Slide 4

Slide 4 text

Does this app use generative AI? (Natural Language, Task Types) Want to build a custom copilot? Does this app have private data? (How do I ground responses?) What models do I need to use? (How do I find or select them?) What architecture should I use? (How do I orchestrate app flow?) What platform should I use? (Go from prompt to production)

Slide 5

Slide 5 text

I build foundation AI Models I ship AI Products “AI Engineering” bridges the skills gap between machine learning (models) and software engineering (products) Amanda Silver - CVP, Developer Division @Microsoft - Here's a brief overview of some insights that our amazing DevDiv User Research team has put together. • Model Selection • Prompt Engineering • Fine Tuning • Retrieval Augmented Generation • AI-Assisted Evaluation • End-to-End Development

Slide 6

Slide 6 text

The AI Engineer’s Journey – From Catalog To Cloud Improve response quality. How do I design prompts? Model Selection Prompt Engineering RAG Architecture Fine Tuning Model Evaluation Azure AI Platform Models drive your AI apps. How do you choose one? Ground response in my data. How do I orchestrate this? Retrain base model with my data for cost & capability. Define quality & safety metrics, AI-assisted flows Streamline development with unified tools & flows CATALOG CLOUD

Slide 7

Slide 7 text

Let’s Talk About Model Selection How do I make the right choice?

Slide 8

Slide 8 text

Oct 2024 | The Future Of AI Is Model Choice - Tech Community The Paradox of Choice – leads to decision fatigue The Developer Challenge – tradeoff quality and cost The Filter Process – reduce catalog to manageable subset

Slide 9

Slide 9 text

Oct 2024 | The Future Of AI Is Model Choice - Tech Community Pick a model that can do the task you want – or support fine-tuning to do it Then see if the precision works for you – or if you need specialization Now you need to make a decision on cost vs. quality The Azure AI Model Catalog can help! Demo: Azure AI Model Catalog

Slide 10

Slide 10 text

Let’s Talk About Development How can I improve the responses from my model choice in a way that lets me tradeoff cost and complexity as needed

Slide 11

Slide 11 text

Let’s Talk About Prompt Engineering User Prompt = the user question asked Model Prompt = the enhanced question send to the model Prompt Engineering is about adding context and instructions to user prompts to improve the quality of final response

Slide 12

Slide 12 text

May 2024 | Generative AI For Beginners - Chapter 4 System Context: define personas and behavior Primary Content: add related knowledge to ground responses Model Configuration: change models or their parameters for quality Prompt Template: have reusable prompts with placeholders for data – for popular tasks or application domains Demo: GitHub Model Marketplace

Slide 13

Slide 13 text

Let’s Talk About RAG Architecture

Slide 14

Slide 14 text

Sep 2024 | RAG With Azure AI Studio | Slides & Video Embedding Model: change text question into vectorized query Retrieval Service: find matching results in a relevant search index Semantic Ranking: put results in order that can fit query context best Prompt Templating: put retrieved knowledge in a prompt template to augment user question

Slide 15

Slide 15 text

Sep 2024 | RAG With Azure AI Studio | Slides & Video Embedding Model: change text question into vectorized query Retrieval Service: find matching results in a relevant search index Semantic Ranking: put results in order that can fit query context best Prompt Templating: put retrieved knowledge in a prompt template to augment user question

Slide 16

Slide 16 text

Let’s Talk About Fine Tuning Base Model = pre- trained on large data Fine-Tuned Variant = trades off quality or cost for complexity Fine Tuning is about retraining that model with your own data set to enhance quality or capabilities

Slide 17

Slide 17 text

May 2024 | Generative AI For Beginners - Chapter 28 Demo: Fine-Tuning With Open AI Concepts: what is fine- tuning & why use it? Process: how do we get started & what are the main challenges faced? Dataset Creation: how do we find data for this (own vs. synthetic) Fine-Tuning Workflow: run job, host variant, compare performance Find & Publish Variants: in community hubs

Slide 18

Slide 18 text

Let’s Talk About Evaluation How do I make the right choice?

Slide 19

Slide 19 text

Application Lifecycle (GenAIOps) Operationalizing Building/ augmenting Ideating/ exploring Managing BUSINESS NEED

Slide 20

Slide 20 text

Yes Yes 1. Ideating/exploring 2. Building/augmenting 3. Operationalizing

Slide 21

Slide 21 text

Let’s Talk About AI-Assisted Evaluation

Slide 22

Slide 22 text

Let’s Talk About Evaluator Metrics & Custom Evaluators for Quality Coherence: does the response make sense for the given question Fluency: is the response grammatically correct and well-written? Relevance: did given response answer the question asked? Groundedness: did the response use the data context we provided? Custom: define metric and its scoring criteria

Slide 23

Slide 23 text

Let’s Talk About Developer Tools How do I streamline my e2e workflow?

Slide 24

Slide 24 text

Let’s Talk About The Azure AI Platform DEMO: AI Apps Gallery Model catalog: go from discovery to evaluation to deployment in mins. Hub & Project: manage AI resources & monitor deployed applications with unified platform Developer Tools: from provisioning to deploy, with CLI, SDK support. Trustworthy AI: built-in safety systems, content filters & evaluators. AI Templates: sample apps for key scenarios

Slide 25

Slide 25 text

TRY IT: Contoso Chat Workshop Nov 2024 | Build a Custom Retail Copilot Code-First With Azure AI Star Repo and watch for updates during Microsoft Ignite Try The Workshop and register for an in-venue session at MSIgnite or on Microsoft AI Tour.

Slide 26

Slide 26 text

ž Microsoft Ignite – LAB 401 · THR 502 ž Microsoft AI Tour – WRK 550 · Track 5 ž Contoso Chat – Self-Guided Workshop ž #30DaysOfIA – Azure AI Week (5 Posts) ž Model Choice – The Future of AI Blog ž Generative AI For Beginners – Free Course Follow Me For AI Updates At: https://linkedin.com/in/nityan Want a copy of theses slides, resource links or high-resolution versions of the sketchnotes from this talk? Follow me on LinkedIn for a blog post! Thank You!

Slide 27

Slide 27 text

From Catalog To Cloud: An AI Engineer’s Journey Nitya Narasimhan, PhD Senior AI Advocate, Microsoft Follow Me For AI Updates At: https://linkedin.com/in/nityan THANK YOU FOR LISTENING!!