Community Days NYC 2024 Keynote Copiloting Your Developer Journey with Azure AI Nitya Narasimhan, PhD Senior AI Advocate, Microsoft @nitya | #in/nityan

Hi, I’m Nitya! PhD & Polyglot (Mobile · Web · Cloud · AI) Research & Development (25+ years) Community & Education (15+ years) Illustrator & Storyteller (10+ years) Cloud Advocacy & Microsoft (5+ years) @nitya | #in/nityan | github/nitya

I’m Developer 0 For Azure AI Product Feedback (Focus on Developer Experience) Content Creation (Focus on Developer Education) Sample Creation (Focus on Developer Productivity) Community Interactions (Focus on Developers) @nitya | #in/nityan | github/nitya Join the Azure AI Discord

How do you see yourself? AI Beginner (How to I skill up on AI fundamentals!) Low-Code Dev (How do I get productive with AI?) Pro-Dev (How can I get control over e2e workflows?) Entrepreneur (How can I prototype my idea quickly?) We’re All On A Journey! @nitya | #in/nityan | github/nitya

Keeping up with Tech is .. Challenging https://aka.ms/bookofnews

Can I help you to see the big picture? @SketchTheDocs

I love to share my insights in Generative AI Responsible AI Agentic AI Careers

I love to build & empower local communities .. https://meetup.com/aide-hub

Join me on my AI Adventure! Copiloting Your Developer Journey With Azure AI 1 | Generative AI – Prompt Engineering 2 | LLM Ops– From Prompt To Production 3 | Copilot Stack - Developer Tools & Workflow 4 | Contoso Chat – Retail Copilot OSS Sample 5 | Responsible AI – From Evaluation to Safety| 6 | Agentic AI – From Assistants to AutoGen 7 | Hugging Face – From Open Source to Ops 8 | Resources – Discord, Collections & More

Generative AI = How Prompts Work PROMPT user request received as a text input Open AI GPT-3.5-turbo RESPONSE generated content returned as user response LANGUAGE MODEL 4097 Tokens Trained to Sep 2021 | Demo – Open AI Tokenizer

Generative AI = What Prompts Do PROMPT user request received as a text input Open AI GPT-3.5-turbo RESPONSE generated content returned as user response LANGUAGE MODEL 4097 Tokens Trained to Sep 2021 | Demo – Hugging Chat (Elementary)

Generative AI = What Prompts Do PROMPT user request received as a text input Open AI GPT-3.5-turbo RESPONSE generated content returned as user response LANGUAGE MODEL 4097 Tokens Trained to Sep 2021 | Demo – Hugging Spaces (AI Comics)

Prompt Engineering = Make it Better! Large Language Model Model Parameters System Context USER PROMPT CHAT HISTORY MODEL PROMPT MODEL RESPONSE PROMPT ENGINEERING VARIABLES RAG DATA PROMPT TEMPLATE | Demo – Open AI (Fine Tuning)

Generative AI For Beginners – Chapter 4 | https://aka.ms/genai-beginners

Generative AI For Beginners – Chapter 18 | https://aka.ms/genai-beginners

Challenge: Grounding Responses In My Data Large Language Model Model Parameters System Context USER PROMPT CHAT HISTORY MODEL PROMPT MODEL RESPONSE PROMPT ENGINEERING VARIABLES RAG DATA PROMPT TEMPLATE | Demo – Hugging Chat (“Hiking Shoes”)

Copilot Stack: Microsoft Runs on Azure AI | Demo – Azure AI Models Hub

Contoso Chat: A RAG-based Retail Copilot Contoso Web: Frontend A web app for user interactions. It is a client for the chat AI API. Contoso Chat: Backend A generative AI app that takes user query & returns response

Experience: Grounded, Personalized, Responsible AI • Grounded in Your Data. Ask a question, verify that answer uses product data. • Validate the Copilot. Ask a question and verify that you get an answer. • Personalized to User. Ask a question, verify if answer uses purchase history. • Default Safety Checks. Try simple jailbreaking to modify rules. It should fail. | Demo – Contoso Chat (“Chat UI”)

Architecture: Retrieval Augmented Generation

App Lifecycle E2E: Prompt Engineering to LLM Ops

Code-First Platform: Azure AI Studio (SDK & Portal)

Workshop: { Build – Evaluate – Deploy – Use }

| Try It Out!  Pre-Reqs: GitHub, Azure, AOAI, Quota  Visit - https://aka.ms/aitour/contoso-chat  Fork it – uncheck box so you get branches  Launch Codespaces – wait till it is ready  Validate setup – azd version  Authenticate – azd auth login  Provision & Deploy – azd up

https://prompty.ai  “Agency with Observability”  New asset class – versionable, actionable  Specification – language agnostic templates  Tooling – developer experience (IDE as playground)  Runtime – engine agnostic execution “It just works” with Azure AI Studio | Demo – Prompty Extension (“Playground In Your IDE”)

| Demo – Flex Flow (“Bring Your Own Framework or Code”) https://aka.ms/promptflow

https://aka.ms/ai-studio/intelligent-apps | 5-part Blog Series Responsible AI: Evaluation Flows & Safety Filters

| Demo – AZD Template Gallery https://aka.ms/ai-studio/azd-templates

If Time Permits 1 | Generative AI – Prompt Engineering 2 | LLM Ops– From Prompt To Production ( 3 | Copilot Stack - Developer Tools & Workflow 4 | Contoso Chat – Retail Copilot OSS Sample 5 | Responsible AI – From Evaluation to Safety| 6 | Agentic AI – From Assistants to AutoGen 7 | Hugging Face – From Open Source to Ops 8 | Resources – Discord, Collections & More See Microsoft Build Session For More Details

Building Generative AI Solutions Can be .. Challenging Large Language Models for rich multi-media content generation Conversational Interactions driven by natural language processing capabilities End-to-End Workflows from prompt-engineering to operationalization Chat-based experiences require human guidance (prompts) and intervention (instructions) to execute complex tasks, creating challenges in workflow automation and optimization

Agentic AI Application Development Can Help Autonomous Agents capable of planning & executing decisions Task Execution Tools identify and execute the right tools for each task Conversational Workflows coordinate actions across agents, user, environment Agentic AI Applications use autonomous agents to execute tasks on behalf of users, interacting with their environment or remote services as needed, and coordinating actions with other agents for efficiency

Building Agentic AI requires new tools & mindset Conversational Agents configurable (LLM) customizable (prompt) chainable (planning) Execution Workflows infer required skill execute relevant tool behave responsibly Conversational Patterns agent-human agent-environment agent-agent Traditional Agentic Application Frameworks are not versed in natural language. We need conversational design patterns and intelligent task inference and execution for generative AI usage scenarios

AutoGen Framework: Multi-Agent Collaboration Open-Source Framework & Samples Docs: https://aka.ms/autogen/website Discord: https://aka.ms/autogen/discord Customizable Conversable Agents, LLMs Research-Driven Tools & Patterns No-Code and Code-First Development

AutoGen Studio: Get Started With Low-Code UI/UX Define Skills Create reusable functions, tools Docs: https://microsoft.github.io/autogen/blog/2023/12/01/AutoGenStudio/ Define Models Define & configure required LLMs Define Workflows Create agents, multi-agent conversations Create Sessions Test and validate agent workflows Define Agents Configure LLM, skills, behaviors Publish Sessions Share sessions to a gallery to revisit

Your Turn! 1 | Generative AI – Prompt Engineering 2 | LLM Ops– From Prompt To Production ( 3 | Copilot Stack - Developer Tools & Workflow 4 | Contoso Chat – Retail Copilot OSS Sample 5 | Responsible AI – From Evaluation to Safety| 6 | Agentic AI – From Assistants to AutoGen 7 | Hugging Face – From Open Source to Ops 8 | Resources – Discord, Collections & More Look For Slides Soon!

Join The Discord 1 | https://aka.ms/genai-beginners 2 | https://aka.ms/ai-studio/collection 3 | https://aka.ms/rai-hub/collection 4 | https://aka.ms/aitour/contoso-chat 5 | https://aka.ms/ai-studio/intelligent-apps 6 | https://aka.ms/ai-studio/code-first-blog 7 | https://aka.ms/ai-studio/azd-templates 8 | Find Me – @nitya #in/nityan github/nitya Look For Slides Soon!

© Copyright Microsoft Corporation. All rights reserved. Thank You!