Microsoft Build 2024 Updates

Slide 1

Slide 1 text

Microsoft Build 2024 Updates SATO Naoki (Neo) Senior Software Engineer Microsoft Corporation

Slide 2

Slide 2 text

The age of AI transformation

Slide 3

Slide 3 text

Scaling laws Training compute of notable machine learning systems over time (FLOP) 1e+2 1e+10 1e+18 1e+26 1950 2010 Today Deep learning era Pre-deep learning era 1.4x/year 4.1x/year Source: Epoch, ‘Parameter, Compute and Data Trends in Machine Learning’

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

Indian teachers are drafting better lesson plans India Switzerland Amid healthcare crunch, AI apps provide more time for patients Indonesia To keep fish and shrimp healthy, farmers now have a copilot to help aka.ms/BuildAIStories

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

Microsoft Copilot Copilot+ PC Copilot stack

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Windows Copilot Runtime Applications + Experiences Windows Copilot Library + On-device models AI frameworks + Toolchains Silicon

Slide 10

Slide 10 text

Applications + Experiences AI frameworks + Toolchains Windows Copilot Library + On-device models Coming June Windows Studio Effects Live Captions Translations Recall User Activity OCR Phi-Silica

Slide 11

Slide 11 text

Applications + Experiences AI frameworks + Toolchains Windows Copilot Library + On-device models Announcing Phi-Silica SOTA SLM included in every Copilot+ PC

Slide 12

Slide 12 text

Applications + Experiences AI frameworks + Toolchains Windows Copilot Library + On-device models Coming later this year RAG Vector Embeddings Text Summarization

Slide 13

Slide 13 text

AI frameworks + Toolchains Developer preview PyTorch now native on Windows powered by DirectML Developer preview Web Neural Network on Windows powered by DirectML Windows Copilot Library + On-device models

Slide 14

Slide 14 text

udio lets developers responsibly build and deploy custom copilots Azure AI Search features search relevance updates and new integrations N Service boosts performance and security for web app creation Azure Functions launches Flex Consumption Plan, extensions Azure Static W Khan Academy and Microsoft announce partnership Azure Compute Fleet now in preview Microsoft Copilot in Azure preview open to all Azure Logic Apps Announcing custom generative mode in preview soon New Azure virtual machine series optimized for AI and cloud-native pabilities in Azure Database for PostgreSQL Azure API Center and generative AI capabilities in Azure API Management now available Visual odel to Pulumi in Azure Deployment Environments Azure Service Bus updates now in preview Microsoft for Startups Founders Hub gains AI c work 2.0 now in preview SharePoint Embedded now generally available New enhancements for custom app experiences connected to Micro wered features and enhanced data protection in Microsoft Teams Premium New features in Microsoft Dataverse in preview Power Apps upd Improvements to the developer experience in Windows Wide array of Windows Developer AI updates Windows Security additions an or Visual Studio Code now in preview Microsoft adds multimodal Phi-3 model Phi-3-vision Real-time video translation in Microsoft Edge com 50+ new updates

Slide 15

Slide 15 text

Copilot stack Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge

Slide 16

Slide 16 text

Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge

Slide 17

Slide 17 text

60+ Azure regions

Slide 18

Slide 18 text

France Largest investment to date to accelerate the adoption of AI, skilling, and innovation Wisconsin $3.3B investment to spur AI innovation and economic growth Mexico First hyper-scale cloud datacenter region Malaysia US$2.2B investme to fuel cloud and AI transformation

Slide 19

Slide 19 text

100% of energy from zero-carbon sources by 2025

Slide 20

Slide 20 text

Microsoft Azure Datacenter CPU AI accelerator IO Network End-to-end systems optimization

Slide 21

Slide 21 text

#1 supercomputer for cloud Microsoft Azure 561.20 PFlop/s TOP500, May 2024

Slide 22

Slide 22 text

November 2023 May 2024 30x Microsoft Azure supercomputers

Slide 23

Slide 23 text

Best AI performance. Lowest cost. Microsoft Copilot Your copilot apps Azure AI

Slide 24

Slide 24 text

March 2023 November 2023 May 2024 GPT-4 GPT-4 Turbo GPT-4o ⅙ ⅟₁₂ Cost Speed 2x 6x

Slide 25

Slide 25 text

Full-stack partnership aka.ms/NVIDIAPartnership

Slide 26

Slide 26 text

Full-stack partnership NVIDIA Blackwell Platform NVIDIA NIM-optimized models NVIDIA Omniverse, NVIDIA DGX Cloud + Fabric Windows 365 and Copilot GPU acceleration Azure AI Platform Microsoft Apps NVIDIA on Azure Microsoft Azure aka.ms/NVIDIAPartnership

Slide 27

Slide 27 text

Expanded partnership aka.ms/AMDPartnership

Slide 28

Slide 28 text

General availability ND MI300X V5 Leading price/performance on GPT-4 Optimized for Microsoft Azure workloads aka.ms/AMDPartnership

Slide 29

Slide 29 text

Updates Azure Maia Integrated system of silicon, racks, and software Designed for Azure OpenAI workloads aka.ms/AzureMaia

Slide 30

Slide 30 text

Public preview Azure Cobalt Up to 40% higher performance Most power-efficient compute offering in Microsoft Azure aka.ms/AzureCobalt

Slide 31

Slide 31 text

Azure Cobalt Customers aka.ms/AzureCobalt

Slide 32

Slide 32 text

Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge

Slide 33

Slide 33 text

Azure AI

Slide 34

Slide 34 text

Azure AI 50,000+ organizations

Slide 35

Slide 35 text

Microsoft OpenAI aka.ms/OpenAI

Slide 36

Slide 36 text

GPT-4o GPT-4o achieves state-of-the-art performance on visual perception benchmarks 100% 0 50% MathVista AI2D ChartQA DocVQA ActivityNet EgoSchema MMMU GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro Claude Opus GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro GPT-4o Gemini 1.0 Ultra Gemini 1.5 Pro

Slide 37

Slide 37 text

Azure OpenAI Service GPT-4 Turbo with Vision GPT-4 GPT-3.5 Turbo DALL·E 3 Whisper aka.ms/AOAIUpdates

Slide 38

Slide 38 text

Azure OpenAI Service GPT-4o Generally available Multimodal input and output Real-time response Safe by design to-speech models Preview aka.ms/AOAIUpdates

Slide 39

Slide 39 text

https://azure.microsoft.com/en-us/blog/openais-fastest-model-gpt-4o-mini-is-now-available-on-azure-ai/

Slide 40

Slide 40 text

Best selection of models Jan 2024 Feb Mar Apr May Deci Mistral-large Databricks DBRX Cohere Command R+ Cohere Embed Meta Llama 3 Snowflake Arctic Phi-3-mini Mistral-small Phi-3-small Phi-3-medium Phi-3-vision

Slide 41

Slide 41 text

New Models-as-a-Service Core42 JAIS New Nixtla TimeGEN-1 New Bria AI Coming soon Gretel Coming soon NTT DATA Coming soon Stability AI Coming soon AI21 Coming soon Cohere Rerank Coming soon aka.ms/ModelCatalog

Slide 42

Slide 42 text

https://azure.microsoft.com/en-us/blog/announcing-phi-3-fine-tuning-new-generative-ai-models- and-other-azure-ai-updates-to-empower-organizations-to-customize-and-scale-ai-applications/ • Serverless fine-tuning for Phi-3-mini and Phi-3-medium models • Updates to Phi-3-mini including significant improvement in core quality, instruction-following, and structured output, • Same day shipping of the latest models from OpenAI (GPT-4o mini), Meta (Llama 3.1 405B), Mistral (Large 2)

Slide 43

Slide 43 text

Microsoft Open source

Slide 44

Slide 44 text

Expanded partnership aka.ms/ExtendedHF

Slide 45

Slide 45 text

aka.ms/ExtendedHF Hugging Face expanded partnership Expanded partnership Deeper integration with Azure AI Studio Additional models coming to Azure AI TGI enabled for optimized runtime

Slide 46

Slide 46 text

Microsoft SLMs

Slide 47

Slide 47 text

Phi-3 Quality vs. size in SLM Model quality measured on MMLU benchmark 0 4 8 28 12 16 20 24 Phi-3-small Phi-3-mini-4k Phi-3-medium Llama-3-8B-In Mixtral 8x7B Gemma 7B Mistral 7B 80 75 65 60 70 Size in billions of parameters (active) Model quality Phi-3-mini-128k Gemma 2

Slide 48

Slide 48 text

Phi-3 Phi-3-mini 3.8B parameters Available on MaaS Phi-3-vision 4.2B parameters New Phi-3-small 7B parameters New Phi-3-medium 14B parameters Available on MaaS New aka.ms/Phi3

Slide 49

Slide 49 text

aka.ms/Phi3 Phi-3 Phi-3-vision 4.2B parameters New Multimodal model with language and vision capabilities Reason over real-world images Optimized for chart and diagram understanding 3-mini arameters e on MaaS Phi-3-s 7B param New

Slide 50

Slide 50 text

aka.ms/Phi3 Phi-3 Phi-3-vision 4.3B parameters Optimized for chart and diagram understanding Reason over real-world images Multimodal model with language and vision capabilities 3-mini arameters e on MaaS Phi-3-s 7B param New New

Slide 51

Slide 51 text

Phi-3 Cloud Local AI PC • Mobile • Edge (iOS, Android)

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

Announcing aka.ms/KhanAcademy

Slide 54

Slide 54 text

Announcing Building the future of education with Khan Academy Providing free access to Khanmigo for teachers in the US Democratizing AI-powered math tutoring with Phi-3 Supporting learners with high-quality education experiences aka.ms/KhanAcademy

Slide 55

Slide 55 text

Azure AI Studio A comprehensive platform to develop and deploy custom copilots

Slide 56

Slide 56 text

Generally available Azure AI Studio API and model choice Complete AI toolchain Responsible AI tools & practices Enterprise-grade production at scale AI.Azure.com

Slide 57

Slide 57 text

No content

Slide 58

Slide 58 text

Announcing Azure AI Content Safety Coming soon Custom categories Preview Prompt shields Preview Groundedness detection aka.ms/ContentSafetyUpdates

Slide 59

Slide 59 text

Azure AI custom models Least customized Prompt engineering Retrieval-Augmented Generation Announcing Co-innovation with Microsoft Most customized

Slide 60

Slide 60 text

Announcing Co-innovation with Microsoft Assisted fine tuning Train a custom language model Ingest new knowledge domains Improve proficiency for specific languages Azure AI custom models

Slide 61

Slide 61 text

Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge

Slide 62

Slide 62 text

Microsoft Intelligent Data Platform Azure SQL DB Azure Cosmos DB Azure for PostgreSQL MySQL Microsoft Fabric Azure Databricks Azure AI Microsoft Purview Databases • Analytics • AI • Governance

Slide 63

Slide 63 text

Microsoft Fabric Data platform for the era of AI Complete analytics platform Lake-centric and open Empower every business user AI-powered

Slide 64

Slide 64 text

11,000+ organizations use Microsoft Fabric

Slide 65

Slide 65 text

No content

Slide 66

Slide 66 text

Public preview Real-Time Intelligence in Microsoft Fabric Instant actionable insights on all streaming data across clouds aka.ms/RealTimeIntelligence

Slide 67

Slide 67 text

Public preview Real-Time Intelligence in Microsoft Fabric Accelerated decisions AI-powered Intuitive Unified data estate aka.ms/RealTimeIntelligence

Slide 68

Slide 68 text

Public preview Microsoft Fabric Workload Development Kit Seamless user experience OneLake integration Enhanced customer discoverability Marketplace-based monetization aka.ms/FabDevKit

Slide 69

Slide 69 text

Public preview Microsoft Fabric Workload Development Kit aka.ms/FabDevKit

Slide 70

Slide 70 text

No content

Slide 71

Slide 71 text

Azure AI Search Revolutionary retrieval State-of-the-art search technology Feature-rich vector database Seamless data and platform integrations Up to 12x more scale and performance

Slide 72

Slide 72 text

No content

Slide 73

Slide 73 text

Microsoft Copilot Your copilots Copilot extensibility AI safety and security Your data Foundation models AI orchestration and toolchain AI infrastructure Cloud and Edge

Slide 74

Slide 74 text

GitHub Copilot

Slide 75

Slide 75 text

1.8 million GitHub Copilot subscribers The world’s most widely adopted AI developer tool

Slide 76

Slide 76 text

50,000 organizations

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

Private preview GitHub Copilot Workspace Copilot-native developer environment aka.ms/GitHubCopilotWorkspace

Slide 79

Slide 79 text

No content

Slide 80

Slide 80 text

Announcing GitHub Copilot Extensions Creating an intelligent third-party ecosystem in GitHub Copilot aka.ms/GitHubCopilotExtensions

Slide 81

Slide 81 text

GitHub Copilot Extensions aka.ms/GitHubCopilotExtensions

Slide 82

Slide 82 text

Preview GitHub Copilot for Azure aka.ms/GitHubCopilotAzure

Slide 83

Slide 83 text

No content

Slide 84

Slide 84 text

AI safety and security Microsoft Copilot Your copilots Copilot extensibility Cloud and Edge Your data Foundation models AI orchestration and toolchain AI infrastructure

Slide 85

Slide 85 text

Microsoft Copilot

Slide 86

Slide 86 text

68% said Copilot helped them jump-start the creative process Marketing 70% of Copilot users said they were more productive Knowledge workers 12% faster at resolving customer issues Customer service

Slide 87

Slide 87 text

Copilot Studio Your Copilot, your way Customize Build Manage

Slide 88

Slide 88 text

Announcing Microsoft Copilot connectors Build and customize your copilot and Microsoft Copilot using your business data, apps, and workflows Knowledge grounding Real-time actions and workflows Q&A over analytical data aka.ms/TryCopilotConnectors

Slide 89

Slide 89 text

Microsoft Copilot Team Personal

Slide 90

Slide 90 text

Announcing Team Copilot A valuable new member of the team Meeting facilitator Group collaborator Project manager aka.ms/TeamCopilot

Slide 91

Slide 91 text

No content

Slide 92

Slide 92 text

Microsoft Copilot Agents Team Personal

Slide 93

Slide 93 text

No content

Slide 94

Slide 94 text

Microsoft Copilot Your copilots Copilot extensibility AI safety and security Cloud and Edge Your data Foundation models AI orchestration and toolchain AI infrastructure

Slide 95

Slide 95 text

Secure Future Initiative Security above all else Secure by design Secure by default Secure operations Protect tenants and isolate production systems Protect identities and secrets Protect networks Protect engineering systems Monitor and detect threats Accelerate response and remediation