AzureBootcamp2023: Use the Power of OpenAI to leverage your business application by David Schneider

In recent months, AI models have made huge strides. In this session, you’ll learn how to make a business application fit for the future thanks to Azure OpenAI Services. We will discuss some uses on how you can empower your business application users with OpenAI. However, the large language models must first build the company’s knowledge. With embeddings, you can “teach” them domain knowledge. You will learn how to create and consume such embeddings but also get some ideas about how to integrate them in your application. There will be a lot of demos, but we won’t go into the details.
🙂 DAVID SCHNEIDER ⚡️ CTO @ isolutions | Microsoft M365 Apps and Services MVP

Transcript

1. Use the power of OpenAI to leverage your business application Azure Bootcamp Switzerland - David Schneider
2. https://linkedin.com/in/david-schneider/ @fiddi
4. OpenAI is great, but… Knowledge • No knowledge about the company, industry and customers Compliance • Data Privacy: Data is leaving “own” network • Copyright Manageability • SLA • Billing • Authentication
5. Missing pieces Enterprise-ready platform Data Privacy, Operation, Secure, Connectivity, SLA, Monitoring, Billing Grounding Link to domain knowledge
7. AI SaaS Market
8. Azure OpenAI
9. Azure OpenAI Offering • Available models: GPT-4*, GPT-3, gpt-35-turbo*, Codex, Embeddings, etc. • Regions: East US, South Central US, West Europe • Managed identities • Virtual network support • Azure Management • Azure Billing • Feature Parity • Same API and SDK as for OpenAI
10. Customer data is… - processed only in the subscription of the customer - not sent to OpenAI - not used to train, retrain or improve the model - stored (prompts and completions) for 30 days to monitor and to detect content violation. Opt-out. Data Privacy @ Azure OpenAI
11. Demo Azure OpenAI
12. Grounding OpenAI
13. Grounding OpenAI Azure OpenAI (GPT-4) App App Server, Orchestrator Prompt Domain Knowledge Response
14. 3 Steps to Ground OpenAI
15. #1: System Message Set the context using a “System Message” “You are …” “Answer I don’t know …” Domain knowledge
16. Demo System Message
17. • GPT-3 supports 4k tokens • GPT-4 supports 32k tokens #2: GPT-4 supports 32k token
18. Demo Large inputs
19. #3: Embedding “Azure Bootcamp” Embedding Model -0.0002 -0.0322 0.0064 -0.0215 -0.0132 …. Text as a vector text-embedding-ada-002 Similarity between two words is typically measured using cosine similarity.
20. Demo Embeddings
21. Solution Blueprint
22. Solution Blueprint Vector DB (Redis, PostgreSQL, Pinecone, etc.) Azure OpenAI (GPT-4) App App Server, Orchestrator Query → Knowledge Prompt + Knowledge → Response Domain Knowledge Generate embeddings
23. Use Cases
24. Product Recommendations: Utilize embeddings to discover similar products that align with a customer's interests.
25. Customer Support Automation: Ground your support bot with the extensive knowledge base of your company.
26. Rules & regulations: Generate embeddings using domain-specific regulations, enabling quick and accurate responses to any questions.
27. Document summarization: Develop a sophisticated Q&A bot capable of comprehensively summarizing vast document sets, including contracts, RfPs, and more
28. Reference Finder: Provide customer references, feedback or rating based on specific topics
29. HR Bot: Equip your HR bot with the employee handbook, allowing it to quickly and accurately answer any HR- related questions.
30. Enterprise search: Improve search functionality within your company's internal databases, intranet, or knowledge base.
31. Anomaly Detection: Monitor and detect unusual patterns or anomalies in text data to proactively identify potential security threats or compliance issues
32. Content Generation: Generate e-mails, product description, blogposts, etc. that includes domain knowledge
33. Now it’s your turn…
34. • Data, privacy, and security for Azure OpenAI Service https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy • Code used in this presentation https://github.com/fiddi/openai-playground • ChatGPT + Enterprise data with Azure OpenAI and Cognitive Search https://github.com/Azure-Samples/azure-search-openai-demo References