AzureBootcamp2023: Use the Power of OpenAI to leverage your business application by David Schneider

Slide 1

Slide 1 text

Use the power of OpenAI to leverage your business application Azure Bootcamp Switzerland - David Schneider

Slide 2

Slide 2 text

https://linkedin.com/in/david-schneider/ @fiddi

Slide 3

Slide 3 text

No content

Slide 4

Slide 4 text

OpenAI is great, but… Knowledge • No knowledge about the company, industry and customers Compliance • Data Privacy: Data is leaving “own” network • Copyright Manageability • SLA • Billing • Authentication

Slide 5

Slide 5 text

Missing pieces Enterprise-ready platform Data Privacy, Operation, Secure, Connectivity, SLA, Monitoring, Billing Grounding Link to domain knowledge

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

AI SaaS Market

Slide 8

Slide 8 text

Azure OpenAI

Slide 9

Slide 9 text

Azure OpenAI Offering • Available models: GPT-4*, GPT-3, gpt-35-turbo*, Codex, Embeddings, etc. • Regions: East US, South Central US, West Europe • Managed identities • Virtual network support • Azure Management • Azure Billing • Feature Parity • Same API and SDK as for OpenAI

Slide 10

Slide 10 text

Customer data is… - processed only in the subscription of the customer - not sent to OpenAI - not used to train, retrain or improve the model - stored (prompts and completions) for 30 days to monitor and to detect content violation. Opt-out. Data Privacy @ Azure OpenAI

Slide 11

Slide 11 text

Demo Azure OpenAI

Slide 12

Slide 12 text

Grounding OpenAI

Slide 13

Slide 13 text

Grounding OpenAI Azure OpenAI (GPT-4) App App Server, Orchestrator Prompt Domain Knowledge Response

Slide 14

Slide 14 text

3 Steps to Ground OpenAI

Slide 15

Slide 15 text

#1: System Message Set the context using a “System Message” “You are …” “Answer I don’t know …” Domain knowledge

Slide 16

Slide 16 text

Demo System Message

Slide 17

Slide 17 text

• GPT-3 supports 4k tokens • GPT-4 supports 32k tokens #2: GPT-4 supports 32k token

Slide 18

Slide 18 text

Demo Large inputs

Slide 19

Slide 19 text

#3: Embedding “Azure Bootcamp” Embedding Model -0.0002 -0.0322 0.0064 -0.0215 -0.0132 …. Text as a vector text-embedding-ada-002 Similarity between two words is typically measured using cosine similarity.

Slide 20

Slide 20 text

Demo Embeddings

Slide 21

Slide 21 text

Solution Blueprint

Slide 22

Slide 22 text

Solution Blueprint Vector DB (Redis, PostgreSQL, Pinecone, etc.) Azure OpenAI (GPT-4) App App Server, Orchestrator Query → Knowledge Prompt + Knowledge → Response Domain Knowledge Generate embeddings

Slide 23

Slide 23 text

Use Cases

Slide 24

Slide 24 text

Product Recommendations: Utilize embeddings to discover similar products that align with a customer's interests.

Slide 25

Slide 25 text

Customer Support Automation: Ground your support bot with the extensive knowledge base of your company.

Slide 26

Slide 26 text

Rules & regulations: Generate embeddings using domain-specific regulations, enabling quick and accurate responses to any questions.

Slide 27

Slide 27 text

Document summarization: Develop a sophisticated Q&A bot capable of comprehensively summarizing vast document sets, including contracts, RfPs, and more

Slide 28

Slide 28 text

Reference Finder: Provide customer references, feedback or rating based on specific topics

Slide 29

Slide 29 text

HR Bot: Equip your HR bot with the employee handbook, allowing it to quickly and accurately answer any HR- related questions.

Slide 30

Slide 30 text

Enterprise search: Improve search functionality within your company's internal databases, intranet, or knowledge base.

Slide 31

Slide 31 text

Anomaly Detection: Monitor and detect unusual patterns or anomalies in text data to proactively identify potential security threats or compliance issues

Slide 32

Slide 32 text

Content Generation: Generate e-mails, product description, blogposts, etc. that includes domain knowledge

Slide 33

Slide 33 text

Now it’s your turn…

Slide 34

Slide 34 text

• Data, privacy, and security for Azure OpenAI Service https://learn.microsoft.com/en-us/legal/cognitive-services/openai/data-privacy • Code used in this presentation https://github.com/fiddi/openai-playground • ChatGPT + Enterprise data with Azure OpenAI and Cognitive Search https://github.com/Azure-Samples/azure-search-openai-demo References