Slide 1

Slide 1 text

AI-DRIVEN SOFTWARE DEVELOPMENT WITH AZURE AI Jörg Neumann

Slide 2

Slide 2 text

TOPICS  AI  Frontend Technologies  Mobile Development  Consulting, Coaching, Training CONTACT  Mail: [email protected]  Twitter: @JoergNeumann  Web: www.neogeeks.de JÖRG NEUMANN Founder & CEO NeoGeeks GmbH Azure OpenAI Service, Azure Machine Learning Platform

Slide 3

Slide 3 text

AI-SCENARIOS POPULAR CASES  Chat with own data  Customer support  Call centers  …

Slide 4

Slide 4 text

RETRIEVAL AUGMENTED GENERATION (RAG) GOAL  Extend the model with external data sources  Integrate domain-specific knowledge  Model doesn’t need to be trained

Slide 5

Slide 5 text

RETRIEVAL AUGMENTED GENERATION question query own data add results to the prompt query model answer

Slide 6

Slide 6 text

RETRIEVAL ARGUMENTED GENERATION (RAG) Chunk Chunk Chunk Chunk Embeddings Embeddings Embeddings Embeddings Vector Database PDF Document user search Embeddings Search answer LLM weight results

Slide 7

Slide 7 text

PROMPT ENGINEERING Answer the question based on the context provided below. If the question cannot be answered with the provided information, respond with 'I don't know’. context: Large Language Models (LLMs) are the latest models used in NLP. Their superior performance over smaller models has made them incredibly useful for developers. These models can be accessed via the 'transformers' library by Hugging Face, via OpenAI with the 'openai' library, and via Cohere with the 'cohere' library. question: Which libraries and model providers offer LLMs?“ instructions prompt external information Which libraries and model providers offer LLMs?

Slide 8

Slide 8 text

IMPLEMENTATION CHALLENGES  Tables and illustrations in documents  Answers have no source citations  Answers can be outdated  The model can hallucinate  Limit context to specific data  Content filters

Slide 9

Slide 9 text

IMPLEMENTATION CODING  Python, C#, Java FRAMEWORKS  LangChain, LlamaIndex, …  Semantic Kernel  Spring AI VECTOR STORES  Pinecone, Chroma, Milvus, Qdrant, …

Slide 10

Slide 10 text

AI IN AZURE AZURE OPENAI SERVICE  Models from OpenAI  Endpoint for API access  Available models depend on region AZURE OPENAI STUDIO  Deployment and configuration of the models

Slide 11

Slide 11 text

Documents Table Storage Blob Storage AI Search OpenAI Service Database Data App ARCHITECTURE

Slide 12

Slide 12 text

DEMO Azure OpenAI Studio Source: https://oai.azure.com

Slide 13

Slide 13 text

AI IN AZURE AZURE AI STUDIO  Development of complex AI solutions  Multi-Model Support  Prompt Flow  Debugging & Metrics  https://ai.azure.com

Slide 14

Slide 14 text

DEMO Azure AI Source: https://oai.azure.com

Slide 15

Slide 15 text

CONCLUSION  RAG is a complex pattern  Numerous frameworks and tools are required  Azure OpenAI and Azure AI Search offers an easy low-code solution for RAG scenarios  Complex scenarios can be implemented with Azure AI Studio

Slide 16

Slide 16 text

Q& A