AI-DRIVEN SOFTWARE DEVELOPMENT WITH AZURE AI Jörg Neumann

TOPICS  AI  Frontend Technologies  Mobile Development  Consulting, Coaching, Training CONTACT  Mail: joerg.neumann@neogeeks.de  Twitter: @JoergNeumann  Web: www.neogeeks.de JÖRG NEUMANN Founder & CEO NeoGeeks GmbH Azure OpenAI Service, Azure Machine Learning Platform

AI-SCENARIOS POPULAR CASES  Chat with own data  Customer support  Call centers  …

RETRIEVAL AUGMENTED GENERATION (RAG) GOAL  Extend the model with external data sources  Integrate domain-specific knowledge  Model doesn’t need to be trained

RETRIEVAL AUGMENTED GENERATION question query own data add results to the prompt query model answer

RETRIEVAL ARGUMENTED GENERATION (RAG) Chunk Chunk Chunk Chunk Embeddings Embeddings Embeddings Embeddings Vector Database PDF Document user search Embeddings Search answer LLM weight results

PROMPT ENGINEERING Answer the question based on the context provided below. If the question cannot be answered with the provided information, respond with 'I don't know’. context: Large Language Models (LLMs) are the latest models used in NLP. Their superior performance over smaller models has made them incredibly useful for developers. These models can be accessed via the 'transformers' library by Hugging Face, via OpenAI with the 'openai' library, and via Cohere with the 'cohere' library. question: Which libraries and model providers offer LLMs?“ instructions prompt external information Which libraries and model providers offer LLMs?

IMPLEMENTATION CHALLENGES  Tables and illustrations in documents  Answers have no source citations  Answers can be outdated  The model can hallucinate  Limit context to specific data  Content filters

IMPLEMENTATION CODING  Python, C#, Java FRAMEWORKS  LangChain, LlamaIndex, …  Semantic Kernel  Spring AI VECTOR STORES  Pinecone, Chroma, Milvus, Qdrant, …

AI IN AZURE AZURE OPENAI SERVICE  Models from OpenAI  Endpoint for API access  Available models depend on region AZURE OPENAI STUDIO  Deployment and configuration of the models

Documents Table Storage Blob Storage AI Search OpenAI Service Database Data App ARCHITECTURE

DEMO Azure OpenAI Studio Source: https://oai.azure.com

AI IN AZURE AZURE AI STUDIO  Development of complex AI solutions  Multi-Model Support  Prompt Flow  Debugging & Metrics  https://ai.azure.com

DEMO Azure AI Source: https://oai.azure.com

CONCLUSION  RAG is a complex pattern  Numerous frameworks and tools are required  Azure OpenAI and Azure AI Search offers an easy low-code solution for RAG scenarios  Complex scenarios can be implemented with Azure AI Studio

Q& A