Wenn die Vibes nicht reichen: Observability für GenAI Anwendungen

Transcript

1. Wenn die Vibes nicht reichen Observability für GenAI-Anwendungen Torsten Bøgh Köster (@tboeghk)
2. @tboeghk Freelancer, Author Search & Observabilitiy
3. Recap: observability Observability for GenAI apps Tracing GenAI apps Acceptance testing Red team testing Integrating observability into the frontier engineering lifecycle
4. Can you understand what's happening inside your systems, just by asking questions from the outside? Can you answer new questions without shipping new code?
7. Observability for GenAi applications
9. ➡ Prompt injection
10. ➡ Non-determinstic
11. ➡ Dedicated OWASP top 10 Motication: OWASP Top 10
12. ➡ External infrastructure
13. ➡ Function calling accuracy
14. How do we tackle this?
15. 💡 LLM provider metrics 💡 LLM application metrics 🤷 LLM interactions
16. 🪧 Demo time
17. Wieviel Milch sollte ein 9 Tage alter Säugling pro Mahlzeit trinken und wieviele Mahlzeiten benötigt er in 24h? RAG cycle
20. 🪧 MidMind demo
21. 🪧 Langfuse Demo 🪧 MidMind demo
22. 💡How to trace application insights?
24. easy OTEL integration
25. Example: RAG retrieval
26. 🪧 Langfuse demo
27. 🙏 Explainability
28. Acceptance testing a RAG application
29. ✅ Acceptance testing ✅ ✅ ✅ ✅
32. red team testing
33. red team testing
34. 🪧 Promptfoo demo
35. Frontier engineering lifecycle
36. Frontier engineering lifecycle
37. Conclusion