Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Sign up for free
Menu
Search
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Features
All features
Private URLs
Password Protection
Custom URLS
Scheduled publishing
Remove Branding
Restrict embedding
Deck Collections
Notes
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Explore
Featured decks
Featured speakers
Programming
Technology
Storyboards
Pricing
Search
Sign in
Sign up for free
Wenn die Vibes nicht reichen: Observability für...
Search
Sponsored
·
Ship Features Fearlessly
Turn features on and off without deploys. Used by thousands of Ruby developers.
→
Torsten Bøgh Köster
October 20, 2026
Technology
20
0
Share
Embed
Copy iframe code
Copy JS code
Copy link
Start on current slide
Wenn die Vibes nicht reichen: Observability für GenAI Anwendungen
Torsten Bøgh Köster
October 20, 2026
More Decks by Torsten Bøgh Köster
See All by Torsten Bøgh Köster
LLMs im Griff: Observability, Tracing und Security
tboeghk
0
98
LLMs im Griff: Observability, Tracing und Security
tboeghk
1
72
Oder mache ich es lieber selbst? Wie sich Kosten und Geopolitik auf Cloud-Betrieb auswirken
tboeghk
0
130
Taking an abandoned Solr search from zero to GenAI hero
tboeghk
0
92
Oder mache ich es lieber selbst? Wie sich Kosten und Geopolitik auf Cloud-Betrieb auswirken
tboeghk
0
68
🔪 How we cut our AWS costs in half
tboeghk
0
480
Shared Nothing Logging Infrastructure
tboeghk
0
140
Beyond Cloud: A road trip into AWS and back to bare metal
tboeghk
1
130
Shared Nothing Logging Infrastructure
tboeghk
0
1.5k
Other Decks in Technology
See All in Technology
自律的なチームを作るためにTechPMがやらないこと
ny7760
1
520
ゼロから作るWebアプリの新定番!?Amplify Gen 2 × AWS Blocksで実現する「開発速度とRDBの柔軟性のいいとこ取り」
amixedcolor
1
170
試験勉強はGrokBotでやれ
maigo999
0
110
Kiro 入門 (1 つのエージェントを、どの画面からでも) | 目指せ Kiro マスター 01
inariku
PRO
5
1k
FinOpsを導入してマルチプロダクトのコスト最適化を実現する
kworkdev
PRO
0
140
登壇はいいぞ! 〜Kaigi on Railsに登壇して起きた3つの変化〜
occhi
0
110
AI AgentによるEmbedded SREの取り組み
tgidgd
1
350
Kiro Crew 入門 (手を離しても進む仕事) | 目指せ Kiro マスター 11
inariku
PRO
4
490
AWS構成から考えるWebセキュリティ ― WordPressの構築経験から考える公開範囲と更新運用【JAWS FESTA 2026 in AKITA】
cscengineer
PRO
1
340
AI Agent 時代のデータ基盤: ClickHouse で実現する 超高速分析とコスト削減の両立
jozono
0
200
公共システムAdventCalendar2026
toru_kubota
0
270
re:Invent の去年のスケジュールを公開します!
miu_crescent
PRO
0
310
Featured
See All Featured
Why Our Code Smells
bkeepers
PRO
340
58k
Building Applications with DynamoDB
mza
96
7.3k
Noah Learner - AI + Me: how we built a GSC Bulk Export data pipeline
techseoconnect
PRO
0
480
Introduction to Domain-Driven Design and Collaborative software design
baasie
1
1.1k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
659
62k
Done Done
chrislema
187
17k
StorybookのUI Testing Handbookを読んだ
zakiyama
31
7k
Technical Leadership for Architectural Decision Making
baasie
3
640
Into the Great Unknown - MozCon
thekraken
41
2.8k
The Hidden Cost of Media on the Web [PixelPalooza 2025]
tammyeverts
2
570
Are puppies a ranking factor?
jonoalderson
2
4.1k
The untapped power of vector embeddings
frankvandijk
2
2k
Transcript
Wenn die Vibes nicht reichen Observability für GenAI-Anwendungen Torsten Bøgh
Köster (@tboeghk)
@tboeghk Freelancer, Author Search & Observabilitiy
Recap: observability Observability for GenAI apps Tracing GenAI apps Acceptance
testing Red team testing Integrating observability into the frontier engineering lifecycle
Can you understand what's happening inside your systems, just by
asking questions from the outside? Can you answer new questions without shipping new code?
None
None
Observability for GenAi applications
None
➡ Prompt injection
➡ Non-determinstic
➡ Dedicated OWASP top 10 Motication: OWASP Top 10
➡ External infrastructure
➡ Function calling accuracy
How do we tackle this?
💡 LLM provider metrics 💡 LLM application metrics 🤷 LLM
interactions
🪧 Demo time
Wieviel Milch sollte ein 9 Tage alter Säugling pro Mahlzeit
trinken und wieviele Mahlzeiten benötigt er in 24h? RAG cycle
None
None
🪧 MidMind demo
🪧 Langfuse Demo 🪧 MidMind demo
💡How to trace application insights?
None
easy OTEL integration
Example: RAG retrieval
🪧 Langfuse demo
🙏 Explainability
Acceptance testing a RAG application
✅ Acceptance testing ✅ ✅ ✅ ✅
None
None
red team testing
red team testing
🪧 Promptfoo demo
Frontier engineering lifecycle
Frontier engineering lifecycle
Conclusion