Cloudy with a Chance of Insights:

Demystifying Observability in the Cloud

Transcript

2. 2 ?
3. “CLOUDY, WITH A CHANCE OF INSIGHTS” Demystifying Observability in the Cloud PHILIPP SEIFERT-KEHRER 4 July 2024
4. 4 WHAT ABOUT ME? • Developer for 7+ years • @ Posedio since 2022 • Blood type AB- • Love cats, crocheting & crocheting cats • Monoliths: hehe • Shared libs: not hehe Philipp Seifert-Kehrer
5. ROADMAP 1. Debugging like it’s 1999 2. Observability Xs and Os 3. Tracing 101 4.Introducing OpenTelemetry 5. Where benefits?
6. 6 SO, WHAT TO EXPECT FROM THIS TALK? (Re)Gain confidence to deploy…
7. DEBUGGING LIKE IT’S 1999 1
8. 8 CLASSICAL DEBUGGING PHD in domain knowledge Sherlock gene
9. 9 LONGEST EMPLOYEE = BEST DEBUGGER? I haven’t moved away from here in 15 years
10. 10 READING THE TEA LEAVES
11. 11 MONITORING = BYOC Bring Your Own Context
12. 12 BUT, BUT, BUT RUNBOOKS • No documentation > wrong documentation
13. 13 HELLO CLOUD
14. 14 BUT WHY MONITORING NOT CUTTIN’ IT? Built upon assumptions Only useful for known unknowns Purely reactive Linear increase of cost Uptime is so 2001
15. 2 OBSERVABILITY XS AND OS
16. 16 WHAT ARE WE TALKIN’ ABOUT? “How well internal states of a system can be inferred from knowledge of its external outputs”
17. 17 PLATO WAS RIGHT
18. 18 BUT FOR SOFTWARE? Understand inner workings of application Understand any system state Understand only by observing/external tools Understand without need to ship new code
19. 19 GANG OF 3 Centralized Logging Metrics & Dashboards Distributed Tracing
20. 20 LET THEM WITHOUT SIN CAST THE FIRST STONE Every transaction No issues with cardinality Proportional growth of cost Centralized Logging ✅ ✅ 🙅 Metrics & Dashboards ✅ 🙅 ✅ Distribruted Tracing 🙅 ✅ ✅
21. 21 GANG OF 3 GANG OF 3 Centralized Logging Metrics & Dashboards Distributed Tracing
22. 22 GIMME SOME EXPLORABILITY
23. TRACING 3
24. 24 SERVICE GRAPH
25. 25 TRACING EXAMPLE
26. 26 ONE TRACE TO RULE ‘EM ALL
27. 27 ANATOMY OF A SPAN
28. 28 EVENTS
29. 29 IT’S ALL ABOUT THE CONTEXT Context propagation Baggage Marshall Context matters (… even after the 14th blunt)
30. OPENTELEMETRY 4
31. 31 OTEL OFFERING … that you can’t refuse Open specification Instrumentation APIs & SDKs Semantic conventions Agent to collect telemetry Optimized protocol OTLP
32. 32 COLLECTOR
33. 33 SEMANTIC CONVENTIONS
34. 34 INSTRUMENTATION Black vs White box Application vs System Distributed vs Monolith
35. 35 DO’S AND DON’TS Do Don’t Feature flags Be aware of PII Verbocity level Start off with automatic instrumentation Ignore potential resource overhead Non-standard format Too high default resolution
36. WHERE BENEFITS 5
37. 37 WHY ALL OF THIS? Less interpretation = less discussion Foster proactive debugging Reliability Good for application, organization & employee (mental) health No more manual, mental joins Learning opportunity instead of learning prerequisite Link anomalies to distinct version
38. 38 A NEW NON-FUNCTIONAL REQUIREMENT Maintainability Availability Scalability Observability in the cloud
39. 39 BUT IT’S SO MUCH OVERHEAD, RIGHT?
40. 40 SAMPLING
41. 41 WHAT IF I JUST KEEP LOGGING EVERYTHING?
42. 42 WRAPPING IT UP? Maintainability Availability Scalability Sustainability Better understanding of architecture More accessible debugging Not all can be fixed… and also should not! Benefits without too much effort possible No panacea
43. 43 Failure is inevitable … so when the shit goes down, you better be ready
44. THANK YOU! POSEDIO GMBH +43 660 9154419 contact@posedio.com Weyringergasse 1-3, 1040 Wien, Millenium Park 4, 6980 Lustenau www.posedio.com Dilly dilly!