Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Observability For Serverless Workloads
Search
Ara
February 21, 2020
Programming
0
78
Observability For Serverless Workloads
What is it observability? How we can improve observability when it comes to serverless workloads?
Ara
February 21, 2020
Tweet
Share
More Decks by Ara
See All by Ara
CloudConf Italy - A Tale of Tail Latency
arapulido
0
79
Abrazando el cambio: Gobierno de Kubernetes como código con OPA y Gatekeeper
arapulido
0
63
Navigating the Sea of Local Kubernetes Clusters
arapulido
0
96
Cloud Services in Kubernetes - Discovering and Using Services with the Service Catalog and Kubeapps
arapulido
0
78
Telepresence - Seamless Development Environments in Kubernetes
arapulido
0
11k
Kubeless demo @ CNCF WG-Serverless meeting
arapulido
0
300
Securing the IoT - J On The Beach 2017
arapulido
1
160
Other Decks in Programming
See All in Programming
Claude Code + Container Use と Cursor で作る ローカル並列開発環境のススメ / ccc local dev
kaelaela
12
7k
Agentic Coding: The Future of Software Development with Agents
mitsuhiko
0
130
PostgreSQLのRow Level SecurityをPHPのORMで扱う Eloquent vs Doctrine #phpcon #track2
77web
2
580
Result型で“失敗”を型にするPHPコードの書き方
kajitack
5
1k
Goで作る、開発・CI環境
sin392
0
260
Porting a visionOS App to Android XR
akkeylab
0
670
Azure AI Foundryではじめてのマルチエージェントワークフロー
seosoft
0
200
Android 16KBページサイズ対応をはじめからていねいに
mine2424
0
420
ソフトウェア品質を数字で捉える技術。事業成長を支えるシステム品質の マネジメント
takuya542
2
15k
Flutterで備える!Accessibility Nutrition Labels完全ガイド
yuukiw00w
0
170
おやつのお供はお決まりですか?@WWDC25 Recap -Japan-\(region).swift
shingangan
0
140
フロントエンドのパフォーマンスチューニング
koukimiura
5
2k
Featured
See All Featured
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
YesSQL, Process and Tooling at Scale
rocio
173
14k
Bootstrapping a Software Product
garrettdimon
PRO
307
110k
Building an army of robots
kneath
306
45k
Music & Morning Musume
bryan
46
6.7k
Chrome DevTools: State of the Union 2024 - Debugging React & Beyond
addyosmani
7
750
StorybookのUI Testing Handbookを読んだ
zakiyama
30
5.9k
A Modern Web Designer's Workflow
chriscoyier
695
190k
Done Done
chrislema
184
16k
Art, The Web, and Tiny UX
lynnandtonic
299
21k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
282
13k
Transcript
Observability for Serverless Workloads Ara Pulido @arapulido
Developer Advocate at Datadog @arapulido at Twitter
[email protected]
About me
Observability
Observability Microservices Containers FaaS Serverless CI/CD ML Big Data
A bit of history
Monoliths
Different user expectations
Infrastructure monitoring
Oops!
Microservices
Infrastructure monitoring is no longer enough
Observability A measure of how well internal states of a
system can be inferred from knowledge of its external outputs
Logs Metrics Tracing
Logs Metrics Tracing Events User/Browser tests Data visualization ML Queries
Serverless
NO ACCESS TO THE UNDERLYING OS CHARGED BY EXECUTION TIME
/ MEMORY ALLOCATED IN MANY CLOUDS, RUNTIMES ARE A BIT OF A BLACKBOX Serverless
Logs Metrics Tracing Demo application https://dtdg.co/faas-sample
Demo application 1 face - No duplicates No faces >
1 face Duplicated
Demo application Search Faces Web app Index Faces Persist Data
Detect Faces
Demo application Detect Faces Search Faces Web app Index Faces
Persist Data
Demo application Search Faces Web app Index Faces Persist Data
Detect Faces
Demo application Search Faces Web app Index Faces Persist Data
Detect Faces
Demo application Search Faces Web app Index Faces Persist Data
Detect Faces
Demo application Search Faces Web app Index Faces Persist Data
Detect Faces
Logs Metrics Tracing
THE PLATFORM LOGS STDOUT, STDERR LOG AS MUCH AS POSSIBLE
(PART OF THE PLATFORM) Logs USE LOG FORWARDERS TO COLLECT THEM ELSEWHERE USE YOUR LANGUAGE LOGGING LIBRARY TO DO MORE COMPLEX STUFF
Logs Cloud Log System Log Forwarder Log system
Logs
Logs
Logs Metrics Tracing
Collecting Metrics
Collecting metrics Send batch Send batch Send batch
Cold start Cold start Execution context Collecting metrics
Cold start EXECUTION CONTEXT SHARES DISK ACROSS INVOCATIONS Cold start
Execution context Collecting metrics
Cold start EXECUTION CONTEXT SHARES DISK ACROSS INVOCATIONS Cold start
Send batch Execution context Collecting metrics
Cold start WE CANNOT KNOW WHEN IT IS GOING TO
BE GARBAGE COLLECTED Cold start Send batch Execution context Collecting metrics EXECUTION CONTEXT SHARES DISK ACROSS INVOCATIONS Send batch
Metrics in Logs Cloud Log System Log Forwarder Metrics
Metrics in Logs Cloud Log System Log Forwarder Metrics
Infrastructure Metrics
4 golden signals LATENCY TRAFFIC SATURATION ERRORS
DURATION INVOCATIONS THROTTLE ERRORS 4 golden signals
Serverless specific COLD STARTS WARM STARTS
BILLED DURATION - DURATION Serverless specific
Use metrics to save $$$ (and time) ESTIMATED COST PER
FUNCTION
Use metrics to save $$$ (and time) 700ms 500ms 300ms
128MB 192MB 320MB 128MB = $ 0.000001465 192MB = $ 0.000001565 320MB = $ 0.000001563
Business Metrics
EACH FUNCTION IS A POTENTIAL BUSINESS METRIC GATHER THOSE AND
USE THEM TO IMPROVE YOUR BUSINESS Business metrics
Business metrics
Logs Metrics Tracing
None
S1 S2 S3 S4 S5
S1 S2 S3 S4 S5 TRACE SPANS
Instrumentation with traces
No OS / No agent
No OS / No agent Use cloud specific libraries for
your functions
X-Ray traces
No OS / No agent Use cloud specific libraries for
your functions … but make sure you don’t break your current traces
Demo application Search Faces Web app Index Faces Persist Data
Detect Faces
Full app trace
Full app trace
Full app trace
Full app trace
Full app trace
Full app trace
No OS / No agent Use cloud specific libraries for
your functions … but make sure you don’t break your current traces or use logs!!
Logs Metrics Tracing
Take aways
USE LOGS FOR METRICS TO AVOID LOSING DATA ADD YOUR
FUNCTIONS TO YOUR CURRENT TRACES (if possible) LOGS ARE CHEAP. LOG A LOT. TRACK THOSE BUSINESS METRICS
Thank you! (we are hiring!) Ara Pulido @arapulido Demo app:
https://dtdg.co/faas-sample