Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Instrumenting Async Runtime
Search
mox692
December 15, 2024
Programming
0
77
Instrumenting Async Runtime
This is a presentation I gave in the Rust.Tokyo conference 2024.
mox692
December 15, 2024
Tweet
Share
More Decks by mox692
See All by mox692
Implementing Async MPMC Channel in Rust
kimuramotoyuki
0
16
Bug-Free Concurrency in Rust
kimuramotoyuki
0
16
Other Decks in Programming
See All in Programming
10 Costly Database Performance Mistakes (And How To Fix Them)
andyatkinson
0
330
なぜ適用するか、移行して理解するClean Architecture 〜構造を超えて設計を継承する〜 / Why Apply, Migrate and Understand Clean Architecture - Inherit Design Beyond Structure
seike460
PRO
3
760
レベル1の開発生産性向上に取り組む − 日々の作業の効率化・自動化を通じた改善活動
kesoji
0
190
0626 Findy Product Manager LT Night_高田スライド_speaker deck用
mana_takada
0
170
Webの外へ飛び出せ NativePHPが切り拓くPHPの未来
takuyakatsusa
2
550
Startups on Rails in Past, Present and Future–Irina Nazarova, RailsConf 2025
irinanazarova
0
100
Hypervel - A Coroutine Framework for Laravel Artisans
albertcht
1
120
イベントストーミング図からコードへの変換手順 / Procedure for Converting Event Storming Diagrams to Code
nrslib
2
810
すべてのコンテキストを、 ユーザー価値に変える
applism118
3
1.3k
技術同人誌をMCP Serverにしてみた
74th
1
640
Claude Code + Container Use と Cursor で作る ローカル並列開発環境のススメ / ccc local dev
kaelaela
9
5.1k
なぜ「共通化」を考え、失敗を繰り返すのか
rinchoku
1
650
Featured
See All Featured
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
46
9.6k
Rails Girls Zürich Keynote
gr2m
95
14k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
How to train your dragon (web standard)
notwaldorf
95
6.1k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
Evolution of real-time – Irina Nazarova, EuRuKo, 2024
irinanazarova
8
820
Adopting Sorbet at Scale
ufuk
77
9.5k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
507
140k
RailsConf 2023
tenderlove
30
1.1k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
970
A designer walks into a library…
pauljervisheath
207
24k
Exploring the Power of Turbo Streams & Action Cable | RailsConf2023
kevinliebholz
34
5.9k
Transcript
Instrumenting Async Runtime Rust.Tokyo 2024 Motoyuki Kimura 1
• Rust Developer 🦀 • Used to use Scala •
A organization member of tokio-rs Hi, I’m Motoyuki(@mox692) 👋 2
Agenda • Share my attempt for async runtime tracing ◦
Recap monitoring ecosystem in Async Rust ◦ What is runtime tracing? ◦ How does it works? ◦ Lessons, challenges 3
Monitoring is important 4
How to achieve a monitoring? • Logging (println!, log! etc…)
• Capture a backtrace • System metrics etc … 5
What about async Rust? 6
… Sometimes, a bit tricky 7
8 Async Task Task1 Task2 Execution
9 Async Task Task1 Task2 Execution .await .await .await
10 Async Task Task1 Task2 .await .await .await Task1 Task1
Task2 Task2
11
12
13
Are there any async-aware tools available today? 14
• Metrics • Tokio-console • Taskdump 15
16 Runtime Metrics
17
18
19
20 Task dump
21 Task dump
Would this be enough? 22
23 Case1: Tasks are not distributed Thread Core
24 Case2: Long running tasks
25 Case2: Long running tasks
26 Case3: Not sure what happens behind the scene 😇
27 • Async runtime tend to be a blackbox •
It would be cool if we have a tool that focuses on the event occurred in the runtime
Prior Art? 28
29
30
31 Goroutine Backtrace
32
33
34 It is integrated into the Java Virtual Machine (JVM)
and causes almost no performance overhead, so it can be used even in heavily loaded production environments.
Can we do something similar with Rust? 35
36
Let’s do a demo to see how it goes 37
Demo time 38
How does it work? 39
High level archtecture 40 Gathering Runtime Event Tokio Runtime Visualize
Log File Runtime thread Runtime thread Runtime thread
41 Runtime Event
42 Runtime Event Runtime Parking Run Task
43 Runtime Event Backtrace Task Name Duration
Visualize 44
Visualize 45
46
Tracing × Perfetto 47
48
49 Hacking Runtime Internal !
50
51
52 Trace Store backtrace
Challenges 53
Backtrace 54
55 Invoked for every task run!!
56 Backtrace • It is crucial to be able to
get backtraces efficiently, • Because the locations where traces are obtained are often hot paths in the runtime 🔥
57 🤔
58
59
60
61
62
Other Subtleties • Symbolize • When to capture a backtrace?
◦ For now, capture backtraces when a future returns Pending ◦ … but this means that we cannot capture a backtrace if that future complete at first poll 63
Trace Log 64
65
66 Trace related stuff
Can we do better? 67
68 Continuous Monitoring with JDK Flight Recorder (JFR)
69 Execution tracer overhaul
70 Trace Log • Should be optimized by e.g., ◦
Using thread local buffer ◦ Using a dedicated trace format
71
72 worker-trace.md
73 Thank you!