Maximize GPU Efficiency with Continuous Profiling for GPUs

Transcript

1. Maximize GPU Efficiency with Continuous Profiling for GPUs by Matthias Loibl
2. Profiling As old as programming
3. What? Profiling is a form of dynamic program analysis that measures resource consumption, for example: • the space (memory) • time complexity of a program (CPU and GPU), • usage of instructions, • frequency and duration of function calls 3 https://en.wikipedia.org/wiki/Profiling_(computer_programming)
4. Why?
5. Improve Performance! 🚀🚀🚀
6. Save Money 💸💸💸
7. Tracing • Recording each and every event constantly • High costs
8. Sampling • Sample for a certain duration ◦ Eg. 10 seconds • Periodically observe function call stack ◦ Eg. 100x per second • Low overhead* ◦ <0.5% CPU ◦ ~4MB memory 8 *https://cloud.google.com/blog/products/management-too ls/in-tests-cloud-profiler-adds-negligible-overhead
9. What is profiling?
10. What is profiling? • Always on • In production • Low Overhead • Zero configuration
11. About me • Director of Polar Signals Cloud • Open Source Maintainer ◦ Prometheus ◦ Prometheus Operator ◦ Parca ◦ Pyrra 11 @metalmatze
12. Get the right metrics to maximize GPU utilization See at a glance when a GPU is not utilized the way you expect using high resolution GPU hardware metrics.
13. Get the right metrics to maximize GPU utilization See at a glance when a GPU is not utilized the way you expect using high resolution GPU hardware metrics.
14. Correlate CPU Usage with CPU strips Understand down to the nanoseconds when the CPU was active and when not.
15. Visualize with flame charts If the GPU wasn't fully utilized, what was the CPU doing that wasn't making the GPU do work?
16. Visualize with flame charts If the GPU wasn't fully utilized, what was the CPU doing that wasn't making the GPU do work?
17. Languages we can see in stacks
18. Introducing GPU-time profiling
19. GPU-time profiling
20. GPU-time profiling https://www.polarsignals.com/blog/posts/2025/06/04/latest-in-continuous-profiling-for-gpus
21. How to get started With just one command and the easiest onboarding guide you’ll ever see, you can start saving costs and optimizing performance in your infrastructure. All it takes is a single command. $ kubectl apply -f https://api.polarsignals.com/api/manifests.yaml?token=<your-token-here> Also Docker, sytemd, and binary
22. Trusted By …and more
23. Thank you! Visit our booth S18 • 2hrs free performance consultation for the first 10 companies • Discounted pricing (up to 50%) for seed & series A startups