Red_Hat_Summit_20224_-_Is_Serverless_Powerfully_Powerless.pdf

Transcript

1. Is Serverless Powerfully Powerless? Kevin Dubois Principal Developer Advocate
2. @kevindubois “If we continue with the existing trajectory of compute energy, by 2040, we are supposed to hit the world’s energy production capacity.The increase in compute energy and demand has been increasing at a much faster rate than the world energy production capacity increase” https://climate.mit.edu/posts/how-can-we-reduce-carbon-footprint-global-computing - Bilge Yildiz Breene M. Kerr Professor in the MIT departments of Nuclear Science and Engineering and Materials Science and Engineering
3. @kevindubois 3
4. @kevindubois https://www.cncf.io/blog/2023/12/20/cncf-cloud-native-finops-cloud-financial-management-microsurvey/
5. @kevindubois Kevin Dubois ★ Principal Developer Advocate at Red Hat ★ Java Champion ★ Based in Belgium 󰎐 ★ Speak English, Dutch, French, Italian ★ Open Source Contributor (Quarkus, Camel, Knative, ..) @kevindubois@mastodon.social youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com
6. @kevindubois overprovisioned & wasting resources underprovisioned & unhappy users! Traditional Deployments
7. @kevindubois Serverless Use only what you need! with Serverless
8. @kevindubois Knative https://github.com/knative
9. @kevindubois • Brings serverless capabilities to Kubernetes • Out-of-the box autoscaling, even down to 0 • Built in load balancer • Server complexity abstracted away • …
10. @kevindubois
11. @kevindubois https://knative.dev/docs/serving/architecture/
12. @kevindubois “To measure is to know”
13. @kevindubois Kubernetes-based Efficient Power Level Exporter
14. @kevindubois • Kepler (Kubernetes-based Efficient Power Level Exporter) that offers a way to estimate power consumption at the process, container, and Kubernetes pod levels. • Kepler provides granular power consumption data for Kubernetes Pods and Nodes. • It uses eBPF to collect energy-related system stats and export them. • RAPL and ACPI when available or trained models for specific hardware otherwise. • Kepler was accepted to CNCF on May 17, 2023 and is at the Sandbox project maturity level. Kepler
15. @kevindubois
16. @kevindubois So… is Serverless Powerfully Powerless?
17. @kevindubois Deployment 1 with istio
18. @kevindubois Deployment 2: serverless
19. @kevindubois Experiment 1 • HTTP/2, 2000 RPS over 1 connection • Alternating CronJobs every ½ hours • 20 mins each • Serverfull: 6 instances • Knative ◦ Max scale: 6 ◦ Concurrency: 4
20. @kevindubois
21. @kevindubois
22. @kevindubois Experiment 1: Results Setup • HTTP/2, 2000 RPS over 1 connection • Alternating CronJobs every ½ hours • 20 mins each • KNative max scale: 6 Results • E(serverless) > E(serverfull) • Knative adds idle power consumption • Serverless overhead comes from knative-serving namespace • E(knative-serving) ~ E(serverless-ns) • Mesh negligible idle power consumption • Contributions (CORE, UNCORE, PKG…) to total energy are similar in both cases • Results constant over time
23. @kevindubois Experiment 2 • Added “plain” setup • HTTP/2, 2000 RPS over 1 connection • Alternating CronJobs every 20 mins • 20 mins each
24. @kevindubois
25. @kevindubois Experiment 2: Results Results • E(serverless) > E(serverfull) > E(plain) • Istio sidecar adds a 2.5x factor in consumption to the mock • Added “plain” setup • HTTP/2, 2000 RPS over 1 connection • Alternating CronJobs every 20 mins • 20 mins each
26. @kevindubois Experiments 3 & 4 • What if we double the load? ◦ Updating containerConcurrency: 8 to prevent errors ◦ 4000 RPS • What if we tune stuff? Maybe it’s the burst capacity! ◦ Burst-capacity default ◦ Go back to HTTP/2, 2000 RPS over 1 connection and concurrency 8
27. @kevindubois Experiment 3 & 4: Results Results • Still E(serverless) > E(serverfull) > E(plain) • Similar ratios apply • Tuning burst capacity does not seem to affect • What if we double the load? ◦ Updating containerConcurrency: 8 to prevent errors ◦ 4000 RPS • What if we tune stuff? ◦ Burst-capacity default ◦ Go back to HTTP/2, 2000 RPS over 1 connection and concurrency 8
28. @kevindubois Supersonic. Subatomic. Java
29. @kevindubois https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf Energy Efficiency of Programming Languages
30. @kevindubois Experiment 5 • Added actual work in the server side • HTTP/2, 6 RPS over 1 connection • Alternating CronJobs every 20 mins • 20 mins each
31. @kevindubois
32. @kevindubois
33. @kevindubois
34. @kevindubois Experiment 5: Results Results • E(serverless) ~ E(serverfull) > E(plain) • Knative-serving contribution is negligible when processing work at low rates • In this type of situations, serverless can be very powerful! • Added actual work • HTTP/2, 6 RPS over 1 connection
35. @kevindubois So… is Serverless Powerfully Powerless?
36. @kevindubois So… is Serverless Powerfully Powerless? It depends 🥲
37. @kevindubois with serverless workloads Efficient Node Utilization == Less Nodes == Less Power
38. @kevindubois Conclusions • Adding more functionality has a price (Istio, Knative or others) • There’s a ~constant overhead in power consumption when deploying background systems like knative ◦ Energy vs traffic rate is not linear: E(6 RPS) <<< E(4000 RPS) ◦ That’s why it’s important to measure! • What your application is doing matters → both idle and under load • One size does not fit all: You can now measure your best setup to ◦ Save power, money or both ◦ Provision accordingly
39. @kevindubois Repo
40. @kevindubois
41. @kevindubois Free Developer e-Books & tutorials! developers.redhat.com/eventtutorials
42. @kevindubois Serverless Java in Action Kevin Dubois & Daniel Oh Coming soon!
43. @kevindubois Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.
44. @kevindubois Thanks! youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com @kevindubois@mastodon.social