Is Serverless Powerfully Powerless? Kevin Dubois Principal Developer Advocate

@kevindubois “If we continue with the existing trajectory of compute energy, by 2040, we are supposed to hit the world’s energy production capacity.The increase in compute energy and demand has been increasing at a much faster rate than the world energy production capacity increase” https://climate.mit.edu/posts/how-can-we-reduce-carbon-footprint-global-computing - Bilge Yildiz Breene M. Kerr Professor in the MIT departments of Nuclear Science and Engineering and Materials Science and Engineering

@kevindubois 3

@kevindubois https://www.cncf.io/blog/2023/12/20/cncf-cloud-native-finops-cloud-financial-management-microsurvey/

@kevindubois Kevin Dubois ★ Principal Developer Advocate at Red Hat ★ Java Champion ★ Based in Belgium 󰎐 ★ Speak English, Dutch, French, Italian ★ Open Source Contributor (Quarkus, Camel, Knative, ..) @kevindubois@mastodon.social youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com

@kevindubois overprovisioned & wasting resources underprovisioned & unhappy users! Traditional Deployments

@kevindubois Serverless Use only what you need! with Serverless

@kevindubois Knative https://github.com/knative

@kevindubois ● Brings serverless capabilities to Kubernetes ● Out-of-the box autoscaling, even down to 0 ● Built in load balancer ● Server complexity abstracted away ● …

@kevindubois

@kevindubois https://knative.dev/docs/serving/architecture/

@kevindubois “To measure is to know”

@kevindubois Kubernetes-based Efficient Power Level Exporter

@kevindubois ● Kepler (Kubernetes-based Efficient Power Level Exporter) that offers a way to estimate power consumption at the process, container, and Kubernetes pod levels. ● Kepler provides granular power consumption data for Kubernetes Pods and Nodes. ● It uses eBPF to collect energy-related system stats and export them. ● RAPL and ACPI when available or trained models for specific hardware otherwise. ● Kepler was accepted to CNCF on May 17, 2023 and is at the Sandbox project maturity level. Kepler

@kevindubois

@kevindubois So… is Serverless Powerfully Powerless?

@kevindubois Deployment 1 with istio

@kevindubois Deployment 2: serverless

@kevindubois Experiment 1 ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every ½ hours ● 20 mins each ● Serverfull: 6 instances ● Knative ○ Max scale: 6 ○ Concurrency: 4

@kevindubois

@kevindubois

@kevindubois Experiment 1: Results Setup ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every ½ hours ● 20 mins each ● KNative max scale: 6 Results ● E(serverless) > E(serverfull) ● Knative adds idle power consumption ● Serverless overhead comes from knative-serving namespace ● E(knative-serving) ~ E(serverless-ns) ● Mesh negligible idle power consumption ● Contributions (CORE, UNCORE, PKG…) to total energy are similar in both cases ● Results constant over time

@kevindubois Experiment 2 ● Added “plain” setup ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every 20 mins ● 20 mins each

@kevindubois

@kevindubois Experiment 2: Results Results ● E(serverless) > E(serverfull) > E(plain) ● Istio sidecar adds a 2.5x factor in consumption to the mock ● Added “plain” setup ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every 20 mins ● 20 mins each

@kevindubois Experiments 3 & 4 ● What if we double the load? ○ Updating containerConcurrency: 8 to prevent errors ○ 4000 RPS ● What if we tune stuff? Maybe it’s the burst capacity! ○ Burst-capacity default ○ Go back to HTTP/2, 2000 RPS over 1 connection and concurrency 8

@kevindubois Experiment 3 & 4: Results Results ● Still E(serverless) > E(serverfull) > E(plain) ● Similar ratios apply ● Tuning burst capacity does not seem to affect ● What if we double the load? ○ Updating containerConcurrency: 8 to prevent errors ○ 4000 RPS ● What if we tune stuff? ○ Burst-capacity default ○ Go back to HTTP/2, 2000 RPS over 1 connection and concurrency 8

@kevindubois Supersonic. Subatomic. Java

@kevindubois https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf Energy Efficiency of Programming Languages

@kevindubois Experiment 5 ● Added actual work in the server side ● HTTP/2, 6 RPS over 1 connection ● Alternating CronJobs every 20 mins ● 20 mins each

@kevindubois

@kevindubois

@kevindubois

@kevindubois Experiment 5: Results Results ● E(serverless) ~ E(serverfull) > E(plain) ● Knative-serving contribution is negligible when processing work at low rates ● In this type of situations, serverless can be very powerful! ● Added actual work ● HTTP/2, 6 RPS over 1 connection

@kevindubois So… is Serverless Powerfully Powerless?

@kevindubois So… is Serverless Powerfully Powerless? It depends 🥲

@kevindubois with serverless workloads Efficient Node Utilization == Less Nodes == Less Power

@kevindubois Conclusions ● Adding more functionality has a price (Istio, Knative or others) ● There’s a ~constant overhead in power consumption when deploying background systems like knative ○ Energy vs traffic rate is not linear: E(6 RPS) <<< E(4000 RPS) ○ That’s why it’s important to measure! ● What your application is doing matters → both idle and under load ● One size does not fit all: You can now measure your best setup to ○ Save power, money or both ○ Provision accordingly

@kevindubois Repo

@kevindubois

@kevindubois Free Developer e-Books & tutorials! developers.redhat.com/eventtutorials

@kevindubois Serverless Java in Action Kevin Dubois & Daniel Oh Coming soon!

@kevindubois Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.

@kevindubois Thanks! youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com @kevindubois@mastodon.social