Slide 1

Slide 1 text

Is Serverless Powerfully Powerless? Kevin Dubois Principal Developer Advocate

Slide 2

Slide 2 text

@kevindubois “If we continue with the existing trajectory of compute energy, by 2040, we are supposed to hit the world’s energy production capacity.The increase in compute energy and demand has been increasing at a much faster rate than the world energy production capacity increase” https://climate.mit.edu/posts/how-can-we-reduce-carbon-footprint-global-computing - Bilge Yildiz Breene M. Kerr Professor in the MIT departments of Nuclear Science and Engineering and Materials Science and Engineering

Slide 3

Slide 3 text

@kevindubois 3

Slide 4

Slide 4 text

@kevindubois https://www.cncf.io/blog/2023/12/20/cncf-cloud-native-finops-cloud-financial-management-microsurvey/

Slide 5

Slide 5 text

@kevindubois Kevin Dubois ★ Principal Developer Advocate at Red Hat ★ Java Champion ★ Based in Belgium 󰎐 ★ Speak English, Dutch, French, Italian ★ Open Source Contributor (Quarkus, Camel, Knative, ..) @[email protected] youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com

Slide 6

Slide 6 text

@kevindubois overprovisioned & wasting resources underprovisioned & unhappy users! Traditional Deployments

Slide 7

Slide 7 text

@kevindubois Serverless Use only what you need! with Serverless

Slide 8

Slide 8 text

@kevindubois Knative https://github.com/knative

Slide 9

Slide 9 text

@kevindubois ● Brings serverless capabilities to Kubernetes ● Out-of-the box autoscaling, even down to 0 ● Built in load balancer ● Server complexity abstracted away ● …

Slide 10

Slide 10 text

@kevindubois

Slide 11

Slide 11 text

@kevindubois https://knative.dev/docs/serving/architecture/

Slide 12

Slide 12 text

@kevindubois “To measure is to know”

Slide 13

Slide 13 text

@kevindubois Kubernetes-based Efficient Power Level Exporter

Slide 14

Slide 14 text

@kevindubois ● Kepler (Kubernetes-based Efficient Power Level Exporter) that offers a way to estimate power consumption at the process, container, and Kubernetes pod levels. ● Kepler provides granular power consumption data for Kubernetes Pods and Nodes. ● It uses eBPF to collect energy-related system stats and export them. ● RAPL and ACPI when available or trained models for specific hardware otherwise. ● Kepler was accepted to CNCF on May 17, 2023 and is at the Sandbox project maturity level. Kepler

Slide 15

Slide 15 text

@kevindubois

Slide 16

Slide 16 text

@kevindubois So… is Serverless Powerfully Powerless?

Slide 17

Slide 17 text

@kevindubois Deployment 1 with istio

Slide 18

Slide 18 text

@kevindubois Deployment 2: serverless

Slide 19

Slide 19 text

@kevindubois Experiment 1 ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every ½ hours ● 20 mins each ● Serverfull: 6 instances ● Knative ○ Max scale: 6 ○ Concurrency: 4

Slide 20

Slide 20 text

@kevindubois

Slide 21

Slide 21 text

@kevindubois

Slide 22

Slide 22 text

@kevindubois Experiment 1: Results Setup ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every ½ hours ● 20 mins each ● KNative max scale: 6 Results ● E(serverless) > E(serverfull) ● Knative adds idle power consumption ● Serverless overhead comes from knative-serving namespace ● E(knative-serving) ~ E(serverless-ns) ● Mesh negligible idle power consumption ● Contributions (CORE, UNCORE, PKG…) to total energy are similar in both cases ● Results constant over time

Slide 23

Slide 23 text

@kevindubois Experiment 2 ● Added “plain” setup ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every 20 mins ● 20 mins each

Slide 24

Slide 24 text

@kevindubois

Slide 25

Slide 25 text

@kevindubois Experiment 2: Results Results ● E(serverless) > E(serverfull) > E(plain) ● Istio sidecar adds a 2.5x factor in consumption to the mock ● Added “plain” setup ● HTTP/2, 2000 RPS over 1 connection ● Alternating CronJobs every 20 mins ● 20 mins each

Slide 26

Slide 26 text

@kevindubois Experiments 3 & 4 ● What if we double the load? ○ Updating containerConcurrency: 8 to prevent errors ○ 4000 RPS ● What if we tune stuff? Maybe it’s the burst capacity! ○ Burst-capacity default ○ Go back to HTTP/2, 2000 RPS over 1 connection and concurrency 8

Slide 27

Slide 27 text

@kevindubois Experiment 3 & 4: Results Results ● Still E(serverless) > E(serverfull) > E(plain) ● Similar ratios apply ● Tuning burst capacity does not seem to affect ● What if we double the load? ○ Updating containerConcurrency: 8 to prevent errors ○ 4000 RPS ● What if we tune stuff? ○ Burst-capacity default ○ Go back to HTTP/2, 2000 RPS over 1 connection and concurrency 8

Slide 28

Slide 28 text

@kevindubois Supersonic. Subatomic. Java

Slide 29

Slide 29 text

@kevindubois https://greenlab.di.uminho.pt/wp-content/uploads/2017/10/sleFinal.pdf Energy Efficiency of Programming Languages

Slide 30

Slide 30 text

@kevindubois Experiment 5 ● Added actual work in the server side ● HTTP/2, 6 RPS over 1 connection ● Alternating CronJobs every 20 mins ● 20 mins each

Slide 31

Slide 31 text

@kevindubois

Slide 32

Slide 32 text

@kevindubois

Slide 33

Slide 33 text

@kevindubois

Slide 34

Slide 34 text

@kevindubois Experiment 5: Results Results ● E(serverless) ~ E(serverfull) > E(plain) ● Knative-serving contribution is negligible when processing work at low rates ● In this type of situations, serverless can be very powerful! ● Added actual work ● HTTP/2, 6 RPS over 1 connection

Slide 35

Slide 35 text

@kevindubois So… is Serverless Powerfully Powerless?

Slide 36

Slide 36 text

@kevindubois So… is Serverless Powerfully Powerless? It depends 🥲

Slide 37

Slide 37 text

@kevindubois with serverless workloads Efficient Node Utilization == Less Nodes == Less Power

Slide 38

Slide 38 text

@kevindubois Conclusions ● Adding more functionality has a price (Istio, Knative or others) ● There’s a ~constant overhead in power consumption when deploying background systems like knative ○ Energy vs traffic rate is not linear: E(6 RPS) <<< E(4000 RPS) ○ That’s why it’s important to measure! ● What your application is doing matters → both idle and under load ● One size does not fit all: You can now measure your best setup to ○ Save power, money or both ○ Provision accordingly

Slide 39

Slide 39 text

@kevindubois Repo

Slide 40

Slide 40 text

@kevindubois

Slide 41

Slide 41 text

@kevindubois Free Developer e-Books & tutorials! developers.redhat.com/eventtutorials

Slide 42

Slide 42 text

@kevindubois Serverless Java in Action Kevin Dubois & Daniel Oh Coming soon!

Slide 43

Slide 43 text

@kevindubois Start exploring in the OpenShift Sandbox. Learn containers, Kubernetes, and OpenShift in your browser. developers.redhat.com/developer-sandbox Try Red Hat's products and technologies without setup or configuration.

Slide 44

Slide 44 text

@kevindubois Thanks! youtube.com/@thekevindubois linkedin.com/in/kevindubois github.com/kdubois @kevindubois.com @[email protected]