These Five Tricks Can Make Your Apps Greener, Cheaper, & Nicer

Slide 1

Slide 1 text

Holly Cummins Senior Principal Software Engineer, Quarkus GOTO Aarhus May 24, 2023 @[email protected] These five tricks can make your apps greener cheaper & nicer

Slide 2

Slide 2 text

@holly_cummins #RedHat

Slide 3

Slide 3 text

@holly_cummins #RedHat compute for training large deep learning models 300,000-fold increase in 6 years Dario Amodei and Daniel Hernandez. 2018. AI and Compute. https://openai.com/blog/ai-and-compute

Slide 4

Slide 4 text

@holly_cummins #RedHat compute for training large deep learning models 300,000-fold increase in 6 years Dario Amodei and Daniel Hernandez. 2018. AI and Compute. https://openai.com/blog/ai-and-compute

Slide 5

Slide 5 text

@holly_cummins #RedHat it’s not just artificial intelligence

Slide 6

Slide 6 text

@holly_cummins #RedHat it’s not just artificial intelligence it’s not just cryptocurrency mining

Slide 7

Slide 7 text

@holly_cummins #RedHat it’s not just artificial intelligence it’s not just cryptocurrency mining it’s all of us

Slide 8

Slide 8 text

@holly_cummins #RedHat Sources: https://www.greenit.fr/wp-content/uploads/2019/11/GREENIT_EENM_etude_EN_accessible.pdf https://ourworldindata.org/ghg-emissions-by-sector the digital world creates more carbon emissions than aviation

Slide 9

Slide 9 text

@holly_cummins #RedHat Lighter area represents high and low estimates, where available. Sources: https://www.iea.org/fuels-and-technologies/data-centres-networks https://ourworldindata.org/grapher/electricity-demand?tab=table&country=USA~GBR~FRA~DEU~IND~BRA data centres use as much electricity as a medium country

Slide 10

Slide 10 text

@holly_cummins #RedHat aaaaaaaargh

Slide 11

Slide 11 text

@holly_cummins #RedHat aaaaaaaargh?

Slide 12

Slide 12 text

@holly_cummins #RedHat be a solutionist

Slide 13

Slide 13 text

@holly_cummins #RedHat how do we do make solutions?

Slide 14

Slide 14 text

@holly_cummins #RedHat green software foundation: principles

Slide 15

Slide 15 text

@holly_cummins #RedHat carbon awareness green software foundation: principles

Slide 16

Slide 16 text

@holly_cummins #RedHat carbon awareness green software foundation: principles where

Slide 17

Slide 17 text

@holly_cummins #RedHat carbon awareness green software foundation: principles where when

Slide 18

Slide 18 text

@holly_cummins #RedHat carbon awareness green software foundation: principles hardware efficiency where when

Slide 19

Slide 19 text

@holly_cummins #RedHat carbon awareness green software foundation: principles hardware efficiency where when elasticity

Slide 20

Slide 20 text

@holly_cummins #RedHat carbon awareness green software foundation: principles hardware efficiency where when elasticity utilisation

Slide 21

Slide 21 text

@holly_cummins #RedHat carbon awareness green software foundation: principles hardware efficiency electricity efficiency where when elasticity utilisation

Slide 22

Slide 22 text

@holly_cummins #RedHat algorithms carbon awareness green software foundation: principles hardware efficiency electricity efficiency where when elasticity utilisation

Slide 23

Slide 23 text

@holly_cummins #RedHat algorithms stack carbon awareness green software foundation: principles hardware efficiency electricity efficiency where when elasticity utilisation

Slide 24

Slide 24 text

@holly_cummins #RedHat algorithms stack carbon awareness hardware efficiency electricity efficiency where when elasticity utilisation green software foundation: principles

Slide 25

Slide 25 text

trick 1: electricity source

Slide 26

Slide 26 text

@holly_cummins #RedHat data center location matters

Slide 27

Slide 27 text

@holly_cummins #RedHat

Slide 28

Slide 28 text

@holly_cummins #RedHat

Slide 29

Slide 29 text

@holly_cummins #RedHat we need to talk about virginia

Slide 30

Slide 30 text

@holly_cummins #RedHat look at the sustainability information before choosing a hosting region

Slide 31

Slide 31 text

@holly_cummins #RedHat look at the sustainability information before choosing a hosting region choose a cloud provider who make this easy

Slide 32

Slide 32 text

@holly_cummins #RedHat time of day matters • (most) renewables are intermittent • if grid load is high, shortfalls are filled by fossil fuels

Slide 33

Slide 33 text

@holly_cummins #RedHat move the workload to the greenest place

Slide 34

Slide 34 text

@holly_cummins #RedHat move the workload to the greenest place

Slide 35

Slide 35 text

@holly_cummins #RedHat move the workload to the greenest place a data problem

Slide 36

Slide 36 text

@holly_cummins #RedHat move the workload to the greenest place a data problem an orchestration problem

Slide 37

Slide 37 text

@holly_cummins #RedHat carbon awareness green software foundation: principles hardware efficiency electricity efficiency

Slide 38

Slide 38 text

@holly_cummins #RedHat carbon awareness green software foundation: principles hardware efficiency electricity efficiency elasticity

Slide 39

Slide 39 text

@holly_cummins #RedHat carbon awareness green software foundation: principles hardware efficiency electricity efficiency elasticity utilisation

Slide 40

Slide 40 text

#RedHat @holly_cummins 2017 server-survey 25% doing no useful work (16,000 sampled)

Slide 41

Slide 41 text

#RedHat @holly_cummins 2017 server-survey 25% doing no useful work (16,000 sampled) “perhaps someone forgot to turn them off”

Slide 42

Slide 42 text

#RedHat @[email protected] https://www.anthesisgroup.com/wp-content/uploads/2019/11/Comatose-Servers-Redux-2017.pdf 2014 server-survey 29% active less than 5% of the time (4,000 sampled)

Slide 43

Slide 43 text

@holly_cummins #RedHat

Slide 44

Slide 44 text

@holly_cummins #RedHat cloud elasticity? https://www.business2community.com/cloud-computing/overprovisioning-always-on-resources-lead-to-26-6-billion-in-public-cloud-waste-expected-in-2021-02381033

Slide 45

Slide 45 text

@holly_cummins #RedHat cloud elasticity? 2021: https://www.business2community.com/cloud-computing/overprovisioning-always-on-resources-lead-to-26-6-billion-in-public-cloud-waste-expected-in-2021-02381033

Slide 46

Slide 46 text

@holly_cummins #RedHat cloud elasticity? 2021: $26.6 billion wasted https://www.business2community.com/cloud-computing/overprovisioning-always-on-resources-lead-to-26-6-billion-in-public-cloud-waste-expected-in-2021-02381033

Slide 47

Slide 47 text

@holly_cummins #RedHat cloud elasticity? 2021: $26.6 billion wasted by always-on cloud instances https://www.business2community.com/cloud-computing/overprovisioning-always-on-resources-lead-to-26-6-billion-in-public-cloud-waste-expected-in-2021-02381033

Slide 48

Slide 48 text

@holly_cummins #RedHat elasticity we used to leave our applications running all the time when we scripted turning them off at night, we reduced our cloud bill by 30% @darkandnerdy, Chicago DevOpsDays

Slide 49

Slide 49 text

@holly_cummins #RedHat it’s not just electricity

Slide 50

Slide 50 text

@holly_cummins #RedHat it’s not just electricity it’s water

Slide 51

Slide 51 text

@holly_cummins #RedHat it’s not just electricity it’s water it’s e-waste

Slide 52

Slide 52 text

@holly_cummins #RedHat yes, turning applications off is scary

Slide 53

Slide 53 text

@holly_cummins #RedHat what if … turning applications off was no more scary than turning the lights off?

Slide 54

Slide 54 text

@holly_cummins #RedHat ultimate elasticity

Slide 55

Slide 55 text

@holly_cummins #RedHat ultimate elasticity

Slide 56

Slide 56 text

@holly_cummins #RedHat turning it off and on again must ultimate elasticity

Slide 57

Slide 57 text

@holly_cummins #RedHat turning it off and on again must • be fast ultimate elasticity

Slide 58

Slide 58 text

@holly_cummins #RedHat turning it off and on again must • be fast • actually work ultimate elasticity

Slide 59

Slide 59 text

@holly_cummins #RedHat turning it off and on again must • be fast • actually work • idempotency ultimate elasticity

Slide 60

Slide 60 text

@holly_cummins #RedHat turning it off and on again must • be fast • actually work • idempotency • resiliency ultimate elasticity

Slide 61

Slide 61 text

@holly_cummins #RedHat architect things to be turned off and on often

Slide 62

Slide 62 text

@holly_cummins #RedHat trick 2: LightSwitchOps architect things to be turned off and on often

Slide 63

Slide 63 text

@holly_cummins #RedHat trick 2: LightSwitchOps architect things to be turned off and on often

Slide 64

Slide 64 text

#Quarkus @holly_cummins you can’t optimise what you can’t measure

Slide 65

Slide 65 text

@holly_cummins #RedHat FinOps figuring out who in your company forgot to turn off their cloud

Slide 66

Slide 66 text

@holly_cummins #RedHat

Slide 67

Slide 67 text

@holly_cummins #RedHat backstage.io

Slide 68

Slide 68 text

@holly_cummins #RedHat backstage.io •cost insights plugin

Slide 69

Slide 69 text

@holly_cummins #RedHat backstage.io •cost insights plugin •cloud carbon footprint plugin

Slide 70

Slide 70 text

@holly_cummins #RedHat algorithms stack carbon awareness hardware efficiency electricity efficiency where when elasticity utilisation green software foundation: principles

Slide 71

Slide 71 text

@holly_cummins #RedHat efficiency

Slide 72

Slide 72 text

@holly_cummins #RedHat what programming languages use the least energy?

Slide 73

Slide 73 text

@holly_cummins #RedHat what programming languages use the most energy?

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

energy efficiency of programming languages

Slide 76

Slide 76 text

@holly_cummins #RedHat

Slide 77

Slide 77 text

@holly_cummins #RedHat

Slide 78

Slide 78 text

@holly_cummins #RedHat what’s the most carbon-efficient java?

Slide 79

Slide 79 text

@holly_cummins #RedHat

Slide 80

Slide 80 text

@holly_cummins #RedHat

Slide 81

Slide 81 text

@holly_cummins #RedHat does being small and fast reduce carbon footprint?

Slide 82

Slide 82 text

#Quarkus @holly_cummins measure, don’t guess.

Slide 83

Slide 83 text

digression: measuring carbon is hard

Slide 84

Slide 84 text

@holly_cummins #RedHat step 1: measure power usage

Slide 85

Slide 85 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement

Slide 86

Slide 86 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement more complete

Slide 87

Slide 87 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement more complete needs access to the wall

Slide 88

Slide 88 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement more complete needs access to the wall and equipment

Slide 89

Slide 89 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement more complete needs access to the wall and equipment RAPL

Slide 90

Slide 90 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement more complete needs access to the wall and equipment RAPL programmatically accessible

Slide 91

Slide 91 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement more complete needs access to the wall and equipment RAPL programmatically accessible misses some components

Slide 92

Slide 92 text

@holly_cummins #RedHat step 1: measure power usage wall power measurement more complete needs access to the wall and equipment data costs carbon RAPL programmatically accessible misses some components

Slide 93

Slide 93 text

@holly_cummins #RedHat

Slide 94

Slide 94 text

@holly_cummins #RedHat

Slide 95

Slide 95 text

@holly_cummins #RedHat

Slide 96

Slide 96 text

@holly_cummins #RedHat load

Slide 97

Slide 97 text

@holly_cummins #RedHat Source: Teads EC2 instances carbon dataset

Slide 98

Slide 98 text

@holly_cummins #RedHat coal wind step 2: convert power usage to carbon solar

Slide 99

Slide 99 text

@holly_cummins #RedHat published energy mixes (these are made-up energy mixes)

Slide 100

Slide 100 text

@holly_cummins #RedHat published energy mixes … but methodologies are not open (these are made-up energy mixes)

Slide 101

Slide 101 text

@holly_cummins #RedHat published energy mixes … but methodologies are not open … or consistent (these are made-up energy mixes)

Slide 102

Slide 102 text

@holly_cummins #RedHat step 3: embedded carbon (manufacturing has costs)

Slide 103

Slide 103 text

@holly_cummins #RedHat simpler models

Slide 104

Slide 104 text

@holly_cummins #RedHat all models are wrong, some are useful

Slide 105

Slide 105 text

@holly_cummins #RedHat trick 3: vrrrrrooooooooom model* * a made-up name

Slide 106

Slide 106 text

@holly_cummins #RedHat

Slide 107

Slide 107 text

@holly_cummins #RedHat these two columns are almost the same

Slide 108

Slide 108 text

@holly_cummins #RedHat energy consumption (sort of, mostly) is proportional to execution time

Slide 109

Slide 109 text

@holly_cummins #RedHat trick 4: economic model*

Slide 110

Slide 110 text

@holly_cummins #RedHat trick 4: economic model* * "economic input-output life cycle assessment"

Slide 111

Slide 111 text

@holly_cummins #RedHat reducing your cloud spend (probably) reducing your carbon footprint* hardware spend electricity bill * if you keep other factors the same

Slide 112

Slide 112 text

@holly_cummins #RedHat worked(ish) example: what’s the carbon footprint of ChatGPT?

Slide 113

Slide 113 text

@holly_cummins #RedHat worked(ish) example: what’s the carbon footprint of ChatGPT?

Slide 114

Slide 114 text

@holly_cummins #RedHat worked(ish) example: what’s the carbon footprint of ChatGPT? $50,000- $700,000 running costs per day https://www.businessinsider.com/how-much-chatgpt-costs-openai-to-run-estimate-report-2023-4

Slide 115

Slide 115 text

@holly_cummins #RedHat worked(ish) example: what’s the carbon footprint of ChatGPT? $3,000,000- $50,000,000 training costs $50,000- $700,000 running costs per day https://www.businessinsider.com/how-much-chatgpt-costs-openai-to-run-estimate-report-2023-4

Slide 116

Slide 116 text

so … quarkus?

Slide 117

Slide 117 text

@holly_cummins #RedHat density Source: Clement Escoffier cost impact of framework choice Setup: • 800 requests/second, over 20 days • SLA > 99% • AWS instances Assumptions: • Costs are for us-east-1 data centre

Slide 118

Slide 118 text

@holly_cummins #RedHat Setup: • 800 requests/second, over 20 days • SLA > 99% Assumptions: • 50% load • us-east-1 data centre • Teads dataset Source: Clement Escoffier x Teads cloud carbon impact of framework choice

Slide 119

Slide 119 text

Slide 120

Slide 120 text

Slide 121

Slide 121 text

@holly_cummins #RedHat Setup: • REST + CRUD • large heap • RAPL energy measurement • multiple instances to support high load   Assumptions: • US energy mix Source: John O’Hara climate impact as a function of load vrrrooom model in action: quarkus on JVM has the smallest footprint … because it has the highest throughput

Slide 122

Slide 122 text

@holly_cummins #RedHat what about memory? more complete model: both throughput (execution time) and memory contribute to carbon footprint

Slide 123

Slide 123 text

@holly_cummins #RedHat what about memory? more complete model: both throughput (execution time) and memory contribute to carbon footprint quarkus is most efficient for • startup time • throughput • memory (RSS size)

Slide 124

Slide 124 text

@holly_cummins #RedHat trick 5: use quarkus

Slide 125

Slide 125 text

@holly_cummins #RedHat trick 5: use quarkus quarkus ‘automatically’ saves

Slide 126

Slide 126 text

@holly_cummins #RedHat trick 5: use quarkus quarkus ‘automatically’ saves • time

Slide 127

Slide 127 text

@holly_cummins #RedHat trick 5: use quarkus quarkus ‘automatically’ saves • time • money

Slide 128

Slide 128 text

@holly_cummins #RedHat trick 5: use quarkus quarkus ‘automatically’ saves • time • money • carbon (~2x)

Slide 129

Slide 129 text

@holly_cummins #RedHat trick 5: use quarkus quarkus ‘automatically’ saves • time • money • carbon (~2x) • … even when Spring compatibility libraries are used (almost no code changes except dependencies and tests)

Slide 130

Slide 130 text

Slide 131

Slide 131 text

@holly_cummins #RedHat aaaaaaaargh?

Slide 132

Slide 132 text

@holly_cummins #RedHat carbon awareness hardware efficiency electricity efficiency green software foundation: principles utility

Slide 133

Slide 133 text

@holly_cummins #RedHat is using this carbon giving value? utility

Slide 134

Slide 134 text

@holly_cummins #RedHat “no-regrets” solutions

Slide 135

Slide 135 text

@holly_cummins #RedHat

Slide 136

Slide 136 text

@holly_cummins #RedHat co-benefits

Slide 137

Slide 137 text

@holly_cummins #RedHat co-benefits the double win

Slide 138

Slide 138 text

@holly_cummins #RedHat co-benefits the double win win-win

Slide 139

Slide 139 text

@holly_cummins #RedHat co-benefits the double win win-win 1 + 1 = 3

Slide 140

Slide 140 text

@holly_cummins #RedHat co-benefits the double win win-win 1 + 1 = 3 twofer

Slide 141

Slide 141 text

@holly_cummins #RedHat co-benefits the double win win-win 1 + 1 = 3 twofer überwinden

Slide 142

Slide 142 text

@holly_cummins #RedHat co-benefits the double win win-win 1 + 1 = 3 twofer überwinden

Slide 143

Slide 143 text

@holly_cummins #RedHat climate solutions can make everything better

Slide 144

Slide 144 text

@holly_cummins #RedHat remember the zombie servers? https://www.business2community.com/cloud-computing/overprovisioning-always-on-resources-lead-to-26-6-billion-in-public-cloud-waste-expected-in-2021-02381033

Slide 145

Slide 145 text

@holly_cummins #RedHat remember the zombie servers? what else could we do with that $26.6 billion of wasted cloud spend? https://www.business2community.com/cloud-computing/overprovisioning-always-on-resources-lead-to-26-6-billion-in-public-cloud-waste-expected-in-2021-02381033

Slide 146

Slide 146 text

@holly_cummins #RedHat the double-win turning things off saves a lot of money

Slide 147

Slide 147 text

@holly_cummins #RedHat the double-win renewable electricity is 9x cheaper

Slide 148

Slide 148 text

@holly_cummins #RedHat the double-win renewable electricity is 9x cheaper hosting in Montreal:

Slide 149

Slide 149 text

@holly_cummins #RedHat the double-win renewable electricity is 9x cheaper hosting in Montreal: 88% less carbon than the same workload in London

Slide 150

Slide 150 text

@holly_cummins #RedHat the double-win renewable electricity is 9x cheaper hosting in Montreal: 88% less carbon than the same workload in London and it’s 15% cheaper

Slide 151

Slide 151 text

@holly_cummins #RedHat remember the vrrrrrooooooooom model? (probably not, it was a made-up name)

Slide 152

Slide 152 text

@holly_cummins #RedHat we need a new mental model for ‘virtuous’ car: high max speed means high fuel usage per mile travelled

Slide 153

Slide 153 text

@holly_cummins #RedHat software: high max transactions means low carbon per transaction we need a new mental model for ‘virtuous’ car: high max speed means high fuel usage per mile travelled

Slide 154

Slide 154 text

@holly_cummins #RedHat the double-win

Slide 155

Slide 155 text

@holly_cummins #RedHat the double-win

Slide 156

Slide 156 text

@holly_cummins #RedHat “this is not sacrifice. it’s advancement.” – Dr. Jonathan Foley

Slide 157

Slide 157 text

@holly_cummins #RedHat tl;dpa (too long; didn’t pay attention) trick 1: choose your hosting wisely trick 2: architect to be able to turn stuff off (LightSwitchOps) trick 3: the vrrrooooom model says faster is greener trick 4: the economic model says cheaper is greener trick 5: choose a fast and energy-efficient framework, such as quarkus