Slide 1

Slide 1 text

Architecting for Sustainability: A Software Architect’s Guide to Green Software Eoin Woods October 2024

Slide 2

Slide 2 text

Image by Markus Kammermann from Pixabay Climate change is one of the most urgent and serious problems facing humanity

Slide 3

Slide 3 text

What does software have to do with climate change? • Anything to do with measuring climate change impact is complicated, but … • Estimates for the impact of ICT are 2-4% of global emissions • By comparison aviation is about 3% ! • And it is growing, at least in part due to AI • Data centres may use 5% of electricity by 2030 • Microsoft’s GHG emissions up 30% last year1 Image by István from Pixabay 1: The Verge: https://tinyurl.com/ms-emissions

Slide 4

Slide 4 text

Data-Based Studies All Point The Same Way https://www.sciencedirect.com/science/article/pii/S2666389921001884#sec1.1 2024

Slide 5

Slide 5 text

Eoin Woods • Endava’s Chief Engineer, based in London (since 2015) • 10+ years in products - Bull, Sybase, InterTrust • 10 years in capital markets - UBS and BGI • PhD in Software Architecture & Energy Efficiency1 (2019) • Author, editor, speaker 1 https://repository.uel.ac.uk/item/8459v

Slide 6

Slide 6 text

Green Software Fundamentals Architectural Principles & Tactics Dealing with AI Case Study Beginning Your Journey 01 02 03 04 05 Agenda

Slide 7

Slide 7 text

Green Software Fundamentals 01

Slide 8

Slide 8 text

Greenhouse Gas (“Carbon”) and Software Supply Chain Construction Hardware Manufacturing Fossil Fuels Embodied Emissions (kgCO2e) Operational Emissions(kgCO2e) Renewables Software Application Energy (kWh) Embodied emissions are the emissions from creating the data centre and hardware Operational emissions are the emissions from the energy required to run the system Credit to James Costerton for the graphical concept) <------------ Grid Carbon Intensity (gCO2e/kWh) -----------> +

Slide 9

Slide 9 text

Embodied Emissions Emissions created by creating and disposing of a piece of hardware Sources: Tech Carbon Standard & Greenly Institute

Slide 10

Slide 10 text

Operational Emissions Emissions created during operation Source: electricitymaps.com 6 September 2024 GHG intensity of energy X Amount of energy used • Demand • Efficiency

Slide 11

Slide 11 text

What can software architects do? • Principles? • Tactics? • Design patterns? • Software tools? Let’s consider some principles and tactics …

Slide 12

Slide 12 text

Some Architectural Principles & Tactics 02

Slide 13

Slide 13 text

GSF Principles of Green Software 1. Emit the least amount of carbon possible. 2. Use the least amount of energy possible. 3. Do more when the electricity is cleaner and do less when the electricity is dirtier. 4. Use the least amount of embodied carbon possible. 5. What you can't measure, you can't improve. 6. Understand the exact mechanism of carbon reduction. https://learn.greensoftware.foundation

Slide 14

Slide 14 text

A Few More Principles for Green Software • Link emissions to business transactions1 • Emissions are a system level concern1 • Emissions need a cross-functional team (applications + infrastructure + business)1 • Focus on high impact changes • Don’t ignore non-technology emissions (e.g. flights) • Understand importance of software emissions in total organisational emissions 1 https://repository.uel.ac.uk/item/8459v, Chapter 5

Slide 15

Slide 15 text

Tactic 1: Demand Shifting • We often have flexibility over when and where we do our processing • Run batches at different times • Execute workload in locations • Electricity grids vary in their GHG intensity https://www.freepik.com/free-vector /clocks-with-different-hand-times-set_263632366.htm

Slide 16

Slide 16 text

Tactic 1: Demand Shifting • Move workload to times or places with lower grid carbon intensity • Move your workload in time to when your local grid is ”greener” • Move some of your workload to another grid which is “greener” • Trade offs: • complexity • hidden emissions (e.g. data movement) • driving cloud vendor behaviour

Slide 17

Slide 17 text

Tactic 2: Demand Shaping • We can often vary intensity of processing in our system • Less precision, less graphics, reduced features, … • We already know the grid varies in GHG intensity …

Slide 18

Slide 18 text

Tactic 2: Demand Shaping • Do less processing when the grid has high GHG intensity, more when it has less • Trade-offs: complexity, reduced service may not be acceptable

Slide 19

Slide 19 text

Tactic 3: Move Work to Client Devices • Desktops and mobile devices are almost idle for most of their lives • High embodied carbon footprints • Nearly zero emissions from additional workload • Moving work to clients reduces server workload and so environmental cost

Slide 20

Slide 20 text

Tactic 3: Move Work to Client Devices Examples: • Consider passing raw results to the client for intensive compute tasks • Cache data on the client and process there to avoid server compute and memory • Move resilience mechanism to the client to simplify server-side resilience Trade-offs: • Complexity, dependencies on client devices, security, data movement

Slide 21

Slide 21 text

Tactic 4: Measurement • All emissions related measurement is difficult and tends to be via estimation • Measuring helps to motivate and direct action • Cloud vendor dashboards provide some assistance • But only GCP’s uses location-based data • Cloud Carbon Footprint is OSS alternative • Data: Climatiq, Watt Time, Electricity Maps

Slide 22

Slide 22 text

Other Tactics • Provision only what is needed when it is needed • e.g. dynamically increase compute when needed • Consider compression for large data sets (at rest and in motion) • Cache static data • Minimise stored data (e.g. retention policies) • Turn off test and dev environments when not in use You will notice that a lot of this is efficient and cost effective too Some more ideas at https://patterns.greensoftware.foundation, but … it depends!

Slide 23

Slide 23 text

Dealing with AI 03

Slide 24

Slide 24 text

Sustainability for AI • AI-based systems require huge amounts of computing resources • processor, storage, memory • To no-one’s surprise they are a sustainability horror show • Why Microsoft’s emissions grew 30% last year1 • Model size and amount of training and inference (querying) are key determinants 1: The Verge: https://tinyurl.com/ms-emissions

Slide 25

Slide 25 text

Sustainability in the AI-System Lifecycle Business & Data Understanding Evaluation Emissions as a quality attribute Use existing data sets Minimise collection Minimise storage Minimise processing Consider pre-trained models Minimise model size Shrink models (quantisation) Demand shift training & eval Specialised hardware Consider shrinking model Deploy to lowest GHG intensity location Specialised hardware Demand shift usage Demand shape usage Monitor GHG emissions Emission level reduction CRISP-ML(Q)

Slide 26

Slide 26 text

Case Study - demand shifting for AI 04

Slide 27

Slide 27 text

Case Study – Demand Shifting AI Workload • Machine learning workloads are resource intensive so can be emissions intensive • This team wanted to minimise the GHG emissions from their ML batches • Example workload chosen was classifying cursive Hiranya (Japanese) characters • Typical execution time of 10 minutes, required many times per day • Goal to move workload in time or across regions to minimise GHG emissions

Slide 28

Slide 28 text

Case Study – Demand Shifting AI Workload Mladen Turanjanin James Costerton Usman Suhail Vladimir Dosen Aleksandar Miodragovic Zoran Janicijevic Miroslav Antalic Nevena Curcic Marko Marjanovic Nikola Ivanisevic Nikola Parezanovic Sladjana Mandic Meet the team …

Slide 29

Slide 29 text

Case Study – Demand Shifting AI Workload • For example, on one specific measured day … • Running at 20:00 in Western-Europe results in estimated emissions of 125g CO2e • Shifting region estimated to save 29g CO2e (~25%) • Shifting time estimated to save 17g CO2E (~15%) • Total possible savings ~40% This is clearly worth exploring !

Slide 30

Slide 30 text

Case Study – Demand Shifting AI Workload Scheduler Electricity Metrics Retriever GSF Carbon-Aware SDK Electricity Maps Region 1 ML Workload GMT runner.py ML Pipeline Green Metrics Tool (UI & API) Electricity Metrics DB Consumption Metrics DB Invocation API Call Region 2 ML Workload GMT runner.py • CO2 emissions from Electricity Maps via Carbon-Aware SDK • Green Metrics Tool used to collect electricity usage • Pipeline invoked via scheduler or manually (for ”urgent” option) • Pipeline can choose region to schedule ML workload • Pipeline can reschedule if not “urgent”

Slide 31

Slide 31 text

Case Study – Demand Shifting AI Workload Pipeline Workflow Logic

Slide 32

Slide 32 text

Case Study – Demand Shifting AI Workload Starting the Pipeline – Finding the Best Region

Slide 33

Slide 33 text

Case Study – Demand Shifting AI Workload Demand Shifting Decision

Slide 34

Slide 34 text

Case Study – Demand Shifting AI Workload • Green Metrics Tool provides insight into energy usage • Can provide comparisons over time

Slide 35

Slide 35 text

Case Study – Demand Shifting AI Workload • Automatic demand shifting across time and geographies, with manual bypass for urgent work • Near real time data-based decision using Electricity Maps to understand grid carbon intensity • Measurement of electricity usage via Green Metrics Tool to allow estimation of emissions

Slide 36

Slide 36 text

Beginning Your Journey 05

Slide 37

Slide 37 text

Getting Started Code Optimisation Architectural Patterns Clean Energy Do Less Learn More Books Training Organisations Hardware Energy Grid carbon intensity Energy consumption e.g. GSF patterns Beware ”it depends” Languages Optimisation Often not a good RoI

Slide 38

Slide 38 text

To Learn More https://tinyurl.com/lsf-gsw-course https://greensoftware.foundation https://sdialliance.org https://www.thegreenwebfoundation.org https://climateaction.tech

Slide 39

Slide 39 text

Useful Resources Software Carbon Intensity Specification (GSF) Tech Carbon Standard (Scott Logic)

Slide 40

Slide 40 text

Thank you … questions? Eoin Woods www.eoinwoods.info [email protected] @eoinwoodz @eoinwoods.bsky.social threads.net/@eoinwoodz

Slide 41

Slide 41 text

No content