Architecting for Sustainability: A Software Architect’s Guide to Green Software Eoin Woods October 2024

Image by Markus Kammermann from Pixabay Climate change is one of the most urgent and serious problems facing humanity

What does software have to do with climate change? • Anything to do with measuring climate change impact is complicated, but … • Estimates for the impact of ICT are 2-4% of global emissions • By comparison aviation is about 3% ! • And it is growing, at least in part due to AI • Data centres may use 5% of electricity by 2030 • Microsoft’s GHG emissions up 30% last year1 Image by István from Pixabay 1: The Verge: https://tinyurl.com/ms-emissions

Data-Based Studies All Point The Same Way https://www.sciencedirect.com/science/article/pii/S2666389921001884#sec1.1 2024

Eoin Woods • Endava’s Chief Engineer, based in London (since 2015) • 10+ years in products - Bull, Sybase, InterTrust • 10 years in capital markets - UBS and BGI • PhD in Software Architecture & Energy Efficiency1 (2019) • Author, editor, speaker 1 https://repository.uel.ac.uk/item/8459v

Green Software Fundamentals Architectural Principles & Tactics Dealing with AI Case Study Beginning Your Journey 01 02 03 04 05 Agenda

Green Software Fundamentals 01

Greenhouse Gas (“Carbon”) and Software Supply Chain Construction Hardware Manufacturing Fossil Fuels Embodied Emissions (kgCO2e) Operational Emissions(kgCO2e) Renewables Software Application Energy (kWh) Embodied emissions are the emissions from creating the data centre and hardware Operational emissions are the emissions from the energy required to run the system Credit to James Costerton for the graphical concept) <------------ Grid Carbon Intensity (gCO2e/kWh) -----------> +

Embodied Emissions Emissions created by creating and disposing of a piece of hardware Sources: Tech Carbon Standard & Greenly Institute

Operational Emissions Emissions created during operation Source: electricitymaps.com 6 September 2024 GHG intensity of energy X Amount of energy used • Demand • Efficiency

What can software architects do? • Principles? • Tactics? • Design patterns? • Software tools? Let’s consider some principles and tactics …

Some Architectural Principles & Tactics 02

GSF Principles of Green Software 1. Emit the least amount of carbon possible. 2. Use the least amount of energy possible. 3. Do more when the electricity is cleaner and do less when the electricity is dirtier. 4. Use the least amount of embodied carbon possible. 5. What you can't measure, you can't improve. 6. Understand the exact mechanism of carbon reduction. https://learn.greensoftware.foundation

A Few More Principles for Green Software • Link emissions to business transactions1 • Emissions are a system level concern1 • Emissions need a cross-functional team (applications + infrastructure + business)1 • Focus on high impact changes • Don’t ignore non-technology emissions (e.g. flights) • Understand importance of software emissions in total organisational emissions 1 https://repository.uel.ac.uk/item/8459v, Chapter 5

Tactic 1: Demand Shifting • We often have flexibility over when and where we do our processing • Run batches at different times • Execute workload in locations • Electricity grids vary in their GHG intensity https://www.freepik.com/free-vector /clocks-with-different-hand-times-set_263632366.htm

Tactic 1: Demand Shifting • Move workload to times or places with lower grid carbon intensity • Move your workload in time to when your local grid is ”greener” • Move some of your workload to another grid which is “greener” • Trade offs: • complexity • hidden emissions (e.g. data movement) • driving cloud vendor behaviour

Tactic 2: Demand Shaping • We can often vary intensity of processing in our system • Less precision, less graphics, reduced features, … • We already know the grid varies in GHG intensity …

Tactic 2: Demand Shaping • Do less processing when the grid has high GHG intensity, more when it has less • Trade-offs: complexity, reduced service may not be acceptable

Tactic 3: Move Work to Client Devices • Desktops and mobile devices are almost idle for most of their lives • High embodied carbon footprints • Nearly zero emissions from additional workload • Moving work to clients reduces server workload and so environmental cost

Tactic 3: Move Work to Client Devices Examples: • Consider passing raw results to the client for intensive compute tasks • Cache data on the client and process there to avoid server compute and memory • Move resilience mechanism to the client to simplify server-side resilience Trade-offs: • Complexity, dependencies on client devices, security, data movement

Tactic 4: Measurement • All emissions related measurement is difficult and tends to be via estimation • Measuring helps to motivate and direct action • Cloud vendor dashboards provide some assistance • But only GCP’s uses location-based data • Cloud Carbon Footprint is OSS alternative • Data: Climatiq, Watt Time, Electricity Maps

Other Tactics • Provision only what is needed when it is needed • e.g. dynamically increase compute when needed • Consider compression for large data sets (at rest and in motion) • Cache static data • Minimise stored data (e.g. retention policies) • Turn off test and dev environments when not in use You will notice that a lot of this is efficient and cost effective too Some more ideas at https://patterns.greensoftware.foundation, but … it depends!

Dealing with AI 03

Sustainability for AI • AI-based systems require huge amounts of computing resources • processor, storage, memory • To no-one’s surprise they are a sustainability horror show • Why Microsoft’s emissions grew 30% last year1 • Model size and amount of training and inference (querying) are key determinants 1: The Verge: https://tinyurl.com/ms-emissions

Sustainability in the AI-System Lifecycle Business & Data Understanding Evaluation Emissions as a quality attribute Use existing data sets Minimise collection Minimise storage Minimise processing Consider pre-trained models Minimise model size Shrink models (quantisation) Demand shift training & eval Specialised hardware Consider shrinking model Deploy to lowest GHG intensity location Specialised hardware Demand shift usage Demand shape usage Monitor GHG emissions Emission level reduction CRISP-ML(Q)

Case Study - demand shifting for AI 04

Case Study – Demand Shifting AI Workload • Machine learning workloads are resource intensive so can be emissions intensive • This team wanted to minimise the GHG emissions from their ML batches • Example workload chosen was classifying cursive Hiranya (Japanese) characters • Typical execution time of 10 minutes, required many times per day • Goal to move workload in time or across regions to minimise GHG emissions

Case Study – Demand Shifting AI Workload Mladen Turanjanin James Costerton Usman Suhail Vladimir Dosen Aleksandar Miodragovic Zoran Janicijevic Miroslav Antalic Nevena Curcic Marko Marjanovic Nikola Ivanisevic Nikola Parezanovic Sladjana Mandic Meet the team …

Case Study – Demand Shifting AI Workload • For example, on one specific measured day … • Running at 20:00 in Western-Europe results in estimated emissions of 125g CO2e • Shifting region estimated to save 29g CO2e (~25%) • Shifting time estimated to save 17g CO2E (~15%) • Total possible savings ~40% This is clearly worth exploring !

Case Study – Demand Shifting AI Workload Scheduler Electricity Metrics Retriever GSF Carbon-Aware SDK Electricity Maps Region 1 ML Workload GMT runner.py ML Pipeline Green Metrics Tool (UI & API) Electricity Metrics DB Consumption Metrics DB Invocation API Call Region 2 ML Workload GMT runner.py • CO2 emissions from Electricity Maps via Carbon-Aware SDK • Green Metrics Tool used to collect electricity usage • Pipeline invoked via scheduler or manually (for ”urgent” option) • Pipeline can choose region to schedule ML workload • Pipeline can reschedule if not “urgent”

Case Study – Demand Shifting AI Workload Pipeline Workflow Logic

Case Study – Demand Shifting AI Workload Starting the Pipeline – Finding the Best Region

Case Study – Demand Shifting AI Workload Demand Shifting Decision

Case Study – Demand Shifting AI Workload • Green Metrics Tool provides insight into energy usage • Can provide comparisons over time

Case Study – Demand Shifting AI Workload • Automatic demand shifting across time and geographies, with manual bypass for urgent work • Near real time data-based decision using Electricity Maps to understand grid carbon intensity • Measurement of electricity usage via Green Metrics Tool to allow estimation of emissions

Beginning Your Journey 05

Getting Started Code Optimisation Architectural Patterns Clean Energy Do Less Learn More Books Training Organisations Hardware Energy Grid carbon intensity Energy consumption e.g. GSF patterns Beware ”it depends” Languages Optimisation Often not a good RoI

To Learn More https://tinyurl.com/lsf-gsw-course https://greensoftware.foundation https://sdialliance.org https://www.thegreenwebfoundation.org https://climateaction.tech

Useful Resources Software Carbon Intensity Specification (GSF) Tech Carbon Standard (Scott Logic)

Thank you … questions? Eoin Woods www.eoinwoods.info eoin.woods@endava.com @eoinwoodz @eoinwoods.bsky.social threads.net/@eoinwoodz

No content