apidays Singapore 2025 - The Quest for the Greenest LLM , Jean Philippe Ehret (Resilio)

The Quest for the Greenest LLM - Insights from the Frontlines
Jean Philippe Ehret, Founder BetterBytes (Now Resilio) - CTO at Emplify

apidays Singapore 2025
Where APIs Meet AI: Building Tomorrow's Intelligent Ecosystems
April 15 & 16, 2025

------

Check out our conferences at https://www.apidays.global/

Do you want to sponsor or talk at one of our conferences?
https://apidays.typeform.com/to/ILJeAaV8

Learn more on APIscene, the global media made by the community for the community:
https://www.apiscene.io

Explore the API ecosystem with the API Landscape:
https://apilandscape.apiscene.io/

Transcript

1. Product Development professionals using AI are most concerned about its environmental impact .
2. The quest for the greenest LLM Insights from the frontlines
3. Kia Ora, I’m Jean -Philippe Founder of BetterBytes by Resilio CTO of Emplify 25 years in IT. Still writing code. MSc (Industrial Science – Renewable Energy) by
4. Even in Middle Earth, We Burn Coal New Zealand: Stunning landscapes, rich in renewables. Yet we’re importing coal.
5. New Zealand’s data centers used to take 1.5% of our electricity. Soon, they’ll take 8% . And they’re eating our renewables. AI’s Quiet Pull on Clean Power
6. When Tech Companies Dig Deeper - Literally Microsoft is helping fund geothermal energy capacity in New Zealand to power new data centres.
7. Impact on your Scope 3 What about your company’s sustainability commitments?
8. First Study of its kind Carbon. Water. Raw Material. Full lifecycle. First-hand data.
9. First Study of its kind Carbon. Water. Raw Material. Full lifecycle. First-hand data.
10. What Models? The Association Of Data Scientist (Thanks Chris Simon) Small -> 3B -> MMLU: 61 (similar to LLAMA 3B) Larger -> 120B -> MMLU: 85 (similar to GPT4o) MMLU Massive Multitask Language Understanding
11. Carbon... Enormous? Or not 120B 3B : 21 500 tCO2e : 250 tCO2e
12. Carbon... Enormous? Or not 120B 3B : 21 500 tCO2e : 250 tCO2e 1.5m tCO2e/year
13. Carbon... Enormous? Or not 120B 3B : 21 500 tCO2e : 250 tCO2e
14. Carbon... Enormous? Or not
15. Electricity Mix -> Relocate Reduction by a factor 10!
16. Training vs Inference: Two Sides of the Emissions Coin Direct emissions: 1.14g (larger) vs 0.16g (small). For a 400 output tokens conversation
17. Training vs Inference: Two Sides of the Emissions Coin Direct emissions: 1.14g (bigger) vs 0.16g (small). Amortized: 37g vs 70g For a 400 output tokens conversation 37 70
18. The Greenest Prompt? None at All. Eco-design your software: Delete non -essentials Allow a budget for it. It is worth it!
19. Water scarcity 120B 3B : 314,000 m3 : 3,400 m3 60% Cooling 30% Hydropower For a 400 output tokens conversation
20. Water Scarcity Freshwater is scarce. Climate location matters. All water on, in, and above the Earth Liquid fresh water Fresh-water lakes and rivers
21. Water Scarcity Freshwater is scarce. Climate location matters. All water on, in, and above the Earth Liquid fresh water Fresh-water lakes and rivers
22. Water Scarcity Freshwater is scarce. Climate location matters. All water on, in, and above the Earth Liquid fresh water Fresh-water lakes and rivers
23. Raw Materials in the Red Zone 120B 3B : 750 kg Sb eq : 10 kg Sb eq 61% Hardware manufacturing
24. Raw Materials in the Red Zone 120B 3B : 750 kg Sb eq : 10 kg Sb eq 61% Hardware manufacturing For a 400 output tokens conversation
25. 25,000 GPUs for Training. GPT-4 used ~60,000 A100s. 5 to 6 years refresh cycle Nvidia H100
26. AI vs. Societal Essentials: A Resource Collision Course Mass Market AI is unsustainable
27. Supply -Demand issue https://www.mckinsey.com/industries/energy-and-materials/our-insights/global-materials-perspective
28. 4 Actions suggested by the Study • Use small/specialized models when possible • Headline generation, sentiment analysis, spam detection
29. 4 Actions suggested by the Study • Use small/specialized models when possible • Select Models trained in appropriate regions. • Which ones? -> Stay tuned
30. 4 Actions suggested by the Study • Use small/specialized models when possible • Select Models trained in appropriate regions. • Run open source models locally when it makes sense
31. 4 Actions suggested by the Study • Use small/specialized models when possible • Select Models trained in appropriate regions. • Run open source models locally when it makes sense • Apply Software eco-design principles for inferences
32. AI Can Accelerate Both Good and Harm • Jevon’s Paradox and rebound effect • Frugal AI and AI for good Efficiency ≠ footprint reduction.
33. AI Can Accelerate Both Good and Harm • Jevon’s Paradox and rebound effect • Frugal AI and AI for good Efficiency ≠ footprint reduction.
34. Thank you!