When to Gen AI? A strategic framework for real-world AI adoption

Generative AI promises to reshape every industry, but a one-size-fits-all approach is a recipe for failure. Where and when should you use what kind of AI? In his session at apidays Australia 2025, Rob dissects the critical trade-offs between probabilistic models that offer creativity and power, and deterministic systems that deliver predictable precision. Navigate the complex landscape of capability, cost, compliance, and risk - helping you choose the right AI for the right job.

------------------------------------

Conference Details:
Conference: apidays Australia 2025
Theme: Platforms, Products, and People: The Power of APIs in the Age of AI
Date: 29 - 30 October 2025 • MCEC, Melbourne Australia

--------------------------

Resources from apidays:
Join our upcoming conferences: https://www.apidays.global/
Read the latest API news: https://www.apiscene.io
Explore the API Landscape: https://apilandscape.apiscene.io/

Transcript

1. When to (Gen) AI? A Strategic Framework for Real-World AI Adoption Rob Valk Engineering CTO @ Deloitte Australia
2. 2023 • All the AI! • The Smartphone Killer • TIME Magazine “The Best Inventions of 2023” 2024 • “It’s… not good” • ”Not even close” • Official fire hazard 2025 • Sold off for parts • Inducted into the Museum of Failure Who am I?
3. (Generative) AI is not always the right answer How can we know where and when to use what kind of AI?
4. Deep Learning models are Probabilistic Code is Deterministic Where? Solution run-time behaviour Nuclear reactor control system AI generated code Diffusion image generation AI systems are a balance of certainty and creativity Sometimes chosen, sometimes forced AI agent AI workflow LLM temperature
5. When? Run-time inference Pre-training AI compute focus Supervised learning Reasoning models RAG systems Large language models Probabilistic Deterministic AI systems are a balance of training and inference Sometimes chosen, sometimes forced
6. Foundational Model Development Medical Imaging Analysis Fine-tuning Foundational Models Drug Discovery & Genomics Demand Forecasting Credit Scoring & Risk Assessment General Predictive Analytics Recommendation Engines Supply Chain Optimisation Basic Chatbots & Digital Assistants Generative AI for Text/Image Creation Advanced Conversational AI Cybersecurity Anomaly Detection Real-time Fraud Detection Retrieval-Augmented Generation (RAG) Algorithmic Trading Single AI Agent Task Automation Multi-Agent System Simulations Autonomous Vehicle Perception Systems solution run-time behaviour AI compute focus Probabilistic Deterministic Pre-training Run-time inference
7. Pretty chart… So what?
8. where and when shapes our strategy for what
9. AI Strategy Framework Functionality Technology Performance Reliability Compliance Cost Risk Skills trade-offs focus big bets - -
10. Functionality Probabilistic Deterministic ✅ consistent ✅ precise ❌ inflexible ✅ creative ✅ ‘human like’ ✅ handles ambiguity ❌ inconsistent ❌ unexplainable Pre-training Run-time inference ✅ broad knowledge ✅ re-usable (transfer learning) ❌ knowledge is static ✅ dynamic reasoning ✅ current information ✅ prompt engineering ❌ prompt brittleness
11. Technology Probabilistic Deterministic ✅ commodity tech stack ✅ easier to test ❌ development complexity at scale ✅ adaptive ✅ conversational ✅ handles ambiguity ❌ harder to test ❌ specialised compute Pre-training Run-time inference ✅ optimised models ✅ simplifies deployment ❌ slow iteration ❌ proprietary compute lock-in ✅ range of compute options ✅ dynamic resource scaling ❌ hardware requirements
12. Performance Probabilistic Deterministic ✅ predictable ✅ resource-efficient ✅ fast (with correct hardware) ❌ resource-intensive ❌ transformers are slow and hungry Pre-training Run-time inference ✅ fast run-time responses ✅ batch training efficiencies ❌ no optimisation at run-time ✅ run-time optimisation possible ❌ increased latency ❌ context limits ❌ highly variable response times
13. Reliability Probabilistic Deterministic ✅ no variance ✅ accountability ❌ limited to pre-defined logic ✅ generalisation ✅ resistance to noise ❌ imperfect ❌ hallucinations Pre-training Run-time inference ✅ testing built in to training process ✅ known accuracy for known scenarios ❌ outdated knowledge ❌ brittle to data changes ✅ dynamic grounding ✅ visible reasoning ❌ prompt sensitivity ❌ injection vulnerabilities
14. Compliance Probabilistic Deterministic ✅ well-understood compliance regimes ✅ auditable, documented ❌ inflexible ✅ guardrails ❌ poor transparency ❌ bias ❌ evolving compliance regimes Pre-training Run-time inference ✅ centralised governance ✅ data provenance ❌ lifecycle compliance ❌ copyright and data removal ✅ dynamic filtering ❌ data exposure
15. Cost Probabilistic Deterministic ✅ lowest operational costs ❌ development & maintenance costs ✅ consumption pricing ✅ FOSS models ❌ GPU compute costs ❌ environmental footprint Pre-training Run-time inference ✅ amortised training costs ❌ Sacrifice 99% of CAPEX to Jensen Huang ✅ token costs tied closer to value ✅ more infra choices for inference ❌ Sacrifice 99% of OPEX to Sam Altm
16. Risk Probabilistic Deterministic ✅ (more) predictable failure modes ✅ explainability ❌ systemic defects ✅ ‘fuzzy’ risk detection ❌ bias, hallucinations ❌ environmental footprint ❌ false negative/positive ❌ explainability Pre-training Run-time inference ✅ accuracy evaluated during training ❌ embedded bias ✅ higher-quality reasoning ✅ adapts to specific environment ❌ uncontrolled cost ❌ data privacy
17. Skills Probabilistic Deterministic ✅ broadest talent pool ❌ declining growth/interest ✅ strong skills investment ❌ skills shortage Pre-training Run-time inference ✅ High skills leverage ❌ Specialised data science skills – premium $$ ✅ easier skills transfer ❌ evolving practices ❌ MLOps skills shortage
18. where and when shapes our strategy for what
19. A place for everything, everything in its place. Benjamin Franklin