Scaling AI at the Dutch Tax Administration: A Risk-Based Control Frameworkvoor implementeren en uitrollen AI

Transcript

1. Scaling AI responsibly From Experimentation to AI in Production Nemko Digital & De Belastingdienst 10 Sep 2026
2. Nice to meet you! Bas Overtoom Managing Director Nemko Digital Hedwich Sijtsema Afdelingshoofd Business Ontwikkeling Analytics Belastingdienst © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
3. Introducing Nemko Digital
4. Nemko: Compliance without Complexity Strong heritage 1933 1991 1992 - 2003 2020 2024 Established by the Norwegian government as Norges Elektriske Materiellkontroll Became independent selfowned private foundation Established offices and laboratories around the world Global reach & local presence 28 locations on 3 continents Proven track record Roster of clients and services (not exhaustive) Over 850 employees worldwide. Offering services in more than 150 countries Serving 7,000 customers across 80 countries. In September 2025, Nemko and IBM jointly organized the AI Trust in Electronics Summit in Oslo http://www.nemko.com/sites/default/files/imagecache/140w/images/Ratt_tgl_0.jpg IECEE Launched Cybersecurity services Established Nemko Digital to consolidate AI & Data Trust services © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission. http://www.nemko.com/sites/default/files/imagecache/140w/images/KCC_small.jpg http://www.nemko.com/sites/default/files/imagecache/140w/images/EMKO.jpg
5. Our popular services We deliver Digital Trust through end-to-end compliance and advisory support, combining technical, regulatory, and process expertise. AI Act Regulatory Compliance ISO Readiness AI, data, cyber AI, data, cyber ISO 42001, ISO 27001… Global Market Access AI, data, cyber Nemko AI Trust Mark © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission. Technical Assurance / AI Testing AI Assurance Tools Governance Maturity Strategy and roadmaps AI, data, cyber AI, data, cyber
6. AI risks in perspective AI Impact assessment AI Impact Potential event A Impact on organisation Impact on individuals Impact on society Potential event B Potential event C … What organizations need to consider in addition (60-80% of attention) What regulators worry about (20-40% of attention) Based on ISO/IEC 42005:2024 Information technology — Artificial intelligence —AI system impact assessment © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission. Likelihood of event occurring
7. Common challenges when scaling AI Who owns AI in the organization? Legal, Compliance, Risk, IT, Data, Security, and Business Units all assume partial responsibility How to get beyond ‘risk v. opportunity’? Perspectives on AI and corresponding risk appetite vary greatly between stakeholders What is needed for AI Trust? Lack of a clear delimitation of ‘when is “good” good enough’ for an exponential technology How do we make it work in practice? Missing approach for cutting across functions and breaking down silos to create business vale
8. Case study: GenAI at scale in Government
9. From governing one AI use case to a repeatable pathway for scaling AI Were is the Dutch Tax Administration today: 1. Growing AI ambition Use AI more broadly to drive efficiency and public value, with risk management and responsible AI as key priorities. 2. Use-case driven approach Start with concrete AI use cases, learn what works in practice, and organize governance around delivery needs. 3. First case exposed the challenge In 2025 the first AI pilots showed that moving PoC to production was difficult due to uncertainty around risks, controls and responsibilities. 4. Built a practical approach Used the first case to develop and validate a practical, risk-based control framework for production. 5. Now scaling across use cases The approach is now being rolled out across multiple AI cases, supporting consistent implementation and production. © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
10. The GenAI Use case(s) Large corporations can ask a government body for a statement providing up-front clarity on how laws and regulations would be applied in specific scenarios – to aid their internal decision making, assure compliance, and avoid surprises. For the government body, this process is slow and time-consuming: it depends on many rules and regulations and that require meticulous verification of formal requirements. Initial application of AI in this process: • Write a report assessing if the formal requirements are met and the request can legally be considered. • Find relevant clauses in regulations and jurisprudence, supporting human assessors.
11. Hurdles to scaling • Promising pilot results and enthusiastic front-line workers, but existing processes do not support a clear path to production • Insufficient AI expertise across the organization to properly identify and delimit AI-related risks and their potential impact • Risk-averse organization with decision makers who are hesitant to ‘green-light’ any novel GenAI applications • A missing shared perspective on how to assess AI risks, nor which controls would be sufficient • Lack of formal ownership for AI-related decisions, sending the product team on a wild goose chase for approvals • Stalled go-live decisions, depriving the organizations and its clients from clear benefits (turn-around speed, efficiency)
12. Eight essentials organization building blocks for your AI success 8. Technology 7. Compliance 1. Leadership & Governance 6. Risk Management 2. AI Lifecycle Management 3. External Stakeholders 5. Operations 4. People & Culture © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
13. ! Attention points for model governance framework Nemko’s AI Maturity Model helps you drive structural improvement 1 Leadership & Governance AI Strategy & Business impact Value/business case & ROI Governance & org ! design Ethics & principles ! Finance & resources Human rights & Sustainability 2 AI Lifecycle Management Use case scoping ! design Data collection & ! quality Development & ! testing Deployment & ! monitoring Decommission & retirement Human in the loop! 3 External Stakeholders Partnership & codevelopment Customers & end-users 4 People & Culture AI trainings & literacy program Change & adoption 5 Operations Processes & procedures Data management AI inventory 6 Risk Management Risk assessments &! mitigation Risk tracking & ! escalation Incident & crisis ! management 7 Compliance Documentation &! record keeping Regulatory monitoring Audit readiness& assurance Data privacy requirements Transparency & ! explainability Fairness 8 Tech (Infra, Data & Cyber) Technical infrastructure Architecture & solution design Cybersecurity Data pipelines & platforms ! ! Internal controls ! ! Supply chain & procurement Culture: AI mindset Incentives & policy! alignment Continuous improvement AI Performance ! management © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission. ! !
14. Approach: Defining a governance control framework for a GenAI Kick-off Document analysis Risk Atlas • What are possible AI-specific risks? • Which risks are relevant for the use case? Risk analysis Control identification Applicable for / Van toepassing op "2. Doorzoeken OECD Guidelines" BD + usecase-specifieke risico's Likelihood / Waarschijnli Impact jkheid Yes Onvolledige of onjuiste OESOrichtlijnen 3 - Medium 1 - Very low Aanwijzen data-eigenaar voor inputbronnen Product owner Business owner De trainings- of fine-tuningdata is niet Nietrepresentatief voor de populatie of het Nauwkeurigheid representatieve Yes fenomeen, waardoor het model systematisch data scheve of onnauwkeurige resultaten geeft. Yes Onvolledige of onjuiste OESOrichtlijnen 3 - Medium 1 - Very low Data-validatie Data engineers Developers Risico's van trainingsdata De trainings- of fine-tuningdata is niet Nietrepresentatief voor de populatie of het Nauwkeurigheid representatieve Yes fenomeen, waardoor het model systematisch data scheve of onnauwkeurige resultaten geeft. Yes Onvolledige of onjuiste OESOrichtlijnen 3 - Medium 1 - Very low Biastoetsing Risico's van trainingsdata Nauwkeurigheid Datavervuiling Onjuiste, corrupte of ongewenste data wordt gebruikt in de training, wat leidt tot Yes vertekende of foutieve modeluitkomsten. Yes Datavervuiling in onderliggende LLM 3 - Medium 2 - Low Architects Training data requirements opnemen in LLM eisen voor Legal experts OSS / licencing Development teams Risico's van trainingsdata Nauwkeurigheid Datavervuiling Yes Yes Datavervuiling in onderliggende LLM 3 - Medium 2 - Low Architectuurstandaarden (bv. RAG vereist voor kennisSolution architect gebaseerde antwoorden, intent regocnition & Development team begrenzing vereist bij chat-interface) Risico's van trainingsdata Eerlijkheid Databias Yes Yes Databias in onderliggende LLM 3 - Medium 2 - Low Architects Training data requirements opnemen in LLM eisen voor Legal experts OSS / licencing Development teams Risico's van trainingsdata Eerlijkheid Databias Yes Yes Databias in onderliggende LLM 3 - Medium 2 - Low Aanvullende testen als use-case daarom vraagt (niet voor GO vooroverleg) Risico's van trainingsdata Eerlijkheid Databias Yes Yes Databias in onderliggende LLM 3 - Medium 2 - Low Biastoetsing Risico's van trainingsdata Robuustheid Datavergiftiging Een aanvaller voegt opzettelijk gemanipuleerde of schadelijke data toe aan Yes de trainingsset om het model te misleiden of te verzwakken. Yes Vervalste OESO-richtlijnen 1 - Very low 3 - Medium 4-ogen principe Data stewards Development team Risico's van trainingsdata Robuustheid Datavergiftiging Een aanvaller voegt opzettelijk gemanipuleerde of schadelijke data toe aan Yes de trainingsset om het model te misleiden of te verzwakken. Yes Vervalste OESO-richtlijnen 1 - Very low 3 - Medium Interne meldingsmogelijkheden Product owner Devewlopment team Business owner Description NL Applicable for / Van toepassing op "1. Toetsen Vereisten" Lifecycle stage NL Risk category NL Risk name NL Risico's van trainingsdata NietDe trainings- of fine-tuningdata is niet Nauwkeurigheid representatieve representatief voor de populatie of het Yes data fenomeen, waardoor het model systematisch Risico's van trainingsdata Onjuiste, corrupte of ongewenste data wordt gebruikt in de training, wat leidt tot vertekende of foutieve modeluitkomsten. Historische of maatschappelijke vooroordelen in de data worden door het model overgenomen en versterkt, met oneerlijke uitkomsten als gevolg. Historische of maatschappelijke vooroordelen in de data worden door het model overgenomen en versterkt, met oneerlijke uitkomsten als gevolg. Historische of maatschappelijke vooroordelen in de data worden door het model overgenomen en versterkt, met oneerlijke uitkomsten als gevolg. Nemko Digital perspectief op operationele controls (governance controls grijpen minder aan op individuele risico's), aanbevolen maatregelen Framework completion Final report & hand-over Key actors (RACI --> RASCI) TEVV (Testing Evaluation, Validation, Verification) experts Governance experts Risk assessment • How likely is it that the risk materializes? • If so, what will be the impact? Controls • Which measures can we implement to mitigate prioritized risks? • Who should be responsible & accountable for these? © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
15. Building on IBM Risk Atlas, we built a client-specific version Bron: IBM AI Risk Atlas © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
16. Based on the use case characteristics, we selected the relevant risks Risks classified by relevance Relevant risks by life stage 15 12 37 9 44 6 5 For both usecases For one usecase For no usecase Training data risks © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission. Inference risks Output risks Non-tech risks
17. Risk estimation based on use case-specific Net Risk GenAI in general Possible risk Conceptual risk Scope Net Risk Considering existing controls Controls This use case • Where Net Risk is not acceptable, additional controls are necessary • Implementation of additional Controls will bring down the Net Risk Gross Risk Theoretical, without controls © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission. • In the end, the remaining Rest Risk has to be accepted
18. For the relevant risks, we determined the likelihood and impact 5 4 Likelihood 3 2 1 1 2 Privacy Robustness Fairness Intellectual property Accuracy Value alignment 3 Impact Legal compliance Explainability ©Societal 2026 Allimpact rights reserved by Nemko Digital: are not to be distributed or shared without prior permission. Misusecontent and materials governance 4 5
19. To make the framework operational, we added a taxonomy of controls User instructions Pillar Strategy Prevention Detection Correction Focus Area Governance Data & Design Technical Defence Human Oversight 22 Testing 20 Monitoring 20 Human validation 14 Architecture guidelines and patterns 14 Documentation 10 Work instructions 9 LLM guidelines 9 Acceptance of residual risk 8 4-eyes principle 7 Access control 7 Data quality and processes 5 Data masking 5 Internal reporting options 4 User notification 4 Logs and data retention policy 3 Bias testing 3 Change management 3 DPIA pre scan 2 Number of risks © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
20. Key metrics 40/87 © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
21. We defined ownership principles and a risk management playbook Implementation steps Ownership principles Risk analysis Control specification Control implementation • Accountability for individual controls (implementation, operation, effectiveness) dependent on the nature of the measure. For example: o Model monitoring: Product Tech Lead o AI literacy: Business Team Lead • Accountability for risk acceptance lies with the Business Owner (with exception of cross-functional risks) • Accountability for oversight and coordination lies with the Product Owner Risk Mitigation Action Plan Risk acceptance Monitoring & feedback © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
22. AI Trust requires strong development and governance flows Impact assessment Use-case approval Use-case creation Use-case onboarding Data access approval Risk evaluation Model approval Risk & Value monitoring AI Governance flow AI Development flow Data access and Feature preparation engineering Model training Model and tuning registration © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission. Model Model serving and deployment monitoring
23. Today we are scaling up With the first GenAI use case successfully in production, Belastingdienst is now scaling a proven approach through a broader organization-wide AI program 01. Proven in production First GenAI use case successfully brought to production using the new riskbased framework and structured approach. 02. Repeatable path established Clear and repeatable approach for assessing and operationalizing additional GenAI use cases, workable for development teams. 03. Scaling across priority use cases Discussion shifted from abstract risks to concrete solutions, enabling teams to collaborate; now applied across 6 priority themes and 12 use cases. 04. Organization-wide AI program established A broader program has been established to scale AI across Belastingdienst, combining strong fundamentals with a focus on business value. © 2026 All rights reserved by Nemko Digital: content and materials are not to be distributed or shared without prior permission.
24. GenAI Control Framework | case study available for download Scan the QR code to download the full case study • Used a real GenAI use case to develop a practical risk and control framework • Identified and assessed 40+ GenAI risks and mapped 150+ controls • Created a repeatable approach to move GenAI use cases from pilot to production • Established the foundation to scale responsible GenAI across the organization