TEST SOFTWARE exactpro.com Using AI for Streamlining Regulatory Compliance Solutions In Financial Organisations Iosif Itkin, CEO & co-founder, Exactpro CTO, Murabex AI Lecture Series University of Luxembourg
SOFTWARE TO TEST SOFTWARE Meet the speaker Iosif Itkin is co-founder and co-CEO of Exactpro – an independent provider of AI-enabled software testing, development and consultancy services for financial organisations. Founded in 2009, Exactpro has a client base of major exchanges, post-trade platform operators, banks and technology vendors across 25 countries. Iosif’s expertise at the intersection of high-availability systems and capital markets has enabled him to successfully implement testing strategies and facilitate technology transformations within exchanges, investment banks and clearing and settlement organisations in London, New York, Milan, Singapore, Sydney and other major financial centres. Iosif is a co-author of ‘Introduction to AI Testing: Guide to ISTQB® CT-AI Certification’ released in September 2025 by BCS Publishing. Iosif regularly uses coding assistants in his day-to-day work for a variety of testing and development purposes. Iosif Itkin CEO & co-Founder, Exactpro CTO, Murabex
Exchange Platforms AI-enabled systems in FMIs – Market Surveillance and Smart Order Router Timeless classic – Knight Capital compliance story The difference between humans and software The Compliance Burden in Financial Technology Regulatory Compliance as a Software Engineering Problem Compliance by Design Example – Murabex Platform Building vs. Testing – Scout mindset Probing the infinite space – Labeling and Coverage Building a Data-intensive Compliance System Questions and Answers
TO TEST SOFTWARE Exactpro is an independent provider of AI-enabled software testing services for financial organisations. Our clients are exchanges, post-trade platform operators and banks across 25 countries. We help our clients to decrease time to market, maintain regulatory compliance, improve scalability, latency and operational resiliency. On top of AI-driven testing and automation capabilities, we offer industry-proven expertise in test strategy development, software development and prototyping, facilitating regulatory compliance and reporting, AI Literacy building and support of enterprise-level AI integration. Headquartered in the UK, Exactpro operates delivery centres in Georgia, Sri Lanka, Armenia, the UK, representative offices in the US, Canada and Italy and a global network of consultants. See more partners & clients Exactpro’s client network See our awards About Exactpro Our Client network spans more than a half of top 20 global exchange groups
2. Liquidity (ability to enter/exit) 3. Price discovery 4. Standardisation and rules 5. Risk distribution 6. Infrastructure and trust 7. Benchmarking and ecosystem effects Why Exchanges are Important
SOFTWARE TO TEST SOFTWARE Load injection toolset th2-shark Trading GWs Lit Book Liquidity Partners CSD Trading Desktops Client Brokerage Systems Market Data GWs Distribution System Surveillance System Dark & Algo- Crossing Book Smart Order Router Reference Pricing Server Static Data MIS Heatmap CCP Dark Markets Lit Markets Clearing GW Brokerage Back Office Market Control Persistence DWH & MIS Surveillance UI connectivity, tags, order validation compare vs. execution reports data, reports, recovery alerts, patterns, book replay post trade processing, reconciliations consolidated book functional correctness and execution efficiency reference data setup market control actions matching rules, periods, order constraints and validity Operability, deployment, configuration, monitoring, fault-tolerance, disaster recovery Market Data Monitoring End points and data reconciliation Latency calculation External price feed simulation Trading Actors Market Data Actors External systems (CCP, SOR) simulation Exchange Platforms
billions of transactions Peak rates – from hundreds of thousands to millions of transactions per second Average round-trip latency – dozens of microseconds > 3,000 trx 2.5 cm <1 mm Latency and Throughput Requirements
Stock Exchanges and Other Systems Challenges and complexities - Principle- and outcome-based regulatory frameworks - Expectation of robust governance and operational resilience - Technological and epistemological complexity - Massive transformations - Adopting the new, aligning with existing and ecosystems - High visibility and impact on national economy and society - Budget and time constraints - Centralisation and no canary deployments
SOFTWARE TO TEST SOFTWARE Load injection toolset th2-shark Trading GWs Lit Book Liquidity Partners CSD Trading Desktops Client Brokerage Systems Market Data GWs Distribution System Surveillance System Dark & Algo- Crossing Book Smart Order Router Reference Pricing Server Static Data MIS Heatmap CCP Dark Markets Lit Markets Clearing GW Brokerage Back Office Market Control Persistence DWH & MIS Surveillance UI connectivity, tags, order validation compare vs. execution reports data, reports, recovery alerts, patterns, book replay post trade processing, reconciliations consolidated book functional correctness and execution efficiency reference data setup market control actions matching rules, periods, order constraints and validity Operability, deployment, configuration, monitoring, fault-tolerance, disaster recovery Market Data Monitoring End points and data reconciliation Latency calculation External price feed simulation Trading Actors Market Data Actors External systems (CCP, SOR) simulation Market Surveillance and Smart Order Routers A.) B.)
SOFTWARE TO TEST SOFTWARE Simplified Market Surveillance System WEB WEB Data Base Real-time Feed Rules Engine Alerts Summary Dashboard WEB Production System Monitoring Desktop Order Book Replay B.)
types of such patterns are known as: • Layering • Spoofing • Marking The Close • Wash Trading • Front Running Exactpro performs the testing of the Market Surveillance System patterns to ensure that the actual results match the business requirements. Market Surveillance Systems Testing – Alerts Testing Marking The Close Front Running Wash Trading Spoofing Layering One of the main purposes of Market Surveillance systems is the prevention and investigation of abusive, manipulative or illegal trading behaviour in the markets. There are a set of pre-configured patterns with known abusive behavior that help streamline regulatory compliance in this area. Market Surveillance systems also contain modules that allow users to create new patterns – or adjust the existent ones – and integrate them into the system. Exactpro has tested a number of Market Surveillance Systems across different markets and locations. Our expertise spans: • Building and maintenance of test/regression libraries covering system platforms E2E; • Automating the testing process using proven tools and methodologies; • Building up a holistic approach encompassing the full cycle of Market Surveillance Systems Testing based on our experience of testing the world’s leading Trading, Exchange and Clearing platforms. B.)
Have We Seen Anything like That Before? Gateways Smart Order Router Complex Events Processor, SOR Algorithms and Control mechanisms (safeguards) Trading Admin FEs Operational Data Analytics Metadata, Historical Data Integrations Clients Exchanges & ATSs Market Participants ML AI A.)
Software can be understood as a collection of instructions and data that direct a computer’s operation. These instructions frequently include control-flow constructs that evaluate incoming data: if the data is accepted, the system’s internal state is modified; if the data is rejected, the state remains unchanged. Unlike human decision-making, these checks are executed automatically and at scale – there is no human in the loop processing each conditional branch. Instead, the programmer encodes policies and rules once, and the machine applies them consistently and tirelessly. At a more fundamental level, software itself is data. The same sequences of bits that represent program instructions can be stored, transmitted and manipulated just like any other data. This equivalence – central to the stored-program computer model – allows programs to process other programs, to be compiled, interpreted or even rewritten dynamically. Thus, software is both instructions for action and data for manipulation, embodying automated decision-making that operates without continuous human oversight.
Intelligence Dichotomy Intelligence is the capability to acquire, process and apply knowledge and skills. Humans: • Humans have intelligence – though we do not yet know what it truly is. • They interact with reality through skills. • They hold knowledge in their minds, in the form of ideas. • They can learn, generalise, isolate and abstract. • They can convert ideas ↔ artifacts in a continuous cycle. • Their interactions with reality and with artifacts serve as a proxy for understanding human intelligence. Software: • Running software executes a set of instructions. • It can process data and produce synthetic artifacts. • It interacts with reality through data flows and control loops defined in its instructions. • Its interactions with reality can be represented as data. • Artificial intelligence is software capable of acquiring, processing and applying knowledge and skills. • Synthetic artifacts provide a proxy for exploring artificial intelligence.
in Financial Technology • Regulatory complexity: standards and laws governing regulatory compliance in the financial space continuously evolve across jurisdictions; rules and requirements may overlap or simply be highly extensive in coverage and numbers of articles. • Financial institutions operate within a complex regulatory environment where legal obligations, risk controls, technology requirements and compliance processes are deeply interconnected. • Regulation is becoming increasingly difficult to track manually, so leveraging AI in the compliance function presents a valuable automation use case. • If introduced correctly and with sufficient guardrails, AI can help significantly streamline the effort spent on updating and maintaining regulatory texts. It can also transform the legal corpora from static text into dynamic data-driven system with executable controls. Technology Legal Risk Compliance • Banking • Trading venues & exchanges • Clearing & settlement • Payment infrastructures • RegTech and SupTech • Digital assets and tokenised finance
Landscape in Financial Organisations For financial market infrastructures, regulatory compliance depends on the ability to align rule books, operational controls, technology platforms and testing activities. AI-driven automation can help organisations establish end-to-end traceability across the software lifecycle, improve coverage visibility and streamline compliance reporting under increasing regulatory scrutiny. Financial regulations rarely say: "Develop software using Java." or “Provision for 5 times the average daily capacity.” Instead they state requirements such as: “Systems must be resilient; transactions must be traceable; customer data must remain confidential; access must be controlled; incidents must be reported; records must be retained.” Legal texts intentionally use generic language to accommodate a wide variety of situations and technology implementations. Each article may also cover many overlapping areas in which a trading venue is subject to regulatory compliance. These legal obligations need to be continuously interpreted and translated into software and the surrounding processes, for to be maintained.
know It When I See It” The most famous opinion from the case is Justice Potter Stewart's concurrence, stating that the Constitution protected all obscenity except "hard-core pornography". He wrote, "I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description; and perhaps I could never succeed in intelligibly doing so. But I know it when I see it, and the motion picture involved in this case is not that”.
Sample Article and Implementation Flow Regulation Business policies Functional requirements Software design Implementation Testing Deployment Operation Audit evidence Every transition introduces the possibility of misunderstanding or error. Source: MIFiD II (Markets in Financial Instruments Directive II).
Regulatory Compliance Historically, regulatory compliance has been viewed as a legal and organisational function. The workflow is typically: Regulation Lawers Compliance Officers Policies Manual Controls Audits Technology appears only near the end of the chain. As a result: • regulations become Word documents • policies become PDFs • controls become spreadsheets • testing becomes manual evidence gathering • audits become expensive reconstruction exercises The software implementing the business often has only a weak connection to the regulatory obligations that motivated it.
Regulatory Compliance Today's financial institutions are software companies that happen to provide financial services. Every regulated activity is ultimately performed by software: • payments • trading • clearing • settlement • custody • lending • onboarding • surveillance • reporting Therefore every regulatory obligation eventually becomes software behaviour. The real question is no longer "Are we compliant?", but "Does our software always behave in a manner consistent with applicable regulation?"
Regulatory Compliance Regulatory Concept Software Equivalent Obligation Requirement Exception Alternative flow Prohibition Validation rule Time limit Timeout / scheduler Threshold Configuration parameter Approval Workflow state Evidence Log / event / document Record retention Data lifecycle policy Segregation of duties Access control Continuous monitoring Runtime verification Almost every regulatory requirement can be decomposed into software concepts. Viewed this way, regulation is no longer mysterious. It is simply another specification that software must satisfy.
Software Architecture The institution does not become compliant because it owns policies. It becomes compliant because thousands of software decisions collectively implement those policies. Examples include: • workflow design • access control • state transitions • data models • audit logging • configuration management • exception handling • deployment controls Every architecture decision may have regulatory implications.
Software Defects This perspective also changes how defects are viewed. Examples include: • missing audit events • incorrect permission checks • inaccurate calculations • broken approval workflows • incorrect data retention • reporting inconsistencies • incomplete customer screening • race conditions creating temporary violations Many compliance failures originate not from misunderstood regulation but from ordinary software bugs. DAYS WITHOUT AN INCIDENT 0
Living System Regulations evolve. Products evolve. Software evolves. Therefore, compliance cannot be a one-time certification exercise. Instead it becomes continuous engineering that should include: • response quality • hallucination rates • latency • user feedback • security incidents • regulatory changes If a regulation changes tomorrow, the AI knowledge base should be updated, and previous responses may need revalidation.
Compliance Lifecycle Where can AI help? AI-assisted regulatory interpretation Analyse large collections of regulations, extract meaning, identify obligations and assess impact AI-assisted software development Accelerate development with AI-generated code, patterns and suggestions Requirements engineering and rule mapping Map regulations to requirements, controls and system behaviour Test generation and validation Suggest/generate test scenarios, validate outcomes and improve coverage Monitoring, reporting and operational compliance Continuously monitor controls, detect inconsistencies and issues early, generate traceability links and compliance reports Regulatory change management Detect regulatory changes, assess impact and drive timely updates Notice that AI does not replace legal interpretation. It helps engineers maintain the enormous knowledge graph connecting regulation to implementation.
Build compliance into the system, not around it Usual approach • Software is developed first. • Compliance is assessed later through reviews, controls and audits. • Gaps are identified after implementation, leading to costly remediation and increased operational risk. Recommended practice • Regulatory obligations are considered from the outset. • Compliance requirements are translated into system requirements, business rules and architecture decisions. • Controls, auditability and evidence generation are embedded directly into the software.
SOFTWARE TO TEST SOFTWARE As emerging technologies become increasingly autonomous and pervasive, the central engineering challenge shifts from mostly focusing on capability to also achieving alignment: ensuring that technological systems operate accurately, consistently and in accordance with legal, ethical, cultural and religious normative frameworks. Compliance by Design – From Capability to Alignment
bank Commodity seller Commodity buyer Customer Payment for commodity Сommodity Сommodity Сash payment for commodity Deferred payment for commodity Сommodity Request for financing Fixed terms
Compliance Discipline • Genuine ownership risk: all participants and their clients bear full ownership risk, including loss, damage and price fluctuation, for the duration of their respective ownership. • Strict transfer of risk: ownership risk transfers only upon legal title transfer and constructive possession, as evidenced through exchange execution, clearing and settlement processes. • No artificial risk elimination: exposure to real commercial risk is a prerequisite for lawful profit and is not neutralised through contractual or structural mechanisms. • Prevention of prohibited structures: workflows that constitute, or are economically equivalent to, sell-and-buy-back arrangements are actively prevented, including across multi-step transaction sequences. • Group-level control enforcement: participants operating under common ownership or control are explicitly identified, enabling compliance checks to be applied at the appropriate aggregation level. • Embedded compliance controls: validation rules and workflow constraints are enforced at each stage of execution, reducing reliance on manual oversight or post-trade review. • Transparency & auditability: all ownership transfers and related actions are fully traceable through exchange and post-trade infrastructure, supporting audit and regulatory review.
Testing Models Abstract Model = theory Concrete Description = data Software = object of study flow of information observe compare Abstract Model = design concept Concrete Description = specification Software = useful product flow of information design produce Software Testing Development The Antiparallel Structures of Software Testing and Development The diagram is inspired by Drexler, K. E. (2013). Radical abundance: How a revolution in nanotechnology will change civilization. Public Affairs. 370 p. https://fs.blog/2013/07/the-difference-between-science-and-engineering/
Development Mindset ” “The scout mindset: the motivation to see things as they are, not as you wish they were. Julia Galef American writer, speaker and co-founder of the Center for Applied Rationality
Coverage – Math vs Business “The phrase ”complete coverage” is misleading. This “completeness” is measured only relative to a specific population of possible test cases, such as lines of code, branches, n-length sub-paths, predicates, etc. Even if you achieve complete coverage for a given population of tests (such as, all lines of code tested), you have not done complete, or even adequate, testing. “We can and should expand the list of populations of possible test cases. We can measure coverage against each of these populations. The decision as to whether to try for 1%, 10%, 50% or 100% coverage against any given population is non-obvious. It involves trade-offs based on thoughtful judgment.” – Cem Kaner, J.D., Ph.D., ‘Software Negligence and Testing Coverage,’ 1995.
Testing Coverage “This appendix lists 101 coverage measures. Coverage measures the amount of testing done of a certain type. Because testing is done to find bugs, coverage is also a measure of your effort to detect a certain class of potential errors. For example, 100% line coverage doesn’t just mean that you’ve executed every line of code; it also means that you’ve tested for every bug that can be revealed by simple execution of a line of code.” – Cem Kaner, J.D., Ph.D. Fig. 5 Excerpt from the 101 coverage measures list put together by Cem Kaner
Testing Workflow ✔ all instruments and their trading control permutations ✔ all users and their trading control permutations ✔ all relevant reference data actions ✔ protocol specifications, machine-readable dictionaries ✔ various input actions check lists ✔ Information obtained _from testing ✔ Calculated weight weights dataset test basis traffic & log files th2-shark Cradle scripts to iterate through available actions and organise them in test cases output dataset interpretation dataset FIX Binary Drop Copy Market Data Internal GW random cartesian all pairs pre-filtering Reconciliation of: ✔ two different output streams ✔ input and output streams ✔ specific properties across the dataset ✔ received data against predefined rule-book Checks for: connection handling and traffic parsing inputs 3 5 4 6 2 1 Property checking Property Checker Rules
for Financial Organisations Compliance – Workflow Management of all controls and tasks from a single interface – formalised, tracked and auditable in one place. Centralised controls Surface alerts, log observations and recommendations appear after issues arise. Real-time monitoring Generate tailored plans and reports on demand, ready to download and share with stakeholders or regulators. Reporting generate execute analyse Define scope, controls, rules, reporting plan Test data execution (iterative cycle, including AB comparison when/if needed) Rules coverage analysis Gaps identification
conformance automation framework, th2-RC, helps teams align rule books, system requirements, testing artefacts and coverage reporting. The solution supports any regulatory framework, trading rules, asset classes and data protocols, providing a scalable, reliable foundation for managing complex compliance obligations while ensuring proper handling of personal and financial data. th2-RC’s AI-assisted coding plugin (that supports various LLMs) underpins rule creation but is not involved in test library execution. This allows the solution to achieve advanced functionality while being resource-light and fully compliant with regulations on handling personal and financial data. th2-RC enables you to: • Turn regulatory obligations into testable evidence • Stay audit-ready as regulations change • Benefit from AI-assisted conformance automation customised for financial systems Regulatory Compliance support for Financial Organisations
Simplify Adoption of the Framework The SUT (System Under Test) domain model is based on posterior traffic data analysis that makes developing the model substantially simpler in comparison to full-scale digital twins implemented in Python or OCaml. Simplification allows to quickly migrate the domain model to a different language using AI-coding assistants and enables software testers to maintain the code of the model themselves. A clear separation between scenario generation, execution and output interpretation make the acceptance framework agnostic to the way test scenarios/execution logs were produced, allowing to ingest the data generated by the existing test libraries and tools. LLM-based code assistants are only used during verification rules development. Users have full control over the snippets of data/code that are provided to the foundation models. Only deterministic code is used during test results analysis. No external API access is required. Software testers and LLMs sometimes produce incomplete or even erroneous rule verification code. The key idea of the acceptance framework is to make it easier for the users to detect and debug such situations using a dedicated user interface integrated with the code assistant tools of their choice.
Rules? State BEFORE State AFTER Input Signal Audit Data Transition and Downstream Effects LLM-generated or hand-crafted code Deterministic verification code
Architecture Overview Compliance Engine Compliance Viewer (WebUI) Conformance Test Results 1. Compliance Viewer is a read-only portal that lets Compliance Analysts browse, filter and export test results produced by external test runners. 2. Results are stored as files on disk (no database involved). 3. The back-end (Compliance) engine reads and processes those files on demand, serving paginated and filterable data to the UI across various data categories: Messages, State Transitions, Scenarios, Annotations, Traceability, Rule Book and Code Assistant.
Testing Workflow generate input dataset execute output dataset analyse interpretation dataset 3 2 1 4 5 6 Evaluate the obtained information and the list of available actions – repeat the steps Select actions to perform • What actions are available? • Which of them to choose? • In what order? Execute actions against the System Under Test (SUT) Action types: • Set-up actions • Experiment actions • Observation actions Interpret the outcome of the actions Non-exclusive possibilities: • Test results match our rules-based checks for the SUT • Something is wrong with our actions • Something is wrong with the SUT • Something is wrong with our rules Provide the relevant information about findings to the stakeholders for them to make decisions Initial coverage based on the provided documentation, specifications, data formats Transactional data coverage based on the annotation mapping, actual data permutations, traffic analysis vs
manage sessions, encoding, decoding, sending messages and storing traffic. Written in Go language Consistency Validation – How do we Execute Testing Against the SUT? FIX ATP Drop Copy MD tcp MD udp tcpdump capture log files traffic files th2-shark Python scripts dataload reference data Uses configuration files with connectivity and submission rates settings input dataset output dataset pre-processing System Under Test (SUT) REST/JSON Websocket Testers
How do we Interpret the Outcome of Test Actions? FIX ATP Drop Copy MD tcp MD udp traffic files output dataset ✓ Check that we are able to maintain connection ✓ Check that we are able to parse the traffic ✓ Check that the parsed data conform to protocol specification ✓ Check for contradictions within each output stream ✓ Reconcile two different output streams ✓ Reconcile input and output streams ✓ Check specific properties across the full dataset ✓ Reconcile via rules-based outcomes ✓ Aggregate the data in the dataset ✓ Reconcile against expected aggregated results ✓ Explore for anomalies and trends ✓ AB comparison (between different release versions) th2 shark code in Go language Simple Python scripts for scenarios and rules validation interpretation dataset System Under Test (SUT) REST/JSON Websocket
• Tier 5 “Coherence Improving”: Focuses on simplifying and unifying the system by removing duplication, unnecessary variation and reducing accidental complexity • Tier 4 “Guardrails Adding”: Establishes automated rules, checks and policies that keep development safe, secure and compliant without blocking progress • Tier 3 “Feature Design”: Identifies gaps, inefficiencies, and missing capabilities before they become problems. It is about proactive thinking and shaping what should be built, not just implementing what is requested • Tier 2 “Feature Testing”: Ensures features behave correctly under real-world conditions by exploring edge cases, failures, and system interactions. It shifts thinking from “Does it work?” to “How can it break?” • Tier 1 “Feature Implementation”: Focuses on rapidly translating clear requirements into working code using AI tools
Your Podcast. Everywhere. Effortlessly.
exactpro
kristinabergwall1
minecr
csswizardry
helmedeiros
hannesfritz
chriscoyier
skoyamalab
ryanjones
yeseniaperezcruz
coloredviolet
inesmontani
holman
![exactpro.com 73 BUILD SOFTWARE TO TEST SOFTWARE Thank You! [email protected]](https://files.speakerdeck.com/presentations/314f0e424ce34a80b4083ca5e8a9ec05/slide_72.jpg){kind=link}