Slide 1

Slide 1 text

exactpro.com 1 BUILD SOFTWARE TO TEST SOFTWARE BUILD SOFTWARE TO TEST SOFTWARE exactpro.com Using AI for Streamlining Regulatory Compliance Solutions In Financial Organisations Iosif Itkin, CEO & co-founder, Exactpro CTO, Murabex AI Lecture Series University of Luxembourg

Slide 2

Slide 2 text

exactpro.com 2 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 2 BUILD SOFTWARE TO TEST SOFTWARE Meet the speaker Iosif Itkin is co-founder and co-CEO of Exactpro – an independent provider of AI-enabled software testing, development and consultancy services for financial organisations. Founded in 2009, Exactpro has a client base of major exchanges, post-trade platform operators, banks and technology vendors across 25 countries. Iosif’s expertise at the intersection of high-availability systems and capital markets has enabled him to successfully implement testing strategies and facilitate technology transformations within exchanges, investment banks and clearing and settlement organisations in London, New York, Milan, Singapore, Sydney and other major financial centres. Iosif is a co-author of ‘Introduction to AI Testing: Guide to ISTQB® CT-AI Certification’ released in September 2025 by BCS Publishing. Iosif regularly uses coding assistants in his day-to-day work for a variety of testing and development purposes. Iosif Itkin CEO & co-Founder, Exactpro CTO, Murabex

Slide 3

Slide 3 text

exactpro.com 3 BUILD SOFTWARE TO TEST SOFTWARE Lecture Contents Introduction Exchange Platforms AI-enabled systems in FMIs – Market Surveillance and Smart Order Router Timeless classic – Knight Capital compliance story The difference between humans and software The Compliance Burden in Financial Technology Regulatory Compliance as a Software Engineering Problem Compliance by Design Example – Murabex Platform Building vs. Testing – Scout mindset Probing the infinite space – Labeling and Coverage Building a Data-intensive Compliance System Questions and Answers

Slide 4

Slide 4 text

exactpro.com 4 BUILD SOFTWARE TO TEST SOFTWARE 4 BUILD SOFTWARE TO TEST SOFTWARE Exactpro is an independent provider of AI-enabled software testing services for financial organisations. Our clients are exchanges, post-trade platform operators and banks across 25 countries. We help our clients to decrease time to market, maintain regulatory compliance, improve scalability, latency and operational resiliency. On top of AI-driven testing and automation capabilities, we offer industry-proven expertise in test strategy development, software development and prototyping, facilitating regulatory compliance and reporting, AI Literacy building and support of enterprise-level AI integration. Headquartered in the UK, Exactpro operates delivery centres in Georgia, Sri Lanka, Armenia, the UK, representative offices in the US, Canada and Italy and a global network of consultants. See more partners & clients Exactpro’s client network See our awards About Exactpro Our Client network spans more than a half of top 20 global exchange groups

Slide 5

Slide 5 text

exactpro.com 5 BUILD SOFTWARE TO TEST SOFTWARE 1. Capital formation 2. Liquidity (ability to enter/exit) 3. Price discovery 4. Standardisation and rules 5. Risk distribution 6. Infrastructure and trust 7. Benchmarking and ecosystem effects Why Exchanges are Important

Slide 6

Slide 6 text

exactpro.com 6 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 6 BUILD SOFTWARE TO TEST SOFTWARE Load injection toolset th2-shark Trading GWs Lit Book Liquidity Partners CSD Trading Desktops Client Brokerage Systems Market Data GWs Distribution System Surveillance System Dark & Algo- Crossing Book Smart Order Router Reference Pricing Server Static Data MIS Heatmap CCP Dark Markets Lit Markets Clearing GW Brokerage Back Office Market Control Persistence DWH & MIS Surveillance UI connectivity, tags, order validation compare vs. execution reports data, reports, recovery alerts, patterns, book replay post trade processing, reconciliations consolidated book functional correctness and execution efficiency reference data setup market control actions matching rules, periods, order constraints and validity Operability, deployment, configuration, monitoring, fault-tolerance, disaster recovery Market Data Monitoring End points and data reconciliation Latency calculation External price feed simulation Trading Actors Market Data Actors External systems (CCP, SOR) simulation Exchange Platforms

Slide 7

Slide 7 text

exactpro.com 7 BUILD SOFTWARE TO TEST SOFTWARE Daily capacity – billions of transactions Peak rates – from hundreds of thousands to millions of transactions per second Average round-trip latency – dozens of microseconds > 3,000 trx 2.5 cm <1 mm Latency and Throughput Requirements

Slide 8

Slide 8 text

exactpro.com 8 BUILD SOFTWARE TO TEST SOFTWARE Trading Systems / Stock Exchanges and Other Systems Challenges and complexities - Principle- and outcome-based regulatory frameworks - Expectation of robust governance and operational resilience - Technological and epistemological complexity - Massive transformations - Adopting the new, aligning with existing and ecosystems - High visibility and impact on national economy and society - Budget and time constraints - Centralisation and no canary deployments

Slide 9

Slide 9 text

exactpro.com 9 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 9 BUILD SOFTWARE TO TEST SOFTWARE Load injection toolset th2-shark Trading GWs Lit Book Liquidity Partners CSD Trading Desktops Client Brokerage Systems Market Data GWs Distribution System Surveillance System Dark & Algo- Crossing Book Smart Order Router Reference Pricing Server Static Data MIS Heatmap CCP Dark Markets Lit Markets Clearing GW Brokerage Back Office Market Control Persistence DWH & MIS Surveillance UI connectivity, tags, order validation compare vs. execution reports data, reports, recovery alerts, patterns, book replay post trade processing, reconciliations consolidated book functional correctness and execution efficiency reference data setup market control actions matching rules, periods, order constraints and validity Operability, deployment, configuration, monitoring, fault-tolerance, disaster recovery Market Data Monitoring End points and data reconciliation Latency calculation External price feed simulation Trading Actors Market Data Actors External systems (CCP, SOR) simulation Market Surveillance and Smart Order Routers A.) B.)

Slide 10

Slide 10 text

exactpro.com 10 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 10 BUILD SOFTWARE TO TEST SOFTWARE Simplified Market Surveillance System WEB WEB Data Base Real-time Feed Rules Engine Alerts Summary Dashboard WEB Production System Monitoring Desktop Order Book Replay B.)

Slide 11

Slide 11 text

exactpro.com 11 BUILD SOFTWARE TO TEST SOFTWARE The most common types of such patterns are known as: ● Layering ● Spoofing ● Marking The Close ● Wash Trading ● Front Running Exactpro performs the testing of the Market Surveillance System patterns to ensure that the actual results match the business requirements. Market Surveillance Systems Testing – Alerts Testing Marking The Close Front Running Wash Trading Spoofing Layering One of the main purposes of Market Surveillance systems is the prevention and investigation of abusive, manipulative or illegal trading behaviour in the markets. There are a set of pre-configured patterns with known abusive behavior that help streamline regulatory compliance in this area. Market Surveillance systems also contain modules that allow users to create new patterns – or adjust the existent ones – and integrate them into the system. Exactpro has tested a number of Market Surveillance Systems across different markets and locations. Our expertise spans: ● Building and maintenance of test/regression libraries covering system platforms E2E; ● Automating the testing process using proven tools and methodologies; ● Building up a holistic approach encompassing the full cycle of Market Surveillance Systems Testing based on our experience of testing the world’s leading Trading, Exchange and Clearing platforms. B.)

Slide 12

Slide 12 text

exactpro.com 12 BUILD SOFTWARE TO TEST SOFTWARE Guiding Policy – Have We Seen Anything like That Before? Gateways Smart Order Router Complex Events Processor, SOR Algorithms and Control mechanisms (safeguards) Trading Admin FEs Operational Data Analytics Metadata, Historical Data Integrations Clients Exchanges & ATSs Market Participants ML AI A.)

Slide 13

Slide 13 text

exactpro.com 13 BUILD SOFTWARE TO TEST SOFTWARE Retrieval-augmented Generation-based System (RAG) RAG - Retrieval-augmented generation ● Multiple Sources ● Nondeterministic ● Adaptable ● Autonomous ● Learning ● Evolving ● Both ML and AI

Slide 14

Slide 14 text

exactpro.com 14 BUILD SOFTWARE TO TEST SOFTWARE Timeless Classic – Knight Capital Compliance Story

Slide 15

Slide 15 text

exactpro.com 15 BUILD SOFTWARE TO TEST SOFTWARE Smart Order Router System

Slide 16

Slide 16 text

exactpro.com 16 BUILD SOFTWARE TO TEST SOFTWARE SEC Order 34-70694

Slide 17

Slide 17 text

exactpro.com 17 BUILD SOFTWARE TO TEST SOFTWARE SEC Order 34-70694

Slide 18

Slide 18 text

exactpro.com 18 BUILD SOFTWARE TO TEST SOFTWARE SEC Order 34-70694

Slide 19

Slide 19 text

exactpro.com 19 BUILD SOFTWARE TO TEST SOFTWARE SEC Order

Slide 20

Slide 20 text

exactpro.com 20 BUILD SOFTWARE TO TEST SOFTWARE What is Software? Software can be understood as a collection of instructions and data that direct a computer’s operation. These instructions frequently include control-flow constructs that evaluate incoming data: if the data is accepted, the system’s internal state is modified; if the data is rejected, the state remains unchanged. Unlike human decision-making, these checks are executed automatically and at scale – there is no human in the loop processing each conditional branch. Instead, the programmer encodes policies and rules once, and the machine applies them consistently and tirelessly. At a more fundamental level, software itself is data. The same sequences of bits that represent program instructions can be stored, transmitted and manipulated just like any other data. This equivalence – central to the stored-program computer model – allows programs to process other programs, to be compiled, interpreted or even rewritten dynamically. Thus, software is both instructions for action and data for manipulation, embodying automated decision-making that operates without continuous human oversight.

Slide 21

Slide 21 text

exactpro.com 21 BUILD SOFTWARE TO TEST SOFTWARE Human-in-the-loop, Human-on-the-loop, Human-out-of-the-loop # human? n = 1000 for i in range(1, n): # human? if i % 137 == 0: call_human() # ??? self.do_something() # human?

Slide 22

Slide 22 text

exactpro.com 22 BUILD SOFTWARE TO TEST SOFTWARE Data Types Dichotomy Hand-Crafted Natural Interactive Attended Manual Synthetic Artificial Generated Unattended Automated

Slide 23

Slide 23 text

exactpro.com 23 BUILD SOFTWARE TO TEST SOFTWARE Data Volumes Dichotomy Artifacts or Snippets Narrow Bandwidth Human-Attended Work Datasets or Corpus Machine-scale Big Data

Slide 24

Slide 24 text

exactpro.com 24 BUILD SOFTWARE TO TEST SOFTWARE Natural and Artificial Intelligence Dichotomy Intelligence is the capability to acquire, process and apply knowledge and skills. Humans: ● Humans have intelligence – though we do not yet know what it truly is. ● They interact with reality through skills. ● They hold knowledge in their minds, in the form of ideas. ● They can learn, generalise, isolate and abstract. ● They can convert ideas ↔ artifacts in a continuous cycle. ● Their interactions with reality and with artifacts serve as a proxy for understanding human intelligence. Software: ● Running software executes a set of instructions. ● It can process data and produce synthetic artifacts. ● It interacts with reality through data flows and control loops defined in its instructions. ● Its interactions with reality can be represented as data. ● Artificial intelligence is software capable of acquiring, processing and applying knowledge and skills. ● Synthetic artifacts provide a proxy for exploring artificial intelligence.

Slide 25

Slide 25 text

exactpro.com 25 BUILD SOFTWARE TO TEST SOFTWARE Reliability/Responsibility Dichotomy Responsible vs. Irresponsible Reliable vs. Unreliable False True

Slide 26

Slide 26 text

exactpro.com 26 BUILD SOFTWARE TO TEST SOFTWARE Humans vs. Automated Irresponsibility ● Can be reasoned with ● Feel pity, remorse and fear ● Will eventually stop

Slide 27

Slide 27 text

exactpro.com 27 BUILD SOFTWARE TO TEST SOFTWARE The Compliance Burden in Financial Technology ● Regulatory complexity: standards and laws governing regulatory compliance in the financial space continuously evolve across jurisdictions; rules and requirements may overlap or simply be highly extensive in coverage and numbers of articles. ● Financial institutions operate within a complex regulatory environment where legal obligations, risk controls, technology requirements and compliance processes are deeply interconnected. ● Regulation is becoming increasingly difficult to track manually, so leveraging AI in the compliance function presents a valuable automation use case. ● If introduced correctly and with sufficient guardrails, AI can help significantly streamline the effort spent on updating and maintaining regulatory texts. It can also transform the legal corpora from static text into dynamic data-driven system with executable controls. Technology Legal Risk Compliance ● Banking ● Trading venues & exchanges ● Clearing & settlement ● Payment infrastructures ● RegTech and SupTech ● Digital assets and tokenised finance

Slide 28

Slide 28 text

exactpro.com 28 BUILD SOFTWARE TO TEST SOFTWARE The Regulatory Compliance Landscape in Financial Organisations For financial market infrastructures, regulatory compliance depends on the ability to align rule books, operational controls, technology platforms and testing activities. AI-driven automation can help organisations establish end-to-end traceability across the software lifecycle, improve coverage visibility and streamline compliance reporting under increasing regulatory scrutiny. Financial regulations rarely say: "Develop software using Java." or “Provision for 5 times the average daily capacity.” Instead they state requirements such as: “Systems must be resilient; transactions must be traceable; customer data must remain confidential; access must be controlled; incidents must be reported; records must be retained.” Legal texts intentionally use generic language to accommodate a wide variety of situations and technology implementations. Each article may also cover many overlapping areas in which a trading venue is subject to regulatory compliance. These legal obligations need to be continuously interpreted and translated into software and the surrounding processes, for to be maintained.

Slide 29

Slide 29 text

exactpro.com 29 BUILD SOFTWARE TO TEST SOFTWARE Human – “I know It When I See It” The most famous opinion from the case is Justice Potter Stewart's concurrence, stating that the Constitution protected all obscenity except "hard-core pornography". He wrote, "I shall not today attempt further to define the kinds of material I understand to be embraced within that shorthand description; and perhaps I could never succeed in intelligibly doing so. But I know it when I see it, and the motion picture involved in this case is not that”.

Slide 30

Slide 30 text

exactpro.com 30 BUILD SOFTWARE TO TEST SOFTWARE MIFiD II – Sample Article and Implementation Flow Regulation Business policies Functional requirements Software design Implementation Testing Deployment Operation Audit evidence Every transition introduces the possibility of misunderstanding or error. Source: MIFiD II (Markets in Financial Instruments Directive II).

Slide 31

Slide 31 text

exactpro.com 31 BUILD SOFTWARE TO TEST SOFTWARE Legacy View on Regulatory Compliance Historically, regulatory compliance has been viewed as a legal and organisational function. The workflow is typically: Regulation Lawers Compliance Officers Policies Manual Controls Audits Technology appears only near the end of the chain. As a result: ● regulations become Word documents ● policies become PDFs ● controls become spreadsheets ● testing becomes manual evidence gathering ● audits become expensive reconstruction exercises The software implementing the business often has only a weak connection to the regulatory obligations that motivated it.

Slide 32

Slide 32 text

exactpro.com 32 BUILD SOFTWARE TO TEST SOFTWARE Legacy View on Regulatory Compliance Today's financial institutions are software companies that happen to provide financial services. Every regulated activity is ultimately performed by software: ● payments ● trading ● clearing ● settlement ● custody ● lending ● onboarding ● surveillance ● reporting Therefore every regulatory obligation eventually becomes software behaviour. The real question is no longer "Are we compliant?", but "Does our software always behave in a manner consistent with applicable regulation?"

Slide 33

Slide 33 text

exactpro.com 33 BUILD SOFTWARE TO TEST SOFTWARE Legacy View on Regulatory Compliance Regulatory Concept Software Equivalent Obligation Requirement Exception Alternative flow Prohibition Validation rule Time limit Timeout / scheduler Threshold Configuration parameter Approval Workflow state Evidence Log / event / document Record retention Data lifecycle policy Segregation of duties Access control Continuous monitoring Runtime verification Almost every regulatory requirement can be decomposed into software concepts. Viewed this way, regulation is no longer mysterious. It is simply another specification that software must satisfy.

Slide 34

Slide 34 text

exactpro.com 34 BUILD SOFTWARE TO TEST SOFTWARE Compliance Architecture is Software Architecture The institution does not become compliant because it owns policies. It becomes compliant because thousands of software decisions collectively implement those policies. Examples include: ● workflow design ● access control ● state transitions ● data models ● audit logging ● configuration management ● exception handling ● deployment controls Every architecture decision may have regulatory implications.

Slide 35

Slide 35 text

exactpro.com 35 BUILD SOFTWARE TO TEST SOFTWARE Compliance Defects are Software Defects This perspective also changes how defects are viewed. Examples include: ● missing audit events ● incorrect permission checks ● inaccurate calculations ● broken approval workflows ● incorrect data retention ● reporting inconsistencies ● incomplete customer screening ● race conditions creating temporary violations Many compliance failures originate not from misunderstood regulation but from ordinary software bugs. DAYS WITHOUT AN INCIDENT 0

Slide 36

Slide 36 text

exactpro.com 36 BUILD SOFTWARE TO TEST SOFTWARE Compliance is a Living System Regulations evolve. Products evolve. Software evolves. Therefore, compliance cannot be a one-time certification exercise. Instead it becomes continuous engineering that should include: ● response quality ● hallucination rates ● latency ● user feedback ● security incidents ● regulatory changes If a regulation changes tomorrow, the AI knowledge base should be updated, and previous responses may need revalidation.

Slide 37

Slide 37 text

exactpro.com 37 BUILD SOFTWARE TO TEST SOFTWARE AI across the Compliance Lifecycle Where can AI help? AI-assisted regulatory interpretation Analyse large collections of regulations, extract meaning, identify obligations and assess impact AI-assisted software development Accelerate development with AI-generated code, patterns and suggestions Requirements engineering and rule mapping Map regulations to requirements, controls and system behaviour Test generation and validation Suggest/generate test scenarios, validate outcomes and improve coverage Monitoring, reporting and operational compliance Continuously monitor controls, detect inconsistencies and issues early, generate traceability links and compliance reports Regulatory change management Detect regulatory changes, assess impact and drive timely updates Notice that AI does not replace legal interpretation. It helps engineers maintain the enormous knowledge graph connecting regulation to implementation.

Slide 38

Slide 38 text

exactpro.com 38 BUILD SOFTWARE TO TEST SOFTWARE Compliance by Design Build compliance into the system, not around it Usual approach ● Software is developed first. ● Compliance is assessed later through reviews, controls and audits. ● Gaps are identified after implementation, leading to costly remediation and increased operational risk. Recommended practice ● Regulatory obligations are considered from the outset. ● Compliance requirements are translated into system requirements, business rules and architecture decisions. ● Controls, auditability and evidence generation are embedded directly into the software.

Slide 39

Slide 39 text

exactpro.com 39 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 39 BUILD SOFTWARE TO TEST SOFTWARE As emerging technologies become increasingly autonomous and pervasive, the central engineering challenge shifts from mostly focusing on capability to also achieving alignment: ensuring that technological systems operate accurately, consistently and in accordance with legal, ethical, cultural and religious normative frameworks. Compliance by Design – From Capability to Alignment

Slide 40

Slide 40 text

exactpro.com 40 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 40 BUILD SOFTWARE TO TEST SOFTWARE Values Alignment https://davidrozado.substack.com/p/the-political-preferences-of-llms

Slide 41

Slide 41 text

exactpro.com 41 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 41 BUILD SOFTWARE TO TEST SOFTWARE Murabex Platform

Slide 42

Slide 42 text

exactpro.com 42 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 42 BUILD SOFTWARE TO TEST SOFTWARE Murabex Platform

Slide 43

Slide 43 text

exactpro.com 43 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 43 BUILD SOFTWARE TO TEST SOFTWARE Murabex Platform

Slide 44

Slide 44 text

exactpro.com 44 BUILD SOFTWARE TO TEST SOFTWARE Murabaha Financing Islamic bank Commodity seller Commodity buyer Customer Payment for commodity Сommodity Сommodity Сash payment for commodity Deferred payment for commodity Сommodity Request for financing Fixed terms

Slide 45

Slide 45 text

exactpro.com 45 BUILD SOFTWARE TO TEST SOFTWARE Ownership Risk and Compliance Discipline ● Genuine ownership risk: all participants and their clients bear full ownership risk, including loss, damage and price fluctuation, for the duration of their respective ownership. ● Strict transfer of risk: ownership risk transfers only upon legal title transfer and constructive possession, as evidenced through exchange execution, clearing and settlement processes. ● No artificial risk elimination: exposure to real commercial risk is a prerequisite for lawful profit and is not neutralised through contractual or structural mechanisms. ● Prevention of prohibited structures: workflows that constitute, or are economically equivalent to, sell-and-buy-back arrangements are actively prevented, including across multi-step transaction sequences. ● Group-level control enforcement: participants operating under common ownership or control are explicitly identified, enabling compliance checks to be applied at the appropriate aggregation level. ● Embedded compliance controls: validation rules and workflow constraints are enforced at each stage of execution, reducing reliance on manual oversight or post-trade review. ● Transparency & auditability: all ownership transfers and related actions are fully traceable through exchange and post-trade infrastructure, supporting audit and regulatory review.

Slide 46

Slide 46 text

exactpro.com 46 BUILD SOFTWARE TO TEST SOFTWARE Engineering Design vs. Scientific Inquiry

Slide 47

Slide 47 text

exactpro.com 47 BUILD SOFTWARE TO TEST SOFTWARE Engineering Design vs. Scientific Inquiry

Slide 48

Slide 48 text

exactpro.com 48 BUILD SOFTWARE TO TEST SOFTWARE Software Development and Testing Models Abstract Model = theory Concrete Description = data Software = object of study flow of information observe compare Abstract Model = design concept Concrete Description = specification Software = useful product flow of information design produce Software Testing Development The Antiparallel Structures of Software Testing and Development The diagram is inspired by Drexler, K. E. (2013). Radical abundance: How a revolution in nanotechnology will change civilization. Public Affairs. 370 p. https://fs.blog/2013/07/the-difference-between-science-and-engineering/

Slide 49

Slide 49 text

exactpro.com 49 BUILD SOFTWARE TO TEST SOFTWARE Testing Mindset vs. Development Mindset ” “The scout mindset: the motivation to see things as they are, not as you wish they were. Julia Galef American writer, speaker and co-founder of the Center for Applied Rationality

Slide 50

Slide 50 text

exactpro.com 50 BUILD SOFTWARE TO TEST SOFTWARE Completeness of Test Coverage – Math vs Business “The phrase ”complete coverage” is misleading. This “completeness” is measured only relative to a specific population of possible test cases, such as lines of code, branches, n-length sub-paths, predicates, etc. Even if you achieve complete coverage for a given population of tests (such as, all lines of code tested), you have not done complete, or even adequate, testing. “We can and should expand the list of populations of possible test cases. We can measure coverage against each of these populations. The decision as to whether to try for 1%, 10%, 50% or 100% coverage against any given population is non-obvious. It involves trade-offs based on thoughtful judgment.” – Cem Kaner, J.D., Ph.D., ‘Software Negligence and Testing Coverage,’ 1995.

Slide 51

Slide 51 text

exactpro.com 51 BUILD SOFTWARE TO TEST SOFTWARE Software Negligence and Testing Coverage “This appendix lists 101 coverage measures. Coverage measures the amount of testing done of a certain type. Because testing is done to find bugs, coverage is also a measure of your effort to detect a certain class of potential errors. For example, 100% line coverage doesn’t just mean that you’ve executed every line of code; it also means that you’ve tested for every bug that can be revealed by simple execution of a line of code.” – Cem Kaner, J.D., Ph.D. Fig. 5 Excerpt from the 101 coverage measures list put together by Cem Kaner

Slide 52

Slide 52 text

exactpro.com 52 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 52 BUILD SOFTWARE TO TEST SOFTWARE Probing an Infinite Space

Slide 53

Slide 53 text

exactpro.com 53 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 53 BUILD SOFTWARE TO TEST SOFTWARE Probing an Infinite Space

Slide 54

Slide 54 text

exactpro.com 54 BUILD SOFTWARE TO TEST SOFTWARE Property Checker Rules-based Testing Workflow ✔ all instruments and their trading control permutations ✔ all users and their trading control permutations ✔ all relevant reference data actions ✔ protocol specifications, machine-readable dictionaries ✔ various input actions check lists ✔ Information obtained _from testing ✔ Calculated weight weights dataset test basis traffic & log files th2-shark Cradle scripts to iterate through available actions and organise them in test cases output dataset interpretation dataset FIX Binary Drop Copy Market Data Internal GW random cartesian all pairs pre-filtering Reconciliation of: ✔ two different output streams ✔ input and output streams ✔ specific properties across the dataset ✔ received data against predefined rule-book Checks for: connection handling and traffic parsing inputs 3 5 4 6 2 1 Property checking Property Checker Rules

Slide 55

Slide 55 text

exactpro.com 55 BUILD SOFTWARE TO TEST SOFTWARE Regulatory Compliance support for Financial Organisations Compliance – Workflow Management of all controls and tasks from a single interface – formalised, tracked and auditable in one place. Centralised controls Surface alerts, log observations and recommendations appear after issues arise. Real-time monitoring Generate tailored plans and reports on demand, ready to download and share with stakeholders or regulators. Reporting generate execute analyse Define scope, controls, rules, reporting plan Test data execution (iterative cycle, including AB comparison when/if needed) Rules coverage analysis Gaps identification

Slide 56

Slide 56 text

exactpro.com 56 BUILD SOFTWARE TO TEST SOFTWARE Regulatory Compliance Our conformance automation framework, th2-RC, helps teams align rule books, system requirements, testing artefacts and coverage reporting. The solution supports any regulatory framework, trading rules, asset classes and data protocols, providing a scalable, reliable foundation for managing complex compliance obligations while ensuring proper handling of personal and financial data. th2-RC’s AI-assisted coding plugin (that supports various LLMs) underpins rule creation but is not involved in test library execution. This allows the solution to achieve advanced functionality while being resource-light and fully compliant with regulations on handling personal and financial data. th2-RC enables you to: ● Turn regulatory obligations into testable evidence ● Stay audit-ready as regulations change ● Benefit from AI-assisted conformance automation customised for financial systems Regulatory Compliance support for Financial Organisations

Slide 57

Slide 57 text

exactpro.com 57 BUILD SOFTWARE TO TEST SOFTWARE Key Features that Simplify Adoption of the Framework The SUT (System Under Test) domain model is based on posterior traffic data analysis that makes developing the model substantially simpler in comparison to full-scale digital twins implemented in Python or OCaml. Simplification allows to quickly migrate the domain model to a different language using AI-coding assistants and enables software testers to maintain the code of the model themselves. A clear separation between scenario generation, execution and output interpretation make the acceptance framework agnostic to the way test scenarios/execution logs were produced, allowing to ingest the data generated by the existing test libraries and tools. LLM-based code assistants are only used during verification rules development. Users have full control over the snippets of data/code that are provided to the foundation models. Only deterministic code is used during test results analysis. No external API access is required. Software testers and LLMs sometimes produce incomplete or even erroneous rule verification code. The key idea of the acceptance framework is to make it easier for the users to detect and debug such situations using a dedicated user interface integrated with the code assistant tools of their choice.

Slide 58

Slide 58 text

exactpro.com 58 BUILD SOFTWARE TO TEST SOFTWARE What are Compliance Rules? State BEFORE State AFTER Input Signal Audit Data Transition and Downstream Effects LLM-generated or hand-crafted code Deterministic verification code

Slide 59

Slide 59 text

exactpro.com 59 BUILD SOFTWARE TO TEST SOFTWARE Sample LLM-code Assistant Conversation

Slide 60

Slide 60 text

exactpro.com 60 BUILD SOFTWARE TO TEST SOFTWARE Regulatory Compliance Solution: Architecture Overview Compliance Engine Compliance Viewer (WebUI) Conformance Test Results 1. Compliance Viewer is a read-only portal that lets Compliance Analysts browse, filter and export test results produced by external test runners. 2. Results are stored as files on disk (no database involved). 3. The back-end (Compliance) engine reads and processes those files on demand, serving paginated and filterable data to the UI across various data categories: Messages, State Transitions, Scenarios, Annotations, Traceability, Rule Book and Code Assistant.

Slide 61

Slide 61 text

exactpro.com 61 BUILD SOFTWARE TO TEST SOFTWARE Property Checker Rules-based Testing Workflow generate input dataset execute output dataset analyse interpretation dataset 3 2 1 4 5 6 Evaluate the obtained information and the list of available actions – repeat the steps Select actions to perform ● What actions are available? ● Which of them to choose? ● In what order? Execute actions against the System Under Test (SUT) Action types: ● Set-up actions ● Experiment actions ● Observation actions Interpret the outcome of the actions Non-exclusive possibilities: ● Test results match our rules-based checks for the SUT ● Something is wrong with our actions ● Something is wrong with the SUT ● Something is wrong with our rules Provide the relevant information about findings to the stakeholders for them to make decisions Initial coverage based on the provided documentation, specifications, data formats Transactional data coverage based on the annotation mapping, actual data permutations, traffic analysis vs

Slide 62

Slide 62 text

exactpro.com 62 BUILD SOFTWARE TO TEST SOFTWARE Contains actors to manage sessions, encoding, decoding, sending messages and storing traffic. Written in Go language Consistency Validation – How do we Execute Testing Against the SUT? FIX ATP Drop Copy MD tcp MD udp tcpdump capture log files traffic files th2-shark Python scripts dataload reference data Uses configuration files with connectivity and submission rates settings input dataset output dataset pre-processing System Under Test (SUT) REST/JSON Websocket Testers

Slide 63

Slide 63 text

exactpro.com 63 BUILD SOFTWARE TO TEST SOFTWARE Consistency Validation – How do we Interpret the Outcome of Test Actions? FIX ATP Drop Copy MD tcp MD udp traffic files output dataset ✓ Check that we are able to maintain connection ✓ Check that we are able to parse the traffic ✓ Check that the parsed data conform to protocol specification ✓ Check for contradictions within each output stream ✓ Reconcile two different output streams ✓ Reconcile input and output streams ✓ Check specific properties across the full dataset ✓ Reconcile via rules-based outcomes ✓ Aggregate the data in the dataset ✓ Reconcile against expected aggregated results ✓ Explore for anomalies and trends ✓ AB comparison (between different release versions) th2 shark code in Go language Simple Python scripts for scenarios and rules validation interpretation dataset System Under Test (SUT) REST/JSON Websocket

Slide 64

Slide 64 text

exactpro.com 64 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 64 BUILD SOFTWARE TO TEST SOFTWARE State Transitions

Slide 65

Slide 65 text

exactpro.com 65 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 65 BUILD SOFTWARE TO TEST SOFTWARE State Transitions

Slide 66

Slide 66 text

exactpro.com 66 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 66 BUILD SOFTWARE TO TEST SOFTWARE State Transitions

Slide 67

Slide 67 text

exactpro.com 67 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 67 BUILD SOFTWARE TO TEST SOFTWARE State Transitions

Slide 68

Slide 68 text

exactpro.com 68 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 68 BUILD SOFTWARE TO TEST SOFTWARE Rule Book

Slide 69

Slide 69 text

exactpro.com 69 BUILD SOFTWARE TO TEST SOFTWARE exactpro.com 69 BUILD SOFTWARE TO TEST SOFTWARE Rule Book

Slide 70

Slide 70 text

exactpro.com 70 BUILD SOFTWARE TO TEST SOFTWARE Contribution Value Framework ● Tier 5 “Coherence Improving”: Focuses on simplifying and unifying the system by removing duplication, unnecessary variation and reducing accidental complexity ● Tier 4 “Guardrails Adding”: Establishes automated rules, checks and policies that keep development safe, secure and compliant without blocking progress ● Tier 3 “Feature Design”: Identifies gaps, inefficiencies, and missing capabilities before they become problems. It is about proactive thinking and shaping what should be built, not just implementing what is requested ● Tier 2 “Feature Testing”: Ensures features behave correctly under real-world conditions by exploring edge cases, failures, and system interactions. It shifts thinking from “Does it work?” to “How can it break?” ● Tier 1 “Feature Implementation”: Focuses on rapidly translating clear requirements into working code using AI tools

Slide 71

Slide 71 text

exactpro.com 71 BUILD SOFTWARE TO TEST SOFTWARE Learn Software Testing and Prepare for ISTQB® Certification https://exactpro.github.io/istqb-practice-hub/

Slide 72

Slide 72 text

exactpro.com 72 BUILD SOFTWARE TO TEST SOFTWARE Software Testing Hub ISTQB® CTFL • CT-AI • GenAI Exam Preparation

Slide 73

Slide 73 text

exactpro.com 73 BUILD SOFTWARE TO TEST SOFTWARE Thank You! [email protected] [email protected]