Slide 1

Slide 1 text

Model-Based Testing in Mission-Critical Financial Market Infrastructures Elena Treshcheva Program Manager, Exactpro

Slide 2

Slide 2 text

Overview ● Definition ● Properties / requirements ● Alternatives to MBT ● 3 problems with MBT ● Processes, platforms and people to make MBT a success What model is a good model? Model-based testing (MBT) MBT challenges and perspectives

Slide 3

Slide 3 text

About me 2005 - 2018 – General and applied linguistics (Saratov State University): - linguistic database engineer - head of the Laboratory of Applied Linguistics - PhD in general linguistics (2012) - associate professor in computer linguistics 2017 – Software testing, QA analyst (Exactpro, Saratov) 2018 – Business Development and Research (Exactpro, USA) https://www.linkedin.com/in/treshcheva/

Slide 4

Slide 4 text

A specialist firm focused on functional and non-functional testing services for financial market infrastructures, financial and non-financial institutions. Headquartered in the UK with operations in the US, Georgia, Russia, Lithuania and Sri Lanka. Most of our clients are regulated market infrastructures. Incorporated in 2009 with 10 people, Exactpro now employs over 700 specialists. Part of the London Stock Exchange Group (LSEG) from May 2015 until January 2018, when the Exactpro management proceeded through the buyout of the company from LSEG. Exactpro provides independent software testing services for mission critical technology that underpins global financial markets. The firm is experienced with trading and clearing and settlement platforms, risk management systems, central data warehouses, accounting and treasury systems, regulatory reporting. About Exactpro

Slide 5

Slide 5 text

Global Client Network – Financial Market Infrastructures

Slide 6

Slide 6 text

6 Build Software to Test Software exactpro.com Skytra selects Exactpro to test its new derivatives trading software Jeremy Norwood, CIO of Skytra: “Our timescales for launch are challenging, and we needed an experienced IT partner who could help us ensure that the delivered software works as per our business, operational and regulatory requirements. Exactpro will conduct a number of independent, unbiased functional and regression test cycles as part of our overall delivery programme. I am confident that Exactpro, with its proven track record of successful delivery of quality assurance projects for the financial services industry, will provide exceptional service for Skytra.” https://exactpro.com/news/exactpro/skytra-selects-exactpro-test-its-new-derivatives-trading-software

Slide 7

Slide 7 text

7 Build Software to Test Software exactpro.com Exactpro Supports Launch of MEMX into the U.S. Equity Market with Quality Assurance Testing “MEMX’s close collaboration with Exactpro provided additional operational resilience and contributed to the exchange’s seamless launch, rollout and, ultimately, live trading in all NMS symbols,” said MEMX Chief Technology Officer Dominick Paniscotti. “We greatly appreciate Exactpro’s partnership in working to ensure the exchange’s technology was ready to perform at full capacity from day one.” Read our latest Case Study: MEMX-Exactpro Collaboration on Exchange Quality Assurance https://exactpro.com/news/exactpro/exactpro-supports-launch-memx-us-equity-market-quality-assurance-testing

Slide 8

Slide 8 text

8 Build Software to Test Software exactpro.com https://exactpro.com/news/exactpro/ledgeredge-selects-exactpro-deliver-resilience-its-distributed -ledger-enabled LedgerEdge Selects Exactpro to Deliver Resilience for its Distributed Ledger Enabled Corporate Bond Trading Ecosystem. Robert Bose, Chief Technology Officer at LedgerEdge: “We are passionate about delivering the future of corporate bond trading. As we get closer to launch, it’s necessary to have a trusted partner to help us test our operational resilience and ensure we can meet the demands of the market. Exactpro’s bespoke tools and methods will enable us to achieve this, and we are excited to see the results of our collaboration.”

Slide 9

Slide 9 text

What is a model? an interpretation of a theory an artifact aimed to describe a system, formulated in a modeling / programming language real-world system model model

Slide 10

Slide 10 text

Model-based testing The MBT flow: - Building a model based on a specification - Automated generation of abstract test scenarios - Abstract test scenarios become executable scripts - Scripts are executed against the system - Test results analysis Requirement/System Specifications MBT Models Abstract Test Case Test Results SUT SystemAdapters ExecutionEnvironment Executable Test Case Mappings Test Selection Criteria 1 2 3 4 5

Slide 11

Slide 11 text

Model-based testing Can be both mental representation or a schema or a code-based artifact Requirement/System Specifications MBT Models Abstract Test Case Test Verdicts SUT SystemAdapters ExecutionEnvironment Executable Test Case Mappings Test Selection Criteria 1 2 3 4 5

Slide 12

Slide 12 text

Some of the existing test approaches: Feasible alternatives? Developer QA BA BDD

Slide 13

Slide 13 text

Some of the existing test approaches: Feasible alternatives? Parallel run / Operational day replay approach

Slide 14

Slide 14 text

All Testing is Model Based Testing

Slide 15

Slide 15 text

Properties of a model mapping feature reduction feature pragmatic feature Herbert Stachowiak. Allgemeine Modelltheorie. Springer-Verlag, Wien and New York, 1973. (Cited by Kühne, Thomas. "What is a Model?." In Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2005).

Slide 16

Slide 16 text

Testing vs. Confirmation pragmatic map reduce

Slide 17

Slide 17 text

The Exactpro Perspective Explicit Pragmatic Multi-Layered

Slide 18

Slide 18 text

Defense in Depth

Slide 19

Slide 19 text

State of the Art Mohd-Shafie M.L., Kadir W.M.N.W, Lichter H., Khatibsyarbini M., Isa M.A. Model‑based test case generation and prioritization: a systematic literature review. 2021, Springer.

Slide 20

Slide 20 text

Taxonomy M. Utting, A. Pretschner, B. Legeard, B., “A taxonomy of model-based testing”, Special Issue: Model-based Testing Volume 1: Foundations and Applications of Model-Based Testing, volume 22, issue 5. August, 2012.

Slide 21

Slide 21 text

Challenges

Slide 22

Slide 22 text

Processes Platforms People Challenges

Slide 23

Slide 23 text

Processes If complex models have to be completed before testing can start, this induces an unacceptable delay for the proper test executions.

Slide 24

Slide 24 text

Platforms For complex SUT, like systems of systems, test models need to abstract from a large amount of detail, because otherwise the resulting test model would become unmanageable.

Slide 25

Slide 25 text

Platforms Delivers end-to-end automated functional and non-functional testing of complex financial systems Enables intelligent interaction with many widely adopted network protocols as well as API, UI, DLT and cloud endpoints Executes sophisticated test algorithms Collects and processes distributed test data (for machine learning and other purposes) Performs model-based testing and analyses the behavior of systems under test Integrates with a variety of widely adopted test tools and frameworks via its open interface Is an open-source solution; th2 source code is available on GitHub Platform and Technology Agnostic Customised for Financial Systems Unified Data Warehouse Deliberate Practice of Software Testing AI-driven Strategic Data Acquisition Pervasive Automation Opportunities “th2 represents the next generation of automated testing platforms. We believe its usage will eventually transform the way firms approach their functional and non-functional testing processes. We look forward to rolling the framework out over the coming months and developing further use cases with our clients and partners.” CLICK TO READ MORE

Slide 26

Slide 26 text

Platforms SUT DW Diversity

Slide 27

Slide 27 text

People The required skills for test engineers writing test models are significantly higher than for test engineers writing sequential test procedures.

Slide 28

Slide 28 text

28 BUILD SOFTWARE TO TEST SOFTWARE Build Software to Test Software exactpro.com FOLLOW US ON SOCIAL MEDIA: exactpro.com