Model-Based Testing in Mission-Critical Financial Market Infrastructures

Elena Treshcheva
Program Manager, Exactpro

The QA Financial Forum: New York 2021
19 October 2021

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Transcript

1. Model-Based Testing
   in Mission-Critical
   Financial Market Infrastructures
   Elena Treshcheva
   Program Manager, Exactpro
   

   View Slide

2. Overview
   ● Definition
   ● Properties / requirements
   ● Alternatives to MBT
   ● 3 problems with MBT
   ● Processes, platforms and people
   to make MBT a success
   What model is a good model?
   Model-based testing (MBT)
   MBT challenges and perspectives
   

   View Slide

3. About me
   2005 - 2018 – General and applied linguistics (Saratov State University):
   - linguistic database engineer
   - head of the Laboratory of Applied Linguistics
   - PhD in general linguistics (2012)
   - associate professor in computer linguistics
   2017 – Software testing, QA analyst (Exactpro, Saratov)
   2018 – Business Development and Research (Exactpro, USA)
   https://www.linkedin.com/in/treshcheva/
   

   View Slide

4. A specialist firm focused on
   functional and non-functional
   testing services for financial
   market infrastructures, financial
   and non-financial institutions.
   Headquartered in the UK with
   operations in the US, Georgia,
   Russia, Lithuania and Sri Lanka.
   Most of our clients are regulated
   market infrastructures.
   Incorporated in 2009 with 10
   people, Exactpro now employs over
   700 specialists.
   Part of the London Stock
   Exchange Group (LSEG) from May
   2015 until January 2018, when the
   Exactpro management proceeded
   through the buyout of the
   company from LSEG.
   Exactpro provides independent
   software testing services for
   mission critical technology that
   underpins global financial
   markets. The firm is
   experienced with trading and
   clearing and settlement
   platforms, risk management
   systems, central data
   warehouses, accounting and
   treasury systems, regulatory
   reporting.
   About Exactpro
   

   View Slide

5. Global Client Network – Financial Market Infrastructures
   

   View Slide

6. 6 Build Software to Test Software exactpro.com
   Skytra selects Exactpro to test its new
   derivatives trading software
   Jeremy Norwood, CIO of Skytra: “Our timescales for launch are challenging, and
   we needed an experienced IT partner who could help us ensure that the
   delivered software works as per our business, operational and regulatory
   requirements. Exactpro will conduct a number of independent, unbiased
   functional and regression test cycles as part of our overall delivery programme. I
   am confident that Exactpro, with its proven track record of successful delivery of
   quality assurance projects for the financial services industry, will provide
   exceptional service for Skytra.”
   https://exactpro.com/news/exactpro/skytra-selects-exactpro-test-its-new-derivatives-trading-software
   

   View Slide

7. 7 Build Software to Test Software exactpro.com
   Exactpro Supports Launch of MEMX into
   the U.S. Equity Market with Quality
   Assurance Testing
   “MEMX’s close collaboration with Exactpro provided
   additional operational resilience and contributed to the
   exchange’s seamless launch, rollout and, ultimately, live
   trading in all NMS symbols,” said MEMX Chief
   Technology Officer Dominick Paniscotti. “We greatly
   appreciate Exactpro’s partnership in working to ensure
   the exchange’s technology was ready to perform at full
   capacity from day one.”
   Read our latest Case Study:
   MEMX-Exactpro Collaboration on Exchange Quality Assurance
   https://exactpro.com/news/exactpro/exactpro-supports-launch-memx-us-equity-market-quality-assurance-testing
   

   View Slide

8. 8 Build Software to Test Software exactpro.com
   https://exactpro.com/news/exactpro/ledgeredge-selects-exactpro-deliver-resilience-its-distributed
   -ledger-enabled
   LedgerEdge Selects Exactpro to Deliver
   Resilience for its Distributed Ledger Enabled
   Corporate Bond Trading Ecosystem.
   Robert Bose, Chief Technology Officer at LedgerEdge:
   “We are passionate about delivering the future of
   corporate bond trading. As we get closer to launch, it’s
   necessary to have a trusted partner to help us test our
   operational resilience and ensure we can meet the
   demands of the market. Exactpro’s bespoke tools and
   methods will enable us to achieve this, and we are
   excited to see the results of our collaboration.”
   

   View Slide

9. What is a model?
   an interpretation
   of a theory
   an artifact aimed to describe a
   system, formulated in a modeling /
   programming language
   real-world system
   model model
   

   View Slide

10. Model-based testing
    The MBT flow:
    - Building a model based on a specification
    - Automated generation of abstract test
    scenarios
    - Abstract test scenarios become executable
    scripts
    - Scripts are executed against the system
    - Test results analysis
    Requirement/System
    Specifications MBT Models Abstract Test Case
    Test Results SUT
    SystemAdapters
    ExecutionEnvironment
    Executable
    Test Case
    Mappings
    Test Selection
    Criteria
    1 2
    3
    4
    5
    

    View Slide

11. Model-based testing
    Can be both mental
    representation or a schema
    or a code-based artifact
    Requirement/System
    Specifications
    MBT Models Abstract Test Case
    Test Verdicts SUT
    SystemAdapters
    ExecutionEnvironment
    Executable
    Test Case
    Mappings
    Test Selection
    Criteria
    1 2
    3
    4
    5
    

    View Slide

12. Some of the existing test approaches: Feasible alternatives?
    Developer QA
    BA
    BDD
    

    View Slide

13. Some of the existing test approaches: Feasible alternatives?
    Parallel run / Operational day replay approach
    

    View Slide

14. All Testing is Model Based Testing
    

    View Slide

15. Properties of a model
    mapping feature reduction feature pragmatic feature
    Herbert Stachowiak. Allgemeine Modelltheorie. Springer-Verlag, Wien and New York, 1973. (Cited by Kühne, Thomas. "What is a Model?." In
    Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2005).
    

    View Slide

16. Testing vs. Confirmation
    pragmatic map reduce
    

    View Slide

17. The Exactpro Perspective
    Explicit Pragmatic Multi-Layered
    

    View Slide

18. Defense in Depth
    

    View Slide

19. State of the Art
    Mohd-Shafie M.L., Kadir W.M.N.W, Lichter H., Khatibsyarbini M., Isa M.A. Model‑based test case generation and prioritization:
    a systematic literature review. 2021, Springer.
    

    View Slide

20. Taxonomy
    M. Utting, A. Pretschner, B.
    Legeard, B., “A taxonomy of
    model-based testing”,
    Special Issue: Model-based
    Testing Volume 1: Foundations
    and Applications of
    Model-Based Testing, volume 22,
    issue 5. August, 2012.
    

    View Slide

21. Challenges
    

    View Slide

22. Processes
    Platforms
    People
    Challenges
    

    View Slide

23. Processes
    If complex models have to be completed before testing can start, this induces an unacceptable delay for the proper test executions.
    

    View Slide

24. Platforms
    For complex SUT, like systems of systems, test models need to abstract from a large amount of detail, because otherwise the
    resulting test model would become unmanageable.
    

    View Slide

25. Platforms
    Delivers end-to-end automated functional and
    non-functional testing of complex financial
    systems
    Enables intelligent interaction with many
    widely adopted network protocols as well as
    API, UI, DLT and cloud endpoints
    Executes sophisticated test algorithms
    Collects and processes distributed test data
    (for machine learning and other purposes)
    Performs model-based testing and analyses
    the behavior of systems under test
    Integrates with a variety of widely adopted
    test tools and frameworks via its open
    interface
    Is an open-source solution; th2 source
    code is available on GitHub
    Platform and
    Technology Agnostic
    Customised for
    Financial Systems
    Unified Data
    Warehouse
    Deliberate Practice of
    Software Testing
    AI-driven Strategic Data
    Acquisition
    Pervasive Automation
    Opportunities
    “th2 represents the next generation of automated testing platforms. We believe
    its usage will eventually transform the way firms approach their functional and
    non-functional testing processes. We look forward to rolling the framework out
    over the coming months and developing further use cases
    with our clients and partners.”
    CLICK TO READ MORE
    

    View Slide

26. Platforms
    SUT
    DW
    Diversity
    

    View Slide

27. People
    The required skills for test engineers writing test models are significantly higher than for test engineers writing sequential test
    procedures.
    

    View Slide

28. 28 BUILD SOFTWARE TO TEST SOFTWARE
    Build Software to Test Software
    exactpro.com
    FOLLOW US ON SOCIAL MEDIA:
    exactpro.com
    

    View Slide