Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Model-Based Testing in Mission-Critical Financial Market Infrastructures

Model-Based Testing in Mission-Critical Financial Market Infrastructures

Elena Treshcheva
Program Manager, Exactpro

The QA Financial Forum: New York 2021
19 October 2021

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Exactpro
PRO

October 19, 2021
Tweet

More Decks by Exactpro

Other Decks in Technology

Transcript

  1. Model-Based Testing
    in Mission-Critical
    Financial Market Infrastructures
    Elena Treshcheva
    Program Manager, Exactpro

    View Slide

  2. Overview
    ● Definition
    ● Properties / requirements
    ● Alternatives to MBT
    ● 3 problems with MBT
    ● Processes, platforms and people
    to make MBT a success
    What model is a good model?
    Model-based testing (MBT)
    MBT challenges and perspectives

    View Slide

  3. About me
    2005 - 2018 – General and applied linguistics (Saratov State University):
    - linguistic database engineer
    - head of the Laboratory of Applied Linguistics
    - PhD in general linguistics (2012)
    - associate professor in computer linguistics
    2017 – Software testing, QA analyst (Exactpro, Saratov)
    2018 – Business Development and Research (Exactpro, USA)
    https://www.linkedin.com/in/treshcheva/

    View Slide

  4. A specialist firm focused on
    functional and non-functional
    testing services for financial
    market infrastructures, financial
    and non-financial institutions.
    Headquartered in the UK with
    operations in the US, Georgia,
    Russia, Lithuania and Sri Lanka.
    Most of our clients are regulated
    market infrastructures.
    Incorporated in 2009 with 10
    people, Exactpro now employs over
    700 specialists.
    Part of the London Stock
    Exchange Group (LSEG) from May
    2015 until January 2018, when the
    Exactpro management proceeded
    through the buyout of the
    company from LSEG.
    Exactpro provides independent
    software testing services for
    mission critical technology that
    underpins global financial
    markets. The firm is
    experienced with trading and
    clearing and settlement
    platforms, risk management
    systems, central data
    warehouses, accounting and
    treasury systems, regulatory
    reporting.
    About Exactpro

    View Slide

  5. Global Client Network – Financial Market Infrastructures

    View Slide

  6. 6 Build Software to Test Software exactpro.com
    Skytra selects Exactpro to test its new
    derivatives trading software
    Jeremy Norwood, CIO of Skytra: “Our timescales for launch are challenging, and
    we needed an experienced IT partner who could help us ensure that the
    delivered software works as per our business, operational and regulatory
    requirements. Exactpro will conduct a number of independent, unbiased
    functional and regression test cycles as part of our overall delivery programme. I
    am confident that Exactpro, with its proven track record of successful delivery of
    quality assurance projects for the financial services industry, will provide
    exceptional service for Skytra.”
    https://exactpro.com/news/exactpro/skytra-selects-exactpro-test-its-new-derivatives-trading-software

    View Slide

  7. 7 Build Software to Test Software exactpro.com
    Exactpro Supports Launch of MEMX into
    the U.S. Equity Market with Quality
    Assurance Testing
    “MEMX’s close collaboration with Exactpro provided
    additional operational resilience and contributed to the
    exchange’s seamless launch, rollout and, ultimately, live
    trading in all NMS symbols,” said MEMX Chief
    Technology Officer Dominick Paniscotti. “We greatly
    appreciate Exactpro’s partnership in working to ensure
    the exchange’s technology was ready to perform at full
    capacity from day one.”
    Read our latest Case Study:
    MEMX-Exactpro Collaboration on Exchange Quality Assurance
    https://exactpro.com/news/exactpro/exactpro-supports-launch-memx-us-equity-market-quality-assurance-testing

    View Slide

  8. 8 Build Software to Test Software exactpro.com
    https://exactpro.com/news/exactpro/ledgeredge-selects-exactpro-deliver-resilience-its-distributed
    -ledger-enabled
    LedgerEdge Selects Exactpro to Deliver
    Resilience for its Distributed Ledger Enabled
    Corporate Bond Trading Ecosystem.
    Robert Bose, Chief Technology Officer at LedgerEdge:
    “We are passionate about delivering the future of
    corporate bond trading. As we get closer to launch, it’s
    necessary to have a trusted partner to help us test our
    operational resilience and ensure we can meet the
    demands of the market. Exactpro’s bespoke tools and
    methods will enable us to achieve this, and we are
    excited to see the results of our collaboration.”

    View Slide

  9. What is a model?
    an interpretation
    of a theory
    an artifact aimed to describe a
    system, formulated in a modeling /
    programming language
    real-world system
    model model

    View Slide

  10. Model-based testing
    The MBT flow:
    - Building a model based on a specification
    - Automated generation of abstract test
    scenarios
    - Abstract test scenarios become executable
    scripts
    - Scripts are executed against the system
    - Test results analysis
    Requirement/System
    Specifications MBT Models Abstract Test Case
    Test Results SUT
    SystemAdapters
    ExecutionEnvironment
    Executable
    Test Case
    Mappings
    Test Selection
    Criteria
    1 2
    3
    4
    5

    View Slide

  11. Model-based testing
    Can be both mental
    representation or a schema
    or a code-based artifact
    Requirement/System
    Specifications
    MBT Models Abstract Test Case
    Test Verdicts SUT
    SystemAdapters
    ExecutionEnvironment
    Executable
    Test Case
    Mappings
    Test Selection
    Criteria
    1 2
    3
    4
    5

    View Slide

  12. Some of the existing test approaches: Feasible alternatives?
    Developer QA
    BA
    BDD

    View Slide

  13. Some of the existing test approaches: Feasible alternatives?
    Parallel run / Operational day replay approach

    View Slide

  14. All Testing is Model Based Testing

    View Slide

  15. Properties of a model
    mapping feature reduction feature pragmatic feature
    Herbert Stachowiak. Allgemeine Modelltheorie. Springer-Verlag, Wien and New York, 1973. (Cited by Kühne, Thomas. "What is a Model?." In
    Dagstuhl Seminar Proceedings. Schloss Dagstuhl-Leibniz-Zentrum für Informatik, 2005).

    View Slide

  16. Testing vs. Confirmation
    pragmatic map reduce

    View Slide

  17. The Exactpro Perspective
    Explicit Pragmatic Multi-Layered

    View Slide

  18. Defense in Depth

    View Slide

  19. State of the Art
    Mohd-Shafie M.L., Kadir W.M.N.W, Lichter H., Khatibsyarbini M., Isa M.A. Model‑based test case generation and prioritization:
    a systematic literature review. 2021, Springer.

    View Slide

  20. Taxonomy
    M. Utting, A. Pretschner, B.
    Legeard, B., “A taxonomy of
    model-based testing”,
    Special Issue: Model-based
    Testing Volume 1: Foundations
    and Applications of
    Model-Based Testing, volume 22,
    issue 5. August, 2012.

    View Slide

  21. Challenges

    View Slide

  22. Processes
    Platforms
    People
    Challenges

    View Slide

  23. Processes
    If complex models have to be completed before testing can start, this induces an unacceptable delay for the proper test executions.

    View Slide

  24. Platforms
    For complex SUT, like systems of systems, test models need to abstract from a large amount of detail, because otherwise the
    resulting test model would become unmanageable.

    View Slide

  25. Platforms
    Delivers end-to-end automated functional and
    non-functional testing of complex financial
    systems
    Enables intelligent interaction with many
    widely adopted network protocols as well as
    API, UI, DLT and cloud endpoints
    Executes sophisticated test algorithms
    Collects and processes distributed test data
    (for machine learning and other purposes)
    Performs model-based testing and analyses
    the behavior of systems under test
    Integrates with a variety of widely adopted
    test tools and frameworks via its open
    interface
    Is an open-source solution; th2 source
    code is available on GitHub
    Platform and
    Technology Agnostic
    Customised for
    Financial Systems
    Unified Data
    Warehouse
    Deliberate Practice of
    Software Testing
    AI-driven Strategic Data
    Acquisition
    Pervasive Automation
    Opportunities
    “th2 represents the next generation of automated testing platforms. We believe
    its usage will eventually transform the way firms approach their functional and
    non-functional testing processes. We look forward to rolling the framework out
    over the coming months and developing further use cases
    with our clients and partners.”
    CLICK TO READ MORE

    View Slide

  26. Platforms
    SUT
    DW
    Diversity

    View Slide

  27. People
    The required skills for test engineers writing test models are significantly higher than for test engineers writing sequential test
    procedures.

    View Slide

  28. 28 BUILD SOFTWARE TO TEST SOFTWARE
    Build Software to Test Software
    exactpro.com
    FOLLOW US ON SOCIAL MEDIA:
    exactpro.com

    View Slide