Iosif Itkin. A Reference Architecture for HiVAT (High Volume Automated Testing)

We invite you to join the International Summer School on Data Science in Software Engineering. The summer school will be held online on 12-16 July, 2021, organized by the Laboratory of Software Testing, Tomsk Polytechnic University.
Participation is free. The official language of the school is English.

Students, young researchers and practitioners interested in applications of modern data science methods to the development and testing of complex software systems are invited to join. Follow the link to learn the full program and register your participation: https://itr-tpu.timepad.ru/event/1629835/
____
To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/
Vkontakte https://vk.com/exactpro_llc

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

Transcript

1. 1 BUILD SOFTWARE TO TEST SOFTWARE
   International Summer School on
   Data Science in
   Software Engineering
   Iosif Itkin, Exactpro
   A Reference Architecture for HiVAT
   (High Volume Automated Testing)
   12.07.2021 — 16.07.2021
   

   View Slide

2. 2 BUILD SOFTWARE TO TEST SOFTWARE
   Contents
   1. Introduction
   2. Build Software to Test Software
   3. High Volumes Automated Testing
   4. Performance and Scalability
   5. Diversity and Unification
   6. Resilience and Antifragility
   7. Reference Implementation (th2)
   8. Q&A
   

   View Slide

3. 3 BUILD SOFTWARE TO TEST SOFTWARE
   Introduction
   ● Started IT career 20+ years ago as a software developer
   ● Moved into performance and load testing in 2006
   ● Co-founded Exactpro on 9th September 2009
   ● Head of LOST (Laboratory of Software Testing) at Tomsk
   Polytechnic University
   ● Co-chair of FIX Trading Community MOST Working
   Group (Monitoring, Onboarding and Software Testing)
   ● TMPA Tools & Methods of Program Analysis Conference
   

   View Slide

4. 4 BUILD SOFTWARE TO TEST SOFTWARE
   TMPA Tools & Methods of Program Analysis Conference
   

   View Slide

5. 5 BUILD SOFTWARE TO TEST SOFTWARE
   About Exactpro
   A specialist firm focused on
   functional and non-functional
   testing services for financial
   market infrastructures
   Headquartered in the UK with
   operations in the US, Georgia,
   Russia. This year Exactpro is
   opening two additional delivery
   centers in Lithuania and Sri Lanka
   Exactpro provides independent
   software testing services for
   mission critical technology
   that underpins global financial
   markets. The firm is
   experienced with trading and
   clearing & settlement
   platforms, risk management
   systems, central data
   warehouses, accounting and
   treasury systems, regulatory
   reporting.
   Incorporated in 2009 with 10
   people, Exactpro now employs
   over 650 specialists.
   Part of the London Stock
   Exchange Group (LSEG) from
   May 2015 until January 2018,
   when the Exactpro management
   proceeded through the buyout of
   the company from LSEG.
   

   View Slide

6. 6 BUILD SOFTWARE TO TEST SOFTWARE
   Our Clients – 10 of the World’s Top 20 Systemically Important
   Exchanges and Clearing Houses
   

   View Slide

7. 7 Build Software to Test Software exactpro.com
   Confidential
   exactpro.com
   Skytra selects Exactpro to test its new
   derivatives trading software
   Jeremy Norwood, CIO of Skytra: “Our timescales for launch are challenging, and we
   needed an experienced IT partner who could help us ensure that the delivered software
   works as per our business, operational and regulatory requirements. Exactpro will
   conduct a number of independent, unbiased functional and regression test cycles as
   part of our overall delivery programme. I am confident that Exactpro, with its proven
   track record of successful delivery of quality assurance projects for the financial
   services industry, will provide exceptional service for Skytra.”
   https://exactpro.com/news/exactpro/skytra-selects-exactpro-test-its-new-derivatives-trading-software
   

   View Slide

8. 8 BUILD SOFTWARE TO TEST SOFTWARE
   8 Build Software to Test Software exactpro.com
   Confidential
   exactpro.com
   Exactpro Supports Launch of MEMX into
   the U.S. Equity Market with Quality
   Assurance Testing
   “MEMX’s close collaboration with Exactpro provided
   additional operational resilience and contributed to the
   exchange’s seamless launch, rollout and, ultimately, live
   trading in all NMS symbols,” said MEMX Chief Technology
   Officer Dominick Paniscotti. “We greatly appreciate
   Exactpro’s partnership in working to ensure the exchange’s
   technology was ready to perform at full capacity from day
   one.”
   Read our latest Case Study:
   MEMX-Exactpro Collaboration on Exchange Quality Assurance
   https://exactpro.com/news/exactpro/exactpro-supports-launch-memx-us-equity-market-quality-assurance-testing
   

   View Slide

9. 9 BUILD SOFTWARE TO TEST SOFTWARE
   Typical Project
   

   View Slide

10. 10 BUILD SOFTWARE TO TEST SOFTWARE
    What makes us different
    The Value of Understanding Your Technology Assets
    Technology is the main asset of every modern company. We help our clients
    build, operate and transform their technology to increase its value. Our
    Deliberate Practice of Software Testing approach is based on modelling the
    entire system under test (SUT), which allows us to create test libraries that
    serve as an executable specification for the highly complex platforms we test.
    This is a ‘deliberate’ practice, meaning that it is systematic and focused on
    achieving the specific goal of improving our capacity of extracting information
    about the SUT.
    https://www.fixtrading.org/wp-content/uploads/2021/06/Pre-Event-Magazine-EMEA-Trading-Confer
    ence-2021.pdf
    Read more in Mikhail Kurkov’s
    article on The Value of
    Understanding Your
    Technology Assets in the
    exclusive FIX Trading
    Community pre-event EMEA
    Trading Conference 2021
    magazine.
    

    View Slide

11. 11 BUILD SOFTWARE TO TEST SOFTWARE
    Software Testing
    Software Testing is an
    ● empirical
    ● technical
    ● investigation
    ● conducted to provide stakeholders
    ● with information
    ● about the quality
    ● of the product or service under test
    Software Testing is exploring software with the intent of
    finding bugs
    

    View Slide

12. 12 BUILD SOFTWARE TO TEST SOFTWARE
    Automated Testing means Computer-Assisted Testing
    

    View Slide

13. 13 BUILD SOFTWARE TO TEST SOFTWARE
    High Volume Automated Testing (HiVAT)
    http://kaner.com/pdfs/highvolCSTER.pdf
    http://kaner.com/?p=278
    HVTA / HiVAT
    

    View Slide

14. 14 BUILD SOFTWARE TO TEST SOFTWARE
    High Volume Automated Testing (HiVAT)
    A family of test techniques that use software to generate,
    execute and interpret arbitrarily many tests
    A family of test techniques that empower a tester to focus
    on interesting results while using software to design,
    implement, execute, and interpret a large suite of tests
    (thousands to billions of tests)
    Automated Testing means
    Computer-Assisted Testing
    

    View Slide

15. 15 BUILD SOFTWARE TO TEST SOFTWARE
    Recent Advances in Machine Learning
    Data Volumes
    Computational Power
    

    View Slide

16. 16 BUILD SOFTWARE TO TEST SOFTWARE
    Performance and Scalability
    Daily capacity - 100+ mln transactions
    Peak rates - 40k+ transactions per second
    Average round-trip latency - <100 microseconds
    3000
    trx
    2.5 cm <1 mm
    

    View Slide

17. 17 BUILD SOFTWARE TO TEST SOFTWARE
    Performance and Scalability
    

    View Slide

18. 18 BUILD SOFTWARE TO TEST SOFTWARE
    Performance and Scalability
    SUT
    DW
    

    View Slide

19. 19 BUILD SOFTWARE TO TEST SOFTWARE
    Diversity and Unification
    Functional
    Testing
    Non-Functional
    Testing
    

    View Slide

20. 20 BUILD SOFTWARE TO TEST SOFTWARE
    Diversity and Unification
    

    View Slide

21. 21 BUILD SOFTWARE TO TEST SOFTWARE
    Diversity and Unification
    

    View Slide

22. 22 BUILD SOFTWARE TO TEST SOFTWARE
    Diversity and Unification
    

    View Slide

23. 23 BUILD SOFTWARE TO TEST SOFTWARE
    Diversity and Unification
    SUT
    DW
    Diversity
    

    View Slide

24. 24 BUILD SOFTWARE TO TEST SOFTWARE
    Resilience and Antifragility
    

    View Slide

25. 25 BUILD SOFTWARE TO TEST SOFTWARE
    Resilience and Antifragility
    SUT
    DW
    

    View Slide

26. 26 BUILD SOFTWARE TO TEST SOFTWARE
    Authonomy
    

    View Slide

27. 27 BUILD SOFTWARE TO TEST SOFTWARE
    th2 Next-generation
    Test Automation Framework
    https://github.com/th2-net
    

    View Slide

28. 28 BUILD SOFTWARE TO TEST SOFTWARE
    Delivers end-to-end automated functional and
    non-functional testing of complex financial systems
    Enables intelligent interaction with many widely
    adopted network protocols as well as API, UI, DLT and
    cloud endpoints
    Executes sophisticated test algorithms
    Collects and processes distributed test data (for
    machine learning and other purposes)
    Performs model-based testing and analyses the
    behaviour of systems under test
    Integrates with a variety of widely adopted test tools
    and frameworks via its open interface
    Is an open-source solution; th2 source code is available
    on GitHub
    exactpro.com
    Platform and
    Technology Agnostic
    Customised for Financial
    Systems
    Unified Data
    Warehouse
    Deliberate Practice of
    Software Testing
    AI-driven Strategic Data
    Acquisition
    Pervasive Automation
    Opportunities
    “th2 represents the next generation of automated testing platforms. We
    believe its usage will eventually transform the way firms approach their
    functional and non-functional testing processes. We look forward to
    rolling the framework out over the coming
    months and developing further use cases
    with our clients and partners.”
    has been successfully deployed in a number of use cases. Its first adopter – Airbus'
    subsidiary Skytra – harnessed the technology for its derivatives trading software. Its most
    recent implementation was for ML-driven testing of Quod Financial’s technology
    READ MORE
    th2 Next-generation
    Test Automation Framework
    

    View Slide

29. 29 BUILD SOFTWARE TO TEST SOFTWARE
    Q&A
    Thank You
    

    View Slide