Software Testing Research

Iosif Itkin, CEO and co-founder, Exactpro

The Software Tester's Day 2021
9 September 2021

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Transcript

1. 1 BUILD SOFTWARE TO TEST SOFTWARE
   Build Software to Test Software
   exactpro.com
   

   View Slide

2. 2 BUILD SOFTWARE TO TEST SOFTWARE
   Introduction
   ● Started IT career 20+ years ago as a software developer
   ● Moved into performance and load testing in 2006
   ● Co-founded Exactpro on 9th September 2009
   ● Head of LOST (Laboratory of Software Testing) at Tomsk Polytechnic University
   ● Co-chair of FIX Trading Community MOST Working Group (Monitoring,
   Onboarding and Software Testing)
   ● TMPA Tools & Methods of Program Analysis Conference
   

   View Slide

3. 3 BUILD SOFTWARE TO TEST SOFTWARE
   12th anniversary of Exactpro
   

   View Slide

4. 4 BUILD SOFTWARE TO TEST SOFTWARE
   4 BUILD SOFTWARE TO TEST SOFTWARE
   About Exactpro
   A specialist firm focused on
   functional and non-functional
   testing services for financial
   market infrastructures. Exactpro
   projects include large-scale
   initiatives worldwide.
   Headquartered in the UK with
   operations in the US, Georgia,
   Russia, Lithuania and Sri-Lanka.
   Most of our clients are regulated
   market infrastructures.
   Exactpro provides independent
   software testing services for
   mission critical technology that
   underpins global financial
   markets. The firm is experienced
   with trading, clearing &
   settlement platforms, risk
   management systems, central
   data warehouses, accounting
   and treasury systems,
   regulatory reporting.
   Incorporated in 2009 with
   10 people, Exactpro is now
   employing over 650
   specialists.
   Part of the London Stock Exchange
   Group (LSEG) from May 2015 until
   January 2018, when the Exactpro
   management proceeded through the
   buyout of the company from LSEG
   

   View Slide

5. 5 BUILD SOFTWARE TO TEST SOFTWARE
   5 BUILD SOFTWARE TO TEST SOFTWARE
   Our clients include 10 of the world’s Top 20 exchange groups
   

   View Slide

6. 6 BUILD SOFTWARE TO TEST SOFTWARE
   Build Software to Test Software
   exactpro.com
   Skytra selects Exactpro to test its new
   derivatives trading software
   Jeremy Norwood, CIO of Skytra: “Our timescales for launch are challenging, and
   we needed an experienced IT partner who could help us ensure that the
   delivered software works as per our business, operational and regulatory
   requirements. Exactpro will conduct a number of independent, unbiased
   functional and regression test cycles as part of our overall delivery programme. I
   am confident that Exactpro, with its proven track record of successful delivery of
   quality assurance projects for the financial services industry, will provide
   exceptional service for Skytra.”
   https://exactpro.com/news/exactpro/skytra-selects-exactpro-test-its-new-derivatives-trading-software
   

   View Slide

7. 7 BUILD SOFTWARE TO TEST SOFTWARE
   7 Build Software to Test Software exactpro.com
   Confidential
   exactpro.com
   Exactpro Supports Launch of MEMX
   into
   the U.S. Equity Market with Quality
   Assurance Testing
   “MEMX’s close collaboration with Exactpro provided
   additional operational resilience and contributed to the
   exchange’s seamless launch, rollout and, ultimately, live
   trading in all NMS symbols,” said MEMX Chief
   Technology Officer Dominick Paniscotti. “We greatly
   appreciate Exactpro’s partnership in working to ensure
   the exchange’s technology was ready to perform at full
   capacity from day one.”
   Read our latest Case Study:
   MEMX-Exactpro Collaboration on Exchange Quality Assurance
   https://exactpro.com/news/exactpro/exactpro-supports-launch-memx-us-equity-market-quality-assurance-testing
   

   View Slide

8. 8 BUILD SOFTWARE TO TEST SOFTWARE
   

   View Slide

9. 9 BUILD SOFTWARE TO TEST SOFTWARE
   ● Art
   ● Craft
   ● Research
   

   View Slide

10. 10 BUILD SOFTWARE TO TEST SOFTWARE
    

    View Slide

11. 11 BUILD SOFTWARE TO TEST SOFTWARE
    Software Testing - Art, Craft, Science, Engineering
    Continuum
    

    View Slide

12. 12 BUILD SOFTWARE TO TEST SOFTWARE
    Software Testing - Art, Craft, Science, Engineering
    Continuum
    

    View Slide

13. 13 BUILD SOFTWARE TO TEST SOFTWARE
    Software Testing is Relentless Learning
    Software Testing is an
    ● empirical
    ● technical
    ● investigation
    ● conducted to provide stakeholders
    ● with information
    ● about the quality
    ● of the product or service under test
    Software Testing is exploring software with the intent of
    finding bugs
    

    View Slide

14. 14 BUILD SOFTWARE TO TEST SOFTWARE
    Software Testing and Automation in Testing
    https://fs.blog/2013/07/the-difference-between-science-and-engineering/
    

    View Slide

15. 15 BUILD SOFTWARE TO TEST SOFTWARE
    Software Testing and Investigative Reporting
    Continuum
    

    View Slide

16. 16 BUILD SOFTWARE TO TEST SOFTWARE
    Peer review process
    Peer review is the evaluation of work by one or more
    people with similar competencies as the producers of the
    work (peers). It functions as a form of self-regulation by
    qualified members of a profession within the relevant field.
    Peer review methods are used to maintain quality
    standards, improve performance, and provide credibility. In
    academia, scholarly peer review is often used to determine
    an academic paper’s suitability for publication
    Wiki:
    

    View Slide

17. 17 BUILD SOFTWARE TO TEST SOFTWARE
    ● Achievements
    ● Challenges
    ● Dreams
    

    View Slide

18. 18 BUILD SOFTWARE TO TEST SOFTWARE
    

    View Slide

19. 19 BUILD SOFTWARE TO TEST SOFTWARE
    

    View Slide

20. 20 BUILD SOFTWARE TO TEST SOFTWARE
    Test Oracles
    

    View Slide

21. 21 BUILD SOFTWARE TO TEST SOFTWARE
    The Oracle Problem in Software Testing: A Survey
    

    View Slide

22. 22 BUILD SOFTWARE TO TEST SOFTWARE
    Subscribe to Exactpro YouTube Channel
    

    View Slide

23. 23 BUILD SOFTWARE TO TEST SOFTWARE
    Machine Learning Testing: Survey, Landscapes and Horizons
    

    View Slide

24. 24 BUILD SOFTWARE TO TEST SOFTWARE
    Machine Learning Testing: Survey, Landscapes and Horizons
    https://www.slideshare.net/HeemengFoo/machine-learning-testing-survey-landscapes-and-horizons-the-cliff-notes
    https://speakerdeck.com/exactpro/tiestirovaniie-mashinnogho-obuchieniia-obzor-pieizazhi-i-ghorizonty
    https://www.eventbrite.com/o/ai-in-testing-and-testing-ai-meetup-group-29640157833
    

    View Slide

25. 25 BUILD SOFTWARE TO TEST SOFTWARE
    https://testingconferences.org/
    https://github.com/TestingConferences/testingconferences.github.io/
    

    View Slide

26. 26 BUILD SOFTWARE TO TEST SOFTWARE
    26 BUILD SOFTWARE TO TEST SOFTWARE
    Top Software Testing Conferences
    

    View Slide

27. 27 BUILD SOFTWARE TO TEST SOFTWARE
    27 BUILD SOFTWARE TO TEST SOFTWARE
    International Symposium on Software Testing and Analysis
    

    View Slide

28. 28 BUILD SOFTWARE TO TEST SOFTWARE
    28 BUILD SOFTWARE TO TEST SOFTWARE
    Artificial Intelligence Testing http://ieeetests.com/
    

    View Slide

29. 29 BUILD SOFTWARE TO TEST SOFTWARE
    IEEE AV Testing Course
    http://av-test-challenge.org/
    

    View Slide

30. 30 BUILD SOFTWARE TO TEST SOFTWARE
    Spring/Summer Young Researchers' Colloquium on
    Software Engineering
    http://syrcose.ispras.ru/
    

    View Slide

31. 31 BUILD SOFTWARE TO TEST SOFTWARE
    tmpaconf.org
    

    View Slide

32. 32 BUILD SOFTWARE TO TEST SOFTWARE
    Build Software to Test Software
    exactpro.com
    

    View Slide

33. 33 BUILD SOFTWARE TO TEST SOFTWARE
    Build Software to Test Software
    exactpro.com
    FOLLOW US ON SOCIAL MEDIA:
    exactpro.com
    exactpro.com
    

    View Slide