Sibos 2017: Disruptive functional testing - the next frontier in post-trade systems verification

16 October 2017
Sibos Toronto
Presentation by Iosif Itkin, Exactpro CEO and co-founder
London Stock Exchange Group

Disruptive functional testing - the next frontier in post-trade systems verification

The operators of complex back office and post-trade systems interconnected by SWIFT and other communication protocols are trying to keep pace with the front office technology advances, rising data volumes and changing regulatory landscape. The industry actively adopts flexible development methodologies to speed up the software delivery process. What aspects of software quality are not covered by regular performance / failover testing and the agile development life cycle? Disruptive functional testing is a way to prepare your mission-critical infrastructure for any challenges the future may bring. Ideas for this presentation are drawn from our work with LCH, T2S, clearing and settlement systems worldwide.

Transcript

1. Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 24 60 , +1 415 830 38 49 www.exactpro.com 1 Disruptive Functional Testing The next frontier in Post-Trade systems verification Open Access Quality Assurance & Related Software Development for Financial Markets Tel: +7 495 640 2460, +1 415 830 38 49 www.exactpro.com 16 October 2017, SIBOS Conference Iosif Itkin, Exactpro CEO and co-founder, London Stock Exchange Group
2. • A specialist firm focused on functional and non-functional testing of systems that process wholesale financial products, particularly market infrastructures • A UK company with operations in the US and four QA & software development centres in Russia • Part of London Stock Exchange Group as of May 29, 2015 • Incorporated in 2009 with 10 people, our company has experienced significant growth as satisfied clients require more services; now employing 536 specialists Front office Listed Derivatives CRM Dark Pool T25 Clearing FX Fixed income SEF Reconciliation Onboarding Ticker Plant Commondities Smart Order Router Trading Desktops Index Dissemination FPGA Equities Connectivity Exchange MTF Swaptions Exactpro Timeline at a Glance
3. Delivering Quality to London Stock Exchange Group Some of our projects (start date): •Turquoise (2010) •Borsa Italiana (2011) •CCP.A (2011) •FPGA-base Ticker Plant (2012) •LSE (2012) •Oslo Bors (2012) •UnaVista (2013) •FTSE (2013) •EuroTLX (2014) •Russell (2015) •IDEM (2015) •LSEDM (2015) •ELITE (since 2015) •CurveGlobal (2016) •GATElab (2016) •LCH (2016) •TRADEcho (2016) •MTS (2017) •RNS (2017)
4. Open Access Quality Assurance – a Global Client Network
5. Faster, not Bigger What is the main difference between incumbent and disruptive? Incumbent means something you already know and expect. Disruptive is when we encounter new things that will re-define what we already know.
6. Post Trade Systems
7. Software Testing Challenges
8. With incumbent testing, you are on the hook of pre-conceptions of how your system under test is supposed to work. It's only when you do disruptive testing that you really gain new knowledge about the system and learn from it. Get Off the Hook
9. Software Testing is Relentless Learning
10. Defect Mining The goal of disruptive testing is to find defects that are unlikely to be found if testing is done with other methods. We see ourselves as a mining operation extracting defects hidden in complex scalable technology platforms.
11. Test Harness
12. Agile Transformation As in any other large organization in the financial services industry, technology used at the London Stock Exchange Group is going through agile transformation.
13. Developing a Test Harness for Agile Process
14. What Agile Development Should Be… and Not NOT LIKE THIS LIKE THIS
15. Testing Challenges in Agile
16. Managing Complex Systems The Law of Requisite Variety: a control system must have at least as many possible states as the system it wants to control. It means that Test Harness development is a truly challenging and continuous software development process.
17. Test Harness Team
18. Testing Critical Infrastructures
19. Testing Critical Infrastructures Safety Rule #1 with Submarines: don't open portholes when underwater!
20. Testing Critical Infrastructures Safety Rule #1 with Submarines: don't open portholes when underwater! Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force
21. Testing Critical Infrastructures Safety Rule #1 with Submarines: don't open portholes when underwater! Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force Disruptive testing: 1) iterate through a huge number of random diverse scenarios under load to prove that it won't open
22. Testing Critical Infrastructures Safety Rule #1 with Submarines: don't open portholes when underwater! Functional testing: iterate through a finite number of scenarios to prove that the porthole won't open Non-Functional testing: iterate through a smaller number of scenarios to prove that it won't open by brute force Disruptive testing: 1) iterate through a huge number of random diverse scenarios under load to prove that it won't open 2) open the porthole
23. Thank you