20 April 2019, Tbilisi, Georgia Behaviour Driven Development (BDD) The Outer Limits Murad Mamedov, QA Analyst, Exactpro Alexandr Banakov, QA Analyst, Exactpro

20 April 2019, Tbilisi, Georgia EXACTPRO Exactpro History at a Glance • A specialist firm focused on functional and non functional testing of exchanges, clearing houses, depositories and other market infrastructures • Incorporated in 2009 with 10 people, our company has experienced significant growth as satisfied clients require more services; now employing 550 specialists. • Part of London Stock Exchange Group (LSEG) from May 2015 till January 2018. Exactpro management buyout from LSEG in January 2018. Headquartered in UK, with operations in US, Georgia and Russia. • We provide software testing services for mission critical technology that underpins global financial markets. Our clients are regulated by FCA, Bank of England and their counterparts from other countries. 2

20 April 2019, Tbilisi, Georgia We have a global software Quality Assurance client network 3

20 April 2019, Tbilisi, Georgia ● Exactpro serves as the final line of defence before Go-Live for many mission-critical systems ● We evaluate the system readiness prior to deployment into external environments ● We review relevant defects and risks with our clients ● System Types: - Major Exchanges - Huge Post-Trade Infrastructures - Market Data Dissemination systems - Clearing and Settlement systems We are 100% Focused on Testing Market Infrastructures 4

20 April 2019, Tbilisi, Georgia We have developed a Test Automation Framework for Distributed Ledger Technology projects Woodpecker - ClearTH Extension: ● Highly customizable (data source, data types, load, time, interface) ● Easy to transfer to different environments due to its ability to extract data right from the system under test ● Testing scenarios created without humans cover a vast majority of diverse conditions/data combinations ● A productive intersection of functional and non-functional testing approaches ClearTH - an innovative way to test Clearing, Settlement and Back-Office systems. It is a unique Exactpro tool able to simultaneously execute multiple end-to-end test scenarios in batches. ClearTH easily detects abnormal behavior in the system under test and effectively predicts potential issues. It offers many built-in actions to cover the majority of activities in post-trade systems. ● Verifies each stage of the DLC ● Has an integrated schedule ● Automatically runs test scripts ● Creates multiple-day test scenarios ● Performs multiple concurrent tests ● Has integrated simulators ● Supports SWIFT ISO protocol 5

20 April 2019, Tbilisi, Georgia Behavior driven development A software development method • Unit tests should be specified based on the unit’s desired behavior • The desired behavior should be specified in the form of user story specifications written in natural human language • Developers, Business Analysts and QA should collaborate on specifying the behavior in user stories 6

20 April 2019, Tbilisi, Georgia Collaboration – “Specification by Example” A collaborative approach to define requirements and business-oriented tests. Capture and illustrate requirements using realistic examples rather than abstract statements Advantages: • Humans understand concrete examples and derive concepts from that • Less feedback loops, less re-work, higher quality, faster turnaround, better alignment of roles on a project. SBE was coined in 2004 by Martin Fowler: https://martinfowler.com/bliki/SpecificationByExample.html One of the authors of the Agile Manifesto 7

20 April 2019, Tbilisi, Georgia 1. Start with the wide end of the tie on the right and the small end on the left. The tip of the small end should rest slightly above your belly-button (this will vary depending on your height and the length & thickness of your tie). Only move the active (wide) end. 2. Wide end over the small end to the left. 3. Up into the neck loop from underneath. 4. Down to the left. 5. Around the back of the small end to the right. 6. Up to the center, towards neck loop. 7. Through the neck loop and down to the right. 8. Across the front to the left. 9. Up into the neck loop from underneath. 10. Down through the loop you've just created in the front. 11. Tighten the knot by pulling down on the wide end. Slide the knot up & adjust. “How to tie a simple necktie knot” 8

20 April 2019, Tbilisi, Georgia “How to tie a simple knot” In Specification By Example format: 1 2 4 6 3 5 9

20 April 2019, Tbilisi, Georgia The rule of THREE 1. Clear, explicit title 2. As a I want So that 3. Acceptance criteria Given – a set of preconditions When – an event occurs Then – an outcome is achieved Developer QA BA 10

20 April 2019, Tbilisi, Georgia Project example - Risk Engine Car insurance premium calculator ??? Requirements 11

20 April 2019, Tbilisi, Georgia ??? Requirements Examples Car insurance premium calculator Project example 12

20 April 2019, Tbilisi, Georgia Executable specification ??? Examples System 13

20 April 2019, Tbilisi, Georgia ??? Examples System ??? Start of the project 14

20 April 2019, Tbilisi, Georgia ??? Examples $100 Version 0.2 Next version 15

20 April 2019, Tbilisi, Georgia $100 Examples Version 1.0 It’s working 16

20 April 2019, Tbilisi, Georgia Examples Working software A Method That Creates Working Software 17

20 April 2019, Tbilisi, Georgia Working software Examples Machine Learning 18

20 April 2019, Tbilisi, Georgia BDD Limitations from our Experience 19

20 April 2019, Tbilisi, Georgia QA Rather than putting the essence of BDD as a method to good use on a project to enjoy its full potential, let’s instead issue a requirement that QA use Gherkin. That way we can say that we have implemented BDD and are in good shape. 1. Misusing a good thing one way… 20

20 April 2019, Tbilisi, Georgia Or, instead of creating a meaningful User Story that’s understood by everyone, let’s make it a requirement to mainly focus on the format (Given, When, Then) and not so much on the essence. As long as we have the right format, we are good. As a User Story producer I want to implement various requirements So that we fully meet all requirements 2. Senseless User story 21

20 April 2019, Tbilisi, Georgia Trading system 22

20 April 2019, Tbilisi, Georgia Trading system 23

20 April 2019, Tbilisi, Georgia The Fourth Amigo Developer QA BA Production Logs 3. Not Using Production Logs 24

20 April 2019, Tbilisi, Georgia BDD limitations for testing trading systems Given – a set of preconditions •The complexity of settings and static data 4. Hard to use for dynamic distributed systems with undetermined results 25

20 April 2019, Tbilisi, Georgia You are in a cockpit 26

20 April 2019, Tbilisi, Georgia Given – a set of preconditions •The complexity of settings and static data When – an event occurs •High load and asynchronous message processing BDD limitations for testing trading systems 27

20 April 2019, Tbilisi, Georgia Given – a set of preconditions •The complexity of settings and static data When – when an event occurs •High load and asynchronous message processing Then – outcome is achieved •Undetermined result •Many details •May need a model BDD limitations for testing trading systems 28

20 April 2019, Tbilisi, Georgia Some testers believe that disclosing all test scenarios to a developer would be cheating. Some developers and agile gurus object saying that it’s silly. Can you imagine what a valid reason could be for hiding test scenarios from the developer? 5. Should they know all scenarios? 29

20 April 2019, Tbilisi, Georgia QA It is sometimes useful to not know the test scenarios QA is going to use as this can stimulate thought process to create a more reliable system. Developer BA Thought process 30

20 April 2019, Tbilisi, Georgia This way? No, this way: I.e. “a product with just enough features to satisfy early customers, and to provide feedback for future product development” Building a minimum viable product 31

20 April 2019, Tbilisi, Georgia Specification by Example 32

20 April 2019, Tbilisi, Georgia Test library applicable to test the minimum viable product may not be entirely applicable to test its current version 6. BDD test library limitations 33

20 April 2019, Tbilisi, Georgia To summarize a few that we see: 1.Misuse of BDD Instruments 2.Senseless User Story - focus on format rather than the meaning; 3.Not using production logs - using imagination instead of real information of how the system actually work; 4.Hard to use for dynamic distributed systems with undetermined results, e.g. trading or post-trade systems; 5.It is sometimes better to not know all scenarios in advance; 6.Test library applicable to test the end product may not applicable to test current version. BDD limitations from our experience 34

35 Build Software to Test Software Confidential exactpro.com EXTENT - Software Testing and Trading Technology Trends September 17 Leadenhall Building, London, 2019 Join us in discussing the newest fintech trends and solutions to the challenges in mission-critical trading and post trade systems! 35