Behaviour Driven Development (BDD): The Outer Limits

EXTENT TALKS: Tbilisi
20.04.2019

Murad Mamedov, QA Analyst, Exactpro
Alexandr Banakov, QA Analyst, Exactpro

Website https://extent.exactpro.com/

Linkedin https://www.linkedin.com/company/exactpro-systems-llc
Instagram https://www.instagram.com/qatbilisi/
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/qatbilisi/
Youtube Channel https://www.youtube.com/c/exactprosystems

Transcript

1. 20 April 2019, Tbilisi, Georgia
   Behaviour Driven Development (BDD)
   The Outer Limits
   Murad Mamedov, QA Analyst, Exactpro
   Alexandr Banakov, QA Analyst, Exactpro
   

   View Slide

2. 20 April 2019, Tbilisi, Georgia
   EXACTPRO
   Exactpro History at a Glance
   • A specialist firm focused on functional and non functional testing of exchanges, clearing
   houses, depositories and other market infrastructures
   • Incorporated in 2009 with 10 people, our company has
   experienced significant growth as satisfied clients require
   more services; now employing 550 specialists.
   • Part of London Stock Exchange Group (LSEG) from May 2015 till January 2018.
   Exactpro management buyout from LSEG in January 2018. Headquartered in UK, with
   operations in US, Georgia and Russia.
   • We provide software testing services for mission critical technology
   that underpins global financial markets. Our clients are regulated by
   FCA, Bank of England and their counterparts from other countries.
   2
   

   View Slide

3. 20 April 2019, Tbilisi, Georgia
   We have a global software Quality Assurance client network
   3
   

   View Slide

4. 20 April 2019, Tbilisi, Georgia
   ● Exactpro serves as the final line of defence before Go-Live for many mission-critical systems
   ● We evaluate the system readiness prior to deployment into external environments
   ● We review relevant defects and risks with our clients
   ● System Types:
   - Major Exchanges
   - Huge Post-Trade Infrastructures
   - Market Data Dissemination systems
   - Clearing and Settlement systems
   We are 100% Focused on Testing Market Infrastructures
   4
   

   View Slide

5. 20 April 2019, Tbilisi, Georgia
   We have developed a Test Automation Framework for
   Distributed Ledger Technology projects
   Woodpecker - ClearTH Extension:
   ● Highly customizable (data
   source, data types, load, time,
   interface)
   ● Easy to transfer to different
   environments due to its ability to
   extract data right from the
   system under test
   ● Testing scenarios created
   without humans cover a vast
   majority of diverse
   conditions/data combinations
   ● A productive intersection of
   functional and non-functional
   testing approaches
   ClearTH - an innovative way to test
   Clearing, Settlement and Back-Office
   systems. It is a unique Exactpro tool able to
   simultaneously execute multiple end-to-end
   test scenarios in batches. ClearTH easily
   detects abnormal behavior in the system
   under test and effectively predicts potential
   issues. It offers many built-in actions to
   cover the majority of activities in post-trade
   systems.
   ● Verifies each stage of the DLC
   ● Has an integrated schedule
   ● Automatically runs test scripts
   ● Creates multiple-day test scenarios
   ● Performs multiple concurrent tests
   ● Has integrated simulators
   ● Supports SWIFT ISO protocol
   5
   

   View Slide

6. 20 April 2019, Tbilisi, Georgia
   Behavior driven development
   A software development method
   • Unit tests should be specified based on the unit’s desired behavior
   • The desired behavior should be specified in the form of user story specifications written in
   natural human language
   • Developers, Business Analysts and QA should collaborate on specifying the behavior in user
   stories
   6
   

   View Slide

7. 20 April 2019, Tbilisi, Georgia
   Collaboration – “Specification by Example”
   A collaborative approach to define requirements and business-oriented tests.
   Capture and illustrate requirements using realistic
   examples rather than abstract statements
   Advantages:
   • Humans understand concrete examples and
   derive concepts from that
   • Less feedback loops, less re-work, higher
   quality, faster turnaround, better alignment of
   roles on a project.
   SBE was coined in 2004 by Martin Fowler:
   https://martinfowler.com/bliki/SpecificationByExample.html
   One of the authors of the Agile Manifesto
   7
   

   View Slide

8. 20 April 2019, Tbilisi, Georgia
   1. Start with the wide end of the tie on the right and the small end on the left. The tip of the small end should
   rest slightly above your belly-button (this will vary depending on your height and the length & thickness of your
   tie). Only move the active (wide) end.
   2. Wide end over the small end to the left.
   3. Up into the neck loop from underneath.
   4. Down to the left.
   5. Around the back of the small end to the right.
   6. Up to the center, towards neck loop.
   7. Through the neck loop and down to the right.
   8. Across the front to the left.
   9. Up into the neck loop from underneath.
   10. Down through the loop you've just created in the front.
   11. Tighten the knot by pulling down on the wide end. Slide the knot up & adjust.
   “How to tie a simple necktie knot”
   8
   

   View Slide

9. 20 April 2019, Tbilisi, Georgia
   “How to tie a simple knot”
   In Specification By Example format:
   1 2 4 6
   3 5
   9
   

   View Slide

10. 20 April 2019, Tbilisi, Georgia
    The rule of THREE
    1. Clear, explicit title
    2. As a
    I want
    So that
    3. Acceptance criteria
    Given – a set of preconditions
    When – an event occurs
    Then – an outcome is achieved
    Developer QA
    BA
    10
    

    View Slide

11. 20 April 2019, Tbilisi, Georgia
    Project example - Risk Engine
    Car insurance premium calculator
    ??? Requirements
    11
    

    View Slide

12. 20 April 2019, Tbilisi, Georgia
    ??? Requirements Examples
    Car insurance premium calculator
    Project example
    12
    

    View Slide

13. 20 April 2019, Tbilisi, Georgia
    Executable specification
    ???
    Examples System
    13
    

    View Slide

14. 20 April 2019, Tbilisi, Georgia
    ???
    Examples System
    ???
    Start of the project
    14
    

    View Slide

15. 20 April 2019, Tbilisi, Georgia
    ???
    Examples
    $100
    Version 0.2
    Next version
    15
    

    View Slide

16. 20 April 2019, Tbilisi, Georgia
    $100
    Examples Version 1.0
    It’s working
    16
    

    View Slide

17. 20 April 2019, Tbilisi, Georgia
    Examples Working
    software
    A Method That Creates Working Software
    17
    

    View Slide

18. 20 April 2019, Tbilisi, Georgia
    Working
    software
    Examples
    Machine Learning
    18
    

    View Slide

19. 20 April 2019, Tbilisi, Georgia
    BDD Limitations from our Experience
    19
    

    View Slide

20. 20 April 2019, Tbilisi, Georgia
    QA
    Rather than putting the essence of BDD as a method to
    good use on a project to enjoy its full potential, let’s instead
    issue a requirement that QA use Gherkin. That way we can
    say that we have implemented BDD and are in good
    shape.
    1. Misusing a good thing one way…
    20
    

    View Slide

21. 20 April 2019, Tbilisi, Georgia
    Or, instead of creating a meaningful User Story that’s
    understood by everyone, let’s make it a requirement to
    mainly focus on the format (Given, When, Then) and not
    so much on the essence. As long as we have the right
    format, we are good.
    As a User Story producer
    I want to implement various requirements
    So that we fully meet all requirements
    2. Senseless User story
    21
    

    View Slide

22. 20 April 2019, Tbilisi, Georgia
    Trading system
    22
    

    View Slide

23. 20 April 2019, Tbilisi, Georgia
    Trading system
    23
    

    View Slide

24. 20 April 2019, Tbilisi, Georgia
    The Fourth Amigo
    Developer QA
    BA Production
    Logs
    3. Not Using Production Logs
    24
    

    View Slide

25. 20 April 2019, Tbilisi, Georgia
    BDD limitations for testing trading systems
    Given – a set of preconditions
    •The complexity of settings and static data
    4. Hard to use for dynamic distributed systems with
    undetermined results
    25
    

    View Slide

26. 20 April 2019, Tbilisi, Georgia
    You are in a cockpit
    26
    

    View Slide

27. 20 April 2019, Tbilisi, Georgia
    Given – a set of preconditions
    •The complexity of settings and static data
    When – an event occurs
    •High load and asynchronous message processing
    BDD limitations for testing trading systems
    27
    

    View Slide

28. 20 April 2019, Tbilisi, Georgia
    Given – a set of preconditions
    •The complexity of settings and static data
    When – when an event occurs
    •High load and asynchronous message processing
    Then – outcome is achieved
    •Undetermined result
    •Many details
    •May need a model
    BDD limitations for testing trading systems
    28
    

    View Slide

29. 20 April 2019, Tbilisi, Georgia
    Some testers believe that disclosing all test scenarios to a developer
    would be cheating.
    Some developers and agile gurus object saying that it’s silly.
    Can you imagine what a valid reason could be for hiding test
    scenarios from the developer?
    5. Should they know all scenarios?
    29
    

    View Slide

30. 20 April 2019, Tbilisi, Georgia
    QA
    It is sometimes useful to not know the
    test scenarios QA is going to use as this
    can stimulate thought process to create
    a more reliable system.
    Developer
    BA
    Thought process
    30
    

    View Slide

31. 20 April 2019, Tbilisi, Georgia
    This way?
    No, this way:
    I.e. “a product with just enough features to satisfy early customers, and
    to provide feedback for future product development”
    Building a minimum viable product
    31
    

    View Slide

32. 20 April 2019, Tbilisi, Georgia
    Specification by Example
    32
    

    View Slide

33. 20 April 2019, Tbilisi, Georgia
    Test library applicable to test the minimum viable
    product may not be entirely applicable to test its
    current version
    6. BDD test library limitations
    33
    

    View Slide

34. 20 April 2019, Tbilisi, Georgia
    To summarize a few that we see:
    1.Misuse of BDD Instruments
    2.Senseless User Story - focus on format rather than the meaning;
    3.Not using production logs - using imagination instead of real information of how the
    system actually work;
    4.Hard to use for dynamic distributed systems with undetermined results, e.g. trading or
    post-trade systems;
    5.It is sometimes better to not know all scenarios in advance;
    6.Test library applicable to test the end product may not applicable to test current version.
    BDD limitations from our experience
    34
    

    View Slide

35. 35 Build Software to Test Software
    Confidential
    exactpro.com
    EXTENT - Software Testing and Trading Technology Trends
    September 17
    Leadenhall Building,
    London, 2019
    Join us in discussing the newest fintech trends
    and solutions to the challenges in
    mission-critical trading and post trade systems!
    35
    

    View Slide