Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Development of Test Suite with Formally Verified FSM Coverage: A Case Study

5206c19df417b8876825b5561344c1a0?s=47 Exactpro
PRO
November 09, 2019

Development of Test Suite with Formally Verified FSM Coverage: A Case Study

Rostislav Yavorskiy and Iosif Itkin

International Conference on Software Testing, Machine Learning and Complex Process Analysis (TMPA-2019)
7-9 November 2019, Tbilisi

Video: https://youtu.be/0FasFDcwYh8

TMPA Conference website https://tmpaconf.org/
TMPA Conference on Facebook https://www.facebook.com/groups/tmpaconf/

5206c19df417b8876825b5561344c1a0?s=128

Exactpro
PRO

November 09, 2019
Tweet

Transcript

  1. Development of Test Suite with Formally Verified FSM Coverage: A

    Case Study Iosif Itkin, Rostislav Yavorskiy TMPA-2019: Software Testing, Machine Learning and Complex Process Analysis Tbilisi, Georgia on 7-9 November 2019
  2. Order book 2

  3. The model restrictions • Bid queue < N1 • Ask

    queue < N2 • Number of the order price levels < N3 3
  4. Hyper-states (equivalence classes) Property definition: [0,1, 2, M] 0|M 0|M

    0|0 0|0 M|0 M|0 M|0 4
  5. Questions Which hyper-states are reachable? Which one-step transitions between the

    hyper-states are possible? 0|M M|M M|0 0|0 0|M M|0 M|0 5
  6. The goal Create test suite with formally verified 100% coverage

    6
  7. Phase 1: Random generation of the tests Advantage: - Fast

    - Easy to implement Disadvantage: - No way to guarantee the completeness 7
  8. Time to discover the hyper-graph links (number of scenarios) 8

  9. Time to discover the hyper-graph links (number of scenarios) 9

  10. Phase 2: Formal verification of the completeness 10

  11. Phase 2: Formal verification of the completeness 11

  12. Conclusion Two reasons of the incompleteness: 1. Time to discover

    reachable transitions between hyper-states grows up exponentially 2. Bugs in the random test generation module. 12