Upgrade to Pro — share decks privately, control downloads, hide ads and more …

EXTENT-2019: Learning from Failure is not just for Humans

5206c19df417b8876825b5561344c1a0?s=47 Exactpro
PRO
September 17, 2019

EXTENT-2019: Learning from Failure is not just for Humans

Learning from Failure is not just for Humans
Eugene Litvinenko
QA Project Manager, Exactpro

EXTENT-2019
17 September 2019, London

Video: https://youtu.be/UJNz_9DwtF8

To learn more about Exactpro, visit our website https://exactpro.com/
EXTENT Conference website: https://extent.exactpro.com/
Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro
Facebook https://www.facebook.com/exactpro/
Instagram https://www.instagram.com/exactpro/

Subscribe to Exactpro YouTube channel https://www.youtube.com/c/exactprosystems

5206c19df417b8876825b5561344c1a0?s=128

Exactpro
PRO

September 17, 2019
Tweet

Transcript

  1. Learning from Failure is not Just for Humans Eugene Litvinenko,

    QA Project Manager, Exactpro
  2. Mistakes make us better 2

  3. Mistakes make us better SUCCESS Failure Failure Failure Failure Failure

    Failure Failure 3
  4. Mistakes make us better SUCCESS Failure Failure Failure Failure Failure

    Failure Failure 4
  5. Silly fish vs. Sailfish 5

  6. What is Sailfish? Web Binary FIX Market Data Database 6

  7. What is Sailfish? #step #description #user_name #action #message_type SecurityID OrdType

    Side OrderQty Price Status Test Case 1: Trade test case start 1 Send BUY Order with Qty = 1000, Price = 10 (O1) User_1 send NewOrderSingle 12345 LIMIT BUY 1000 10 2 Receive Execution Report with Status = NEW (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 NEW 3 Send SELL Order with Qty = 1000, Price = 10 (O2) User_2 send NewOrderSingle 12345 LIMIT SELL 1000 10 4 Receive Execution Report with Status = FILLED (O2) User_2 receive ExecutionReport 12345 LIMIT SELL 1000 10 FILLED 5 Receive Execution Report with Status = FILLED (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 FILLED test case end 7
  8. The main actions of Sailfish #step #description #user_name #action #message_type

    SecurityID OrdType Side OrderQty Price Status Test Case 1: Trade test case start 1 Send BUY Order with Qty = 1000, Price = 10 (O1) User_1 send NewOrderSingle 12345 LIMIT BUY 1000 10 2 Receive Execution Report with Status = NEW (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 NEW 3 Send SELL Order with Qty = 1000, Price = 10 (O2) User_2 send NewOrderSingle 12345 LIMIT SELL 1000 10 4 Receive Execution Report with Status = FILLED (O2) User_2 receive ExecutionReport 12345 LIMIT SELL 1000 10 FILLED 5 Receive Execution Report with Status = FILLED (O1) User_1 receive ExecutionReport 12345 LIMIT BUY 1000 10 FILLED test case end 8
  9. Passed and Failed test steps send NewOrderSingle (PASSED) [0.001s] -

    receive ExecutionReport (PASSED) [0.001s]: + Input Parameters - Verification: Message (PASSED): Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW NEW PASSED ... ... ... ... 9
  10. Passed and Failed test steps send NewOrderSingle (PASSED) [0.001s] -

    receive ExecutionReport (FAILED) [0.421s]: + Input Parameters + Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED) + Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED) + Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED) - Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED): Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW REJECTED FAILED ... ... ... ... 10
  11. Supervised Machine Learning through unpleasant experience send NewOrderSingle (PASSED) [0.001s]

    - receive ExecutionReport (FAILED) [0.421s]: + Input Parameters + Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED) + Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED) + Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED) - Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED): Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW REJECTED FAILED ... ... ... ... 11
  12. Supervised Machine Learning through unpleasant experience send NewOrderSingle (PASSED) [0.001s]

    - receive ExecutionReport (FAILED) [0.421s]: + Input Parameters - Verification: Similar message [1]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED): 89% + Verification: Similar message [2]. Failed/Passed/Conditionally Passed/NA: 4/26/0/6 (FAILED) 81% + Verification: Similar message [3]. Failed/Passed/Conditionally Passed/NA: 2/28/0/6 (FAILED) 54% + Verification: Similar message [4]. Failed/Passed/Conditionally Passed/NA: 1/29/0/6 (FAILED) 42% Field Expected Result Actual Result Status ... ... ... ... OrdType LIMIT LIMIT PASSED Status NEW REJECTED FAILED ... ... ... ... 12
  13. Supervised Machine Learning: going forward tens of thousands of tests

    thousands of raised bugs 13
  14. Supervised Machine Learning: going forward This test step failed with

    the following results... This test step failed with the following results… And the reason is... 14
  15. Thank you! 15