TMPA-2021: Searching for Deviations in Trading Systems: Combining Control-Flow and Data Perspectives

Julio Cesar Carrasquel and Irina Lomazova, HSE

Searching for Deviations in Trading Systems: Combining Control-Flow and Data Perspectives

TMPA is an annual International Conference on Software Testing, Machine Learning and Complex Process Analysis. The conference will focus on the application of modern methods of data science to the analysis of software quality.

To learn more about Exactpro, visit our website https://exactpro.com/

Follow us on
LinkedIn https://www.linkedin.com/company/exactpro-systems-llc
Twitter https://twitter.com/exactpro

Transcript

1. Searching for Deviations in Trading Systems: Combining Control-Flow and Data Perspectives J.C. Carrasquel and I.A. Lomazova jcarrasquel@hse.ru HSE University Laboratory of Process-Aware Information Systems (PAIS Lab) TMPA — November 2021
2. Trading System A multi-agent system where traders submit orders into a platform to exchange securities. ben sam ... buy_order sec=ROSNEFT qty=4 price=20.0$ side=BUY order book ROSNEFT order book ... ... trading platform sell_order sec=ROSNEFT qty=4 price=20.0$ side=SELL The platform places orders in order books and performs priority-based matching of buy and sell orders. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 1 / 15
3. Searching for Deviations in Trading Systems via Conformance Checking Conformance checking — process mining1 method to check if runs of a system (recorded in event logs) comply with the system’s specification. event log real behavior system specification trading system Colored Petri net model expected behavior conformance checking deviations Deviations — differences detected between the system and its specification. In which event, order book, order, activity, etc. [1] van der Aalst, W.: Process Mining: Data Science in Action. Springer, 2016. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 2 / 15
4. Petri Nets: An Introduction Petri Nets — a formalism for modeling and analysis of distributed systems. accept accepted rejected reject expired canceled partially filled cancel expire filled trade trade trade replace replace trade cancel expire Places Local states Transitions Activities Tokens Control threads Example: workflow of an order in a trading system Focus on control-flow — activities and their causal dependence. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 3 / 15
5. Petri Nets: An Introduction Petri Nets are executable — the firing of a transition removes tokens from input places and produces tokens in output places. accept accepted rejected reject expired canceled partially filled cancel expire filled trade trade trade replace replace trade cancel expire Places Local states Transitions Activities Tokens Control threads Example: workflow of an order in a trading system Focus on control-flow — activities and their causal dependence. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 3 / 15
6. Petri Nets: An Introduction Petri Nets are executable — the firing of a transition removes tokens from input places and produces tokens in output places. accept accepted rejected reject expired canceled partially filled cancel expire filled trade trade trade replace replace trade cancel expire Places Local states Transitions Activities Tokens Control threads Example: workflow of an order in a trading system Focus on control-flow — activities and their causal dependence. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 3 / 15
7. Colored Petri Nets: Modeling the Management of an Order Book p 1 OB source place buy orders source place sell orders OS (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1) p 1 p 2 Place p1 contains a buy order with id. b1, submitted in time t = 1, buying 5 stocks at a price 22.0$ per stock. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
8. Colored Petri Nets: Modeling the Management of an Order Book p 1 t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders p 3 OB t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders p 4 OS OS (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1) p 1 p 2 J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
9. Colored Petri Nets: Modeling the Management of an Order Book p 1 t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 OS p 6 order book sell side (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1) p 1 p 2 J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
10. Colored Petri Nets: Modeling the Management of an Order Book p 1 t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 5 trade1 (o2,ts2,pr2,q2) (o,ts,pr,q) OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1) p 1 p 2 trade1 — matching between orders with the same stock quantity. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
11. Colored Petri Nets: Modeling the Management of an Order Book t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 5 trade1 (o2,ts2,pr2,q2) (o,ts,pr,q) OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (b1,1,10.0$,7) (s1,2,10.0$,7) p 1 p 2 trade1 — matching between orders with the same stock quantity. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
12. Colored Petri Nets: Modeling the Management of an Order Book t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 5 trade1 (o2,ts2,pr2,q2) (o,ts,pr,q) OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (b1,1,10.0$,0) (s1,2,10.0$,0) p 1 p 2 trade1 — matching between orders with the same stock quantity. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
13. Colored Petri Nets: Modeling the Management of an Order Book t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 (o2,ts2,pr2,q2) (o,ts,pr,q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side p 1 p 2 (o2,ts2,pr2,0) trade2 — the sell order is filled, whereas the buy order returns with its stock remainder to the buy side. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
14. Colored Petri Nets: Modeling the Management of an Order Book p 1 t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 (o2,ts2,pr2,q2) (o,ts,pr,q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (b1,1,10.0$,7) (s1,2,10.0$,1) p 1 p 2 (o2,ts2,pr2,0) trade2 — the sell order is filled, whereas the buy order returns with its stock remainder to the buy side. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
15. Colored Petri Nets: Modeling the Management of an Order Book p 1 t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 (o2,ts2,pr2,q2) (o,ts,pr,q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (b1,1,10.0$,6) (s1,2,10.0$,0) p 1 p 2 (o2,ts2,pr2,0) trade2 — the sell order is filled, whereas the buy order returns with its stock remainder to the buy side. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
16. Colored Petri Nets: Modeling the Management of an Order Book t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 trade3 — the buy order order is filled, whereas the sell order returns with its stock remainder to the sell side. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
17. Colored Petri Nets: Modeling the Management of an Order Book t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 t 8 discard buy order t 9 discard sell order OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
18. Colored Petri Nets: Modeling the Management of an Order Book t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 t 8 discard buy order t 9 discard sell order OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 The model represents an order book for limit orders, where orders have no amendments. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 4 / 15
19. Event Logs An event log consists of cases, each of them related to a run in an order book. Each case has associated an ordered sequence of events (called a trace). Example of a trace. Each event indicates an activity executed and a set of objects involved. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 5 / 15
20. Checking Conformance between a Trace and a Colored Petri Net Execute the colored Petri net as indicated by the events in a trace of an event log. event log real behavior system specification trading system Colored Petri net model expected behavior conformance checking deviations Deviations are found when the model can’t be executed as indicated by some events. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 6 / 15
21. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Distinct objects observed in the trace are inserted as tokens in dedicated source places of the model.
22. Colored Petri Net Trace p 1 t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1) p 1 p 2 Distinct objects observed in the trace are inserted as tokens in dedicated source places of the model.
23. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
24. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
25. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
26. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
27. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
28. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
29. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
30. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 For every event of the trace, execute the model as indicated by the event. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
31. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Control-Flow Deviation detected! In event e 5 , new sell order was invoked for order s2, skipping activity submit sell order. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
32. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Control-Flow Deviation detected! In event e 5 , new sell order was invoked for order s2, skipping activity submit sell order. To force the model execution, order s2 jumps to place p 4 p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
33. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
34. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Rule Violation detected! The highest ranked sell order s2 with price 19.0$ should have traded before sell order s1 with price 21.0$, but in event e 6 order s1 traded first. p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
35. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Fire transition trade2 involving orders b1 and s1, exchanging 2 stocks. (b1, 1, 22.0$, 5) changes to (b1, 1, 22.0$, 3) (s1, 2, 21.0$, 2) changes to (s1, 2, 21.0$, 0) p 1 (b1,1,22.0$,5) (s1,2,21.0$,2) (s2,3,19.0$,1)
36. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Fire transition trade2 involving orders b1 and s1, exchanging 2 stocks. (b1, 1, 22.0$, 5) changes to (b1, 1, 22.0$, 3) (s1, 2, 21.0$, 2) changes to (s1, 2, 21.0$, 0) p 1 (b1,1,22.0$,3) (s1,2,21.0$,0) (s2,3,19.0$,1)
37. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Resource corruption! In the model, order b1 has state (b1,1,22.0$,3) But In the event, order b1 has state (b1,1,22.0$,4) The real system (as observed in the event) erroneously updated the stock quantity. p 1 (b1,1,22.0$,3) (s1,2,21.0$,0) (s2,3,19.0$,1)
38. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Resource corruption! In the model, order b1 has state (b1,1,22.0$,3) But In the event, order b1 has state (b1,1,22.0$,4) The real system (as observed in the event) erroneously updated the stock quantity. p 1 (b1,1,22.0$,4) (s1,2,21.0$,0) (s2,3,19.0$,1) To replicate this event in the model, we update attributes as observed in the event
39. Colored Petri Net Trace t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 Non-proper termination! At the end of the method, we check whether or not each individual object arrived to its sink. We see that orders b1 and s2 remained in the order book at the end of the trace. p 1 (b1,1,22.0$,4) (s1,2,21.0$,0) (s2,3,19.0$,1)
40. Checking Conformance between a Trace and a Colored Petri Net Deviation Types: Summary 1 Control-Flow Deviation: An activity is executed processing a given object, but illegally skipping activities that must have processed such an object before. 2 Priority Rule Violation: An object with less priority has been processed before an object with higher prority. 3 Resource Corruption: The real system (as observed in an event) has transformed attributes of an object in a different way than the model specifies. 4 Non-proper termination: Objects not fully processed by the system at the end of a run. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 7 / 15
41. Experimental Evaluation — Setting The management of order books in a real-life trading system is validated by extracting an event log from a set of Financial Information Exchange (FIX) protocol messages. FIX channel Colored Petri net model conformance checking ... pre-processing/ filtering trading system Event Log Each trace relates to an order book. system specification deviations FIX Messages = 552935 Selected Order books (Traces) = 73 Events = 2259 Avg. Events per Trace = 30.94 For this experiment we considered order books with only day limit orders with no amendments. ∗https://github.com/jcarrasquel/hse-uamc-conformance-checking J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 8 / 15
42. Experimental Evaluation — Event Log Extracting an event log from Financial Information Exchange (FIX) protocol messages... J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 9 / 15
43. Experimental Evaluation — Event Log The event log is formated as a Comma-Separated-Values (CSV) file. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 10 / 15
44. Experimental Evaluation — Colored Petri Net t 1 submit buy order (o,ts,pr,q) OB source place buy orders (o,ts,pr,q) submitted buy orders t 3 new buy order (o,ts,pr,q) (o,ts,pr,q) p 3 OB order book buy side t 2 submit sell order (o2,ts2,pr2,q2) source place sell orders (o2,ts2,pr2,q2) submitted sell orders t 4 new sell order p 4 OS (o2,ts2,pr2,q2) (o2,ts2,pr2,q2) OS OB p 5 t 6 t 5 trade1 t 7 (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2-q) (o,ts,pr,q-q2) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,q) (o2,ts2,pr2,q2) trade3 trade2 t 8 discard buy order t 9 discard sell order OS sink place buy orders (o2,ts2,pr2,0) OB p 7 OS (o,ts,pr,0) (o,ts,pr,q) (o2,ts2,pr2,q2) (o,ts,pr,0) p 8 sink place sell orders p 6 order book sell side (o2,ts2,pr2,0) (o2,ts2,pr2,0) (o,ts,pr,0) p 1 p 2 The model represents an order book for limit orders, where orders have no amendments. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 11 / 15
45. Experimental Evaluation — Colored Petri Net The conformance method and colored Petri nets are programmed in Python using SNAKES1 — a library for construction and simulation of high-level Petri nets. [1] https://snakes.ibisc.univ-evry.fr/ J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 12 / 15
46. Experimental Evaluation — Deviations Detected deviations TRACE EVENT TIMESTAMP ACTIVITY OBJECT DEV. DEVIATION DESCRIPTION 1488058 1781 05:52:58.18 trade2 bSovX RC 1488058 1782 05:52:58.18 trade1 bSovX RC 1488061 1792 05:53:23.38 trade1 sSowK RV 1488061 1792 05:53:23.38 trade1 sSowK RC 1488061 1792 05:53:23.38 trade1 bSowJ RC 1488061 1793 05:53:23.38 trade2 bSowJ CF resource with id: b00d0PhqYSowJ was not in location p5 but in p7 1488061 1793 05:53:23.38 trade2 bSowJ RC 1488061 1793 05:53:23.38 trade2 sSowL RC 1488061 end - - bSowJ NT resource with id: b00d0PhqYSowJ was not in final location p7 but in p5 1488062 1803 05:53:31.38 trade2 bSowN RC 1488062 1804 05:53:31.38 trade1 bSowN RC 9088012 end - - bmkq9 NT resource with id: b00d0PiS3mkq9 was not in final location p7 but in p5 9088012 end - - smkqA NT resource with id: s00d0PiS3mkqA was not in final location p8 but in p6 9088015 end - - sSSZd NT resource with id: s00d0Pi88SSZd was not in final location p8 but in p6 resource has event-state: ('b00d0PhqYSovX' 1550491266 100.0 100) ,but model-state is: ('b00d0PhqYSovX' 1550491266 105.0 100) resource has event-state: ('b00d0PhqYSovX' 1550491266 101.0 0) ,but model-state is: ('b00d0PhqYSovX' 1550491266 100.0 0) resource with id: s00d0PhqYSowK did not have priority ,over other resources in the same place. resource has event-state: ('s00d0PhqYSowK' 1550490938 101.0 0) ,but model-state is: ('s00d0PhqYSowK' 1550490938 101.0 -100) resource has event-state: ('b00d0PhqYSowJ' 1550490919 101.0 0) ,but model-state is: ('b00d0PhqYSowJ' 1550490919 105.0 100) resource has event-state: ('b00d0PhqYSowJ' 1550490919 105.0 100) ,but model-state is: ('b00d0PhqYSowJ' 1550490919 101.0 -100) resource has event-state: ('s00d0PhqYSowL' 1550490947 105.0 0) ,but model-state is: ('s00d0PhqYSowL' 1550490947 100.0 0) resource has event-state: ('b00d0PhqYSowN' 1550490899 100.0 100) ,but model-state is: ('b00d0PhqYSowN' 1550490899 105.0 100) resource has event-state: ('b00d0PhqYSowN' 1550490899 101.0 0) ,but model-state is: ('b00d0PhqYSowN' 1550490899 100.0 0) RC: Object corrupted. RV: Rule violation. CF: Control-flow deviation. NT: Non-proper termination. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 13 / 15
47. Concluding Remarks (1/2) On the side of Trading Systems and Testing... A method to search deviations in trading systems, by checking conformance between an event log (real behavior) and a colored Petri net (expected behavior). Conformance checking can be seen as a passive analysis1 technique. Experts seek to employ passive analysis grounded on data science for validating trading systems2. The file of deviations becomes attractive to be leveraged by data mining or machine learning methods, e.g., to find unsuspected relationships or patterns between the deviations. [1] Itkin, I. et al. Overview of Applications of Passive Testing Techniques, MACSPro 2019, CEUR vol. 2478 (2019). [2] Itkin, I. et al. User-Assisted Log Analysis for Quality Control of Distributed Fintech Applications, IEEE AITest 2019, 45-51 (2019). J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 14 / 15
48. Concluding Remarks (2/2) On the side of Process Mining... A novel conformance method that leverages formal semantics of colored Petri nets. The use of high-level classes of Petri nets which can capture multiple perspectives of systems is an active research direction in process mining1. In our research, we have considered: nested Petri nets for modeling and analyzing agent behavior2. Models that seek to describe agent behavior and data together 3. [1] Wednesday Nights of Petri Nets and Their Extensions, PAIS Lab - HSE. https://pais.hse.ru/en/seminar-pne [2] K. Mecheraoui, J.C. Carrasquel, I.A. Lomazova. Compositional Conformance Checking of Nested Petri Nets and Event Logs of Multi-Agent Systems, Modeling and Analysis of Complex Systems and Processes (MACSPro 2020), CEUR, vol. 2795, 2020. [3] J.C. Carrasquel, I.A. Lomazova, A. Rivkin. Modeling Trading Systems using Petri Net Extensions, Int. Workshop on Petri Nets and Software Engineering (PNSE 2020), CEUR, vol. 2651, 2020. J.C. Carrasquel, I.A. Lomazova - HSE Searching for Deviations in Trading Systems 15 / 15
49. Thank you! Searching for Deviations in Trading Systems: Combining Control-Flow and Data Perspectives J.C. Carrasquel and I.A. Lomazova jcarrasquel@hse.ru HSE University Laboratory of Process-Aware Information Systems (PAIS Lab) TMPA — November 2021