Petri Nets Based Testing

A talk about automated exploratory testing based on Petri net models for Saint P RubyConf 2018. Pretty much the English version of https://speakerdeck.com/p0deje/tiestirovaniie-na-osnovie-sietiei-pietri.

Transcript

1. PETRI NETS BASED TESTING
2. 👋 Alex Rodionov 2
3. 3 👋 Alex Rodionov
4. 👋 Alex Rodionov @p0deje 4
5. 5 👋 Alex Rodionov @p0deje
6. 6 👋 Alex Rodionov @p0deje
7. 7 👋 Alex Rodionov @p0deje
8. PETRI NETS BASED TESTING
9. 1. Insuf fi ciency of usual tests 
 
 

10. 1. Insuf fi ciency of usual tests 2. Model-based testing 
 

11. 1. Insuf fi ciency of usual tests 2. Model-based testing 3. Petri nets 

12. 1. Insuf fi ciency of usual tests 2. Model-based testing 3. Petri nets 4. Petri nets based testing
13. 1. Insuf fi ciency of usual tests 2. Model-based testing 3. Petri nets 4. Petri nets based testing
14. USUAL TEST A A A 14
15. Arrange: set up preconditions A A 15 USUAL TEST
16. Arrange: set up preconditions Act: execute operation we’re testing A 16 USUAL TEST
17. Arrange: set up preconditions Act: execute operation we’re testing Assert: verify results 17 USUAL TEST
18. 18 USUAL TEST Arrange: set up preconditions Act: execute operation we’re testing Assert: verify results
19. 19 class User < ApplicationRecord end
20. 20 class User < ApplicationRecord end it 'destroys user’ do user = User.create! user.destroy expect(User.count).to eq(0) end
21. 21 arrange act assert class User < ApplicationRecord end it 'destroys user’ do user = User.create! user.destroy expect(User.count).to eq(0) end
22. 22 class User < ApplicationRecord end it 'destroys user’ do user = User.create! user.destroy expect(User.count).to eq(0) end
23. 23 class User < ApplicationRecord has_many :notes end class Note < ApplicationRecord belongs_to :user end it 'destroys user’ do user = User.create! user.destroy expect(User.count).to eq(0) end
24. 24 class User < ApplicationRecord has_many :notes end class Note < ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end
25. 25 class User < ApplicationRecord has_many :notes end class Note < ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end
26. 26 class User < ApplicationRecord has_many :notes end class Note < ApplicationRecord belongs_to :user end ActiveRecord : : InvalidForeignKey: PG : : ForeignKeyViolation: ERROR : update or delete on table "users" violates foreign key constraint “fk_rails_83d9817008" on table "notes" DETAIL : Key (id)=(1) is still referenced from table “notes".
27. 27 class User < ApplicationRecord has_many :notes, dependent: :destroy end class Note < ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end
28. 28 class User < ApplicationRecord has_many :notes, dependent: :destroy end class Note < ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end
29. STATE COMBINATIONS 29
30. STATE COMBINATIONS 30
31. STATE COMBINATIONS 31
32. STATE COMBINATIONS 32
33. STATE COMBINATIONS 33
34. STATE COMBINATIONS 34
35. 35 1.Write more tests? STATE COMBINATIONS
36. 36 1.Write more tests? 2.Randomize test data? STATE COMBINATIONS
37. 1. Insuf fi ciency of usual tests 2. Model-based testing 3. Petri nets 4. Petri nets based testing
38. MODEL-BASED TESTING 38 1.Create model of the system.
39. MODEL-BASED TESTING 39 1.Create model of the system. 2.Generate tests based on this model.
40. GRAPHWALKER 40
41. yEd 41
42. EXAMPLE 42
43. 43 EXAMPLE
44. 44 EXAMPLE
45. 45 EXAMPLE
46. 46 EXAMPLE
47. 47 EXAMPLE
48. 48 EXAMPLE
49. GRAPHWALKER 1.Java or Python. 2.Finite-state machine. 49
50. FINITE-STATE MACHINE 50
51. 51 FINITE-STATE MACHINE
52. 52 FINITE-STATE MACHINE
53. 53 FINITE-STATE MACHINE
54. 54 FINITE-STATE MACHINE
55. 55 FINITE-STATE MACHINE
56. 1.Duplication. 2.Concurrency = state explosion. 56 FINITE-STATE MACHINE
57. 1. Insuf fi ciency of usual tests 2. Model-based testing 3. Petri nets 4. Petri nets based testing
58. PETRI NET 58 not created Create User created
59. 59 places store tokens not created Create User created PETRI NET
60. 60 places store tokens transitions move tokens not created Create User created PETRI NET
61. 61 places store tokens arcs connect places and transitions transitions move tokens not created Create User created PETRI NET
62. 62 places store tokens arcs connect places and transitions transitions move tokens not created Create User created PETRI NET
63. 63 places store tokens arcs connect places and transitions transitions move tokens not created Create User created PETRI NET
64. 64 place not created has a token not created Create User created PETRI NET
65. 65 transition Create User is enabled (can fi re) not created Create User created PETRI NET
66. 66 fi ring removes token from not created and adds token to created not created Create User created PETRI NET
67. 67 1. Places can only be connected to transitions (and vice versa). PETRI NET
68. 68 1. Places can only be connected to transitions (and vice versa). 2. Transitions can fi re if every incoming place has at least one token. PETRI NET
69. 69 1. Places can only be connected to transitions (and vice versa). 2. Transitions can fi re if every incoming place has at least one token. 3. Firing transition consumes (removes) one token from each incoming place and produces (adds) one token to each outgoing place. PETRI NET
70. not created Create User created 70 ip allowed PETRI NET
71. 71 not created Create User created ip allowed not created Create User created ip allowed PETRI NET
72. 72 pending review not created Create User created PETRI NET
73. 73 pending review not created Create User created pending review not created Create User created PETRI NET
74. 74 not created Create User created PETRI NET
75. 75 PETRI NET not created Create User created not created Create User created
76. 76 FINITE-STATE MACHINE
77. 77 user not created user created note created PETRI NET
78. 78 user not created user created note created Create User PETRI NET
79. 79 user not created user created note created Create User Destroy User PETRI NET
80. 80 user not created user created note created Create User Destroy User Create Note PETRI NET
81. 81 user not created user created note created Create User Destroy User Create Note Destroy Note PETRI NET
82. 82 user not created user created note created Create User Destroy User Create Note Destroy Note PETRI NET
83. 83 user not created user created note created Create User Destroy User Create Note Destroy Note PETRI NET
84. 84 user not created user created note created Create User Destroy User Create Note Destroy Note PETRI NET
85. FINITE-STATE MACHINE 85
86. 86 user not created user created note created Create User Destroy User Create Note Destroy Note post created PETRI NET
87. 87 user not created user created note created Create User Destroy User Create Note Destroy Note post created Create Post PETRI NET
88. 88 user not created user created note created Create User Destroy User Create Note Destroy Note post created Create Post Destroy Post PETRI NET
89. 89 FINITE-STATE MACHINE
90. 90 user not created user created note created Create User Destroy User Create Note Destroy Note post created Create Post Destroy Post video created PETRI NET
91. 91 user not created user created note created Create User Destroy User Create Note Destroy Note post created Create Post Destroy Post Create Video video created PETRI NET
92. 92 user not created user created note created Create User Destroy User Create Note Destroy Note post created Create Post Destroy Post Create Video Destroy Video video created PETRI NET
93. 1. Insuf fi ciency of usual tests 2. Model-based testing 3. Petri nets 4. Petri nets based testing
94. PETRI NETS BASED TESTING 94
95. 1.Create model of the system using Petri net. 95 PETRI NETS BASED TESTING
96. 1.Create model of the system using Petri net. 2.Generate tests based on this model. 96 PETRI NETS BASED TESTING
97. CREATE MODEL 97
98. 98 { "places": [ { "guid": "e57c0830-148f-11e8-bb7f - adf9986e1043", "identifier": "user created" } ], "transitions": [ { "guid": “df65a6e0-148f-11e8-bb7f - adf9986e1043" "identifier": "Create User", } ], "arcs": [ { "from_guid": "e57c0830-148f-11e8-bb7f - adf9986e1043", "to_guid": “df65a6e0-148f-11e8-bb7f - adf9986e1043" } ] } CREATE MODEL
99. LOAD NET 99
100. 100 class Node def input_arcs # .. . end def output_arcs # .. . end end LOAD NET
101. 101 class Node def input_arcs # .. . end def output_arcs # .. . end end class Place < Node end LOAD NET
102. 102 class Node def input_arcs # .. . end def output_arcs # .. . end end class Place < Node end class Transition < Node end LOAD NET
103. 103 class Node def input_arcs # .. . end def output_arcs # .. . end end class Place < Node end class Transition < Node end class Arc def from # .. . end def to # .. . end end LOAD NET
104. 104 class Runner def enabled?(transition) # .. . end def fire(transition) # .. . end def marking # .. . end end LOAD NET
105. LINK NET TO TEST CODE 105
106. 106 user not created user created note created Create User Destroy User Create Note Destroy Note LINK NET TO TEST CODE
107. 107 user not created user created note created Create User Destroy User Create Note Destroy Note class UserNoteNet < Rhizome :: Net end
108. 108 user not created user created note created Create User Destroy User Create Note Destroy Note class UserNoteNet < Rhizome :: Net runner { Runner.new('user - note.bpf', marking: ['user not created']) } end
109. 109 class UserNoteNet < Rhizome :: Net runner { Runner.new('user - note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } end user not created user created note created Create User Destroy User Create Note Destroy Note
110. 110 class UserNoteNet < Rhizome :: Net runner { Runner.new('user - note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } end user not created user created note created Create User Destroy User Create Note Destroy Note
111. 111 class UserNoteNet < Rhizome :: Net runner { Runner.new('user - note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } place('user not created') do empty { expect(User.count).to be > 0 } filled { expect(User.count).to eq(0) } end end user not created user created note created Create User Destroy User Create Note Destroy Note
112. 112 class UserNoteNet < Rhizome :: Net runner { Runner.new('user - note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } place('user not created') do empty { expect(User.count).to be > 0 } filled { expect(User.count).to eq(0) } end place('user created') do empty { expect(User.count).to eq(0) } filled { expect(User.count).to be > 0 } end end user not created user created note created Create User Destroy User Create Note Destroy Note
113. class UserNoteNet < Rhizome :: Net runner { Runner.new('user - note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } place('user not created') do empty { expect(User.count).to be > 0 } filled { expect(User.count).to eq(0) } end place('user created') do empty { expect(User.count).to eq(0) } filled { expect(User.count).to be > 0 } end place('note created') do empty { expect(Note.count).to eq(0) } filled { expect(Note.count).to be > 0 } end end user not created user created note created Create User Destroy User Create Note Destroy Note 113
114. TRAVERSE NET 1.Give every transition same probability (4 transitions = 0.25 to each). 114
115. TRAVERSE NET 1.Give every transition same probability (4 transitions = 0.25 to each). 2.Generate random number - a threshold of transition selection. 115
116. TRAVERSE NET 1.Give every transition same probability (4 transitions = 0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 116
117. TRAVERSE NET 1.Give every transition same probability (4 transitions = 0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 117
118. TRAVERSE NET 1.Give every transition same probability (4 transitions = 0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 5.Verify empty and fi lled places. 118
119. TRAVERSE NET 1.Give every transition same probability (4 transitions = 0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 5.Verify empty and fi lled places. 6.Halve selected transition probability. 119
120. TRAVERSE NET 1.Give every transition same probability (4 transitions = 0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 5.Verify empty and fi lled places. 6.Halve selected transition probability. 7.If all transitions fi red at least once - stop. Otherwise, repeat from point 2. 120
121. 121 $ bin/rhizome Traversing user_note net: Initial marking: user not created
122. 122 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note
123. 123 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note
124. 124 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note Executing Destroy Note New marking: user created Transitions left: Destroy User
125. 125 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note Executing Destroy Note New marking: user created Transitions left: Destroy User Executing Destroy User New marking: user not created Finished traversing user_note net.
126. 126 $ bin/rhizome Traversing user_note net: Initial marking: user not created
127. 127 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note
128. 128 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note
129. 129 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note Executing Create User New marking: user created Transitions left: Create Note, Destroy Note
130. 130 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note Executing Create User New marking: user created Transitions left: Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy Note
131. 131 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note Executing Create User New marking: user created Transitions left: Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy Note Executing Destroy Note New marking: user created Finished traversing user_note net.
132. 132 $ bin/rhizome Traversing user_note net: Initial marking: user not created
133. 133 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note
134. 134 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note
135. 135 $ bin/rhizome Traversing user_note net: Initial marking: user not created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note Executing Destroy User 
 ActiveRecord :: InvalidForeignKey: PG : : ForeignKeyViolation: ERROR : update or delete on table "users" violates foreign key constraint "fk_rails_83d9817008" on table "notes" DETAIL : Key (id)=(1) is still referenced from table “notes".
136. RHIZOME 1. Reproducible tests using seed. 136
137. RHIZOME 1. Reproducible tests using seed. 2. Parallelize like any other test framework (parallel_tests). 137
138. RHIZOME 1. Reproducible tests using seed. 2. Parallelize like any other test framework (parallel_tests). 3. General purpose framework (API, UI). 138
139. USUAL TEST (RSPEC) 139 RHIZOME
140. RHIZOME 1000 LoC 140 100 LoC USUAL TEST (RSPEC)
141. RHIZOME 1000 LoC 1m 141 100 LoC 1m USUAL TEST (RSPEC)
142. 1. Test coverage increased. 142 PETRI NETS BASED TESTING
143. 1. Test coverage increased. 2. Time to run tests is the same. 143 PETRI NETS BASED TESTING
144. 1. Test coverage increased. 2. Time to run tests is the same. 3. Time to write and support tests decreased. 144 PETRI NETS BASED TESTING
145. P.S. What else?
146. 146 user not created user created note created Create User Destroy User Create Note Destroy Note
147. 147 user not created user created note created Create User Destroy User Create Note Destroy Note
148. 148 user not created user created note created Create User Destroy User Create Note Destroy Note
149. 149 user not created user created note created Create User Destroy User Create Note Destroy Note
150. 150 user not created user created note created Create User Destroy User Create Note Destroy Note
151. RESET ARC 151 user not created user created note created Create User Destroy User Create Note Destroy Note clears place after fi ring
152. INHIBITOR ARC 152 user created note created Create User Destroy User Create Note Destroy Note enables transition only if connected place has no tokens
153. user not created user created Create User Destroy User HIERARCHICAL PETRI NET 153 Note note created Create Note Destroy Note
154. PETRI NET 154 user not created user created note created Create User Destroy User Create Note Destroy Note
155. COLORED PETRI NET 155 user not created user created note created Create User Destroy User Create Note Destroy Note token stores data
156. DELAYED TRANSITIONS TIMED PETRI NETS STOCHASTIC PETRI NETS ANALYSIS ALGORITHMS … 156
157. 🙇 Alex Rodionov @p0deje