Share
Embed
Petri Nets Based Testing
A talk about automated exploratory testing based on Petri net models for Saint P RubyConf 2018. Pretty much the English version of https://speakerdeck.com/p0deje/tiestirovaniie-na-osnovie-sietiei-pietri.
Alex Rodionov
June 10, 2018
Transcript
157 slides
-
1. Insuf fi ciency of usual tests 2. Model-based testing
3. Petri nets -
1. Insuf fi ciency of usual tests 2. Model-based testing
3. Petri nets 4. Petri nets based testing -
1. Insuf fi ciency of usual tests 2. Model-based testing
3. Petri nets 4. Petri nets based testing -
Arrange: set up preconditions Act: execute operation we’re testing Assert:
verify results 17 USUAL TEST -
18 USUAL TEST Arrange: set up preconditions Act: execute operation
we’re testing Assert: verify results -
20 class User < ApplicationRecord end it 'destroys user’ do
user = User.create! user.destroy expect(User.count).to eq(0) end -
21 arrange act assert class User < ApplicationRecord end it
'destroys user’ do user = User.create! user.destroy expect(User.count).to eq(0) end -
22 class User < ApplicationRecord end it 'destroys user’ do
user = User.create! user.destroy expect(User.count).to eq(0) end -
23 class User < ApplicationRecord has_many :notes end class Note
< ApplicationRecord belongs_to :user end it 'destroys user’ do user = User.create! user.destroy expect(User.count).to eq(0) end -
24 class User < ApplicationRecord has_many :notes end class Note
< ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end -
25 class User < ApplicationRecord has_many :notes end class Note
< ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end -
26 class User < ApplicationRecord has_many :notes end class Note
< ApplicationRecord belongs_to :user end ActiveRecord : : InvalidForeignKey: PG : : ForeignKeyViolation: ERROR : update or delete on table "users" violates foreign key constraint “fk_rails_83d9817008" on table "notes" DETAIL : Key (id)=(1) is still referenced from table “notes". -
27 class User < ApplicationRecord has_many :notes, dependent: :destroy end
class Note < ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end -
28 class User < ApplicationRecord has_many :notes, dependent: :destroy end
class Note < ApplicationRecord belongs_to :user end it 'destroys user with a note’ do user = User.create!(note: Note.create!) user.destroy expect(User.count).to eq(0) end -
1. Insuf fi ciency of usual tests 2. Model-based testing
3. Petri nets 4. Petri nets based testing -
MODEL-BASED TESTING 39 1.Create model of the system. 2.Generate tests
based on this model. -
1. Insuf fi ciency of usual tests 2. Model-based testing
3. Petri nets 4. Petri nets based testing -
60 places store tokens transitions move tokens not created Create
User created PETRI NET -
61 places store tokens arcs connect places and transitions transitions
move tokens not created Create User created PETRI NET -
62 places store tokens arcs connect places and transitions transitions
move tokens not created Create User created PETRI NET -
63 places store tokens arcs connect places and transitions transitions
move tokens not created Create User created PETRI NET -
64 place not created has a token not created Create
User created PETRI NET -
65 transition Create User is enabled (can fi re) not
created Create User created PETRI NET -
66 fi ring removes token from not created and adds
token to created not created Create User created PETRI NET -
67 1. Places can only be connected to transitions (and
vice versa). PETRI NET -
68 1. Places can only be connected to transitions (and
vice versa). 2. Transitions can fi re if every incoming place has at least one token. PETRI NET -
69 1. Places can only be connected to transitions (and
vice versa). 2. Transitions can fi re if every incoming place has at least one token. 3. Firing transition consumes (removes) one token from each incoming place and produces (adds) one token to each outgoing place. PETRI NET -
71 not created Create User created ip allowed not created
Create User created ip allowed PETRI NET -
73 pending review not created Create User created pending review
not created Create User created PETRI NET -
75 PETRI NET not created Create User created not created
Create User created -
79 user not created user created note created Create User
Destroy User PETRI NET -
80 user not created user created note created Create User
Destroy User Create Note PETRI NET -
81 user not created user created note created Create User
Destroy User Create Note Destroy Note PETRI NET -
82 user not created user created note created Create User
Destroy User Create Note Destroy Note PETRI NET -
83 user not created user created note created Create User
Destroy User Create Note Destroy Note PETRI NET -
84 user not created user created note created Create User
Destroy User Create Note Destroy Note PETRI NET -
86 user not created user created note created Create User
Destroy User Create Note Destroy Note post created PETRI NET -
87 user not created user created note created Create User
Destroy User Create Note Destroy Note post created Create Post PETRI NET -
88 user not created user created note created Create User
Destroy User Create Note Destroy Note post created Create Post Destroy Post PETRI NET -
90 user not created user created note created Create User
Destroy User Create Note Destroy Note post created Create Post Destroy Post video created PETRI NET -
91 user not created user created note created Create User
Destroy User Create Note Destroy Note post created Create Post Destroy Post Create Video video created PETRI NET -
92 user not created user created note created Create User
Destroy User Create Note Destroy Note post created Create Post Destroy Post Create Video Destroy Video video created PETRI NET -
1. Insuf fi ciency of usual tests 2. Model-based testing
3. Petri nets 4. Petri nets based testing -
1.Create model of the system using Petri net. 95 PETRI
NETS BASED TESTING -
1.Create model of the system using Petri net. 2.Generate tests
based on this model. 96 PETRI NETS BASED TESTING -
98 { "places": [ { "guid": "e57c0830-148f-11e8-bb7f - adf9986e1043", "identifier":
"user created" } ], "transitions": [ { "guid": “df65a6e0-148f-11e8-bb7f - adf9986e1043" "identifier": "Create User", } ], "arcs": [ { "from_guid": "e57c0830-148f-11e8-bb7f - adf9986e1043", "to_guid": “df65a6e0-148f-11e8-bb7f - adf9986e1043" } ] } CREATE MODEL -
100 class Node def input_arcs # .. . end def
output_arcs # .. . end end LOAD NET -
101 class Node def input_arcs # .. . end def
output_arcs # .. . end end class Place < Node end LOAD NET -
102 class Node def input_arcs # .. . end def
output_arcs # .. . end end class Place < Node end class Transition < Node end LOAD NET -
103 class Node def input_arcs # .. . end def
output_arcs # .. . end end class Place < Node end class Transition < Node end class Arc def from # .. . end def to # .. . end end LOAD NET -
104 class Runner def enabled?(transition) # .. . end def
fire(transition) # .. . end def marking # .. . end end LOAD NET -
106 user not created user created note created Create User
Destroy User Create Note Destroy Note LINK NET TO TEST CODE -
107 user not created user created note created Create User
Destroy User Create Note Destroy Note class UserNoteNet < Rhizome :: Net end -
108 user not created user created note created Create User
Destroy User Create Note Destroy Note class UserNoteNet < Rhizome :: Net runner { Runner.new('user - note.bpf', marking: ['user not created']) } end -
109 class UserNoteNet < Rhizome :: Net runner { Runner.new('user
- note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } end user not created user created note created Create User Destroy User Create Note Destroy Note -
110 class UserNoteNet < Rhizome :: Net runner { Runner.new('user
- note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } end user not created user created note created Create User Destroy User Create Note Destroy Note -
111 class UserNoteNet < Rhizome :: Net runner { Runner.new('user
- note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } place('user not created') do empty { expect(User.count).to be > 0 } filled { expect(User.count).to eq(0) } end end user not created user created note created Create User Destroy User Create Note Destroy Note -
112 class UserNoteNet < Rhizome :: Net runner { Runner.new('user
- note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } place('user not created') do empty { expect(User.count).to be > 0 } filled { expect(User.count).to eq(0) } end place('user created') do empty { expect(User.count).to eq(0) } filled { expect(User.count).to be > 0 } end end user not created user created note created Create User Destroy User Create Note Destroy Note -
class UserNoteNet < Rhizome :: Net runner { Runner.new('user -
note.bpf', marking: ['user not created']) } transition('Create User') { @user = User.create! } transition('Destroy User') { @user.destroy } transition('Create Note') { @note = Note.create!(user: @user) } transition('Destroy Note') { @note.destroy } place('user not created') do empty { expect(User.count).to be > 0 } filled { expect(User.count).to eq(0) } end place('user created') do empty { expect(User.count).to eq(0) } filled { expect(User.count).to be > 0 } end place('note created') do empty { expect(Note.count).to eq(0) } filled { expect(Note.count).to be > 0 } end end user not created user created note created Create User Destroy User Create Note Destroy Note 113 -
TRAVERSE NET 1.Give every transition same probability (4 transitions =
0.25 to each). 114 -
TRAVERSE NET 1.Give every transition same probability (4 transitions =
0.25 to each). 2.Generate random number - a threshold of transition selection. 115 -
TRAVERSE NET 1.Give every transition same probability (4 transitions =
0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 116 -
TRAVERSE NET 1.Give every transition same probability (4 transitions =
0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 117 -
TRAVERSE NET 1.Give every transition same probability (4 transitions =
0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 5.Verify empty and fi lled places. 118 -
TRAVERSE NET 1.Give every transition same probability (4 transitions =
0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 5.Verify empty and fi lled places. 6.Halve selected transition probability. 119 -
TRAVERSE NET 1.Give every transition same probability (4 transitions =
0.25 to each). 2.Generate random number - a threshold of transition selection. 3.Select the fi rst transition which is enabled and has probability suitable for the threshold. 4.Fire transition. 5.Verify empty and fi lled places. 6.Halve selected transition probability. 7.If all transitions fi red at least once - stop. Otherwise, repeat from point 2. 120 -
122 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note -
123 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note -
124 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note Executing Destroy Note New marking: user created Transitions left: Destroy User -
125 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note Executing Destroy Note New marking: user created Transitions left: Destroy User Executing Destroy User New marking: user not created Finished traversing user_note net. -
127 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note -
128 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note -
129 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note Executing Create User New marking: user created Transitions left: Create Note, Destroy Note -
130 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note Executing Create User New marking: user created Transitions left: Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy Note -
131 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Destroy User New marking: user not created Transitions left: Create Note, Destroy Note Executing Create User New marking: user created Transitions left: Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy Note Executing Destroy Note New marking: user created Finished traversing user_note net. -
133 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note -
134 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note -
135 $ bin/rhizome Traversing user_note net: Initial marking: user not
created Executing Create User New marking: user created Transitions left: Destroy User, Create Note, Destroy Note Executing Create Note New marking: user created, note created Transitions left: Destroy User, Destroy Note Executing Destroy User ActiveRecord :: InvalidForeignKey: PG : : ForeignKeyViolation: ERROR : update or delete on table "users" violates foreign key constraint "fk_rails_83d9817008" on table "notes" DETAIL : Key (id)=(1) is still referenced from table “notes". -
RHIZOME 1. Reproducible tests using seed. 2. Parallelize like any
other test framework (parallel_tests). 137 -
RHIZOME 1. Reproducible tests using seed. 2. Parallelize like any
other test framework (parallel_tests). 3. General purpose framework (API, UI). 138 -
1. Test coverage increased. 2. Time to run tests is
the same. 143 PETRI NETS BASED TESTING -
1. Test coverage increased. 2. Time to run tests is
the same. 3. Time to write and support tests decreased. 144 PETRI NETS BASED TESTING -
146 user not created user created note created Create User
Destroy User Create Note Destroy Note -
147 user not created user created note created Create User
Destroy User Create Note Destroy Note -
148 user not created user created note created Create User
Destroy User Create Note Destroy Note -
149 user not created user created note created Create User
Destroy User Create Note Destroy Note -
150 user not created user created note created Create User
Destroy User Create Note Destroy Note -
RESET ARC 151 user not created user created note created
Create User Destroy User Create Note Destroy Note clears place after fi ring -
INHIBITOR ARC 152 user created note created Create User Destroy
User Create Note Destroy Note enables transition only if connected place has no tokens -
user not created user created Create User Destroy User HIERARCHICAL
PETRI NET 153 Note note created Create Note Destroy Note -
PETRI NET 154 user not created user created note created
Create User Destroy User Create Note Destroy Note -
COLORED PETRI NET 155 user not created user created note
created Create User Destroy User Create Note Destroy Note token stores data