test "adding 2 numbers" do
assert add(1, 1) == 2
assert add(0, 2) == 2
assert add(3, 4) == 7
assert add(-1, 4) == 3
end
def add(x, _) when x < 0, do: 3
def add(3, _), do: 7
def add(_x, _y), do: 2
Slide 59
Slide 59 text
No content
Slide 60
Slide 60 text
No content
Slide 61
Slide 61 text
No content
Slide 62
Slide 62 text
No content
Slide 63
Slide 63 text
What have we gained?
Slide 64
Slide 64 text
Validation
Protection From Regression
Design
TDD
Slide 65
Slide 65 text
Validation
Protection From Regression
Design
TDD
Slide 66
Slide 66 text
Validation
Protection From Regression
Design
TDD
Slide 67
Slide 67 text
Validation
Protection From Regression
Design ?
TDD
Slide 68
Slide 68 text
There are bugs in your code
Slide 69
Slide 69 text
Writing tests for one feature
Slide 70
Slide 70 text
Writing tests for one feature
o(n)
Slide 71
Slide 71 text
Writing tests for two features
o(n)
o(n^2)
Slide 72
Slide 72 text
Writing tests for three features
o(n)
o(n^2)
o(n^3)
Slide 73
Slide 73 text
Race conditions?
Slide 74
Slide 74 text
Property tests
Slide 75
Slide 75 text
No content
Slide 76
Slide 76 text
Expected == Actual
Slide 77
Slide 77 text
Expected == Actual
Overly Specific
Slide 78
Slide 78 text
No content
Slide 79
Slide 79 text
Property Tests
Slide 80
Slide 80 text
Property Tests
Int
Slide 81
Slide 81 text
Property Tests
Int
p(x)
Slide 82
Slide 82 text
Property Tests
Int
p(x) ?
Slide 83
Slide 83 text
Property Tests
Int
p(x) ?
Invariant
Slide 84
Slide 84 text
Invariant: “a function, quantity,
or property that remains
unchanged when a specified
transformation is applied.”
Slide 85
Slide 85 text
Basic Property
Tests
Slide 86
Slide 86 text
What is true about addition?
Slide 87
Slide 87 text
x + 0 == x
Slide 88
Slide 88 text
test "addition with zero returns the same number" do
end
def add(_x, _y) do
end
Slide 89
Slide 89 text
test "addition with zero returns the same number" do
ptest do
end
end
def add(_x, _y) do
end
Slide 90
Slide 90 text
test "addition with zero returns the same number" do
ptest x: int() do
end
end
def add(_x, _y) do
end
Slide 91
Slide 91 text
test "addition with zero returns the same number" do
ptest x: int() do
assert add(x, 0) == x
end
end
def add(_x, _y) do
end
Slide 92
Slide 92 text
No content
Slide 93
Slide 93 text
test "addition with zero returns the same number" do
ptest x: int() do
assert add(x, 0) == x
end
end
def add(_x, _y) do
end
Slide 94
Slide 94 text
test "addition with zero returns the same number" do
ptest x: int() do
assert add(x, 0) == x
end
end
def add(x, _y) do
x
end
Slide 95
Slide 95 text
x + y == y + x
Slide 96
Slide 96 text
test "addition is commutative" do
end
def add(x, _y) do
x
end
Slide 97
Slide 97 text
test "addition is commutative" do
ptest x: int(), y: int() do
end
end
def add(x, _y) do
x
end
Slide 98
Slide 98 text
test "addition is commutative" do
ptest x: int(), y: int() do
assert add(x, y) == add(y, x)
end
end
def add(x, _y) do
x
end
Slide 99
Slide 99 text
No content
Slide 100
Slide 100 text
test "addition is commutative" do
ptest x: int(), y: int() do
assert add(x, y) == add(y, x)
end
end
def add(x, _y) do
x
end
Slide 101
Slide 101 text
test "addition is commutative" do
ptest x: int(), y: int() do
assert add(x, y) == add(y, x)
end
end
def add(x, y) do
x * y
end
Slide 102
Slide 102 text
No content
Slide 103
Slide 103 text
test "addition is commutative" do
ptest x: int(), y: int() do
assert add(x, y) == add(y, x)
end
end
def add(x, y) do
x * y
end
Slide 104
Slide 104 text
test "addition is commutative" do
ptest x: int(), y: int() do
assert add(x, y) == add(y, x)
end
end
def add(x, y) do
x * y
end
def add(x, 0), do: x
Slide 105
Slide 105 text
No content
Slide 106
Slide 106 text
(1 + x) + y == x + (1 + y)
Slide 107
Slide 107 text
def add(x, y) do
x * y
end
def add(x, 0), do: x
test "addition is asociative" do
end
Slide 108
Slide 108 text
def add(x, y) do
x * y
end
def add(x, 0), do: x
test "addition is asociative" do
ptest x: int(), y: int(), z: int() do
end
end
Slide 109
Slide 109 text
def add(x, y) do
x * y
end
def add(x, 0), do: x
test "addition is asociative" do
ptest x: int(), y: int(), z: int() do
assert add(x, add(y, z)) == add(add(x, y), z)
end
end
Slide 110
Slide 110 text
def add(x, y) do
x * y
end
def add(x, 0), do: x
test "addition is asociative" do
ptest x: int(), y: int(), z: int() do
assert add(x, add(y, z)) == add(add(x, y), z)
end
end
Slide 111
Slide 111 text
No content
Slide 112
Slide 112 text
def add(x, 0), do: x
def add(x, y) do
x * y
end
Slide 113
Slide 113 text
def add(x, y) do
x + y
end
Slide 114
Slide 114 text
A Real-ish
Example
Slide 115
Slide 115 text
No content
Slide 116
Slide 116 text
Modeling the application
?
p(x) ?
Slide 117
Slide 117 text
Modeling the application
?
p(x) ?
What is the input domain?
Slide 118
Slide 118 text
Vote Vote Vote
User
Slide 119
Slide 119 text
Vote Vote Vote
User
Slide 120
Slide 120 text
Vote Vote Vote
User
Slide 121
Slide 121 text
Vote Vote Vote
User
Slide 122
Slide 122 text
Modeling Users as FSMs
logged_out logged_in
login
logout
vote
Property: Users votes should increase
test “users votes increase after voting" do
end
Slide 133
Slide 133 text
Property: Users votes should increase
test “users votes increase after voting" do
ptest [commands: gen_commands("chris")] do
end
end
Slide 134
Slide 134 text
Property: Users votes should increase
test “users votes increase after voting" do
ptest [commands: gen_commands("chris")] do
VoteCounter.reset()
end
end
Slide 135
Slide 135 text
Property: Users votes should increase
test “users votes increase after voting" do
ptest [commands: gen_commands("chris")] do
VoteCounter.reset()
{_state, result} = run_commands(commands, Client)
end
end
Slide 136
Slide 136 text
Property: Users votes should increase
test “users votes increase after voting" do
ptest [commands: gen_commands("chris")] do
VoteCounter.reset()
{_state, result} = run_commands(commands, Client)
assert result
end
end
Slide 137
Slide 137 text
Property: Users votes should increase
test “users votes increase after voting" do
ptest [commands: gen_commands("chris")] do
VoteCounter.reset()
{_state, result} = run_commands(commands, Client)
assert result
end
end
def run_commands(commands, module) do
Enum.reduce(
commands,
{0, true},
& run_command(module, &1, &2) )
end
Slide 138
Slide 138 text
def gen_commands(name) do
end
Command Generators
Slide 139
Slide 139 text
def gen_commands(name) do
list(of: gen_vote(name), max: 20)
end
Command Generators
Slide 140
Slide 140 text
def gen_commands(name) do
list(of: gen_vote(name), max: 20)
end
def gen_vote(name) do
end
Command Generators
Slide 141
Slide 141 text
def gen_commands(name) do
list(of: gen_vote(name), max: 20)
end
def gen_vote(name) do
tuple(like: {
)})
end
Command Generators
Slide 142
Slide 142 text
def gen_commands(name) do
list(of: gen_vote(name), max: 20)
end
def gen_vote(name) do
tuple(like: {
value(:vote),
)})
end
Command Generators
Slide 143
Slide 143 text
def gen_commands(name) do
list(of: gen_vote(name), max: 20)
end
def gen_vote(name) do
tuple(like: {
value(:vote),
value(name),
)})
end
Command Generators
Slide 144
Slide 144 text
def gen_commands(name) do
list(of: gen_vote(name), max: 20)
end
def gen_vote(name) do
tuple(like: {
value(:vote),
value(name),
choose(from: [value(1), value(2), value(3)])})
end
Command Generators
Slide 145
Slide 145 text
defmodule ClientStateMachine do
end
Slide 146
Slide 146 text
defmodule ClientStateMachine do
def vote(name, id) do
%{"votes" => new_votes} = post(id, name)
{:ok, new_votes}
end
end
Slide 147
Slide 147 text
defmodule ClientStateMachine do
def vote(name, id) do
%{"votes" => new_votes} = post(id, name)
{:ok, new_votes}
end
def vote_next(state, [id, name], _result) do
{:ok, update_in(state, [name, to_string(id)], &(&1 + 1))}
end
end
Slide 148
Slide 148 text
defmodule ClientStateMachine do
def vote(name, id) do
%{"votes" => new_votes} = post(id, name)
{:ok, new_votes}
end
def vote_next(state, [id, name], _result) do
{:ok, update_in(state, [name, to_string(id)], &(&1 + 1))}
end
def vote_post(state, [id, name], actual_result) do
expected_result = get_in(state, [name, to_string(id)]) + 1
{:ok, actual_result == expected_result}
end
end
Slide 149
Slide 149 text
Property: Users votes should increase
test “users votes increase after voting" do
ptest [commands: gen_commands("chris")] do
VoteCounter.reset()
{_state, result} = run_commands(commands, Client)
assert result
end
end
Slide 150
Slide 150 text
No content
Slide 151
Slide 151 text
Property: Users shouldn’t effect
other users votes
Slide 152
Slide 152 text
Property: Users should get the correct votes
test "users don't effect each others votes" do
end
Slide 153
Slide 153 text
Property: Users should get the correct votes
test "users don't effect each others votes" do
ptest [chris: gen_commands("chris"), jane: gen_commands("jane")] do
end
end
Slide 154
Slide 154 text
Property: Users should get the correct votes
test "users don't effect each others votes" do
ptest [chris: gen_commands("chris"), jane: gen_commands("jane")] do
VoteCounter.reset()
end
end
Slide 155
Slide 155 text
Property: Users should get the correct votes
test "users don't effect each others votes" do
ptest [chris: gen_commands("chris"), jane: gen_commands("jane")] do
VoteCounter.reset()
{_state, result} = run_commands([chris, jane], Client)
end
end
Slide 156
Slide 156 text
Property: Users should get the correct votes
test "users don't effect each others votes" do
ptest [chris: gen_commands("chris"), jane: gen_commands("jane")] do
VoteCounter.reset()
{_state, result} = run_commands([chris, jane], Client)
assert result
end
end
Slide 157
Slide 157 text
Property: Users should get the correct votes
test "users don't effect each others votes" do
ptest [chris: gen_commands("chris"), jane: gen_commands("jane")] do
VoteCounter.reset()
{_state, result} = run_parallel_commands([chris, jane], Client)
assert result
end
end
The Bug
def new(conn, %{"id" => id, "name" => name}) do
{:ok, current_votes} = VoteCounter.get(id)
new_votes = [name | current_votes]
VoteCounter.put(id, new_votes)
# Other nonsense
end
Slide 162
Slide 162 text
The Bug
Votes
Slide 163
Slide 163 text
The Bug
Votes
3
3
Slide 164
Slide 164 text
The Bug
Votes
3
3
Slide 165
Slide 165 text
The Bug
Votes
4
4
Slide 166
Slide 166 text
The Bug
Votes
4
4
Slide 167
Slide 167 text
The Bug
3 + 1 + 1 == 4
?
Slide 168
Slide 168 text
The Bug
def new(conn, %{"id" => id, "name" => name}) do
{:ok, current_votes} = VoteCounter.get(id)
new_votes = [name | current_votes]
VoteCounter.put(id, new_votes)
# Other nonsense
end
Slide 169
Slide 169 text
The Bug
def new(conn, %{"id" => id, "name" => name}) do
{:ok, new_votes} = VoteCounter.incr(id, name)
# Other nonsense
end
Slide 170
Slide 170 text
No content
Slide 171
Slide 171 text
Conclusion
Slide 172
Slide 172 text
How to think in properties
Generating data
Generate commands
Model users as FSMs
Slide 173
Slide 173 text
Resources:
“Finding Race conditions in Erlang with QuickCheck and PULSE”
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.724.3518&rep=rep1&type=pdf
Testing Async apis with QuickCheck
https://www.youtube.com/watch?v=iW2J7Of8jsE&t=272s
“QuickCheck: A lightweight tool for Random Testing of Haskell Programs”
http://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf
Composing Test Generators
https://www.youtube.com/watch?v=4-sPhFtGwZk
Property based testing for better code
https://www.youtube.com/watch?v=shngiiBfD80
Slide 174
Slide 174 text
If you could write less code and find
more bugs would you do that?