Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Property-based Testing is a Mindset
Search
Andrea Leopardi
December 06, 2017
Programming
0
690
Property-based Testing is a Mindset
Andrea Leopardi
December 06, 2017
Tweet
Share
More Decks by Andrea Leopardi
See All by Andrea Leopardi
The Umbrella and the Range
whatyouhide
0
17
gen_statem - OTP's Unsung Hero
whatyouhide
2
270
The World is a Network (and We Are Just Nodes)
whatyouhide
1
230
BEAM: The Perfect Fit for Networks
whatyouhide
1
210
Update from the Elixir team - 2022
whatyouhide
0
420
Testing Asynchronous OTP
whatyouhide
1
530
Elixir Sightseeing Tour
whatyouhide
0
440
Mint - Disrupting HTTP clients
whatyouhide
0
260
BEAM Architecture Handbook
whatyouhide
7
2.9k
Other Decks in Programming
See All in Programming
CSC509 Lecture 04
javiergs
PRO
0
300
CSC509 Lecture 03
javiergs
PRO
0
330
Go Conference 2025: Goで体感するMultipath TCP ― Go 1.24 時代の MPTCP Listener を理解する
takehaya
7
1.6k
(Extension DC 2025) Actor境界を越える技術
teamhimeh
1
230
階層構造を表現するデータ構造とリファクタリング 〜1年で10倍成長したプロダクトの変化と課題〜
yuhisatoxxx
3
920
ポスターセッション: 「まっすぐ行って、右!」って言ってラズパイカーを動かしたい 〜生成AI × Raspberry Pi Pico × Gradioの試作メモ〜
komofr
0
970
Serena MCPのすすめ
wadakatu
4
900
Introducing ReActionView: A new ActionView-Compatible ERB Engine @ Kaigi on Rails 2025, Tokyo, Japan
marcoroth
3
920
エンジニアとして高みを目指す、 利益を生み出す設計の考え方 / design-for-profit
minodriven
23
12k
CSC305 Lecture 03
javiergs
PRO
0
230
プログラミングどうやる? ~テスト駆動開発から学ぶ達人の型~
a_okui
0
190
ソフトウェア設計の実践的な考え方
masuda220
PRO
3
490
Featured
See All Featured
Embracing the Ebb and Flow
colly
88
4.8k
4 Signs Your Business is Dying
shpigford
185
22k
Faster Mobile Websites
deanohume
310
31k
Writing Fast Ruby
sferik
629
62k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
19
1.2k
10 Git Anti Patterns You Should be Aware of
lemiorhan
PRO
657
61k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
54
3k
Bash Introduction
62gerente
615
210k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Context Engineering - Making Every Token Count
addyosmani
5
190
Optimising Largest Contentful Paint
csswizardry
37
3.4k
Typedesign – Prime Four
hannesfritz
42
2.8k
Transcript
I S A M I N D S E T
P R O P E R T Y - B A S E D T E S T I N G
@whatyouhide
None
Elixir
T E S T I N G
why do we test? no tests yes tests
unit tests
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end example-based
table-based Input Output [] [] [1, 2, 3] [1, 2,
3] [2, 1, 3] [1, 2, 3]
unit tests , but also
P R O P E R T I E S
hard to write , but...
valid inputs properties of output testing framework
github.com/whatyouhide/stream_data
example time: sorting lists
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end
lists of integers
it's a list has the same elements it's ordered
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
def sort(list), do: list
[32, 2, 44, -12] [1, 0] *shrinking
P A T T E R N S
circular code
decode(encode(term)) == term
property "encoding->decoding is circular" do check all bin <- binary()
do encoded = Huffman.encode(bin) assert is_binary(encoded) assert Huffman.decode(encoded) == bin end end Huffman encoding
oracle model
my_code() == oracle_code()
older system less performant implementation
property "gives same results as Erlang impl" do check all
bin <- binary() do assert Huffman.encode(bin) == :huffman.encode(bin) end end
smoke tests https://www.youtube.com/watch?v=jvwfDdgg93E
API: 200, 201, 400, 404 https://www.youtube.com/watch?v=jvwfDdgg93E
https://www.youtube.com/watch?v=jvwfDdgg93E property "only expected codes are returned" do check all
request <- request() do response = HTTP.perform(request) assert response.status in [200, 201, 400, 404] end end
locally , CI
if System.get_env("CI") == "true" do config :stream_data, max_runs: 500 else
config :stream_data, max_runs: 25 end
unit + properties
R E A L - W O R L D
E X A M P L E
Redis parser
continuation parser parse(bytes) {:ok, command} {:more, function}
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
S T A T E F U L T E
S T I N G
model valid commands +
model: state + state transformations
commands: calls + preconditions
bounded-size queue
Max size model system
•push(queue, value) •resize(queue, max_size) commands
def push(model, _value), do: model def resize(_model, max_size), do: max_size
push(queue, {:some, "value"}) push(queue, [123, 3.2]) resize(queue, 1) push(queue, %{})
assert size(queue) <= model
LevelDB
17 (seventeen) calls 33 (thirty three) calls
C O N C L U S I O N
find obscure bugs reduce to minimal failing input find specification
errors keep increasing input space
v1.7
use stream_data
use property-based testing
@whatyouhide github.com/whatyouhide/stream_data