Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Property-based Testing is a Mindset
Search
Andrea Leopardi
December 06, 2017
Programming
0
670
Property-based Testing is a Mindset
Andrea Leopardi
December 06, 2017
Tweet
Share
More Decks by Andrea Leopardi
See All by Andrea Leopardi
The World is a Network (and We Are Just Nodes)
whatyouhide
0
180
BEAM: The Perfect Fit for Networks
whatyouhide
1
170
Update from the Elixir team - 2022
whatyouhide
0
370
Testing Asynchronous OTP
whatyouhide
0
490
Elixir Sightseeing Tour
whatyouhide
0
390
Mint - Disrupting HTTP clients
whatyouhide
0
230
BEAM Architecture Handbook
whatyouhide
7
2.7k
The Evolution of a Language
whatyouhide
0
130
Elixir - functional, concurrent, distributed programming for the rest of us
whatyouhide
2
310
Other Decks in Programming
See All in Programming
Kanzawa.rbのLT大会を支える技術の裏側を変更する Ruby on Rails + Litestream 編
muryoimpl
0
170
第3回 Snowflake 中部ユーザ会- dbt × Snowflake ハンズオン
hoto17296
4
320
Spring gRPC について / About Spring gRPC
mackey0225
0
200
Djangoアプリケーション 運用のリアル 〜問題発生から可視化、最適化への道〜 #pyconshizu
kashewnuts
1
160
Compose でデザインと実装の差異を減らすための取り組み
oidy
1
290
CloudNativePGがCNCF Sandboxプロジェクトになったぞ! 〜CloudNativePGの仕組みの紹介〜
nnaka2992
0
210
動作確認やテストで漏れがちな観点3選
starfish719
6
960
Kubernetes History Inspector(KHI)を触ってみた
bells17
0
180
AWS Organizations で実現する、 マルチ AWS アカウントのルートユーザー管理からの脱却
atpons
0
110
バックエンドのためのアプリ内課金入門 (サブスク編)
qnighy
8
1.7k
最近のVS Codeで気になるニュース 2025/01
74th
1
250
知られざるDMMデータエンジニアの生態 〜かつてツチノコと呼ばれし者〜
takaha4k
3
1.2k
Featured
See All Featured
Rails Girls Zürich Keynote
gr2m
94
13k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
98
18k
Typedesign – Prime Four
hannesfritz
40
2.5k
ピンチをチャンスに:未来をつくるプロダクトロードマップ #pmconf2020
aki_iinuma
113
50k
Automating Front-end Workflow
addyosmani
1367
200k
Product Roadmaps are Hard
iamctodd
PRO
50
11k
Designing on Purpose - Digital PM Summit 2013
jponch
117
7.1k
The Art of Programming - Codeland 2020
erikaheidi
53
13k
Rebuilding a faster, lazier Slack
samanthasiow
79
8.8k
Build The Right Thing And Hit Your Dates
maggiecrowley
34
2.5k
The Psychology of Web Performance [Beyond Tellerrand 2023]
tammyeverts
45
2.3k
Faster Mobile Websites
deanohume
306
31k
Transcript
I S A M I N D S E T
P R O P E R T Y - B A S E D T E S T I N G
@whatyouhide
None
Elixir
T E S T I N G
why do we test? no tests yes tests
unit tests
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end example-based
table-based Input Output [] [] [1, 2, 3] [1, 2,
3] [2, 1, 3] [1, 2, 3]
unit tests , but also
P R O P E R T I E S
hard to write , but...
valid inputs properties of output testing framework
github.com/whatyouhide/stream_data
example time: sorting lists
test "sorting" do assert sort([]) == [] assert sort([1, 2,
3]) == [1, 2, 3] assert sort([2, 1, 3]) == [1, 2, 3] end
lists of integers
it's a list has the same elements it's ordered
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
check all list <- list_of(int()) do sorted = sort(list) assert
is_list(sorted) assert same_elements?(list, sorted) assert ordered?(sorted) end
def sort(list), do: list
[32, 2, 44, -12] [1, 0] *shrinking
P A T T E R N S
circular code
decode(encode(term)) == term
property "encoding->decoding is circular" do check all bin <- binary()
do encoded = Huffman.encode(bin) assert is_binary(encoded) assert Huffman.decode(encoded) == bin end end Huffman encoding
oracle model
my_code() == oracle_code()
older system less performant implementation
property "gives same results as Erlang impl" do check all
bin <- binary() do assert Huffman.encode(bin) == :huffman.encode(bin) end end
smoke tests https://www.youtube.com/watch?v=jvwfDdgg93E
API: 200, 201, 400, 404 https://www.youtube.com/watch?v=jvwfDdgg93E
https://www.youtube.com/watch?v=jvwfDdgg93E property "only expected codes are returned" do check all
request <- request() do response = HTTP.perform(request) assert response.status in [200, 201, 400, 404] end end
locally , CI
if System.get_env("CI") == "true" do config :stream_data, max_runs: 500 else
config :stream_data, max_runs: 25 end
unit + properties
R E A L - W O R L D
E X A M P L E
Redis parser
continuation parser parse(bytes) {:ok, command} {:more, function}
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
property "splitting commands at random" do check all cmd <-
command(), split_cmd <- random_splits(cmd) do assert {:ok, _, ""} = parse_many(split_cmd) end end
S T A T E F U L T E
S T I N G
model valid commands +
model: state + state transformations
commands: calls + preconditions
bounded-size queue
Max size model system
•push(queue, value) •resize(queue, max_size) commands
def push(model, _value), do: model def resize(_model, max_size), do: max_size
push(queue, {:some, "value"}) push(queue, [123, 3.2]) resize(queue, 1) push(queue, %{})
assert size(queue) <= model
LevelDB
17 (seventeen) calls 33 (thirty three) calls
C O N C L U S I O N
find obscure bugs reduce to minimal failing input find specification
errors keep increasing input space
v1.7
use stream_data
use property-based testing
@whatyouhide github.com/whatyouhide/stream_data