$ ./fuzz -int | program
634111569742810193727424069509
741355925061499451162464719526
615957331924826555590537407605
181400079803446874252046374716
740973770255348279425601333144
152724057932073828569041216191
099859446496509919024810271242
622974988671421938012464630138
735355134599327240920259675263
574528613057084231370741920902
794677842164654990353575580453
777282305855352378119038096476
699871306655084953377039862387
924957554389878352934547664240
082431556093837288597262675598
630851919061829885048834738832
677022429414980917053939970795
722006987916088650168665471731 yes
9
Feedback Driven Fuzzing
def is_prime(n: int) -> bool:
"""Primality test using 6k+-1 optimization."""
if n <= 3:
return n > 1
if n % 2 == 0 or n % 3 == 0:
return False
i = 5
while i ** 2 <= n:
if n % i == 0 or n % (i + 2) == 0:
return False
i += 6
return True
def main():
num = stdin.read()
print(num, is_prime(num))
15
Constraining the Search Space
with
Input Grammars
Slide 29
Slide 29 text
16
Grammar
Slide 30
Slide 30 text
17
Formal Languages
Formal Language Descriptions
Slide 31
Slide 31 text
17
Formal Languages
Formal Language Descriptions
3. Regular
(Chomsky,1956)
Slide 32
Slide 32 text
17
Formal Languages
Formal Language Descriptions
3. Regular
Context Free
(Chomsky,1956)
Argument Stack
Slide 33
Slide 33 text
17
Formal Languages
Formal Language Descriptions
3. Regular
Context Free
Recursively Enumerable
(Chomsky,1956)
Argument Stack
Return Stack
Slide 34
Slide 34 text
17
Formal Languages
Formal Language Descriptions
3. Regular
Context Free
Recursively Enumerable
(Chomsky,1956)
Easy to produce and parse
Argument Stack
Return Stack
29
Where to Get the Grammar From?
Hand-written parsers already encode the grammar
Slide 59
Slide 59 text
29
Where to Get the Grammar From?
1. Extract the input string accesses
2. Attach control
fl
ow information (context-managers)
Hand-written parsers already encode the grammar
Slide 60
Slide 60 text
30
How to Extract This Grammar?
Slide 61
Slide 61 text
30
How to Extract This Grammar?
• Inputs + control
fl
ow -> Dynamic Control Dependence Trees
Slide 62
Slide 62 text
30
How to Extract This Grammar?
• Inputs + control
fl
ow -> Dynamic Control Dependence Trees
• DCD Trees -> Parse Tree
Slide 63
Slide 63 text
31
Control Dependence Graph
Statement B is control dependent on A if A determines whether B executes.
def parse_csv(s,i):
while s[i:]:
if is_digit(s[i]):
n,j = num(s[i:])
i = i+j
else:
comma(s[i])
i += 1
Slide 64
Slide 64 text
31
Control Dependence Graph
Statement B is control dependent on A if A determines whether B executes.
def parse_csv(s,i):
while s[i:]:
if is_digit(s[i]):
n,j = num(s[i:])
i = i+j
else:
comma(s[i])
i += 1
CDG for parse_csv
Slide 65
Slide 65 text
31
Control Dependence Graph
Statement B is control dependent on A if A determines whether B executes.
def parse_csv(s,i):
while s[i:]:
if is_digit(s[i]):
n,j = num(s[i:])
i = i+j
else:
comma(s[i])
i += 1
CDG for parse_csv
while: determines
whether
if: executes
Slide 66
Slide 66 text
32
def parse_csv(s,i):
while s[i:]:
if is_digit(s[i]):
n,j = num(s[i:])
i = i+j
else:
comma(s[i])
i += 1
CDG for parse_csv
Dynamic Control Dependence Tree
Each statement execution is represented as a separate node
Slide 67
Slide 67 text
32
def parse_csv(s,i):
while s[i:]:
if is_digit(s[i]):
n,j = num(s[i:])
i = i+j
else:
comma(s[i])
i += 1
CDG for parse_csv
Dynamic Control Dependence Tree
Each statement execution is represented as a separate node
DCD Tree for call parse_csv()
Slide 68
Slide 68 text
33
def parse_csv(s,i):
while s[i:]:
if is_digit(s[i]):
n,j = num(s[i:])
i = i+j
else:
comma(s[i])
i += 1
DCD Tree ~ Parse Tree
•No tracking beyond input bu
ff
er
•Characters are attached to nodes where they are accessed last
"12,"
"12,"
Slide 69
Slide 69 text
33
def parse_csv(s,i):
while s[i:]:
if is_digit(s[i]):
n,j = num(s[i:])
i = i+j
else:
comma(s[i])
i += 1
'1' '2' ','
DCD Tree ~ Parse Tree
•No tracking beyond input bu
ff
er
•Characters are attached to nodes where they are accessed last
"12,"
"12,"
Slide 70
Slide 70 text
34
def is_digit(i): return i in '0123456789'
def parse_num(s,i):
n = ''
while s[i:] and is_digit(s[i]):
n += s[i]
i = i +1
return i,n
def parse_paren(s, i):
assert s[i] == '('
i, v = parse_expr(s, i+1)
if s[i:] == '': raise Ex(s, i)
assert s[i] == ')'
return i+1, v
def parse_expr(s, i = 0):
expr, is_op = [], True
while s[i:]:
c = s[i]
if isdigit(c):
if not is_op: raise Ex(s,i)
i,num = parse_num(s,i)
expr.append(num)
is_op = False
elif c in ['+', '-', '*', '/']:
if is_op: raise Ex(s,i)
expr.append(c)
is_op, i = True, i + 1
elif c == '(':
if not is_op: raise Ex(s,i)
i, cexpr = parse_paren(s, i)
expr.append(cexpr)
is_op = False
elif c == ')': break
else: raise Ex(s,i)
if is_op: raise Ex(s,i)
return i, expr
9+3/4
Parse tree for parse_expr('9+3/4')
Slide 71
Slide 71 text
34
def is_digit(i): return i in '0123456789'
def parse_num(s,i):
n = ''
while s[i:] and is_digit(s[i]):
n += s[i]
i = i +1
return i,n
def parse_paren(s, i):
assert s[i] == '('
i, v = parse_expr(s, i+1)
if s[i:] == '': raise Ex(s, i)
assert s[i] == ')'
return i+1, v
def parse_expr(s, i = 0):
expr, is_op = [], True
while s[i:]:
c = s[i]
if isdigit(c):
if not is_op: raise Ex(s,i)
i,num = parse_num(s,i)
expr.append(num)
is_op = False
elif c in ['+', '-', '*', '/']:
if is_op: raise Ex(s,i)
expr.append(c)
is_op, i = True, i + 1
elif c == '(':
if not is_op: raise Ex(s,i)
i, cexpr = parse_paren(s, i)
expr.append(cexpr)
is_op = False
elif c == ')': break
else: raise Ex(s,i)
if is_op: raise Ex(s,i)
return i, expr
9+3/4
Parse tree for parse_expr('9+3/4')
52
Issue 386 from Rhino
var A = class extends (class {}){};
Issue 2937 from Closure
const [y,y] = [];
var {baz:{} = baz => {}} = baz => {};
Issue 385 from Rhino
{while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}}
Issue 2842 from Closure
Slide 112
Slide 112 text
52
Issue 386 from Rhino
var A = class extends (class {}){};
Issue 2937 from Closure
const [y,y] = [];
var {baz:{} = baz => {}} = baz => {};
Issue 385 from Rhino
{while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}}
Issue 2842 from Closure
Slide 113
Slide 113 text
52
Issue 386 from Rhino
var A = class extends (class {}){};
Issue 2937 from Closure
const [y,y] = [];
var {baz:{} = baz => {}} = baz => {};
Issue 385 from Rhino
{while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}}
Issue 2842 from Closure
Slide 114
Slide 114 text
52
Issue 386 from Rhino
var A = class extends (class {}){};
Issue 2937 from Closure
const [y,y] = [];
var {baz:{} = baz => {}} = baz => {};
Issue 385 from Rhino
{while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}}
Issue 2842 from Closure
Slide 115
Slide 115 text
52
Issue 386 from Rhino
var A = class extends (class {}){};
Issue 2937 from Closure
const [y,y] = [];
var {baz:{} = baz => {}} = baz => {};
Issue 385 from Rhino
{while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}}
Issue 2842 from Closure
Delta Minimization is useful but not su
ff
i
cient
3 * 4
:=
:= ' + '
| ' - '
|
:= ' * '
| ' / '
|
:= '+'
| '-'
| '(' ')'
| '.'
|
:=
|
:= [0-9]
c
c
✓ Did not reproduce the failure
60
Slide 129
Slide 129 text
( ( 4 ) )
:=
:= ' + '
| ' - '
|
:= ' * '
| ' / '
|
:= '+'
| '-'
| '(' ')'
| '.'
|
:=
|
:= [0-9]
c
c
c
c
c
c
c
61
Slide 130
Slide 130 text
( ( 1 - 2 ) )
:=
:= ' + '
| ' - '
|
:= ' * '
| ' / '
|
:= '+'
| '-'
| '(' ')'
| '.'
|
:=
|
:= [0-9]
c
c
c
c
c
c
c
( ( 1 - 2 ) )
62
Slide 131
Slide 131 text
( ( 1 - 2 ) )
:=
:= ' + '
| ' - '
|
:= ' * '
| ' / '
|
:= '+'
| '-'
| '(' ')'
| '.'
|
:=
|
:= [0-9]
c
c
c
c
c
c
c
✘ reproduced the failure
( ( 1 - 2 ) )
62
Slide 132
Slide 132 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
( ( 1 - 2 ) )
63
Slide 133
Slide 133 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
✘
( ( 1 - 2 ) )
63
Slide 134
Slide 134 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
✘
( ( 1 - 2 ) )
( ( 2 * 3 + 4 ) )
64
Slide 135
Slide 135 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
✘
( ( 1 - 2 ) )
✘
( ( 2 * 3 + 4 ) )
64
Slide 136
Slide 136 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
✘
( ( 1 - 2 ) )
✘
( ( 2 * 3 + 4 ) )
( ( - 2 / 1 ) )
65
Slide 137
Slide 137 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
✘
( ( 1 - 2 ) )
✘
( ( 2 * 3 + 4 ) )
✘
( ( - 2 / 1 ) )
65
Slide 138
Slide 138 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
✘
( ( 1 - 2 ) )
✘
( ( 2 * 3 + 4 ) )
✘
( ( - 2 / 1 ) )
( ( 98 - 0 ) )
66
Slide 139
Slide 139 text
( ( 1 - 2 ) )
c
c
c
c
c
c
c
✘
( ( 1 - 2 ) )
✘
( ( 2 * 3 + 4 ) )
✘
( ( - 2 / 1 ) )
✘
( ( 98 - 0 ) )
66
Slide 140
Slide 140 text
)
(
( )
( ( )
4 )
( ( 4 ) )
c
c
c
c
c
c
c
A
67
Slide 141
Slide 141 text
)
(
( )
( ( )
4 )
( ( 4 ) )
c
c
c
c
c
c
c
A
68
Slide 142
Slide 142 text
( ( 4 ) )
c
c
c
c
c
c
c
A
( ( ) )
( ( ) )
4
Minimized Input
Abstract Failure Inducing Input
def check(parsed):
if parsed.is_nested() and parsed.child.is_nested():
raise Exception()
return input
69
def jsoncheck(json):
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
{"abc": null}
✘
74
Slide 149
Slide 149 text
def jsoncheck(json):
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
{"abc": null}
✘
is : null
74
Slide 150
Slide 150 text
{"abc": []}
✔
def jsoncheck(json):
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
75
Slide 151
Slide 151 text
{"abc": []}
✔
no is : null
def jsoncheck(json):
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
75
Slide 152
Slide 152 text
{"abc": 124}
✘
no is "" :
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
process(json)
76
Slide 153
Slide 153 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
process(json)
{"": 124}
✔
is "" :
77
Slide 154
Slide 154 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
78
Slide 155
Slide 155 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
{"": 124}
✔
is "" :
no is : null
&
79
Slide 156
Slide 156 text
def jsoncheck(json):
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
{"abc": null}
✘
is : null
Start Symbol
80
Slide 157
Slide 157 text
def jsoncheck(json):
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
{"abc": []}
no is : null
✔
Start Symbol
81
Slide 158
Slide 158 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
process(json)
{"abc": 124}
✘
is "" :
Start Symbol
82
Slide 159
Slide 159 text
is "" :
no is : null
&
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
{"": 124}
✔
Start Symbol
83
Slide 160
Slide 160 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
Evogram
84
Slide 161
Slide 161 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
Evogram
84
Slide 162
Slide 162 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
Evogram
Automatically Derived
84
Slide 163
Slide 163 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
85
Slide 164
Slide 164 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
85
Slide 165
Slide 165 text
def jsoncheck(json):
if no_key_is_empty_string(json):
fail(’one key must be empty’)
if any_key_has_null_value(json):
fail(’key value must not be null’)
process(json)
Automatically Derived
85
Slide 166
Slide 166 text
Supercharged Pattern Matchers
where
is "":
is :null
where
is (())
is / 0
where
is "0"
is "0x"
where
is ";;"
is "()"
is "()"
Alternative to Regular Expressions
86