Rahul Gopinath
July 19, 2022
9

# Building Blocks for Fuzzing

Talk at IIT-Gandhinagar

July 19, 2022

## Transcript

3

5. ### 500 BC Aṣṭādhyāyī Dakṣiputra Pāṇini Ad hoc rules Formal specification

Vedic Sanskrit Classical Sanskrit 5

Era

19. ### def process_input(input) : try : ✘val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r SYNTAX ERROR 19 Parser
20. ### SYNTAX ERROR def process_input(input) : try : ✘val = parse(input

) res = process(val ) return re s except SyntaxError : return Erro r 20 The Core
21. ### 21 def process_input(input) : try : val = parse(input )

res = process(val ) return re s except SyntaxError : return Erro r SYNTAX ERROR 21
22. ### • Insert Instrumentatio n • Generate input s • Collect

execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Feedback Driven Fuzzing 22 22 https://www.fuzzingbook.org/html/MutationFuzzer.html
23. ### 23 def triangle(a, b, c): if a == b: if

b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene def triangle(a, b, c): __probe_enter() if a == b: __probe_1() if b == c: __probe_2() return Equilateral else: __probe_3() return Isosceles else: __probe_4() if b == c: __probe_5() return Isosceles else: __probe_6() if a == c: __probe_7() return Isosceles else: __probe_8() return Scalene def triangle(a, b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene Feedback Driven Fuzzing • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 23 https://www.fuzzingbook.org/html/MutationFuzzer.html
24. ### 24 Feedback Driven Fuzzing triangle (1,1,1) def triangle(a, b, c):

if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 24 https://www.fuzzingbook.org/html/MutationFuzzer.html
25. ### 25 Feedback Driven Fuzzing triangle (1,1,1) def triangle(a, b, c):

if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 25 https://www.fuzzingbook.org/html/MutationFuzzer.html
26. ### triangle (1,1,1) 26 Feedback Driven Fuzzing triangle (1,1,2) def triangle(a,

b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Mutated 26 https://www.fuzzingbook.org/html/MutationFuzzer.html
27. ### 27 Feedback Driven Fuzzing triangle (1,1,3) def triangle(a, b, c):

if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Mutated 27 https://www.fuzzingbook.org/html/MutationFuzzer.html
28. ### 28 Feedback Driven Fuzzing triangle (1,1,2) def triangle(a, b, c):

if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 28 https://www.fuzzingbook.org/html/MutationFuzzer.html triangle (1,1,1) triangle (1,1,3)

30. ### 30 Feedback Driven Fuzzing Weakness: static int is_reserved_word_token(const char *s,

int len) { const char *reserved[] = { "break", "case", "catch", "continue", "debugger", "default", "delete", "do", "else", "false", "finally", "for", "function", "if", "in", "instanceof", "new", "null", "return", "switch", "this", "throw", "true", "try", "typeof", "var", "void", "while", "with", "let", "undefined", ((void *)0)}; int i; if (!mjs_is_alpha(s[0])) return 0; for (i = 0; reserved[i] != ((void *)0); i++) { if (len == (int)strlen(reserved[i]) && strncmp(s, reserved[i], len) == 0) return i + 1; } return 0; } Tokens if (x > 100) { } coverage: 20% if (x > 100) { } e coverage: 5% if (x > 100) { } el coverage: 5% if (x > 100) { } els coverage: 5% if (x > 100) { } else coverage: 25% No smooth coverage gradient in parsers 30
31. ### 31 Feedback Driven Fuzzing def json_raw(stm) : while True :

stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 31
32. ### 32 Feedback Driven Fuzzing def json_raw(stm) : while True :

stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 32
33. ### 33 Feedback Driven Fuzzing def json_raw(stm) : while True :

stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 33
34. ### 34 Feedback Driven Fuzzing 34 • Insert Instrumentatio n •

Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage
35. ### 35 Solver Directed Fuzzing • Collect path constraint s •

Solve negated constraints for new inputs (a == b) (b == c) (b != c) triangle(1,2,1) https://www.fuzzingbook.org/html/ConcolicFuzzer.html 35
36. ### 36 Solver Directed Fuzzing • Collect path constraint s •

Solve constraints for new inputs void next_sym() { while(!eof) { switch (ch){ case '{': next_ch(); sym = LBRA; return; case '}': next_ch(); sym = RBRA; return; case '(': next_ch(); sym = LPAR; return; case ')': next_ch(); sym = RPAR; return; case '+': next_ch(); sym = PLUS; return; case '-': next_ch(); sym = MINUS; return; case '<': next_ch(); sym = LESS; return; case ';': next_ch(); sym = SEMI; return; case '=': next_ch(); sym = EQUAL; return; default: if (ch >= '0' && ch <= '9') { int_val = 0; /* missing overflow check */ while (ch >= '0' && ch <= '9') { int_val = int_val*10 + (ch - '0'); next_ch(); } sym = INT; } else if (ch >= 'a' && ch <= 'z') { int i = 0; /* missing overflow check */ while ((ch >= 'a' && ch <= 'z') || ch == '_'){ id_name[i++] = ch; next_ch(); } id_name[i] = '\0'; sym = 0; while (words[sym] != NULL && strcmp(words[sym], id_name) != 0) sym++; if (words[sym] == NULL) if (id_name[1] == '\0') sym = ID; else syntax_error(); } else syntax_error(); return; } Weak point: Path explosion https://www.fuzzingbook.org/html/ConcolicFuzzer.html 36
37. ### 37 Solver Directed Fuzzing • Collect path constraint s •

Solve negated constraints for new inputs 37

41. ### 41 Formal Languages Language Descriptions: Grammars 3. Regular Context Free

Recursively Enumerable (Chomsky,1956) Easy to produce and parse Argument Stack Return Stack 41
42. ### 42 Grammar JSON grammar De f inition for <elt> <elt>

key <json> ::= <elt >   <elt> ::= <object>  | <array>  | <string>  | <number>  | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> 42
43. ### 43 Grammar JSON grammar <json> ::= <elt >   <elt>

::= <object>  | <array>  | <string>  | <number>  | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> Expansion Rule Terminal Symbol Nonterminal Symbol 43
44. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Tree 44
45. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <json> Tree <elt> 45
46. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <elt> <object> Tree 46
47. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <object> { <items> } Tree 47
48. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <items> } { <item> } Tree 48
49. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <item> } { <string> : <elt> } Tree 49
50. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <string> : <elt> } { "" : <elt> } Tree 50
51. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { "" : <elt> } Tree 51
52. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { "" : <elt> } { "" : true } Tree 52
53. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] {"":true} Derivation Tree 53
54. ### {  '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

, ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]  } Grammar Fuzzer 54
55. ### General Parser {  '<json>' : [['<elt>']] , '<elt>' : [['<object>']

, ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]  } 55
56. ### {  '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

, ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]  } Lang Fuzzer https://www.fuzzingbook.org/html/LangFuzzer.html 56
57. ### def process_input(input) : try : ✔val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r {  '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]  } 57

61. ### 61 "Be liberal in what you accept, and conservative in

what you send" Postel's Law 61
62. ### QUIRK_ALLOW_ASCII_CONTROL_CODES QUIRK_ALLOW_BACKSLASH_A QUIRK_ALLOW_BACKSLASH_CAPITAL_U QUIRK_ALLOW_BACKSLASH_E QUIRK_ALLOW_BACKSLASH_NEW_LINE QUIRK_ALLOW_BACKSLASH_QUESTION_MARK QUIRK_ALLOW_BACKSLASH_SINGLE_QUOTE QUIRK_ALLOW_BACKSLASH_V QUIRK_ALLOW_BACKSLASH_X_AS_BYTE S

QUIRK_ALLOW_BACKSLASH_X_AS_CODE_POINTS QUIRK_ALLOW_BACKSLASH_ZERO QUIRK_ALLOW_COMMENT_BLOCK QUIRK_ALLOW_COMMENT_LINE QUIRK_ALLOW_EXTRA_COMMA QUIRK_ALLOW_INF_NAN_NUMBERS QUIRK_ALLOW_LEADING_ASCII_RECORD_SEPARATOR QUIRK_ALLOW_LEADING_UNICODE_BYTE_ORDER_MARK QUIRK_ALLOW_TRAILING_FILLER QUIRK_EXPECT_TRAILING_NEW_LINE_OR_EOF QUIRK_JSON_POINTER_ALLOW_TILDE_N_TILDE_R_TILDE_T QUIRK_REPLACE_INVALID_UNICODE JSON common quirks from https://github.com/google/wuffs 62
63. ### "Be liberal in what you accept, and conservative in what

you send"  Postel's Law The Specification The Implementation Extra "Features" Where to Get the Grammar From? 63
64. ### 64 Where to Get the Grammar From? Hand-written parsers already

encode the grammar 64
65. ### def json_raw(stm) : while True : stm.skipspaces( ) c =

stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw>::= <json_object>  | <json_list>  | <json_string>  | <json_number>  | <json_fixed(true)> | <json_fixed(false) > | <json_fixed(null) > <json_string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char > <json_object> ::= `{`<items>`} ` | `{} ` <items> ::= <item>`,`<items > | <item > <item> ::= <string>`:`<elt > <json_list> ::= `[`<elts>`] ` | `[] ` <elts> ::= <elt>`,`<elts > | <elt > <json_number> ::= <digits > <digits> ::= <digit><digits> | <digit> https://github.com/phensley/microjson MicroJSON 65 65
66. ### def json_raw(stm) : while True : stm.skipspaces( ) c =

stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw>::= <json_object>  | <json_list>  | <json_string>  | <json_number>  | <json_fixed(true)> | <json_fixed(false) > | <json_fixed(null)> <json_string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <json_object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <json_list> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <json_number> ::= <digits > <digits> ::= <digit><digits> | <digit> https://github.com/phensley/microjson MicroJSON 66
67. ### <F> := <A > | <B> <F> := <A> <B>

<C> <Fs> := <B> <Fs > | <empty> Structured Control Flow to Grammar Sequence A B C [F] Selection cond A B [F] F T Iteration cond B [F] 67
68. ### def json_raw(stm) : while True : stm.skipspaces( ) c =

stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_object(stm ) elif c == '[': return json_array(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw> ::= <loop1> <loop1> ::= <iter1> <loop1 > | <empty > <iter1>::= <json_fixed(true) > | <json_fixed(false) > | <json_fixed(null) >   | `"` <json_string > | `{` <json_object >   | `[` <json_array >   | [[1-9] <json_number>  68
69. ### def json_raw(stm) : while True : stm.skipspaces( ) c =

stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_object(stm ) elif c == '[': return json_array(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw> ::= <loop1> <loop1> ::= <iter1> <loop1 > | <empty > <iter1>::= <json_fixed(true) > | <json_fixed(false) > | <json_fixed(null) >   | `"` <json_string > | `{` <json_object >   | `[` <json_array >   | [[1-9] <json_number>  69
70. ### def json_string(stm) : # skip over '" ' stm.next() r

= [ ] while True : c = stm.next( ) if c == '' : raise JSONError(E_TRUNC ) elif c == '\\' : c = stm.next( ) r.append(decode_escape(c, stm) ) elif c == '"' : return ''.join(r ) else : r.append(c) <json_raw> ::= <loop1> ... <iter1>::= `"` <json_string > .. . <json_string> ::= <loop2 > <loop2> ::= <iter2> <loop2 > | <empty > <iter2>::= `\\` <decode_escape > | `" ` | <other_char> "ab" <json_string> ::= <loop2> <loop2_end > <loop2> ::= <iter2> <loop2 > | <empty > <iter2>::= `\\` <decode_escape > | <other_char > <loop2_end> ::= <iter2_end > <iter2_end>::=`"` 70
71. ### 71 <START> ::= <json_raw > <json_raw> ::= '"' <json_string' >

| '[' <json_list' > | '{' <json_dict' > | <json_number' > | 'true ' | 'false ' | 'null ' <json_number'> ::= <json_number> + | <json_number>+ 'e' <json_number> + <json_number> ::= '+' | '-' | '.' | [0-9] | 'E' | 'e ' <json_string'> ::= <json_string>* '" ' <json_list'> ::= '] ' | <json_raw> (','<json_raw>)* '] ' | ( ',' <json_raw>)+ (',' <json_raw>)* '] ' <json_dict'> ::= '} ' | ( '"' <json_string'> ':' <json_raw> ',' )* '"'<json_string'> ':' <json_raw> '} ' <json_string> ::= ' ' | '!' | '#' | '\$' | '%' | '&' | '' ' | '*' | '+' | '-' | ',' | '.' | '/' | ':' | '; ' | '<' | '=' | '>' | '?' | '@' | '[' | ']' | '^ ' | '_', ''',| '{' | '|' | '}' | '~ ' | '[A-Za-z0-9] ' | '\' <decode_escape > <decode_escape> ::= '"' | '/' | 'b' | 'f' | 'n' | 'r' | 't' stm.next()
72. ### Recall Subjects Mimid calc.py 100.0 % mathexpr.py 87.5 % cgidecode.py

100.0 % urlparse.py 100.0 % microjson.py 98.7 % parseclisp.py 99.3 % jsonparser.c 100.0 % tiny.c 100.0 % mjs.c 95.4 % Inputs generated by inferred grammar that were accepted by the program Subjects Mimid calc.py 100.0 % mathexpr.py 92.7 % cgidecode.py 100.0 % urlparse.py 96.4 % microjson.py 93.0 % parseclisp.py 80.6 % jsonparser.c 83.8 % tiny.c 92.8 % mjs.c 95.9 % Inputs generated by golden grammar that were accepted by the inferred grammar parser Precision Evaluation: Accuracy 72 72

75. ### 75 HTTP Parser XML Parser SOAP Parser RPC Parser C

Parser Check Declarations Check Types Static Checks Challenges Compilers Servers Semantics Application

78. ### {  '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

, ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'], ['{}']] , '<items>' : [['<item>,',',<items>'], ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'], ['[]']] , '<elts>' : [['<elt>,',',<elts>'], ['<elt>']] , '<string>' : [['"', '<chars>', '"'], ['""']] , '<chars>' : [['<char>','<chars>'], ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'], ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]  } def process_input(input) : try : ✔val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r Fuzz Programs? 78
79. ### Grammar Fuzzer Fast Fuzzers def json() : elt( ) def

elt() : match (random() % 7) : case 0: object( ) case 1: array( ) case 2: string( ) case 3: number( ) case 4: print('true' ) case 5: print('false' ) case 5: print('null')  def object() : match (random() % 2) : case 0: print('{'); items(); print('}' ) case 1: print('{');print('}' ) def array() : match (random() % 2) : case 0: print('['); elts(); print(']' ) case 1: print('[');print(']' ) def items() : match (random() % 2) : case 0: item(); print(','); items( ) case 1: item() Compiled Grammar (F1) Building Fast Fuzzers Gopinath and Zeller 2019 def json_0(rops) : elt_0(rops ) def elt_0(rops) : r = next(rops ) if 0 <= r < 36: elt_0( ) elif 36 <= r < 73: elt_1( ) elif 73 <= r < 109: elt_2( ) elif 109 <= r <146: elt_3( ) elif 146 <= r < 182: elt_4( ) elif 182 <= r < 219: elt_5( ) else: elt_6( ) def elt_0(rops) : r = next(rops ) if 0 <= r < 43: expr_0( ) elif 43 <= r < 85: expr_1( ) elif 85 <= r < 128: expr_2( ) elif 128 <= r < 171: expr_3( ) elif 171 <= r < 213: expr_4( ) else: expr_5( ) print('+' ) elif 171 <= r < 213: expr_4( ) else: expr_5() Grammar VM (F1) 79 {  '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]  }
80. ### 80 Focused Fuzzing Generate inputs that focus on speci fi

c system parts • Covering speci fi c portions of code • Speci fi c paths through the code • Performing speci fi c functions Without damaging the nondeterminism. 80
81. ### def process_input(input) : try : val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} ✓ 81
82. ### def process_input(input) : try : val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ✘ 82
83. ### def process_input(input) : try : val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ✓ ["A", "B", "C"] 83
84. ### def process_input(input) : try : val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ["A", "B", "C"] [{"": [1,2,3,4]}] ✘ 84
85. ### def process_input(input) : try : val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ["A", "B", "C"] [{"": [1,2,3,4]}] if json.has_key("") : raise Exception() 85
86. ### def process_input(input) : try : val = parse(input ) res

= process(val ) return re s except SyntaxError : return Erro r {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," ✘ 86
87. ### {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971

7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": What is the smallest failure inducing input? Delta Debugging 87
88. ### <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>

| <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]`| `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> Hierarchical Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": 88
89. ### {"":[]} ✘ 89 Test Minimization {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi

llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": Hierarchical Delta Debugging 89
90. ### 90 Why? [12345] {"":[]} {"":0} {"x":[]} {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000","

fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": Hierarchical Delta Debugging 90
91. ### DDSET Gopinath, Kampmann, Havrikov, Soremekun, and Zeller. Abstracting Failure Inducing

Inputs. ISSTA 2020. https://github.com/vrthra/ddset 91
92. ### {"": []} DDSET: <json> ::= <elt>  <elt> ::= <object>  |

<array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> 92
93. ### {"": []} 122489 {"A":{}, {"23": {"P":[]}}] [[], [[[]],[]],{"A":{}, {"23": {"P":[]}}]

<json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 93
94. ### {"": []} "XYZR389" {"A":{}, {"23": {"P":[]}}} [[], [[[]],[]],{"A":{}, {"23": {"P":[]}}]

<json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 94
95. ### {"": []} [true, null] {"A":{}, {"23": {"P":[]}}} [[], [[[]],[]],{"A":{}, {"23":

{"P":[]}}] <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 95
96. ### {"": []} {"__": [[]]} {"?P":[{}], {"|": {"":[]}}} {"X":[[],[]],{"A":{}, {"2": {"R":[]}}}

<json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 96
97. ### {"": []} {"": [[]]} {"?P":[{}], {"|": {"P":[]}}} {"X":[[],[]],{"A":{}, {"2": {"R":[]}}}

97 <json> ::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
98. ### {"": []} {"7897A": []} {"klnm,.qer;dfs?P":[]} {"123KOUIJ!qR30578950":[]} 98 <json> ::= <elt>

<elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
99. ### {"": []} {"": true} {"":[1,2,445,"x"]} {"":{"PQ":[true, false, 223,"a"]}} 99 <json>

::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
100. ### {"": []} Abstraction {"": <elt>} Abstract Input 100 <json> ::=

<elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
101. ### {"": <elt>} Abstract Input {"": []} Minimized Input 101 <json>

::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
102. ### Issue 386 from Rhino var A = class extends (class

{}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure <varModifier> <Identifier> = class extends (class {}){} var {<\$Id1>:{} = <\$Id1> => {}} <variableDeclaration>; const [<\$Id1>,<\$Id1>] = [] {while ((<\$Id1>)){ if ((<\$Id1>)) {break;;var <\$Id1>; continue }0}} 102
103. ### {"": <elt>} Abstract Input {"": []} Minimized Input 103 <json>

::= <elt>  <elt> ::= <object>  | <array>  | <string>  | <number>  | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> <json E> ::= <elt E>  <elt E> ::= <object E>  | <array E > | <string E>  | <number E>  <object E> ::= `{`<items E>`}`  <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>  <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>  <array E> ::= `[`<elts E>`]`  <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>  <elt> ::= <object>  | <array > | <string>  | <number>  | `true` | `false` | `null`  <object> ::= `{`<items>`}` | `{}`  <items> ::= <item> | <item>`,`<items>  <item> ::= <string>`:`<elt>  <array> ::= `[`<elts>`]` | `[]`  <elts> ::= <elt> | <elt>`,`<elts>  <string> ::= `"` <chars> `"` | `""`  <chars> ::= <char><chars>  <char> ::= [A-Za-z0-9]  <number> ::= <digits>  <digits> ::= <digit><digits> | <digit>  <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Pattern Evocative Grammar
104. ### 104 {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[],

{"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ <json E> ::= <elt E>  <elt E> ::= <object E>  | <array E > | <string E>  | <number E>  <object E> ::= `{`<items E>`}`  <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>  <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>  <array E> ::= `[`<elts E>`]`  <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>  <elt> ::= <object>  | <array > | <string>  | <number>  | `true` | `false` | `null`  <object> ::= `{`<items>`}` | `{}`  <items> ::= <item> | <item>`,`<items>  <item> ::= <string>`:`<elt>  <array> ::= `[`<elts>`]` | `[]`  <elts> ::= <elt> | <elt>`,`<elts>  <string> ::= `"` <chars> `"` | `""`  <chars> ::= <char><chars>  <char> ::= [A-Za-z0-9]  <number> ::= <digits>  <digits> ::= <digit><digits> | <digit>  <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Grammar
105. ### 105 <json E> where <item E> is "":<elt> 1. We

can produce any and all instances of the failure inducing pattern . 2. We can recognize any input that contains the failure inducing pattern . 3. The grammar will reject any input that doesn't contain the failure inducing pattern. <json E> ::= <elt E>  <elt E> ::= <object E>  | <array E > | <string E>  | <number E>  <object E> ::= `{`<items E>`}`  <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>  <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>  <array E> ::= `[`<elts E>`]`  <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>  <elt> ::= <object>  | <array > | <string>  | <number>  | `true` | `false` | `null`  <object> ::= `{`<items>`}` | `{}`  <items> ::= <item> | <item>`,`<items>  <item> ::= <string>`:`<elt>  <array> ::= `[`<elts>`]` | `[]`  <elts> ::= <elt> | <elt>`,`<elts>  <string> ::= `"` <chars> `"` | `""`  <chars> ::= <char><chars>  <char> ::= [A-Za-z0-9]  <number> ::= <digits>  <digits> ::= <digit><digits> | <digit>  <digit> ::= [0-9] Evocative Grammar
106. ### <json E> where <item E> is "":<elt> if json.has_key("") :

raise Exception() if json.has_key_value(null) : raise Exception() <json N> where <item N> is <string>: null 106 Combining Evocative Patterns
107. ### if json.has_key("") and json.has_key_value(null) : raise Exception() <json E &

N> where <item E> is "":<elt>   <item N> is <string>: null 107 Combining Evocative Patterns
108. ### if json.has_key("") and not json.has_key_value(null) : raise Exception() <json E

& not(N)> where <item E> is "":<elt> <item N> is <string>: null 108 Combining Evocative Patterns
109. ### if json.has_key("") : raise Exception( ) if json.has_key_value(null) : raise

Exception() <json not(E) & not(N)> where <item E> is "":<elt> <item N> is <string>: null 109 Combining Evocative Patterns
110. ### <json E&N> := <elt E&N > <elt E&N> := <object

E&N > | <array E&N > <array E&N>:= '[' <elts E&N> '] ' <object E&N>:= '{' <items E&N> '} ' <elts E&N> := <elt E&N > | <elt E&N>','<elts N > | <elt N>','<elts E&N > <items E&N> := <item E&N > | <item E&N>','<items N > | <item N>','<items E&N > <item E&N> := <string E1>':'<elt N&N1 > | <string>':'<elt E&N&N1 > <elt E&N&N1> := <object E&N> <array E&N > <elt N> := 'false' | 'true ' | <number> | <string > | <object N> <array N > <array N> := '[]' | '[' <elts N> '] ' <object N> := '{}' | '{' <items N> '} ' <elts N> := <elt N > | <elt N>','<elts N > <items N> := <item N > | <item N>','<items N > <item N> := <string>':'<elt N&N1 > <elt N&N1> := 'false' | 'true ' | <number> | <string > | <object N> | <array N> <json E & N> where <item E> is "":<elt> <item N> is <string>: null {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[], {"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E&N>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ 110
111. ### Issue 386 from Rhino var A = class extends (class

{}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure <varModifier> <Identifier> = class extends (class {}){} var {<\$Id1>:{} = <\$Id1> => {}} <variableDeclaration>; const [<\$Id1>,<\$Id1>] = [] {while ((<\$Id1>)){ if ((<\$Id1>)) {break;;var <\$Id1>; continue }0}} 111
112. ### where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class

{}){} <iterationStatement C2842> is {while ((<\$Id1>)){ if ((<\$Id1>)) {break;;var <\$Id1>; continue }0}} <variableStatement R385> is var {<\$Id2>:{} = <\$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<\$Id3>,<\$Id3>] = [] Evocative Expressions <JavaScript C2937 and C2842> <JavaScript R385 and R386> <JavaScript (C2937 or C2842) and (R385 or R386)> <JavaScript not(C2937 or C2842 or R385 or R386)> 112
113. ### 113 Evocative Expressions: Validating Inputs where <variableDeclarationList C2937> is <varModifier>

<Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<\$Id1>)){ if ((<\$Id1>)) {break;;var <\$Id1>; continue }0}} <variableStatement R385> is var {<\$Id2>:{} = <\$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<\$Id3>,<\$Id3>] = [] <JavaScript not(C2937 or C2842 or R385 or R386)>
114. ### Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB )

self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) 114

assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"role" : "admin"} Evocative Expressions for Focused Fuzzing
116. ### 116 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"method":"remove_table","args":["orders", "inventory"]} <json DBDT> where <items DBDT> is "method":"remove_table","args":<elt> Evocative Expressions for Focused Fuzzing
117. ### 117 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"], "role":"admin"} {"role":"admin", "method":"remove_table","args":["orders", "inventory"]} {"method":"remove_table","role":"admin","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"admin"}]} Evocative Expressions for Focused Fuzzing
118. ### 118 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"], "role":"guest"} {"role":"guest", "method":"remove_table","args":["orders", "inventory"]} {"method":"remove_table","role":"guest","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"guest"}]} Evocative Expressions for Focused Fuzzing
119. ### Algebraic Data Types 119 <my_struct> ::= <stype> <stype> struct mystruct

{ stype m1 ; stype m2 ; }; union myunion { utype m1 ; utype m2 ; }; <my_union> ::= <utype> | <utype> Data Structures Context Free Grammar Evocative Expressions: Data Structures
120. ### Search Source Code For Fault Patterns • Structure aware •

Semantics can be mined from bugs! • No need to specify by hand • Combine different patterns using `or`, `and` and `not` 120 Evocative Expressions: Semantic Search

121
122. ### if json.has_key("") : raise Exception() if not json.has_key_value(null) : raise

Exception() 122 if json.has_key("") and not json.has_key_value(null) : raise Exception() Future: Decomposition with DDSet 122

126. ### 126 • Differentiate incomplete and incorrect input s • Solve

one character at a time systematically Key Idea (Monotonic Failure Property) 126
127. ### 127 Sample Free Generators A ( 2 - B 9

) 4 ) A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0 B ∉ +,-,1,2,3,4,5,6,7,8,9,0,) ) ∉ +,-,1,2,3,4,5,6,7,8,9,0 (2-94) 127
128. ### 128 Mathis, Gopinath, Mera, Kampmann, Höschele, and Zeller. Parser Directed

Fuzzing. PLDI 2019 . Mathis, Gopinath and Zeller Learning Input Tokens for Effective Fuzzing. ISSTA 2020. Sample Free Generators A ( 2 - B 9 ) 4 )
129. ### 129 Blackbox Sample Free Generators • Monotonic Failure Propert y

• Differentiate incomplete and incorrect input s • Solve one character at a time systematically Bendrissou, Gopinath, Mathis, and Zeller Failure Feedback for Fast Fuzzing. (arXiv). 129

131. ### 131 HTTP Parser XML Parser SOAP Parser RPC Parser C

Parser Check Declarations Check Types Static Checks Compilers Servers Semantics Application Layered Programs 131

133. ### 133 QS: Computer Science 22 in the world One in

the Group of Eight (Go8) in Australia