Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Blocks for Fuzzing

Building Blocks for Fuzzing

Talk at IIT-Gandhinagar

D27cb84e0d30e2778e9b66d6a5f42106?s=128

Rahul Gopinath

July 19, 2022
Tweet

More Decks by Rahul Gopinath

Other Decks in Research

Transcript

  1. 1 Building Blocks for Fuzzing Rahul Gopinath https://rahul.gopinath.org @_rahulgopinath

  2. Rahul Gopinath Building Blocks for Fuzzing https://rahul.gopinath.org @_rahulgopinath 2

  3. The story begins 2500 years ago. Vedic Era 500 BC

    3
  4. Classical Era 500 BC 4

  5. 500 BC Aṣṭādhyāyī Dakṣiputra Pāṇini Ad hoc rules Formal specification

    Vedic Sanskrit Classical Sanskrit 5
  6. 6 2500 years later.... 2022 CE Dawn of a New

    Era
  7. 7 New Challenges

  8. Bugs 8

  9. 9

  10. 10 Input ✓ ✘ Testing @app.route('/admin') def admin(): username =

    request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500
  11. HTTP/1.1 401 Not Authorize d Content-Type: application/jso n { "Error"

    : "Assignment of internal rol e 'superadmin' is forbidden " } HTTP/1.1 200 O K Content-type: application/jso n { "result": "OK: Updated use r 'exampleUser' with rol e 'superadmin' " } 11 Input ✓ ✘ Testing POST /user/update HTTP/1. 1 { "user": "exampleUser", "roles": [ "superadmin " ] } @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 HTTP/1.1 500 Internal Server Error ✘
  12. 12 Verify Behavior Input ✓ ✘ Automatic Testing @app.route('/admin') def

    admin(): username = request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500
  13. 13 (Oracle) Input ✓ ✘ Automatic Testing Oracles require domain

    expertise @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 Verify Behavior
  14. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 Fuzzing Trash deck technique: 1950s - Gerald Weinberg Crash? 14
  15. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 Fuzzing Crash? • Memory Bounds Violation • Privilege Escalation • Safety Violations • Metamorphic Relations • Differential Execution ASAN,MSAN,TSAN,NSan,FuZZan,UBSan 15
  16. 16 https://www.fuzzingbook.org

  17. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 [ ; x 1 - G P Z + w c c k c ] ; , N 9 J + ? # 6 ^ 6 \ e ? ] 9 l u 2 _ % ' 4 G X " 0 V U B [ E / r ~ f A p u 6 b 8 < { % s i q 8 Z h . 6 { V , h r ? ; {Ti.r3PIxMMMv6{xS^+'Hq!AxB"YXRS@! Kd6;wtAMefFWM(`|J_<1~o}z3K(CCzRH J I I v H z > _ * . \ > J r l U 3 2 ~ e G P ? lR=bF3+;y$3lodQ<B89!5"W2fK*vE7v{')KC- i,c{<[~m!]o;{.'}Gj\(X}EtYetrpbY@aGZ1{P! A Z U 7 x # 4 ( R t n ! q 4 n C w q o l ^ y 6 } 0 | Ko=*JK~;zMKV=9Nai:wxu{J&UV#HaU)*Bi C < ) , ` + t * g k a < W = Z . % T 5 W G H Z p I 3 0 D < P q > & ] B S 6 R & j ? # t P 7 i a V } - } ` \ ? [ _ [ Z ^ L B M P G - FKj'\xwuZ1=Q`^`5,$N$Q@[!CuRzJ2D|vBy! ^ z k h d f 3 C 5 P A k R ? V ( ( - % > < h n | 3='i2Qx]D$qs4O`1@fevnG'2\11Vf3piU37@ 5 : d f d 4 5 * ( 7 ^ % 5 a p \ z I y l " ' f , $ee,J4Gw:cgNKLie3nx9(`efSlg6#[K"@Wjh Z}r[Scun&sBCS,T[/3]KAeEnQ7lU)3Pn,0)G/ 6N-wyzj/MTd#A;r Program Traditional Fuzzing 17 https://www.fuzzingbook.org/html/Fuzzer.html
  18. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 [ ; x 1 - G P Z + w c c k c ] ; , N 9 J + ? # 6 ^ 6 \ e ? ] 9 l u 2 _ % ' 4 G X " 0 V U B [ E / r ~ f A p u 6 b 8 < { % s i q 8 Z h . 6 { V , h r ? ; {Ti.r3PIxMMMv6{xS^+'Hq!AxB"YXRS@! Kd6;wtAMefFWM(`|J_<1~o}z3K(CCzRH J I I v H z > _ * . \ > J r l U 3 2 ~ e G P ? lR=bF3+;y$3lodQ<B89!5"W2fK*vE7v{')KC- i,c{<[~m!]o;{.'}Gj\(X}EtYetrpbY@aGZ1{P! A Z U 7 x # 4 ( R t n ! q 4 n C w q o l ^ y 6 } 0 | Ko=*JK~;zMKV=9Nai:wxu{J&UV#HaU)*Bi C < ) , ` + t * g k a < W = Z . % T 5 W G H Z p I 3 0 D < P q > & ] B S 6 R & j ? # t P 7 i a V } - } ` \ ? [ _ [ Z ^ L B M P G - FKj'\xwuZ1=Q`^`5,$N$Q@[!CuRzJ2D|vBy! ^ z k h d f 3 C 5 P A k R ? V h n | 3='i2Qx]D$qs4O`1@fevnG'2\11Vf3piU37@ 5 5 a p \ z I y l " ' f , $ee,J4Gw:cgNKLie3nx9(`efSlg6#[K"@Wjh Z}r[Scun&sBCS,T[/3]KAeEnQ7lU)3Pn,0)G/ 6N-wyzj/MTd#A;r Structured Inputs SYNTAX ERROR ✘ 18
  19. def process_input(input) : try : ✘val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r SYNTAX ERROR 19 Parser
  20. SYNTAX ERROR def process_input(input) : try : ✘val = parse(input

    ) res = process(val ) return re s except SyntaxError : return Erro r 20 The Core
  21. 21 def process_input(input) : try : val = parse(input )

    res = process(val ) return re s except SyntaxError : return Erro r SYNTAX ERROR 21
  22. • Insert Instrumentatio n • Generate input s • Collect

    execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Feedback Driven Fuzzing 22 22 https://www.fuzzingbook.org/html/MutationFuzzer.html
  23. 23 def triangle(a, b, c): if a == b: if

    b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene def triangle(a, b, c): __probe_enter() if a == b: __probe_1() if b == c: __probe_2() return Equilateral else: __probe_3() return Isosceles else: __probe_4() if b == c: __probe_5() return Isosceles else: __probe_6() if a == c: __probe_7() return Isosceles else: __probe_8() return Scalene def triangle(a, b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene Feedback Driven Fuzzing • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 23 https://www.fuzzingbook.org/html/MutationFuzzer.html
  24. 24 Feedback Driven Fuzzing triangle (1,1,1) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 24 https://www.fuzzingbook.org/html/MutationFuzzer.html
  25. 25 Feedback Driven Fuzzing triangle (1,1,1) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 25 https://www.fuzzingbook.org/html/MutationFuzzer.html
  26. triangle (1,1,1) 26 Feedback Driven Fuzzing triangle (1,1,2) def triangle(a,

    b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Mutated 26 https://www.fuzzingbook.org/html/MutationFuzzer.html
  27. 27 Feedback Driven Fuzzing triangle (1,1,3) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Mutated 27 https://www.fuzzingbook.org/html/MutationFuzzer.html
  28. 28 Feedback Driven Fuzzing triangle (1,1,2) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 28 https://www.fuzzingbook.org/html/MutationFuzzer.html triangle (1,1,1) triangle (1,1,3)
  29. • Insert Instrumentatio

  30. 30 Feedback Driven Fuzzing Weakness: static int is_reserved_word_token(const char *s,

    int len) { const char *reserved[] = { "break", "case", "catch", "continue", "debugger", "default", "delete", "do", "else", "false", "finally", "for", "function", "if", "in", "instanceof", "new", "null", "return", "switch", "this", "throw", "true", "try", "typeof", "var", "void", "while", "with", "let", "undefined", ((void *)0)}; int i; if (!mjs_is_alpha(s[0])) return 0; for (i = 0; reserved[i] != ((void *)0); i++) { if (len == (int)strlen(reserved[i]) && strncmp(s, reserved[i], len) == 0) return i + 1; } return 0; } Tokens if (x > 100) { } coverage: 20% if (x > 100) { } e coverage: 5% if (x > 100) { } el coverage: 5% if (x > 100) { } els coverage: 5% if (x > 100) { } else coverage: 25% No smooth coverage gradient in parsers 30
  31. 31 Feedback Driven Fuzzing def json_raw(stm) : while True :

    stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 31
  32. 32 Feedback Driven Fuzzing def json_raw(stm) : while True :

    stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 32
  33. 33 Feedback Driven Fuzzing def json_raw(stm) : while True :

    stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 33
  34. 34 Feedback Driven Fuzzing 34 • Insert Instrumentatio n •

    Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage
  35. 35 Solver Directed Fuzzing • Collect path constraint s •

    Solve negated constraints for new inputs (a == b) (b == c) (b != c) triangle(1,2,1) https://www.fuzzingbook.org/html/ConcolicFuzzer.html 35
  36. 36 Solver Directed Fuzzing • Collect path constraint s •

    Solve constraints for new inputs void next_sym() { while(!eof) { switch (ch){ case '{': next_ch(); sym = LBRA; return; case '}': next_ch(); sym = RBRA; return; case '(': next_ch(); sym = LPAR; return; case ')': next_ch(); sym = RPAR; return; case '+': next_ch(); sym = PLUS; return; case '-': next_ch(); sym = MINUS; return; case '<': next_ch(); sym = LESS; return; case ';': next_ch(); sym = SEMI; return; case '=': next_ch(); sym = EQUAL; return; default: if (ch >= '0' && ch <= '9') { int_val = 0; /* missing overflow check */ while (ch >= '0' && ch <= '9') { int_val = int_val*10 + (ch - '0'); next_ch(); } sym = INT; } else if (ch >= 'a' && ch <= 'z') { int i = 0; /* missing overflow check */ while ((ch >= 'a' && ch <= 'z') || ch == '_'){ id_name[i++] = ch; next_ch(); } id_name[i] = '\0'; sym = 0; while (words[sym] != NULL && strcmp(words[sym], id_name) != 0) sym++; if (words[sym] == NULL) if (id_name[1] == '\0') sym = ID; else syntax_error(); } else syntax_error(); return; } Weak point: Path explosion https://www.fuzzingbook.org/html/ConcolicFuzzer.html 36
  37. 37 Solver Directed Fuzzing • Collect path constraint s •

    Solve negated constraints for new inputs 37
  38. 38 Specialized Generators • Specialize generation for a domain 38

  39. 39 Overcoming Parsers

  40. Needed: Input Language 40 40

  41. 41 Formal Languages Language Descriptions: Grammars 3. Regular Context Free

    Recursively Enumerable (Chomsky,1956) Easy to produce and parse Argument Stack Return Stack 41
  42. 42 Grammar JSON grammar De f inition for <elt> <elt>

    key <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> 42
  43. 43 Grammar JSON grammar <json> ::= <elt > 
 <elt>

    ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> Expansion Rule Terminal Symbol Nonterminal Symbol 43
  44. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Tree 44
  45. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <json> Tree <elt> 45
  46. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <elt> <object> Tree 46
  47. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <object> { <items> } Tree 47
  48. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <items> } { <item> } Tree 48
  49. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <item> } { <string> : <elt> } Tree 49
  50. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <string> : <elt> } { "" : <elt> } Tree 50
  51. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { "" : <elt> } Tree 51
  52. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { "" : <elt> } { "" : true } Tree 52
  53. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] {"":true} Derivation Tree 53
  54. {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

    , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } Grammar Fuzzer 54
  55. General Parser {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>']

    , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } 55
  56. {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

    , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } Lang Fuzzer https://www.fuzzingbook.org/html/LangFuzzer.html 56
  57. def process_input(input) : try : ✔val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } 57
  58. Where to Get the Grammar From? 58

  59. Almost Everyone Uses Handwritten Parsers https://notes.eatonphil.com/parser-generators-vs-handwritten-parsers-survey-2021.html 59

  60. Where to Get the Grammar From? 60

  61. 61 "Be liberal in what you accept, and conservative in

    what you send" Postel's Law 61
  62. QUIRK_ALLOW_ASCII_CONTROL_CODES QUIRK_ALLOW_BACKSLASH_A QUIRK_ALLOW_BACKSLASH_CAPITAL_U QUIRK_ALLOW_BACKSLASH_E QUIRK_ALLOW_BACKSLASH_NEW_LINE QUIRK_ALLOW_BACKSLASH_QUESTION_MARK QUIRK_ALLOW_BACKSLASH_SINGLE_QUOTE QUIRK_ALLOW_BACKSLASH_V QUIRK_ALLOW_BACKSLASH_X_AS_BYTE S

    QUIRK_ALLOW_BACKSLASH_X_AS_CODE_POINTS QUIRK_ALLOW_BACKSLASH_ZERO QUIRK_ALLOW_COMMENT_BLOCK QUIRK_ALLOW_COMMENT_LINE QUIRK_ALLOW_EXTRA_COMMA QUIRK_ALLOW_INF_NAN_NUMBERS QUIRK_ALLOW_LEADING_ASCII_RECORD_SEPARATOR QUIRK_ALLOW_LEADING_UNICODE_BYTE_ORDER_MARK QUIRK_ALLOW_TRAILING_FILLER QUIRK_EXPECT_TRAILING_NEW_LINE_OR_EOF QUIRK_JSON_POINTER_ALLOW_TILDE_N_TILDE_R_TILDE_T QUIRK_REPLACE_INVALID_UNICODE JSON common quirks from https://github.com/google/wuffs 62
  63. "Be liberal in what you accept, and conservative in what

    you send"
 Postel's Law The Specification The Implementation Extra "Features" Where to Get the Grammar From? 63
  64. 64 Where to Get the Grammar From? Hand-written parsers already

    encode the grammar 64
  65. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw>::= <json_object>
 | <json_list>
 | <json_string>
 | <json_number>
 | <json_fixed(true)> | <json_fixed(false) > | <json_fixed(null) > <json_string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char > <json_object> ::= `{`<items>`} ` | `{} ` <items> ::= <item>`,`<items > | <item > <item> ::= <string>`:`<elt > <json_list> ::= `[`<elts>`] ` | `[] ` <elts> ::= <elt>`,`<elts > | <elt > <json_number> ::= <digits > <digits> ::= <digit><digits> | <digit> https://github.com/phensley/microjson MicroJSON 65 65
  66. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw>::= <json_object>
 | <json_list>
 | <json_string>
 | <json_number>
 | <json_fixed(true)> | <json_fixed(false) > | <json_fixed(null)> <json_string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <json_object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <json_list> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <json_number> ::= <digits > <digits> ::= <digit><digits> | <digit> https://github.com/phensley/microjson MicroJSON 66
  67. <F> := <A > | <B> <F> := <A> <B>

    <C> <Fs> := <B> <Fs > | <empty> Structured Control Flow to Grammar Sequence A B C [F] Selection cond A B [F] F T Iteration cond B [F] 67
  68. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_object(stm ) elif c == '[': return json_array(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw> ::= <loop1> <loop1> ::= <iter1> <loop1 > | <empty > <iter1>::= <json_fixed(true) > | <json_fixed(false) > | <json_fixed(null) > 
 | `"` <json_string > | `{` <json_object > 
 | `[` <json_array > 
 | [[1-9] <json_number>
 68
  69. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_object(stm ) elif c == '[': return json_array(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw> ::= <loop1> <loop1> ::= <iter1> <loop1 > | <empty > <iter1>::= <json_fixed(true) > | <json_fixed(false) > | <json_fixed(null) > 
 | `"` <json_string > | `{` <json_object > 
 | `[` <json_array > 
 | [[1-9] <json_number>
 69
  70. def json_string(stm) : # skip over '" ' stm.next() r

    = [ ] while True : c = stm.next( ) if c == '' : raise JSONError(E_TRUNC ) elif c == '\\' : c = stm.next( ) r.append(decode_escape(c, stm) ) elif c == '"' : return ''.join(r ) else : r.append(c) <json_raw> ::= <loop1> ... <iter1>::= `"` <json_string > .. . <json_string> ::= <loop2 > <loop2> ::= <iter2> <loop2 > | <empty > <iter2>::= `\\` <decode_escape > | `" ` | <other_char> "ab" <json_string> ::= <loop2> <loop2_end > <loop2> ::= <iter2> <loop2 > | <empty > <iter2>::= `\\` <decode_escape > | <other_char > <loop2_end> ::= <iter2_end > <iter2_end>::=`"` 70
  71. 71 <START> ::= <json_raw > <json_raw> ::= '"' <json_string' >

    | '[' <json_list' > | '{' <json_dict' > | <json_number' > | 'true ' | 'false ' | 'null ' <json_number'> ::= <json_number> + | <json_number>+ 'e' <json_number> + <json_number> ::= '+' | '-' | '.' | [0-9] | 'E' | 'e ' <json_string'> ::= <json_string>* '" ' <json_list'> ::= '] ' | <json_raw> (','<json_raw>)* '] ' | ( ',' <json_raw>)+ (',' <json_raw>)* '] ' <json_dict'> ::= '} ' | ( '"' <json_string'> ':' <json_raw> ',' )* '"'<json_string'> ':' <json_raw> '} ' <json_string> ::= ' ' | '!' | '#' | '$' | '%' | '&' | '' ' | '*' | '+' | '-' | ',' | '.' | '/' | ':' | '; ' | '<' | '=' | '>' | '?' | '@' | '[' | ']' | '^ ' | '_', ''',| '{' | '|' | '}' | '~ ' | '[A-Za-z0-9] ' | '\' <decode_escape > <decode_escape> ::= '"' | '/' | 'b' | 'f' | 'n' | 'r' | 't' stm.next()
  72. Recall Subjects Mimid calc.py 100.0 % mathexpr.py 87.5 % cgidecode.py

    100.0 % urlparse.py 100.0 % microjson.py 98.7 % parseclisp.py 99.3 % jsonparser.c 100.0 % tiny.c 100.0 % mjs.c 95.4 % Inputs generated by inferred grammar that were accepted by the program Subjects Mimid calc.py 100.0 % mathexpr.py 92.7 % cgidecode.py 100.0 % urlparse.py 96.4 % microjson.py 93.0 % parseclisp.py 80.6 % jsonparser.c 83.8 % tiny.c 92.8 % mjs.c 95.9 % Inputs generated by golden grammar that were accepted by the inferred grammar parser Precision Evaluation: Accuracy 72 72
  73. 73 <START> ::= <json_raw

  74. 74

  75. 75 HTTP Parser XML Parser SOAP Parser RPC Parser C

    Parser Check Declarations Check Types Static Checks Challenges Compilers Servers Semantics Application
  76. 76 76

  77. What Can We Do With Grammars? 77 77

  78. {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

    , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'], ['{}']] , '<items>' : [['<item>,',',<items>'], ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'], ['[]']] , '<elts>' : [['<elt>,',',<elts>'], ['<elt>']] , '<string>' : [['"', '<chars>', '"'], ['""']] , '<chars>' : [['<char>','<chars>'], ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'], ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } def process_input(input) : try : ✔val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r Fuzz Programs? 78
  79. Grammar Fuzzer Fast Fuzzers def json() : elt( ) def

    elt() : match (random() % 7) : case 0: object( ) case 1: array( ) case 2: string( ) case 3: number( ) case 4: print('true' ) case 5: print('false' ) case 5: print('null')
 def object() : match (random() % 2) : case 0: print('{'); items(); print('}' ) case 1: print('{');print('}' ) def array() : match (random() % 2) : case 0: print('['); elts(); print(']' ) case 1: print('[');print(']' ) def items() : match (random() % 2) : case 0: item(); print(','); items( ) case 1: item() Compiled Grammar (F1) Building Fast Fuzzers Gopinath and Zeller 2019 def json_0(rops) : elt_0(rops ) def elt_0(rops) : r = next(rops ) if 0 <= r < 36: elt_0( ) elif 36 <= r < 73: elt_1( ) elif 73 <= r < 109: elt_2( ) elif 109 <= r <146: elt_3( ) elif 146 <= r < 182: elt_4( ) elif 182 <= r < 219: elt_5( ) else: elt_6( ) def elt_0(rops) : r = next(rops ) if 0 <= r < 43: expr_0( ) elif 43 <= r < 85: expr_1( ) elif 85 <= r < 128: expr_2( ) elif 128 <= r < 171: expr_3( ) elif 171 <= r < 213: expr_4( ) else: expr_5( ) print('+' ) elif 171 <= r < 213: expr_4( ) else: expr_5() Grammar VM (F1) 79 {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 }
  80. 80 Focused Fuzzing Generate inputs that focus on speci fi

    c system parts • Covering speci fi c portions of code • Speci fi c paths through the code • Performing speci fi c functions Without damaging the nondeterminism. 80
  81. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} ✓ 81
  82. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ✘ 82
  83. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ✓ ["A", "B", "C"] 83
  84. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ["A", "B", "C"] [{"": [1,2,3,4]}] ✘ 84
  85. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ["A", "B", "C"] [{"": [1,2,3,4]}] if json.has_key("") : raise Exception() 85
  86. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," ✘ 86
  87. {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971

    7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": What is the smallest failure inducing input? Delta Debugging 87
  88. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]`| `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> Hierarchical Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": 88
  89. {"":[]} ✘ 89 Test Minimization {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi

    llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": Hierarchical Delta Debugging 89
  90. 90 Why? [12345] {"":[]} {"":0} {"x":[]} {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000","

    fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": Hierarchical Delta Debugging 90
  91. DDSET Gopinath, Kampmann, Havrikov, Soremekun, and Zeller. Abstracting Failure Inducing

    Inputs. ISSTA 2020. https://github.com/vrthra/ddset 91
  92. {"": []} DDSET: <json> ::= <elt>
 <elt> ::= <object>
 |

    <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> 92
  93. {"": []} 122489 {"A":{}, {"23": {"P":[]}}] [[], [[[]],[]],{"A":{}, {"23": {"P":[]}}]

    <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 93
  94. {"": []} "XYZR389" {"A":{}, {"23": {"P":[]}}} [[], [[[]],[]],{"A":{}, {"23": {"P":[]}}]

    <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 94
  95. {"": []} [true, null] {"A":{}, {"23": {"P":[]}}} [[], [[[]],[]],{"A":{}, {"23":

    {"P":[]}}] <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 95
  96. {"": []} {"__": [[]]} {"?P":[{}], {"|": {"":[]}}} {"X":[[],[]],{"A":{}, {"2": {"R":[]}}}

    <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 96
  97. {"": []} {"": [[]]} {"?P":[{}], {"|": {"P":[]}}} {"X":[[],[]],{"A":{}, {"2": {"R":[]}}}

    97 <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  98. {"": []} {"7897A": []} {"klnm,.qer;dfs?P":[]} {"123KOUIJ!qR30578950":[]} 98 <json> ::= <elt>


    <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  99. {"": []} {"": true} {"":[1,2,445,"x"]} {"":{"PQ":[true, false, 223,"a"]}} 99 <json>

    ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  100. {"": []} Abstraction {"": <elt>} Abstract Input 100 <json> ::=

    <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  101. {"": <elt>} Abstract Input {"": []} Minimized Input 101 <json>

    ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
  102. Issue 386 from Rhino var A = class extends (class

    {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure <varModifier> <Identifier> = class extends (class {}){} var {<$Id1>:{} = <$Id1> => {}} <variableDeclaration>; const [<$Id1>,<$Id1>] = [] {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} 102
  103. {"": <elt>} Abstract Input {"": []} Minimized Input 103 <json>

    ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Pattern Evocative Grammar
  104. 104 {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[],

    {"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Grammar
  105. 105 <json E> where <item E> is "":<elt> 1. We

    can produce any and all instances of the failure inducing pattern . 2. We can recognize any input that contains the failure inducing pattern . 3. The grammar will reject any input that doesn't contain the failure inducing pattern. <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] Evocative Grammar
  106. <json E> where <item E> is "":<elt> if json.has_key("") :

    raise Exception() if json.has_key_value(null) : raise Exception() <json N> where <item N> is <string>: null 106 Combining Evocative Patterns
  107. if json.has_key("") and json.has_key_value(null) : raise Exception() <json E &

    N> where <item E> is "":<elt> 
 <item N> is <string>: null 107 Combining Evocative Patterns
  108. if json.has_key("") and not json.has_key_value(null) : raise Exception() <json E

    & not(N)> where <item E> is "":<elt> <item N> is <string>: null 108 Combining Evocative Patterns
  109. if json.has_key("") : raise Exception( ) if json.has_key_value(null) : raise

    Exception() <json not(E) & not(N)> where <item E> is "":<elt> <item N> is <string>: null 109 Combining Evocative Patterns
  110. <json E&N> := <elt E&N > <elt E&N> := <object

    E&N > | <array E&N > <array E&N>:= '[' <elts E&N> '] ' <object E&N>:= '{' <items E&N> '} ' <elts E&N> := <elt E&N > | <elt E&N>','<elts N > | <elt N>','<elts E&N > <items E&N> := <item E&N > | <item E&N>','<items N > | <item N>','<items E&N > <item E&N> := <string E1>':'<elt N&N1 > | <string>':'<elt E&N&N1 > <elt E&N&N1> := <object E&N> <array E&N > <elt N> := 'false' | 'true ' | <number> | <string > | <object N> <array N > <array N> := '[]' | '[' <elts N> '] ' <object N> := '{}' | '{' <items N> '} ' <elts N> := <elt N > | <elt N>','<elts N > <items N> := <item N > | <item N>','<items N > <item N> := <string>':'<elt N&N1 > <elt N&N1> := 'false' | 'true ' | <number> | <string > | <object N> | <array N> <json E & N> where <item E> is "":<elt> <item N> is <string>: null {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[], {"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E&N>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ 110
  111. Issue 386 from Rhino var A = class extends (class

    {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure <varModifier> <Identifier> = class extends (class {}){} var {<$Id1>:{} = <$Id1> => {}} <variableDeclaration>; const [<$Id1>,<$Id1>] = [] {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} 111
  112. where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class

    {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Evocative Expressions <JavaScript C2937 and C2842> <JavaScript R385 and R386> <JavaScript (C2937 or C2842) and (R385 or R386)> <JavaScript not(C2937 or C2842 or R385 or R386)> 112
  113. 113 Evocative Expressions: Validating Inputs where <variableDeclarationList C2937> is <varModifier>

    <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] <JavaScript not(C2937 or C2842 or R385 or R386)>
  114. Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB )

    self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) 114
  115. 115 <json ADM> where <item ADM> is "role": "admin" def

    assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"role" : "admin"} Evocative Expressions for Focused Fuzzing
  116. 116 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

    self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"method":"remove_table","args":["orders", "inventory"]} <json DBDT> where <items DBDT> is "method":"remove_table","args":<elt> Evocative Expressions for Focused Fuzzing
  117. 117 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

    self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"], "role":"admin"} {"role":"admin", "method":"remove_table","args":["orders", "inventory"]} {"method":"remove_table","role":"admin","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"admin"}]} Evocative Expressions for Focused Fuzzing
  118. 118 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

    self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"], "role":"guest"} {"role":"guest", "method":"remove_table","args":["orders", "inventory"]} {"method":"remove_table","role":"guest","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"guest"}]} Evocative Expressions for Focused Fuzzing
  119. Algebraic Data Types 119 <my_struct> ::= <stype> <stype> struct mystruct

    { stype m1 ; stype m2 ; }; union myunion { utype m1 ; utype m2 ; }; <my_union> ::= <utype> | <utype> Data Structures Context Free Grammar Evocative Expressions: Data Structures
  120. Search Source Code For Fault Patterns • Structure aware •

    Semantics can be mined from bugs! • No need to specify by hand • Combine different patterns using `or`, `and` and `not` 120 Evocative Expressions: Semantic Search
  121. Gopinath, Nemati, Zeller. Input Algebras. ICSE 2021. Evocative Expressions https://rahul.gopinath.org/posts/

    121
  122. if json.has_key("") : raise Exception() if not json.has_key_value(null) : raise

    Exception() 122 if json.has_key("") and not json.has_key_value(null) : raise Exception() Future: Decomposition with DDSet 122
  123. 123 123

  124. Generating Unbiased Samples 124

  125. Finding Good Samples Seed corpus? (Blind spots) 125

  126. 126 • Differentiate incomplete and incorrect input s • Solve

    one character at a time systematically Key Idea (Monotonic Failure Property) 126
  127. 127 Sample Free Generators A ( 2 - B 9

    ) 4 ) A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0 B ∉ +,-,1,2,3,4,5,6,7,8,9,0,) ) ∉ +,-,1,2,3,4,5,6,7,8,9,0 (2-94) 127
  128. 128 Mathis, Gopinath, Mera, Kampmann, Höschele, and Zeller. Parser Directed

    Fuzzing. PLDI 2019 . Mathis, Gopinath and Zeller Learning Input Tokens for Effective Fuzzing. ISSTA 2020. Sample Free Generators A ( 2 - B 9 ) 4 )
  129. 129 Blackbox Sample Free Generators • Monotonic Failure Propert y

    • Differentiate incomplete and incorrect input s • Solve one character at a time systematically Bendrissou, Gopinath, Mathis, and Zeller Failure Feedback for Fast Fuzzing. (arXiv). 129
  130. 130

  131. 131 HTTP Parser XML Parser SOAP Parser RPC Parser C

    Parser Check Declarations Check Types Static Checks Compilers Servers Semantics Application Layered Programs 131
  132. 132

  133. 133 QS: Computer Science 22 in the world One in

    the Group of Eight (Go8) in Australia
  134. 134

  135. 135 Mining Evocative Expressions Blakbox Generation