Upgrade to PRO for Only $50/Year—Limited-Time Offer! 🔥

Building Blocks for Fuzzing

Building Blocks for Fuzzing

Talk at IIT-Gandhinagar

Rahul Gopinath

July 19, 2022
Tweet

More Decks by Rahul Gopinath

Other Decks in Research

Transcript

  1. 9

  2. 10 Input ✓ ✘ Testing @app.route('/admin') def admin(): username =

    request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500
  3. HTTP/1.1 401 Not Authorize d Content-Type: application/jso n { "Error"

    : "Assignment of internal rol e 'superadmin' is forbidden " } HTTP/1.1 200 O K Content-type: application/jso n { "result": "OK: Updated use r 'exampleUser' with rol e 'superadmin' " } 11 Input ✓ ✘ Testing POST /user/update HTTP/1. 1 { "user": "exampleUser", "roles": [ "superadmin " ] } @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 HTTP/1.1 500 Internal Server Error ✘
  4. 12 Verify Behavior Input ✓ ✘ Automatic Testing @app.route('/admin') def

    admin(): username = request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500
  5. 13 (Oracle) Input ✓ ✘ Automatic Testing Oracles require domain

    expertise @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 Verify Behavior
  6. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 Fuzzing Trash deck technique: 1950s - Gerald Weinberg Crash? 14
  7. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 Fuzzing Crash? • Memory Bounds Violation • Privilege Escalation • Safety Violations • Metamorphic Relations • Differential Execution ASAN,MSAN,TSAN,NSan,FuZZan,UBSan 15
  8. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 [ ; x 1 - G P Z + w c c k c ] ; , N 9 J + ? # 6 ^ 6 \ e ? ] 9 l u 2 _ % ' 4 G X " 0 V U B [ E / r ~ f A p u 6 b 8 < { % s i q 8 Z h . 6 { V , h r ? ; {Ti.r3PIxMMMv6{xS^+'Hq!AxB"YXRS@! Kd6;wtAMefFWM(`|J_<1~o}z3K(CCzRH J I I v H z > _ * . \ > J r l U 3 2 ~ e G P ? lR=bF3+;y$3lodQ<B89!5"W2fK*vE7v{')KC- i,c{<[~m!]o;{.'}Gj\(X}EtYetrpbY@aGZ1{P! A Z U 7 x # 4 ( R t n ! q 4 n C w q o l ^ y 6 } 0 | Ko=*JK~;zMKV=9Nai:wxu{J&UV#HaU)*Bi C < ) , ` + t * g k a < W = Z . % T 5 W G H Z p I 3 0 D < P q > & ] B S 6 R & j ? # t P 7 i a V } - } ` \ ? [ _ [ Z ^ L B M P G - FKj'\xwuZ1=Q`^`5,$N$Q@[!CuRzJ2D|vBy! ^ z k h d f 3 C 5 P A k R ? V ( ( - % > < h n | 3='i2Qx]D$qs4O`1@fevnG'2\11Vf3piU37@ 5 : d f d 4 5 * ( 7 ^ % 5 a p \ z I y l " ' f , $ee,J4Gw:cgNKLie3nx9(`efSlg6#[K"@Wjh Z}r[Scun&sBCS,T[/3]KAeEnQ7lU)3Pn,0)G/ 6N-wyzj/MTd#A;r Program Traditional Fuzzing 17 https://www.fuzzingbook.org/html/Fuzzer.html
  9. @app.route('/admin') def admin(): username = request.cookies.get("username") if not username: return

    {"Error": "Specify username in Cookie"} username = urllib.quote(os.path.basename(username)) url = "http://permissions:5000/permissions/{}".format(username) resp = requests.request(method="GET", url=url) # "superadmin\ud888" will be simpli fi ed to "superadmin" ret = ujson.loads(resp.text) if resp.status_code == 200: if "superadmin" in ret["roles"]: return {"OK": "Superadmin Access granted"} else: e = u"Access denied. User has following roles: {}".format(ret["roles"]) return {"Error": e}, 401 else:return {"Error": ret["Error"]}, 500 [ ; x 1 - G P Z + w c c k c ] ; , N 9 J + ? # 6 ^ 6 \ e ? ] 9 l u 2 _ % ' 4 G X " 0 V U B [ E / r ~ f A p u 6 b 8 < { % s i q 8 Z h . 6 { V , h r ? ; {Ti.r3PIxMMMv6{xS^+'Hq!AxB"YXRS@! Kd6;wtAMefFWM(`|J_<1~o}z3K(CCzRH J I I v H z > _ * . \ > J r l U 3 2 ~ e G P ? lR=bF3+;y$3lodQ<B89!5"W2fK*vE7v{')KC- i,c{<[~m!]o;{.'}Gj\(X}EtYetrpbY@aGZ1{P! A Z U 7 x # 4 ( R t n ! q 4 n C w q o l ^ y 6 } 0 | Ko=*JK~;zMKV=9Nai:wxu{J&UV#HaU)*Bi C < ) , ` + t * g k a < W = Z . % T 5 W G H Z p I 3 0 D < P q > & ] B S 6 R & j ? # t P 7 i a V } - } ` \ ? [ _ [ Z ^ L B M P G - FKj'\xwuZ1=Q`^`5,$N$Q@[!CuRzJ2D|vBy! ^ z k h d f 3 C 5 P A k R ? V h n | 3='i2Qx]D$qs4O`1@fevnG'2\11Vf3piU37@ 5 5 a p \ z I y l " ' f , $ee,J4Gw:cgNKLie3nx9(`efSlg6#[K"@Wjh Z}r[Scun&sBCS,T[/3]KAeEnQ7lU)3Pn,0)G/ 6N-wyzj/MTd#A;r Structured Inputs SYNTAX ERROR ✘ 18
  10. def process_input(input) : try : ✘val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r SYNTAX ERROR 19 Parser
  11. SYNTAX ERROR def process_input(input) : try : ✘val = parse(input

    ) res = process(val ) return re s except SyntaxError : return Erro r 20 The Core
  12. 21 def process_input(input) : try : val = parse(input )

    res = process(val ) return re s except SyntaxError : return Erro r SYNTAX ERROR 21
  13. • Insert Instrumentatio n • Generate input s • Collect

    execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Feedback Driven Fuzzing 22 22 https://www.fuzzingbook.org/html/MutationFuzzer.html
  14. 23 def triangle(a, b, c): if a == b: if

    b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene def triangle(a, b, c): __probe_enter() if a == b: __probe_1() if b == c: __probe_2() return Equilateral else: __probe_3() return Isosceles else: __probe_4() if b == c: __probe_5() return Isosceles else: __probe_6() if a == c: __probe_7() return Isosceles else: __probe_8() return Scalene def triangle(a, b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene Feedback Driven Fuzzing • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 23 https://www.fuzzingbook.org/html/MutationFuzzer.html
  15. 24 Feedback Driven Fuzzing triangle (1,1,1) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 24 https://www.fuzzingbook.org/html/MutationFuzzer.html
  16. 25 Feedback Driven Fuzzing triangle (1,1,1) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 25 https://www.fuzzingbook.org/html/MutationFuzzer.html
  17. triangle (1,1,1) 26 Feedback Driven Fuzzing triangle (1,1,2) def triangle(a,

    b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Mutated 26 https://www.fuzzingbook.org/html/MutationFuzzer.html
  18. 27 Feedback Driven Fuzzing triangle (1,1,3) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage Mutated 27 https://www.fuzzingbook.org/html/MutationFuzzer.html
  19. 28 Feedback Driven Fuzzing triangle (1,1,2) def triangle(a, b, c):

    if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene • Insert Instrumentatio n • Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage 28 https://www.fuzzingbook.org/html/MutationFuzzer.html triangle (1,1,1) triangle (1,1,3)
  20. 30 Feedback Driven Fuzzing Weakness: static int is_reserved_word_token(const char *s,

    int len) { const char *reserved[] = { "break", "case", "catch", "continue", "debugger", "default", "delete", "do", "else", "false", "finally", "for", "function", "if", "in", "instanceof", "new", "null", "return", "switch", "this", "throw", "true", "try", "typeof", "var", "void", "while", "with", "let", "undefined", ((void *)0)}; int i; if (!mjs_is_alpha(s[0])) return 0; for (i = 0; reserved[i] != ((void *)0); i++) { if (len == (int)strlen(reserved[i]) && strncmp(s, reserved[i], len) == 0) return i + 1; } return 0; } Tokens if (x > 100) { } coverage: 20% if (x > 100) { } e coverage: 5% if (x > 100) { } el coverage: 5% if (x > 100) { } els coverage: 5% if (x > 100) { } else coverage: 25% No smooth coverage gradient in parsers 30
  21. 31 Feedback Driven Fuzzing def json_raw(stm) : while True :

    stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 31
  22. 32 Feedback Driven Fuzzing def json_raw(stm) : while True :

    stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 32
  23. 33 Feedback Driven Fuzzing def json_raw(stm) : while True :

    stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) Weak points : • Need for smooth coverage gradien t • Coverage only provides first level guidance 1. {"abc":[]} 2. [{"a":[]}, {"b":[]}, {"c":["ab","c"]}] 33
  24. 34 Feedback Driven Fuzzing 34 • Insert Instrumentatio n •

    Generate input s • Collect execution feedbac k • Branches covered during executio n • Slightly Mutate Input and try agai n Collect inputs obtaining new coverage
  25. 35 Solver Directed Fuzzing • Collect path constraint s •

    Solve negated constraints for new inputs (a == b) (b == c) (b != c) triangle(1,2,1) https://www.fuzzingbook.org/html/ConcolicFuzzer.html 35
  26. 36 Solver Directed Fuzzing • Collect path constraint s •

    Solve constraints for new inputs void next_sym() { while(!eof) { switch (ch){ case '{': next_ch(); sym = LBRA; return; case '}': next_ch(); sym = RBRA; return; case '(': next_ch(); sym = LPAR; return; case ')': next_ch(); sym = RPAR; return; case '+': next_ch(); sym = PLUS; return; case '-': next_ch(); sym = MINUS; return; case '<': next_ch(); sym = LESS; return; case ';': next_ch(); sym = SEMI; return; case '=': next_ch(); sym = EQUAL; return; default: if (ch >= '0' && ch <= '9') { int_val = 0; /* missing overflow check */ while (ch >= '0' && ch <= '9') { int_val = int_val*10 + (ch - '0'); next_ch(); } sym = INT; } else if (ch >= 'a' && ch <= 'z') { int i = 0; /* missing overflow check */ while ((ch >= 'a' && ch <= 'z') || ch == '_'){ id_name[i++] = ch; next_ch(); } id_name[i] = '\0'; sym = 0; while (words[sym] != NULL && strcmp(words[sym], id_name) != 0) sym++; if (words[sym] == NULL) if (id_name[1] == '\0') sym = ID; else syntax_error(); } else syntax_error(); return; } Weak point: Path explosion https://www.fuzzingbook.org/html/ConcolicFuzzer.html 36
  27. 37 Solver Directed Fuzzing • Collect path constraint s •

    Solve negated constraints for new inputs 37
  28. 41 Formal Languages Language Descriptions: Grammars 3. Regular Context Free

    Recursively Enumerable (Chomsky,1956) Easy to produce and parse Argument Stack Return Stack 41
  29. 42 Grammar JSON grammar De f inition for <elt> <elt>

    key <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> 42
  30. 43 Grammar JSON grammar <json> ::= <elt > 
 <elt>

    ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> Expansion Rule Terminal Symbol Nonterminal Symbol 43
  31. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Tree 44
  32. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <json> Tree <elt> 45
  33. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <elt> <object> Tree 46
  34. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] <object> { <items> } Tree 47
  35. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <items> } { <item> } Tree 48
  36. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <item> } { <string> : <elt> } Tree 49
  37. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { <string> : <elt> } { "" : <elt> } Tree 50
  38. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { "" : <elt> } Tree 51
  39. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] { "" : <elt> } { "" : true } Tree 52
  40. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] {"":true} Derivation Tree 53
  41. {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

    , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } Grammar Fuzzer 54
  42. General Parser {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>']

    , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } 55
  43. {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

    , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } Lang Fuzzer https://www.fuzzingbook.org/html/LangFuzzer.html 56
  44. def process_input(input) : try : ✔val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } 57
  45. QUIRK_ALLOW_ASCII_CONTROL_CODES QUIRK_ALLOW_BACKSLASH_A QUIRK_ALLOW_BACKSLASH_CAPITAL_U QUIRK_ALLOW_BACKSLASH_E QUIRK_ALLOW_BACKSLASH_NEW_LINE QUIRK_ALLOW_BACKSLASH_QUESTION_MARK QUIRK_ALLOW_BACKSLASH_SINGLE_QUOTE QUIRK_ALLOW_BACKSLASH_V QUIRK_ALLOW_BACKSLASH_X_AS_BYTE S

    QUIRK_ALLOW_BACKSLASH_X_AS_CODE_POINTS QUIRK_ALLOW_BACKSLASH_ZERO QUIRK_ALLOW_COMMENT_BLOCK QUIRK_ALLOW_COMMENT_LINE QUIRK_ALLOW_EXTRA_COMMA QUIRK_ALLOW_INF_NAN_NUMBERS QUIRK_ALLOW_LEADING_ASCII_RECORD_SEPARATOR QUIRK_ALLOW_LEADING_UNICODE_BYTE_ORDER_MARK QUIRK_ALLOW_TRAILING_FILLER QUIRK_EXPECT_TRAILING_NEW_LINE_OR_EOF QUIRK_JSON_POINTER_ALLOW_TILDE_N_TILDE_R_TILDE_T QUIRK_REPLACE_INVALID_UNICODE JSON common quirks from https://github.com/google/wuffs 62
  46. "Be liberal in what you accept, and conservative in what

    you send"
 Postel's Law The Specification The Implementation Extra "Features" Where to Get the Grammar From? 63
  47. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw>::= <json_object>
 | <json_list>
 | <json_string>
 | <json_number>
 | <json_fixed(true)> | <json_fixed(false) > | <json_fixed(null) > <json_string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char > <json_object> ::= `{`<items>`} ` | `{} ` <items> ::= <item>`,`<items > | <item > <item> ::= <string>`:`<elt > <json_list> ::= `[`<elts>`] ` | `[] ` <elts> ::= <elt>`,`<elts > | <elt > <json_number> ::= <digits > <digits> ::= <digit><digits> | <digit> https://github.com/phensley/microjson MicroJSON 65 65
  48. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw>::= <json_object>
 | <json_list>
 | <json_string>
 | <json_number>
 | <json_fixed(true)> | <json_fixed(false) > | <json_fixed(null)> <json_string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <json_object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <json_list> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <json_number> ::= <digits > <digits> ::= <digit><digits> | <digit> https://github.com/phensley/microjson MicroJSON 66
  49. <F> := <A > | <B> <F> := <A> <B>

    <C> <Fs> := <B> <Fs > | <empty> Structured Control Flow to Grammar Sequence A B C [F] Selection cond A B [F] F T Iteration cond B [F] 67
  50. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_object(stm ) elif c == '[': return json_array(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw> ::= <loop1> <loop1> ::= <iter1> <loop1 > | <empty > <iter1>::= <json_fixed(true) > | <json_fixed(false) > | <json_fixed(null) > 
 | `"` <json_string > | `{` <json_object > 
 | `[` <json_array > 
 | [[1-9] <json_number>
 68
  51. def json_raw(stm) : while True : stm.skipspaces( ) c =

    stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_object(stm ) elif c == '[': return json_array(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) <json_raw> ::= <loop1> <loop1> ::= <iter1> <loop1 > | <empty > <iter1>::= <json_fixed(true) > | <json_fixed(false) > | <json_fixed(null) > 
 | `"` <json_string > | `{` <json_object > 
 | `[` <json_array > 
 | [[1-9] <json_number>
 69
  52. def json_string(stm) : # skip over '" ' stm.next() r

    = [ ] while True : c = stm.next( ) if c == '' : raise JSONError(E_TRUNC ) elif c == '\\' : c = stm.next( ) r.append(decode_escape(c, stm) ) elif c == '"' : return ''.join(r ) else : r.append(c) <json_raw> ::= <loop1> ... <iter1>::= `"` <json_string > .. . <json_string> ::= <loop2 > <loop2> ::= <iter2> <loop2 > | <empty > <iter2>::= `\\` <decode_escape > | `" ` | <other_char> "ab" <json_string> ::= <loop2> <loop2_end > <loop2> ::= <iter2> <loop2 > | <empty > <iter2>::= `\\` <decode_escape > | <other_char > <loop2_end> ::= <iter2_end > <iter2_end>::=`"` 70
  53. 71 <START> ::= <json_raw > <json_raw> ::= '"' <json_string' >

    | '[' <json_list' > | '{' <json_dict' > | <json_number' > | 'true ' | 'false ' | 'null ' <json_number'> ::= <json_number> + | <json_number>+ 'e' <json_number> + <json_number> ::= '+' | '-' | '.' | [0-9] | 'E' | 'e ' <json_string'> ::= <json_string>* '" ' <json_list'> ::= '] ' | <json_raw> (','<json_raw>)* '] ' | ( ',' <json_raw>)+ (',' <json_raw>)* '] ' <json_dict'> ::= '} ' | ( '"' <json_string'> ':' <json_raw> ',' )* '"'<json_string'> ':' <json_raw> '} ' <json_string> ::= ' ' | '!' | '#' | '$' | '%' | '&' | '' ' | '*' | '+' | '-' | ',' | '.' | '/' | ':' | '; ' | '<' | '=' | '>' | '?' | '@' | '[' | ']' | '^ ' | '_', ''',| '{' | '|' | '}' | '~ ' | '[A-Za-z0-9] ' | '\' <decode_escape > <decode_escape> ::= '"' | '/' | 'b' | 'f' | 'n' | 'r' | 't' stm.next()
  54. Recall Subjects Mimid calc.py 100.0 % mathexpr.py 87.5 % cgidecode.py

    100.0 % urlparse.py 100.0 % microjson.py 98.7 % parseclisp.py 99.3 % jsonparser.c 100.0 % tiny.c 100.0 % mjs.c 95.4 % Inputs generated by inferred grammar that were accepted by the program Subjects Mimid calc.py 100.0 % mathexpr.py 92.7 % cgidecode.py 100.0 % urlparse.py 96.4 % microjson.py 93.0 % parseclisp.py 80.6 % jsonparser.c 83.8 % tiny.c 92.8 % mjs.c 95.9 % Inputs generated by golden grammar that were accepted by the inferred grammar parser Precision Evaluation: Accuracy 72 72
  55. 74

  56. 75 HTTP Parser XML Parser SOAP Parser RPC Parser C

    Parser Check Declarations Check Types Static Checks Challenges Compilers Servers Semantics Application
  57. {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>']

    , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'], ['{}']] , '<items>' : [['<item>,',',<items>'], ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'], ['[]']] , '<elts>' : [['<elt>,',',<elts>'], ['<elt>']] , '<string>' : [['"', '<chars>', '"'], ['""']] , '<chars>' : [['<char>','<chars>'], ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'], ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } def process_input(input) : try : ✔val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r Fuzz Programs? 78
  58. Grammar Fuzzer Fast Fuzzers def json() : elt( ) def

    elt() : match (random() % 7) : case 0: object( ) case 1: array( ) case 2: string( ) case 3: number( ) case 4: print('true' ) case 5: print('false' ) case 5: print('null')
 def object() : match (random() % 2) : case 0: print('{'); items(); print('}' ) case 1: print('{');print('}' ) def array() : match (random() % 2) : case 0: print('['); elts(); print(']' ) case 1: print('[');print(']' ) def items() : match (random() % 2) : case 0: item(); print(','); items( ) case 1: item() Compiled Grammar (F1) Building Fast Fuzzers Gopinath and Zeller 2019 def json_0(rops) : elt_0(rops ) def elt_0(rops) : r = next(rops ) if 0 <= r < 36: elt_0( ) elif 36 <= r < 73: elt_1( ) elif 73 <= r < 109: elt_2( ) elif 109 <= r <146: elt_3( ) elif 146 <= r < 182: elt_4( ) elif 182 <= r < 219: elt_5( ) else: elt_6( ) def elt_0(rops) : r = next(rops ) if 0 <= r < 43: expr_0( ) elif 43 <= r < 85: expr_1( ) elif 85 <= r < 128: expr_2( ) elif 128 <= r < 171: expr_3( ) elif 171 <= r < 213: expr_4( ) else: expr_5( ) print('+' ) elif 171 <= r < 213: expr_4( ) else: expr_5() Grammar VM (F1) 79 {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 }
  59. 80 Focused Fuzzing Generate inputs that focus on speci fi

    c system parts • Covering speci fi c portions of code • Speci fi c paths through the code • Performing speci fi c functions Without damaging the nondeterminism. 80
  60. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} ✓ 81
  61. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ✘ 82
  62. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ✓ ["A", "B", "C"] 83
  63. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ["A", "B", "C"] [{"": [1,2,3,4]}] ✘ 84
  64. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"a": ["key"]} {"": [1,2,"k"]} ["A", "B", "C"] [{"": [1,2,3,4]}] if json.has_key("") : raise Exception() 85
  65. def process_input(input) : try : val = parse(input ) res

    = process(val ) return re s except SyntaxError : return Erro r {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," ✘ 86
  66. {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971

    7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": What is the smallest failure inducing input? Delta Debugging 87
  67. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>


    | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]`| `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> Hierarchical Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": 88
  68. {"":[]} ✘ 89 Test Minimization {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi

    llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": Hierarchical Delta Debugging 89
  69. 90 Why? [12345] {"":[]} {"":0} {"x":[]} {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000","

    fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": Hierarchical Delta Debugging 90
  70. {"": []} DDSET: <json> ::= <elt>
 <elt> ::= <object>
 |

    <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> 92
  71. {"": []} 122489 {"A":{}, {"23": {"P":[]}}] [[], [[[]],[]],{"A":{}, {"23": {"P":[]}}]

    <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 93
  72. {"": []} "XYZR389" {"A":{}, {"23": {"P":[]}}} [[], [[[]],[]],{"A":{}, {"23": {"P":[]}}]

    <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 94
  73. {"": []} [true, null] {"A":{}, {"23": {"P":[]}}} [[], [[[]],[]],{"A":{}, {"23":

    {"P":[]}}] <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 95
  74. {"": []} {"__": [[]]} {"?P":[{}], {"|": {"":[]}}} {"X":[[],[]],{"A":{}, {"2": {"R":[]}}}

    <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET: 96
  75. {"": []} {"": [[]]} {"?P":[{}], {"|": {"P":[]}}} {"X":[[],[]],{"A":{}, {"2": {"R":[]}}}

    97 <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  76. {"": []} {"7897A": []} {"klnm,.qer;dfs?P":[]} {"123KOUIJ!qR30578950":[]} 98 <json> ::= <elt>


    <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  77. {"": []} {"": true} {"":[1,2,445,"x"]} {"":{"PQ":[true, false, 223,"a"]}} 99 <json>

    ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  78. {"": []} Abstraction {"": <elt>} Abstract Input 100 <json> ::=

    <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> DDSET:
  79. {"": <elt>} Abstract Input {"": []} Minimized Input 101 <json>

    ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
  80. Issue 386 from Rhino var A = class extends (class

    {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure <varModifier> <Identifier> = class extends (class {}){} var {<$Id1>:{} = <$Id1> => {}} <variableDeclaration>; const [<$Id1>,<$Id1>] = [] {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} 102
  81. {"": <elt>} Abstract Input {"": []} Minimized Input 103 <json>

    ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Pattern Evocative Grammar
  82. 104 {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[],

    {"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Grammar
  83. 105 <json E> where <item E> is "":<elt> 1. We

    can produce any and all instances of the failure inducing pattern . 2. We can recognize any input that contains the failure inducing pattern . 3. The grammar will reject any input that doesn't contain the failure inducing pattern. <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] Evocative Grammar
  84. <json E> where <item E> is "":<elt> if json.has_key("") :

    raise Exception() if json.has_key_value(null) : raise Exception() <json N> where <item N> is <string>: null 106 Combining Evocative Patterns
  85. if json.has_key("") and json.has_key_value(null) : raise Exception() <json E &

    N> where <item E> is "":<elt> 
 <item N> is <string>: null 107 Combining Evocative Patterns
  86. if json.has_key("") and not json.has_key_value(null) : raise Exception() <json E

    & not(N)> where <item E> is "":<elt> <item N> is <string>: null 108 Combining Evocative Patterns
  87. if json.has_key("") : raise Exception( ) if json.has_key_value(null) : raise

    Exception() <json not(E) & not(N)> where <item E> is "":<elt> <item N> is <string>: null 109 Combining Evocative Patterns
  88. <json E&N> := <elt E&N > <elt E&N> := <object

    E&N > | <array E&N > <array E&N>:= '[' <elts E&N> '] ' <object E&N>:= '{' <items E&N> '} ' <elts E&N> := <elt E&N > | <elt E&N>','<elts N > | <elt N>','<elts E&N > <items E&N> := <item E&N > | <item E&N>','<items N > | <item N>','<items E&N > <item E&N> := <string E1>':'<elt N&N1 > | <string>':'<elt E&N&N1 > <elt E&N&N1> := <object E&N> <array E&N > <elt N> := 'false' | 'true ' | <number> | <string > | <object N> <array N > <array N> := '[]' | '[' <elts N> '] ' <object N> := '{}' | '{' <items N> '} ' <elts N> := <elt N > | <elt N>','<elts N > <items N> := <item N > | <item N>','<items N > <item N> := <string>':'<elt N&N1 > <elt N&N1> := 'false' | 'true ' | <number> | <string > | <object N> | <array N> <json E & N> where <item E> is "":<elt> <item N> is <string>: null {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[], {"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E&N>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ 110
  89. Issue 386 from Rhino var A = class extends (class

    {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure <varModifier> <Identifier> = class extends (class {}){} var {<$Id1>:{} = <$Id1> => {}} <variableDeclaration>; const [<$Id1>,<$Id1>] = [] {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} 111
  90. where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class

    {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Evocative Expressions <JavaScript C2937 and C2842> <JavaScript R385 and R386> <JavaScript (C2937 or C2842) and (R385 or R386)> <JavaScript not(C2937 or C2842 or R385 or R386)> 112
  91. 113 Evocative Expressions: Validating Inputs where <variableDeclarationList C2937> is <varModifier>

    <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] <JavaScript not(C2937 or C2842 or R385 or R386)>
  92. Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB )

    self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) 114
  93. 115 <json ADM> where <item ADM> is "role": "admin" def

    assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"role" : "admin"} Evocative Expressions for Focused Fuzzing
  94. 116 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

    self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"method":"remove_table","args":["orders", "inventory"]} <json DBDT> where <items DBDT> is "method":"remove_table","args":<elt> Evocative Expressions for Focused Fuzzing
  95. 117 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

    self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"], "role":"admin"} {"role":"admin", "method":"remove_table","args":["orders", "inventory"]} {"method":"remove_table","role":"admin","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"admin"}]} Evocative Expressions for Focused Fuzzing
  96. 118 def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W

    self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"], "role":"guest"} {"role":"guest", "method":"remove_table","args":["orders", "inventory"]} {"method":"remove_table","role":"guest","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"guest"}]} Evocative Expressions for Focused Fuzzing
  97. Algebraic Data Types 119 <my_struct> ::= <stype> <stype> struct mystruct

    { stype m1 ; stype m2 ; }; union myunion { utype m1 ; utype m2 ; }; <my_union> ::= <utype> | <utype> Data Structures Context Free Grammar Evocative Expressions: Data Structures
  98. Search Source Code For Fault Patterns • Structure aware •

    Semantics can be mined from bugs! • No need to specify by hand • Combine different patterns using `or`, `and` and `not` 120 Evocative Expressions: Semantic Search
  99. if json.has_key("") : raise Exception() if not json.has_key_value(null) : raise

    Exception() 122 if json.has_key("") and not json.has_key_value(null) : raise Exception() Future: Decomposition with DDSet 122
  100. 126 • Differentiate incomplete and incorrect input s • Solve

    one character at a time systematically Key Idea (Monotonic Failure Property) 126
  101. 127 Sample Free Generators A ( 2 - B 9

    ) 4 ) A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0 B ∉ +,-,1,2,3,4,5,6,7,8,9,0,) ) ∉ +,-,1,2,3,4,5,6,7,8,9,0 (2-94) 127
  102. 128 Mathis, Gopinath, Mera, Kampmann, Höschele, and Zeller. Parser Directed

    Fuzzing. PLDI 2019 . Mathis, Gopinath and Zeller Learning Input Tokens for Effective Fuzzing. ISSTA 2020. Sample Free Generators A ( 2 - B 9 ) 4 )
  103. 129 Blackbox Sample Free Generators • Monotonic Failure Propert y

    • Differentiate incomplete and incorrect input s • Solve one character at a time systematically Bendrissou, Gopinath, Mathis, and Zeller Failure Feedback for Fast Fuzzing. (arXiv). 129
  104. 130

  105. 131 HTTP Parser XML Parser SOAP Parser RPC Parser C

    Parser Check Declarations Check Types Static Checks Compilers Servers Semantics Application Layered Programs 131
  106. 132

  107. 133 QS: Computer Science 22 in the world One in

    the Group of Eight (Go8) in Australia
  108. 134