Input Languages for Effective and Focused Fuzzing

FuzzCon Europe 2021

Transcript

1. Rahul Gopinath https://rahul.gopinath.org @_rahulgopinath Rafael Dutra https://cispa.de/en/people/c01radu/ 1 Input Languages for 
 Effective and Focused Fuzzing
2. Rahul Gopinath https://rahul.gopinath.org @_rahulgopinath Rafael Dutra https://cispa.de/en/people/c01radu/ FuzzCon Europe 2021 1 Input Languages for 
 Effective and Focused Fuzzing
3. 2 2 Feedback-Directed Fuzzers: Challenges file.png AFL readpng PNG corpus save inputs that increase coverage
4. 3 3 Feedback-Directed Fuzzers: Challenges file.png AFL readpng PNG corpus save inputs that increase coverage Most mutations make the file invalid Hard to find deeper semantic bugs
5. 4 4 Feedback-Directed Fuzzers: Challenges file.png AFL readpng PNG corpus save inputs that increase coverage Highly dependent on initial corpus
6. 5 5 Feedback-Directed Fuzzers: Challenges file.png AFL readpng PNG corpus save inputs that increase coverage Require feedback from the program
7. 6 6 What About a Format-specic Fuzzer?
8. 7 7 How to Generate a PNG le?
9. 8 8 Read the Spec?
10. 9 9 How Can We Even Inspect a PNG?
11. 10 10 How Can We Even Inspect a PNG? 010 Editor
12. 11 11 Binary Templates typedef struct { uint32 length; char type[4]; ubyte data[length]; uint32 crc; } PNG_CHUNK;
13. 12 12 Binary Templates typedef struct { uint32 length; char type[4]; ubyte data[length]; uint32 crc; } PNG_CHUNK;
14. 13 13 Binary Templates typedef struct { uint32 length; local int64 start = FTell(); char type[4]; ubyte data[length]; local int64 end = FTell(); local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
15. 14 14 Binary Templates typedef struct { uint32 length; local int64 start = FTell(); char type[4]; ubyte data[length]; local int64 end = FTell(); local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
16. 15 15 Binary Templates typedef struct { uint32 length; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
17. 16 16 200+ Binary Templates Available
18. 17 17 Binary Templates as Generators?
19. 18 18 Genera.ng Valid Files typedef struct { uint32 length; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
20. 19 19 Genera.ng Valid Files typedef struct { uint32 length; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
21. 20 20 Genera.ng Valid Files typedef struct { uint32 length; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
22. 21 21 Genera.ng Valid Files typedef struct { uint32 length <min=1, max=16>; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); /* ← Fix length here */ local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
23. 22 22 Genera.ng Valid Files typedef struct { uint32 length <min=1, max=16>; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); /* ← Fix length here */ local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
24. 23 23 Genera.ng Valid Files typedef struct { uint32 length <min=1, max=16>; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); /* ← Fix length here */ local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
25. 24 24 Genera.ng Valid Files typedef struct { uint32 length <min=1, max=16>; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); /* ← Fix length here */ local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK; IHDR PLTE IDAT IEND
26. 25 25 FormatFuzzer
27. 26 26 FormatFuzzer decision seed file.png generate mutate parse png-fuzzer png.bt C++
28. 27 27 Decision Seeds 89 P N G 0d 0a 1a 0a 00 00 00 00 Decision Seed Generated PNG File
29. 28 28 Decision Seeds 89 P N G 0d 0a 1a 0a 00 00 00 0d I H D R 00 00 00 01 00 00 00 01 08 02 00 00 00 90 77 53 de 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 08 00 01 00 00 00 00 00 00 00 00 Decision Seed Generated PNG File
30. 29 29 Decision Seeds 89 P N G 0d 0a 1a 0a 00 00 00 0d I H D R 00 00 00 01 00 00 00 01 08 02 00 00 00 90 77 53 de 00 00 00 0c I D A T 78 da 63 f8 cf c0 00 00 03 01 01 00 f7 03 41 43 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 08 00 01 00 00 00 00 00 00 00 00 00 00 01 00 0b 00 29 24 21 34 00 ff 00 00 08 00 00 Decision Seed Generated PNG File
31. 30 30 Decision Seeds 89 P N G 0d 0a 1a 0a 00 00 00 0d I H D R 00 00 00 01 00 00 00 01 08 02 00 00 00 90 77 53 de 00 00 00 0c I D A T 78 da 63 f8 cf c0 00 00 03 01 01 00 f7 03 41 43 00 00 00 00 I E N D ae 42 60 82 00 00 00 00 00 00 00 0c 00 00 00 00 00 00 08 00 01 00 00 00 00 00 00 00 00 00 00 01 00 0b 00 29 24 21 34 00 ff 00 00 08 00 00 00 00 7f 00 00 00 00 00 00 00 Decision Seed Generated PNG File
32. 31 31 Smart Muta.ons
33. 32 32 Smart Muta.ons Respect Context typedef struct { uint32 length <min=1, max=16>; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); /* ← Fix length here */ local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
34. 33 33 Smart Muta.ons Respect Context typedef struct { uint32 length <min=1, max=16>; local int64 start = FTell(); char type[4]; if (type == "IHDR") PNG_CHUNK_IHDR ihdr; else if (type == "PLTE") PNG_CHUNK_PLTE plte(length); /* … */ local int64 end = FTell(); /* ← Fix length here */ local uint32 crc_calc = Checksum(CHECKSUM_CRC32, start, end-start); uint32 crc = { crc_calc }; if (crc != crc_calc) Warning("Bad CRC %08x; expected: %08x", crc, crc_calc); } PNG_CHUNK;
35. 34 34 How to use FormatFuzzer?
36. 35 35 Fuzzing Strategies: Black-box Genera.on decision seed file.png generate mutate parse png-fuzzer /dev/urandom readpng FFGen
37. 36 36 Fuzzing Strategies: Black-box Muta.on decision seed corpus PNG corpus generate mutate parse png-fuzzer FFMut
38. 37 37 Fuzzing Strategies: Black-box Muta.on decision seed corpus file.png generate mutate parse png-fuzzer FFMut readpng
39. 38 38 Feedback-Directed Fuzzers file.png AFL AFL readpng PNG corpus save inputs that increase coverage
40. 39 39 Fuzzing Strategies: Generator-based Fuzzing AFL AFL+FFGen readpng save inputs that increase coverage decision seed corpus decision seed generate mutate parse png-fuzzer file.png
41. 40 40 Fuzzing Strategies: Muta.on-based Fuzzing file.png AFL AFL+FFMut readpng PNG corpus save inputs that increase coverage generate mutate parse png-fuzzer
42. 41 41 Evalua.on • Effort: most lines in the binary templates remain unchanged • Speed: ~7000 files/s generated or parsed • Success: 97% generations are successful • Accuracy: 76% of generated files are valid (82% without evil decisions)
43. 42 42 Fuzzing Results (Line Coverage %) PNG JPG GIF MIDI MP4 ZIP PCAP AVI BMP FFGen 22.3 24.2 68.7 12.3 5.6 33.7 11.5 5.6 27.8 FFMut 22.5 24.1 70.7 10.4 6.9 34.8 7.8 6.7 27.8 AFL 17.6 29.0 73.3 11.7 10.3 36.1 24.0 9.3 30.7 AFL+FFGen 23.6 26.7 71.6 11.9 9.0 36.5 21.1 9.1 27.9 AFL+FFMut 26.0 33.1 73.2 12.2 10.2 37.1 23.4 10.1 30.7 AFLSmart 18.0 29.4 12.1 10.8 36.2 24.1 10.7
44. 43 43 Bugs Found • FFmpeg: 8 distinct bugs already fixed by FFmpeg developers • Most are segmentation faults related to allocation • TiMidity: 19 distinct memory errors found
45. 44 44 FormatFuzzer • FormatFuzzer 1.0 released today • Includes integration with AFL++ 2.60c • Project page: https://uds-se.github.io/FormatFuzzer/ • Source: https://github.com/uds-se/FormatFuzzer • Preprint: http://arxiv.org/abs/2109.11277
46. 45 Current Research
47. 45 Current Research Mining Input
 Formats Patterns of Failure Composable Fuzzers Specifying Constraints Learning from Inputs Input Coverage Fuzzing Digital Certificates
48. 46 Current Research Mining Input
 Formats Patterns of Failure Composable Fuzzers Specifying Constraints Learning from Inputs Input Coverage Fuzzing Digital Certificates
49. 47 Current Research Mining Input
 Formats
50. def process_input(input) : try : ✘val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r 48 Parser
51. SYNTAX ERROR def process_input(input) : try : ✘val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r 49 The Core
52. def process_input(input) : try : ✔val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r { "store": { "book": [ { "category":"reference" , "author":"Nigel Rees" , "title":"Sayings of the Century" , "price":8.9 5 } , { "category":"fiction" , "author":"Evelyn Waugh" , "title":"Sword of Honour" , "price":12.9 9 } , { "category":"fiction" , "author":"J. R. R. Tolkien" , "title":"The Lord of the Rings" , "isbn":"0-395-19395-8" , "price":22.9 9 } ] , "bicycle": { "color":"red" , "price":19.9 5 } } } 50 Input Format
53. def process_input(input) : try : ✔val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null` <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <string> ::= `"` <chars> `" ` | `""` <chars> ::= <char><chars > | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits > | <digit> <digit> ::= [0-9] 51 Input Grammar
54. 52 Grammar JSON grammar <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9]
55. 52 Grammar JSON grammar <elt> key <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9]
56. 52 Grammar JSON grammar De f inition for <elt> <elt> key <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9]
57. 53 Grammar JSON grammar <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9]
58. 53 Grammar JSON grammar <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Expansion Rule
59. 53 Grammar JSON grammar <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Expansion Rule Terminal Symbol
60. 53 Grammar JSON grammar <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Expansion Rule Terminal Symbol Nonterminal Symbol
61. 53 Grammar JSON grammar <json> ::= <elt > 
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true ` | `false ` | `null ` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Expansion Rule Terminal Symbol Nonterminal Symbol
62. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Derivation Tree 54
63. <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false` | `null` <object> ::= `{`<items>`}`| `{}` <items> ::= <item>`,`<items> | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`]` | `[]` <elts> ::= <elt>`,`<elts> | <elt> <string> ::= `"` <chars> `"` | `""` <chars> ::= <char><chars> | <char> <char> ::= [A-Za-z0-9] <number> ::= <digits> <digits> ::= <digit><digits> | <digit> <digit> ::= [0-9] Derivation Tree {"":true} 54
64. {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } Fuzzer 55
65. Parser 56 {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 }
66. 57 {
 '<json>' : [['<elt>']] , '<elt>' : [['<object>'] , ['<array>'] , ['<string>'] , ['<number>'] , ['true'], ['false'], ['null']] , '<object>' : [['{', '<items>','}'] , ['{}']] , '<items>' : [['<item>,',',<items>'] , ['<item>']] , '<item>' : [['<string>',':', '<elt>']] , '<array>' : [['[', '<elts>', ']'] , ['[]']] , '<elts>' : [['<elt>,',',<elts>'] , ['<elt>']] , '<string>' : [['"', '<chars>', '"'] , ['""']] , '<chars>' : [['<char>','<chars>'] , ['<char>']] , '<number>' : [['<digits>']] , '<digits>' : [['<digit>','<digits>'] , ['<digit>']] , '<char>' : [[c] for c in string.characters ] '<digit>' : [[c] for c in string.digits]
 } https://www.fuzzingbook.org/html/LangFuzzer.html
67. 58 Where to Get the Grammar From?
68. 59 Almost Everyone Uses Handwritten Parsers https://notes.eatonphil.com/parser-generators-vs-handwritten-parsers-survey-2021.html
69. 60
70. 61
71. 61 "Be liberal in what you accept, and conservative in what you send" Postel's Law
72. 62 "Be liberal in what you accept, and conservative in what you send"
 Postel's Law The Specification Where to Get the Grammar From?
73. 62 "Be liberal in what you accept, and conservative in what you send"
 Postel's Law The Specification The Implementation Extra "Features" Where to Get the Grammar From?
74. 63 Where to Get an Accurate Grammar?
75. 64 Where to Get an Accurate Grammar? Hand-written parsers already encode the grammar
76. def json_raw(stm) : while True : stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) https://github.com/phensley/microjson 65 Source to Control Flow
77. def json_raw(stm) : while True : stm.skipspaces( ) c = stm.peek( ) if c == 't' : return json_fixed(stm, 'true' ) elif c == 'f' : return json_fixed(stm, 'false' ) elif c == 'n': return json_fixed(stm, 'null' ) elif c == '"': return json_string(stm ) elif c == '{': return json_dict(stm ) elif c == '[': return json_list(stm ) elif c in NUMSTART : return json_number(stm ) raise JSONError(E_MALF, stm, stm.pos) https://github.com/phensley/microjson 65 Control Flow Graph Source to Control Flow
78. Grammar Mining From Control Flow Sequence A B C [F] Selection cond A B [F] F T Iteration cond B [F] 66
79. <F> := <A> <B> <C> Grammar Mining From Control Flow Sequence A B C [F] Selection cond A B [F] F T Iteration cond B [F] 66
80. <F> := <A > | <B> <F> := <A> <B> <C> Grammar Mining From Control Flow Sequence A B C [F] Selection cond A B [F] F T Iteration cond B [F] 66
81. <F> := <A > | <B> <F> := <A> <B> <C> <F> := <B> <F > | <empty> Grammar Mining From Control Flow Sequence A B C [F] Selection cond A B [F] F T Iteration cond B [F] 66
82. 67 https://github.com/vrthra/mimid/ Gopinath, Mathis, and Zeller. Mining Input Grammars from Dynamic Control Flow. ESEC/FSE 2020. Mimid
83. Recall Subjects Mimid calc.py 100.0 % mathexpr.py 87.5 % cgidecode.py 100.0 % urlparse.py 100.0 % microjson.py 98.7 % parseclisp.py 99.3 % jsonparser.c 100.0 % tiny.c 100.0 % mjs.c 95.4 % Inputs generated by inferred grammar that were accepted by the program Subjects Mimid calc.py 100.0 % mathexpr.py 92.7 % cgidecode.py 100.0 % urlparse.py 96.4 % microjson.py 93.0 % parseclisp.py 80.6 % jsonparser.c 83.8 % tiny.c 92.8 % mjs.c 95.9 % Inputs generated by golden grammar that were accepted by the inferred grammar parser Precision Evaluation: Accuracy 68
84. 69 Sample Free Generators
85. 69 Sample Free Generators A [ 2 , B 9 ) 4 ] A ∉ [,+,-,1,2,3,4,5,6,7,8,9,0 B ∉ +,-,1,2,3,4,5,6,7,8,9,0,) ) ∉ +,-,1,2,3,4,5,6,7,8,9,0 [2,94]
86. Grammar Miner Program Under Test Sample Free Generator Grammar Fuzzer Inputs 70
87. 71 Extract The Program Input Format Automatically What Does This Mean For You? • Fuzzer agnostic • Easy to use • Open Source https://github.com/vrthra/mimid
88. 72 Current Research Mining Input
 Formats
89. 73 Current Research Mining Input
 Formats Patterns of Failure Composable
 Fuzzers Specifying Constraints Learning from Inputs Input Coverage Fuzzing Digital Certificates
90. Patterns of Failure 74 Current Research
91. def process_input(input) : try : val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r 75
92. def process_input(input) : try : val = parse(input ) res = process(val ) return re s except SyntaxError : return Erro r {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," ✘ 75
93. {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," What is the smallest failure inducing input? 76 Delta Debugging
94. {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," 77 Grammar Based Delta Debugging
95. {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> 77 Grammar Based Delta Debugging
96. {"":[]} 78 Grammar Based Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," Test Minimization
97. {"":[]} ✘ 78 Grammar Based Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," Test Minimization
98. 79 Grammar Based Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," Why? {"":[]}
99. 79 Grammar Based Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," Why? [12345] {"":[]}
100. 79 Grammar Based Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," Why? [12345] {"":[]} {"":0}
101. 79 Grammar Based Delta Debugging {"type":"PathNode","matrix": {"m11":-0.6630394213564543,"m12":0,"m21":0,"m22":0.5236476835782672,"dx":565.5201948 628471,"dy":371.5686591257294},"children": [],"strokeStyle":"#000000"," fi llStyle":"#e1e1e1","lineWidth":4,"smoothness":0.3,"sloppiness":0.5, "startX":50,"startY":0,"closed":true,"segments":[{"type":3,"x":100,"y":50,"x1":100,"y1":0,"r": [-0.3779207859188318,0.07996635790914297,-0.47163885831832886,-0.0710031278431415 6]},{"type":3,"x":50,"y":100,"x1":100,"y1":100,"r": [0.24857700895518064,0.030472169630229473,0.49844827968627214,0.1326016811653971 7]},{"type":3,"x":0,"y":50,"x1":0,"y1":100,"r": [0.1751830680295825,-0.18606301862746477,-0.4092112798243761,-0.4790717279538512]} ,{"type":3,"x":50,"y":0,"x1":0,"y1":0,"r": [0.37117584701627493,0.3612578883767128,0.0462839687243104,-0.1564063960686326]}], "shadow":false},{"type":"PathNode","matrix": {"m11":-1.475090930376591,"m12":0,"m21":0,"m22":1.2306765694828008,"dx":700.13810328 55618,"dy":133.20628077515605},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3,"sloppiness":0.5,"star tX":126.25,"startY":127.50445838342671,"closed":true,"segments": [{"type":3,"x":146.01190476190476,"y":147.5936260519611,"x1":146.01190476190476,"y1":127 .50445838342671,"r": [-0.1750196823850274,-0.05804965365678072,-0.3536788672208786,0.05322327278554439 5]}, {"type":3,"x":126.25,"y":167.6827937204955,"x1":146.01190476190476,"y1":167.68279372049 55,"r": [-0.32906053867191076,-0.11536165233701468,0.35579121299088,0.38731588050723076]}, {"type":3,"x":108,"y":147,"x1":106.48809523809524,"y1":167.6827937204955,"r": [0.08825046103447676,0.011088204570114613,0.43411328736692667,-0.133069220930337 9]}, {"type":3,"x":126.25,"y":127.50445838342671,"x1":106.48809523809524,"y1":127.5044583834 2671,"r": [0.42778260353952646,0.24726040940731764,0.3631806019693613,0.05325550492852926]} ],"shadow":false},{"type":"TextNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":543,"dy":225},"children": []," fi llStyle":"#000000","text":"Y","fontName":"FG Virgil","fontSize":20}, {"type":"TextNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":559,"dy":144},"children": []," fi llStyle":"#000000","text":"x","fontName":"FG Virgil","fontSize":20}, {"type":"ArrowNode","matrix":{"m11":1,"m12":0,"m21":0,"m22":1,"dx":0,"dy":0},"children": [],"arrowSize":10,"path":{"type":"PathNode","matrix": {"m11":1,"m12":0,"m21":0,"m22":1,"dx":464,"dy":-3},"children": [],"strokeStyle":"#000000"," fi llStyle":"#ffffff","lineWidth":2,"smoothness":0.3," Why? [12345] {"":[]} {"":0} {"x":[]}
102. 80 DDSET Gopinath, Kampmann, Havrikov, Soremekun, and Zeller. Abstracting Failure Inducing Inputs. ISSTA 2020. https://github.com/vrthra/ddset
103. {"": []} DDSET: 81 <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
104. {"": []} 82 DDSET: <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
105. {"": []} {"7897A": []} {"klnm,.qer;dfs?P":[]} {"123KOUIJ!qR30578950":[]} 82 DDSET: <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
106. {"": []} 83 DDSET: <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
107. {"": []} {"": true} {"":[1,2,445,"x"]} {"":{"PQ":[true, false, 223,"a"]}} 83 DDSET: <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
108. {"": []} Abstraction {"": <elt>} Abstract Input 84 DDSET: <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
109. {"": <elt>} Abstract Input {"": []} Minimized Input 85 <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit>
110. Issue 2842 from Closure var A = class extends (class {}){}; Issue 2937 from Closure {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino const [y,y] = []; Issue 386 from Rhino 86
111. Issue 2842 from Closure var A = class extends (class {}){}; Issue 2937 from Closure {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino const [y,y] = []; Issue 386 from Rhino <varModifier> <Identifier> = class extends (class {}){} var {<$Id1>:{} = <$Id1> => {}} <variableDeclaration>; {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} const [<$Id1>,<$Id1>] = [] 86
112. {"": <elt>} Abstract Input {"": []} Minimized Input 87 <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> Evocative Grammar
113. {"": <elt>} Abstract Input {"": []} Minimized Input 87 <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> <json E> where <item E> is "":<elt> Evocative Pattern Evocative Grammar
114. {"": <elt>} Abstract Input {"": []} Minimized Input 87 <json> ::= <elt>
 <elt> ::= <object>
 | <array>
 | <string>
 | <number>
 | `true` | `false ` | `null ` <string> ::= `"` <chars> `" ` | `"" ` <chars> ::= <char><chars> | <char> <object> ::= `{`<items>`} ` | `{}` <items> ::= <item>`,`<items > | <item> <item> ::= <string>`:`<elt> <array> ::= `[`<elts>`] ` | `[]` <elts> ::= <elt>`,`<elts > | <elt> <number> ::= <digits > <digits> ::= <digit><digits> | <digit> <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Pattern Evocative Grammar
115. 88 generate(<json E>) <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Grammar
116. 88 {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[], {"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] <json E> where <item E> is "":<elt> Evocative Grammar
117. 89 <json E> where <item E> is "":<elt> 1. We can produce any and all instances of the failure inducing pattern . 2. We can recognize any input that contains the failure inducing pattern . 3. The grammar will reject any input that doesn't contain the failure inducing pattern. <json E> ::= <elt E>
 <elt E> ::= <object E>
 | <array E > | <string E>
 | <number E>
 <object E> ::= `{`<items E>`}`
 <items E> ::= <item E> | <item E>`,`<items > | <item>`,`<items E>
 <item E> ::= <string>`:`<elt E > | <string E1>`:`<elt>
 <array E> ::= `[`<elts E>`]`
 <elts E> ::= <elt E > | <elt E>`,`<elts > | <elt>`,`<elts E > <string E1> ::= `"" ` <json> ::= <elt>
 <elt> ::= <object>
 | <array > | <string>
 | <number>
 | `true` | `false` | `null`
 <object> ::= `{`<items>`}` | `{}`
 <items> ::= <item> | <item>`,`<items>
 <item> ::= <string>`:`<elt>
 <array> ::= `[`<elts>`]` | `[]`
 <elts> ::= <elt> | <elt>`,`<elts>
 <string> ::= `"` <chars> `"` | `""`
 <chars> ::= <char><chars>
 <char> ::= [A-Za-z0-9]
 <number> ::= <digits>
 <digits> ::= <digit><digits> | <digit>
 <digit> ::= [0-9] Evocative Grammar (Statistical Guarantees based on the accuracy of the evocative pattern)
118. 90 Evocative Patterns What Does This Mean For You? • Automatically Mined • Produce Specialized Fuzzers • Open Source https://github.com/vrthra/ddset
119. Patterns of Failure 91 Current Research
120. 92 Current Research Mining Input
 Formats Patterns of Failure Composable
 Fuzzers Specifying Constraints Learning from Inputs Input Coverage Fuzzing Digital Certificates
121. 93 Current Research Composable
 Fuzzers
122. <json E> where <item E> is "":<elt> if json.has_key("") : raise Exception() 94 Evocative Pattern
123. <json E> where <item E> is "":<elt> if json.has_key("") : raise Exception() 95 Composing Evocative Patterns
124. <json E> where <item E> is "":<elt> if json.has_key("") : raise Exception() if json.has_key_value(null) : raise Exception() 95 Composing Evocative Patterns
125. <json E> where <item E> is "":<elt> if json.has_key("") : raise Exception() if json.has_key_value(null) : raise Exception() <json N> where <item N> is <string>: null 95 Composing Evocative Patterns
126. if json.has_key("") and json.has_key_value(null) : raise Exception() 96 Composing Evocative Patterns
127. if json.has_key("") and json.has_key_value(null) : raise Exception() <json E & N> where <item E> is "":<elt> 
 <item N> is <string>: null 96 Composing Evocative Patterns
128. if json.has_key("") and not json.has_key_value(null) : raise Exception() 97 Composing Evocative Patterns
129. if json.has_key("") and not json.has_key_value(null) : raise Exception() <json E & not(N)> where <item E> is "":<elt> <item N> is <string>: null 97 Composing Evocative Patterns
130. if json.has_key("") : raise Exception( ) if json.has_key_value(null) : raise Exception() 98 Composing Evocative Patterns
131. if json.has_key("") : raise Exception( ) if json.has_key_value(null) : raise Exception() <json not(E) & not(N)> where <item E> is "":<elt> <item N> is <string>: null 98 Composing Evocative Patterns
132. <json E&N> := <elt E&N > <elt E&N> := <object E&N > | <array E&N > <array E&N>:= '[' <elts E&N> '] ' <object E&N>:= '{' <items E&N> '} ' <elts E&N> := <elt E&N > | <elt E&N>','<elts N > | <elt N>','<elts E&N > <items E&N> := <item E&N > | <item E&N>','<items N > | <item N>','<items E&N > <item E&N> := <string E1>':'<elt N&N1 > | <string>':'<elt E&N&N1 > <elt E&N&N1> := <object E&N> <array E&N > <elt N> := 'false' | 'true ' | <number> | <string > | <object N> <array N > <array N> := '[]' | '[' <elts N> '] ' <object N> := '{}' | '{' <items N> '} ' <elts N> := <elt N > | <elt N>','<elts N > <items N> := <item N > | <item N>','<items N > <item N> := <string>':'<elt N&N1 > <elt N&N1> := 'false' | 'true ' | <number> | <string > | <object N> | <array N> <json E & N> where <item E> is "":<elt> <item N> is <string>: null generate(<json E&N>) 99
133. <json E&N> := <elt E&N > <elt E&N> := <object E&N > | <array E&N > <array E&N>:= '[' <elts E&N> '] ' <object E&N>:= '{' <items E&N> '} ' <elts E&N> := <elt E&N > | <elt E&N>','<elts N > | <elt N>','<elts E&N > <items E&N> := <item E&N > | <item E&N>','<items N > | <item N>','<items E&N > <item E&N> := <string E1>':'<elt N&N1 > | <string>':'<elt E&N&N1 > <elt E&N&N1> := <object E&N> <array E&N > <elt N> := 'false' | 'true ' | <number> | <string > | <object N> <array N > <array N> := '[]' | '[' <elts N> '] ' <object N> := '{}' | '{' <items N> '} ' <elts N> := <elt N > | <elt N>','<elts N > <items N> := <item N > | <item N>','<items N > <item N> := <string>':'<elt N&N1 > <elt N&N1> := 'false' | 'true ' | <number> | <string > | <object N> | <array N> <json E & N> where <item E> is "":<elt> <item N> is <string>: null {"": 100} {"": [343,{},44998]} [{"": {"xxy":44998, {"b":[1,2,3]}}},[],[]] {"_": {"ket":[], {"":[],"y",[[],[1,2,3,455,6]]}}} {".":[{3243435656:"xy,zzzpqiu"},[{"":[112]},{"d":[[]]},{}]]} [{"": [1,2,3,4]}] {"pqr": {"": [1,2,3,4]}, "abc":[]} [[1132],{"xx":[{6:"dafjli;y,zzzdfaiu"},[{"__":[1{}{}]},{"":[[444456]]},{}]]} generate(<json E&N>) ✘ ✘ ✘ ✘ ✘ ✘ ✘ ✘ 99
134. Issue 386 from Rhino var A = class extends (class {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure <varModifier> <Identifier> = class extends (class {}){} var {<$Id1>:{} = <$Id1> => {}} <variableDeclaration>; const [<$Id1>,<$Id1>] = [] {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} 100
135. where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Evocative Expressions 101
136. where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Evocative Expressions <JavaScript C2937 and C2842> 101
137. where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Evocative Expressions <JavaScript C2937 and C2842> <JavaScript R385 and R386> 101
138. where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Evocative Expressions <JavaScript C2937 and C2842> <JavaScript R385 and R386> <JavaScript (C2937 or C2842) and (R385 or R386)> 101
139. where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Evocative Expressions <JavaScript C2937 and C2842> <JavaScript R385 and R386> <JavaScript (C2937 or C2842) and (R385 or R386)> <JavaScript not(C2937 or C2842 or R385 or R386)> 101
140. Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt )
141. 103 Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"role" : "admin"}
142. 103 <json ADM> where <item ADM> is "role": "admin" Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"role" : "admin"}
143. 104 Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) {"method":"remove_table","args":["orders", "inventory"]} <json DBDT> where <items DBDT> is "method":"remove_table","args":<elt>
144. 105 Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>}
145. 105 Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"], "role":"admin"} {"role":"admin", "method":"remove_table","args":["orders", "inventory"]} {"method":"remove_table","role":"admin","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"admin"}]}
146. 106 Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>}
147. 106 Evocative Expressions for Focused Fuzzing def assign_admin_rights() : self.db_rights.add(MODIFY_DB ) self.fs_rights = R W self.timeout = 6 0 self.deploy = Tru e def assign_guest_rights() : self.db_rights = [QUERY_DB ] self.fs_rights = Non e self.timeout = Non e self.deploy = Fals e def modify_db(stmt) : if ADMIN in self.db_rights : process(stmt ) else : raise Error( ) def query_db(stmt) : process(stmt ) <json ADM & DBDT> where <item ADM> is "role": "admin" <object DBDT> is {"method":"remove_table","args":<elt>} {"method":"remove_table","args":["orders", "inventory"]} {"role":"guest", "method":"remove_table","args":{"role":"guest"}} {"method":"remove_table","role":"guest","args":["orders", "inventory"]} {"method":"remove_table","args":["orders","inventory",{"role":"guest"}]}
148. Evocative Expressions: Data Structures Algebraic Data Types 107 <my_struct> ::= <stype> <stype> struct mystruct { stype m1 ; stype m2 ; }; union myunion { utype m1 ; utype m2 ; }; <my_union> ::= <utype> | <utype> Data Structures Context Free Grammar
149. 108 Composable Fuzzers REST with a REST vulnerability and a SQL Injection SQL but doesn't Induce BugB BugB or cover function A FnA I want an XML fuzzer XML
150. XMLREST 108 Composable Fuzzers REST with a REST vulnerability and a SQL Injection SQL but doesn't Induce BugB BugB or cover function A FnA I want an XML fuzzer XML
151. XMLREST XMLSQL 108 Composable Fuzzers REST with a REST vulnerability and a SQL Injection SQL but doesn't Induce BugB BugB or cover function A FnA I want an XML fuzzer XML
152. XMLREST XMLBugB XMLSQL 108 Composable Fuzzers REST with a REST vulnerability and a SQL Injection SQL but doesn't Induce BugB BugB or cover function A FnA I want an XML fuzzer XML
153. XMLREST XMLFnA XMLBugB XMLSQL 108 Composable Fuzzers REST with a REST vulnerability and a SQL Injection SQL but doesn't Induce BugB BugB or cover function A FnA I want an XML fuzzer XML
154. XMLREST XMLFnA XMLBugB XMLSQL 109 Composable Fuzzers REST with a REST vulnerability and a SQL Injection SQL but doesn't Induce BugB BugB or cover function A FnA & & not[ ] | I want an XML fuzzer XML
155. 110 Evocative Expressions = Composable Fuzzers I want a fuzzer that targets a REST server wit h • SQL Injections A,B, and C • But does not go through the input sanitizer code I want a fuzzer that targets a C compiler wit h • No undefined behaviors in the produced input s • But contain at least one function pointer declaration I want a fuzzer that targets a database wit h • Each input containing previously fixed bugs A and B • But does not induce a known bug C • And does not cover the function X in the database source code I want a fuzzer that targets a JSON parser wit h • Each input containing at least one known quirk from other parsers I want a fuzzer that targets my applicatio n • Each input exercising the code I just fixe d • And also other known bug pattern s • But does not consume resource A
156. 110 Evocative Expressions = Composable Fuzzers I want a fuzzer that targets a REST server wit h • SQL Injections A,B, and C • But does not go through the input sanitizer code I want a fuzzer that targets a C compiler wit h • No undefined behaviors in the produced input s • But contain at least one function pointer declaration I want a fuzzer that targets a database wit h • Each input containing previously fixed bugs A and B • But does not induce a known bug C • And does not cover the function X in the database source code I want a fuzzer that targets a JSON parser wit h • Each input containing at least one known quirk from other parsers I want a fuzzer that targets my applicatio n • Each input exercising the code I just fixe d • And also other known bug pattern s • But does not consume resource A All w ithout w riting a single line of code
157. Gopinath, Nemati, Zeller. Input Algebras. ICSE 2021. Evocative Expressions https://rahul.gopinath.org/posts/ 111
158. 112 Evocative Expressions What Does This Mean For You? • Mix and match specialized fuzzers • Use historical bugs • An ecosystem of targeted fuzzers • Open Source https://github.com/vrthra/Ewoks
159. 113 Current Research Composable
 Fuzzers
160. 114 Current Research Mining Input
 Formats Patterns of Failure Composable
 Fuzzers Specifying Constraints Learning from Inputs Input Coverage Fuzzing Digital Certificates
161. 115 Current Research Mining Input
 Formats Patterns of Failure Composable
 Fuzzers Specifying Constraints Learning from Inputs Input Coverage Fuzzing Digital Certificates
162. 116