Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Fuzzing -- From Alchemy to a Science

Fuzzing -- From Alchemy to a Science

Talk at Passau University

D27cb84e0d30e2778e9b66d6a5f42106?s=128

Rahul Gopinath

June 30, 2021
Tweet

Transcript

  1. Fuzzing Rahul Gopinath

  2. Fuzzing from Alchemy Rahul Gopinath

  3. Fuzzing from Alchemy Rahul Gopinath

  4. Fuzzing from Alchemy to Science Rahul Gopinath

  5. Fuzzing from Alchemy to Science Rahul Gopinath

  6. 4

  7. 4 The story begins 2500 years ago. 500 BC

  8. 4 Vedic Sanskrit 500 BC

  9. 4 Vedic Sanskrit Classical Sanskrit 500 BC

  10. 5 500 BC Vedic Sanskrit Classical Sanskrit

  11. 5 500 BC Aṣṭādhyāyī Dakṣiputra Pāṇini Vedic Sanskrit Classical Sanskrit

  12. 5 500 BC Aṣṭādhyāyī Dakṣiputra Pāṇini Ad hoc rules Formal

    specification Vedic Sanskrit Classical Sanskrit
  13. 6

  14. 6 2500 years later.... 2021 CE The world is governed

    by software
  15. 6 2500 years later.... 2021 CE We have a crisis

    The world is governed by software
  16. 7 Debian 5 ~ 70 million lines Smart cars ~

    100 million lines Google is ~ 2 Billion lines 5 M 10 M 15 M 20 M 25 M 30 M 35 M (Source: Wikipedia) #Linux Kernel Size in Millions of Lines of Code Linux Releases Growth in software complexity
  17. 7 Debian 5 ~ 70 million lines Smart cars ~

    100 million lines Google is ~ 2 Billion lines 1994 1996 1.0.0 2.0.0 2.1.0 2.2.0 2.4.0 2.6.0 2003 2011 3.0 2015 4.0 2019 5.0 2019 5.7 2001 5 M 10 M 15 M 20 M 25 M 30 M 35 M (Source: Wikipedia) #Linux Kernel Size in Millions of Lines of Code Linux Releases Growth in software complexity
  18. 8 Growth in vulnerabilities #Vulnerabilities Year 16k 14k 12k 10k

    8k 6k 4k 2k (Source: NIST)
  19. 8 Growth in vulnerabilities #Vulnerabilities Year 2019 2018 2017 2016

    2015 2014 2013 2012 2011 2010 2009 2008 2007 2006 2005 2004 2003 2002 2001 16k 14k 12k 10k 8k 6k 4k 2k (Source: NIST)
  20. Fuzzing Program Trash deck technique: 1950s - Gerald Weinberg

  21. Fuzzing Program Trash deck technique: 1950s - Gerald Weinberg

  22. Fuzzing Crash? Program Trash deck technique: 1950s - Gerald Weinberg

  23. 10 Fuzzing Random Inputs Program Automatic Checks

  24. 10 Fuzzing • Memory Bounds Violation • Privilege Escalation •

    Safety Violations • Metamorphic Relations • Differential Execution Random Inputs Program Automatic Checks
  25. 11 Random Fuzzing Program

  26. 11 Random Fuzzing $ ./fuzz [;x1-GPZ+wcckc];,N9J+?#6^6\e?]9lu
 2_%'4GX"0VUB[E/r ~fApu6b8<{%siq8Z
 h.6{V,hr?;{Ti.r3PIxMMMv6{xS^+'Hq!
 AxB"YXRS@!Kd6;wtAMefFWM(`|J_<1~o}


    z3K(CCzRH JIIvHz>_*.\>JrlU32~eGP?
 lR=bF3+;y$3lodQ<B89!5"W2fK*vE7v{'
 )KC-i,c{<[~m!]o;{.'}Gj\(X}EtYetrp
 bY@aGZ1{P!AZU7x#4(Rtn!q4nCwqol^y6
 }0|Ko=*JK~;zMKV=9Nai:wxu{J&UV#HaU
 )*BiC<),`+t*gka<W=Z.%T5WGHZpI30D<
 Pq>&]BS6R&j?#tP7iaV}-}`\?[_[Z^LBM
 PG-FKj'\xwuZ1=Q`^`5,$N$Q@[!CuRzJ2
 D|vBy!^zkhdf3C5PAkR?V((-%><hn|3='
 i2Qx]D$qs4O`1@fevnG'2\11Vf3piU37@
 5:dfd45*(7^%5ap\zIyl"'f,$ee,J4Gw:
 cgNKLie3nx9(`efSlg6#[K"@WjhZ}r[Sc
 un&sBCS,T[/3]KAeEnQ7lU)3Pn,0)G/6N
 -wyzj/MTd#A;r*(ds./df3r8Odaf?/<#r Program ✘
  27. $ ./fuzz -int | program 634111569742810193727424069509 741355925061499451162464719526 615957331924826555590537407605 181400079803446874252046374716 740973770255348279425601333144

    152724057932073828569041216191 099859446496509919024810271242 622974988671421938012464630138 735355134599327240920259675263 574528613057084231370741920902 794677842164654990353575580453 777282305855352378119038096476 699871306655084953377039862387 924957554389878352934547664240 082431556093837288597262675598 630851919061829885048834738832 677022429414980917053939970795 722006987916088650168665471731 yes 12 Random Fuzzing def is_prime(n: int) -> bool: """Primality test using 6k+-1 optimization.""" if n <= 3: return n > 1 if n % 2 == 0 or n % 3 == 0: return False i = 5 while i ** 2 <= n: if n % i == 0 or n % (i + 2) == 0: return False i += 6 return True def main(): num = stdin.read() print(num, is_prime(num))
  28. $ ./fuzz -int | program 634111569742810193727424069509 741355925061499451162464719526 615957331924826555590537407605 181400079803446874252046374716 740973770255348279425601333144

    152724057932073828569041216191 099859446496509919024810271242 622974988671421938012464630138 735355134599327240920259675263 574528613057084231370741920902 794677842164654990353575580453 777282305855352378119038096476 699871306655084953377039862387 924957554389878352934547664240 082431556093837288597262675598 630851919061829885048834738832 677022429414980917053939970795 722006987916088650168665471731 yes 12 Random Fuzzing def is_prime(n: int) -> bool: """Primality test using 6k+-1 optimization.""" if n <= 3: return n > 1 if n % 2 == 0 or n % 3 == 0: return False i = 5 while i ** 2 <= n: if n % i == 0 or n % (i + 2) == 0: return False i += 6 return True def main(): num = stdin.read() print(num, is_prime(num)) Weak point: unequal divisions in input space Input space n > 3 n <= 3
  29. 13 Advanced Fuzzing: Instrumentation

  30. 14 Advanced Fuzzing: Instrumentation def triangle(a, b, c): if a

    == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene def triangle(a, b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene
  31. 14 Advanced Fuzzing: Instrumentation def triangle(a, b, c): __probe_enter() if

    a == b: __probe_1() if b == c: __probe_2() return Equilateral else: __probe_3() return Isosceles else: __probe_4() if b == c: __probe_5() return Isosceles else: __probe_6() if a == c: __probe_7() return Isosceles else: __probe_8() return Scalene def triangle(a, b, c): if a == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene
  32. 15 Instrumentation Based Fuzzing def triangle(a, b, c): if a

    == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene
  33. 15 Instrumentation Based Fuzzing def triangle(a, b, c): if a

    == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene triangle (1,1,1)
  34. 16 Instrumentation Based Fuzzing def triangle(a, b, c): if a

    == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene
  35. 16 Instrumentation Based Fuzzing def triangle(a, b, c): if a

    == b: if b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene triangle (1,1,2)
  36. 17 Instrumentation Guided Fuzzing • Coverage Guided • Solver Directed

  37. 18 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage
  38. 18 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage AFL
  39. 19 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage AFL First Release: 2013 AFL Trophy Case
  40. 20 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage AFL Pulling JPEGs out of thin air https://lcamtuf.blogspot.com/2014/11/pulling-jpegs-out-of-thin-air.html Valid JPEG in 6 hours in an 8 core machine!
  41. 21 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage AFL static int is_reserved_word_token(const char *s, int len) { const char *reserved[] = { "break", "case", "catch", "continue", "debugger", "default", "delete", "do", "else", "false", "finally", "for", "function", "if", "in", "instanceof", "new", "null", "return", "switch", "this", "throw", "true", "try", "typeof", "var", "void", "while", "with", "let", "undefined", ((void *)0)}; int i; if (!mjs_is_alpha(s[0])) return 0; for (i = 0; reserved[i] != ((void *)0); i++) { if (len == (int)strlen(reserved[i]) && strncmp(s, reserved[i], len) == 0) return i + 1; } return 0; }
  42. 21 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage AFL static int is_reserved_word_token(const char *s, int len) { const char *reserved[] = { "break", "case", "catch", "continue", "debugger", "default", "delete", "do", "else", "false", "finally", "for", "function", "if", "in", "instanceof", "new", "null", "return", "switch", "this", "throw", "true", "try", "typeof", "var", "void", "while", "with", "let", "undefined", ((void *)0)}; int i; if (!mjs_is_alpha(s[0])) return 0; for (i = 0; reserved[i] != ((void *)0); i++) { if (len == (int)strlen(reserved[i]) && strncmp(s, reserved[i], len) == 0) return i + 1; } return 0; } Weak point: Magic bytes
  43. 22 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage
  44. 22 Coverage Guided Fuzzing • Randomly generate inputs • Choose

    seeds with new coverage
  45. 23 Solver Directed Fuzzing • Collect path constraints • Solve

    negated constraints for new inputs
  46. 23 Solver Directed Fuzzing • Collect path constraints • Solve

    negated constraints for new inputs (a == b) (b == c)
  47. 23 Solver Directed Fuzzing • Collect path constraints • Solve

    negated constraints for new inputs (a == b) (b == c) (b != c)
  48. 23 Solver Directed Fuzzing • Collect path constraints • Solve

    negated constraints for new inputs (a == b) (b == c) (b != c) triangle(1,2,1)
  49. 23 Solver Directed Fuzzing • Collect path constraints • Solve

    negated constraints for new inputs (a == b) (b == c) (b != c) triangle(1,2,1)
  50. 24 Solver Directed Fuzzing • Collect path constraints • Solve

    constraints for new inputs void next_sym() { while(1) { switch (ch){ case '{': next_ch(); sym = LBRA; return; case '}': next_ch(); sym = RBRA; return; case '(': next_ch(); sym = LPAR; return; case ')': next_ch(); sym = RPAR; return; case '+': next_ch(); sym = PLUS; return; case '-': next_ch(); sym = MINUS; return; case '<': next_ch(); sym = LESS; return; case ';': next_ch(); sym = SEMI; return; case '=': next_ch(); sym = EQUAL; return; default: if (ch >= '0' && ch <= '9') { int_val = 0; /* missing overflow check */ while (ch >= '0' && ch <= '9') { int_val = int_val*10 + (ch - '0'); next_ch(); } sym = INT; } else if (ch >= 'a' && ch <= 'z') { int i = 0; /* missing overflow check */ while ((ch >= 'a' && ch <= 'z') || ch == '_'){ id_name[i++] = ch; next_ch(); } id_name[i] = '\0'; sym = 0; while (words[sym] != NULL && strcmp(words[sym], id_name) != 0) sym++; if (words[sym] == NULL) if (id_name[1] == '\0') sym = ID; else syntax_error(); } else syntax_error(); return; }
  51. 24 Solver Directed Fuzzing • Collect path constraints • Solve

    constraints for new inputs void next_sym() { while(1) { switch (ch){ case '{': next_ch(); sym = LBRA; return; case '}': next_ch(); sym = RBRA; return; case '(': next_ch(); sym = LPAR; return; case ')': next_ch(); sym = RPAR; return; case '+': next_ch(); sym = PLUS; return; case '-': next_ch(); sym = MINUS; return; case '<': next_ch(); sym = LESS; return; case ';': next_ch(); sym = SEMI; return; case '=': next_ch(); sym = EQUAL; return; default: if (ch >= '0' && ch <= '9') { int_val = 0; /* missing overflow check */ while (ch >= '0' && ch <= '9') { int_val = int_val*10 + (ch - '0'); next_ch(); } sym = INT; } else if (ch >= 'a' && ch <= 'z') { int i = 0; /* missing overflow check */ while ((ch >= 'a' && ch <= 'z') || ch == '_'){ id_name[i++] = ch; next_ch(); } id_name[i] = '\0'; sym = 0; while (words[sym] != NULL && strcmp(words[sym], id_name) != 0) sym++; if (words[sym] == NULL) if (id_name[1] == '\0') sym = ID; else syntax_error(); } else syntax_error(); return; } Weak point: Path explosion
  52. 25 Solver Directed Fuzzing • Collect path constraints • Solve

    negated constraints for new inputs
  53. 25 Solver Directed Fuzzing • Collect path constraints • Solve

    negated constraints for new inputs
  54. 26 Fuzzing Parsers $ ./fuzz [;x1-GPZ+wcckc];,N9J+?#6^6\e?]9lu
 2_%'4GX"0VUB[E/r ~fApu6b8<{%siq8Z
 h.6{V,hr?;{Ti.r3PIxMMMv6{xS^+'Hq!
 AxB"YXRS@!Kd6;wtAMefFWM(`|J_<1~o}


    z3K(CCzRH JIIvHz>_*.\>JrlU32~eGP?
 lR=bF3+;y$3lodQ<B89!5"W2fK*vE7v{'
 )KC-i,c{<[~m!]o;{.'}Gj\(X}EtYetrp
 bY@aGZ1{P!AZU7x#4(Rtn!q4nCwqol^y6
 }0|Ko=*JK~;zMKV=9Nai:wxu{J&UV#HaU
 )*BiC<),`+t*gka<W=Z.%T5WGHZpI30D<
 Pq>&]BS6R&j?#tP7iaV}-}`\?[_[Z^LBM
 PG-FKj'\xwuZ1=Q`^`5,$N$Q@[!CuRzJ2
 D|vBy!^zkhdf3C5PAkR?V((-%><hn|3='
 i2Qx]D$qs4O`1@fevnG'2\11Vf3piU37@
 5:dfd45*(7^%5ap\zIyl"'f,$ee,J4Gw:
 cgNKLie3nx9(`efSlg6#[K"@WjhZ}r[Sc
 un&sBCS,T[/3]KAeEnQ7lU)3Pn,0)G/6N
 -wyzj/MTd#A;r*(ds./df3r8Odaf?/<#r Interpreter
  55. 26 Fuzzing Parsers $ ./fuzz [;x1-GPZ+wcckc];,N9J+?#6^6\e?]9lu
 2_%'4GX"0VUB[E/r ~fApu6b8<{%siq8Z
 h.6{V,hr?;{Ti.r3PIxMMMv6{xS^+'Hq!
 AxB"YXRS@!Kd6;wtAMefFWM(`|J_<1~o}


    z3K(CCzRH JIIvHz>_*.\>JrlU32~eGP?
 lR=bF3+;y$3lodQ<B89!5"W2fK*vE7v{'
 )KC-i,c{<[~m!]o;{.'}Gj\(X}EtYetrp
 bY@aGZ1{P!AZU7x#4(Rtn!q4nCwqol^y6
 }0|Ko=*JK~;zMKV=9Nai:wxu{J&UV#HaU
 )*BiC<),`+t*gka<W=Z.%T5WGHZpI30D<
 Pq>&]BS6R&j?#tP7iaV}-}`\?[_[Z^LBM
 PG-FKj'\xwuZ1=Q`^`5,$N$Q@[!CuRzJ2
 D|vBy!^zkhdf3C5PAkR?V((-%><hn|3='
 i2Qx]D$qs4O`1@fevnG'2\11Vf3piU37@
 5:dfd45*(7^%5ap\zIyl"'f,$ee,J4Gw:
 cgNKLie3nx9(`efSlg6#[K"@WjhZ}r[Sc
 un&sBCS,T[/3]KAeEnQ7lU)3Pn,0)G/6N
 -wyzj/MTd#A;r*(ds./df3r8Odaf?/<#r Parser Syntax Error Interpreter #
  56. 27 Advanced Fuzzing: Specialized Generators • Specialize generation for a

    domain
  57. 27 Advanced Fuzzing: Specialized Generators • Specialize generation for a

    domain
  58. 27 Advanced Fuzzing: Specialized Generators • Specialize generation for a

    domain 80,000 lines of code
  59. 28 The Holy Grail of Fuzzing

  60. 28 The Holy Grail of Fuzzing Parsers

  61. Overcoming Parsers

  62. Overcoming Parsers

  63. 30 Overcoming Parsers

  64. 30 Overcoming Parsers

  65. 30 Monolithic Overcoming Parsers

  66. 31

  67. 32 The Missing Piece: Formal Specification

  68. 33 Grammar

  69. 34 Formal Languages Formal Language Descriptions

  70. 34 Formal Languages Formal Language Descriptions 3. Regular (Chomsky,1956)

  71. 34 Formal Languages Formal Language Descriptions 3. Regular Context Free

    (Chomsky,1956) Argument Stack
  72. 34 Formal Languages Formal Language Descriptions 3. Regular Context Free

    Recursively Enumerable (Chomsky,1956) Argument Stack Return Stack
  73. 34 Formal Languages Formal Language Descriptions 3. Regular Context Free

    Recursively Enumerable (Chomsky,1956) Easy to produce and parse Argument Stack Return Stack
  74. 35 Grammar <start> := <expr> <expr> := <expr> '+' <expr>

    | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Arithmetic expression grammar
  75. 35 Grammar <start> := <expr> <expr> := <expr> '+' <expr>

    | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Arithmetic expression grammar
  76. 35 Grammar <start> := <expr> <expr> := <expr> '+' <expr>

    | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Arithmetic expression grammar <expr> key
  77. 35 Grammar <start> := <expr> <expr> := <expr> '+' <expr>

    | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Arithmetic expression grammar Definition for <expr> <expr> key
  78. 36 <start> := <expr> <expr> := <expr> '+' <expr> |

    <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Grammar Arithmetic expression grammar
  79. 36 <start> := <expr> <expr> := <expr> '+' <expr> |

    <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Grammar Arithmetic expression grammar Expansion Rule
  80. 36 <start> := <expr> <expr> := <expr> '+' <expr> |

    <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Grammar Arithmetic expression grammar Expansion Rule Terminal Symbol
  81. 36 <start> := <expr> <expr> := <expr> '+' <expr> |

    <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Grammar Arithmetic expression grammar Expansion Rule Terminal Symbol Nonterminal Symbol
  82. 36 <start> := <expr> <expr> := <expr> '+' <expr> |

    <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Grammar Arithmetic expression grammar Expansion Rule Terminal Symbol Nonterminal Symbol
  83. 37 Grammars As recognizers <start> := <expr> <expr> := <expr>

    '+' <expr> | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
  84. 37 Grammars As recognizers (8 / 3) * 49 <start>

    := <expr> <expr> := <expr> '+' <expr> | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
  85. 37 Grammars As recognizers (8 / 3) * 49 <start>

    := <expr> <expr> := <expr> '+' <expr> | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
  86. 38 Grammars <start> := <expr> <expr> := <expr> '+' <expr>

    | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] As producers (Hanford 1970) (Purdom 1972)
  87. 38 Grammars8.2 - 27 - -9 / +((+9 * --2

    + --+-+- ((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4) )))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * + (8 - 5 - 6)) * (-(a-+(((+(4))))) - + +4) / +(-+---((5.6 - --(3 * -1.8 * + (6 * +-(((-(-6) * ---+6)) / +--(+-+- 7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(- -2 - -++-9.0)))) / 5 * --++090 + * - +5 + 7.513)))) - (+1 / ++((-84)))))) )) * 8.2 - 27 - -9 / +((+9 * --2 + - -+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(( (+(4))))) - ++4) / +(-+---((5.6 - -- (3 * -1.8 * +(6 * +-(((-(-6) * ---+6 )) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6 .37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 <start> := <expr> <expr> := <expr> '+' <expr> | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] As producers (Hanford 1970) (Purdom 1972)
  88. 39 Grammars As effective producers 8.2 - 27 - -9

    / +((+9 * --2 + --+-+- ((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4) )))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * + (8 - 5 - 6)) * (-(a-+(((+(4))))) - + +4) / +(-+---((5.6 - --(3 * -1.8 * + (6 * +-(((-(-6) * ---+6)) / +--(+-+- 7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(- -2 - -++-9.0)))) / 5 * --++090 + * - +5 + 7.513)))) - (+1 / ++((-84)))))) )) * 8.2 - 27 - -9 / +((+9 * --2 + - -+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(( (+(4))))) - ++4) / +(-+---((5.6 - -- (3 * -1.8 * +(6 * +-(((-(-6) * ---+6 )) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6 .37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090
  89. 39 Grammars As effective producers Interpreter Parser ✘ ✔ 8.2

    - 27 - -9 / +((+9 * --2 + --+-+- ((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4) )))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * + (8 - 5 - 6)) * (-(a-+(((+(4))))) - + +4) / +(-+---((5.6 - --(3 * -1.8 * + (6 * +-(((-(-6) * ---+6)) / +--(+-+- 7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(- -2 - -++-9.0)))) / 5 * --++090 + * - +5 + 7.513)))) - (+1 / ++((-84)))))) )) * 8.2 - 27 - -9 / +((+9 * --2 + - -+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(( (+(4))))) - ++4) / +(-+---((5.6 - -- (3 * -1.8 * +(6 * +-(((-(-6) * ---+6 )) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6 .37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090
  90. 40 Where to Get the Grammar From?

  91. 41 •Reference Specification?

  92. 41 The standard spec •Reference Specification?

  93. 41 The standard spec Buggy Implementation •Reference Specification?

  94. 41 The standard spec Buggy Implementation "Extra" Features •Reference Specification?

  95. 41 The standard spec Buggy Implementation "Extra" Features "Be liberal

    in what you accept, and conservative in what you send"
 (the cause of trouble) Postel's Law "Accepted" Bugs •Reference Specification?
  96. 42 https://www.json.org

  97. 42 object { } { members } members pair pair

    , members pair string : value array [ ] [ elements ] elements value value , elements value string number object array true false null string " " " chars " chars char char chars char UNICODE \ [",\,CTRL] \" \\ \/ \b \f \n \r \t \u hex hex hex hex number int int frac int exp int frac exp int digit onenine digits - digit - onenine digits frac . digits exp e digits hex digit A - F a - f digits digit digit digits e e e+ e- E E+ E- https://www.json.org
  98. 43 Parsing JSON is a Minefield http://seriot.ch/

  99. 43 Parsing JSON is a Minefield http://seriot.ch/

  100. 43 Parsing JSON is a Minefield http://seriot.ch/

  101. 43 Parsing JSON is a Minefield http://seriot.ch/

  102. 43 Parsing JSON is a Minefield http://seriot.ch/ Expected Parse Fail

    (Expect Success) Parse Success (Expect Fail) Parse Success (Undefined) Parse Fail (Undefined) Parser Crash Timeout
  103. 43 Parsing JSON is a Minefield http://seriot.ch/ Expected Parse Fail

    (Expect Success) Parse Success (Expect Fail) Parse Success (Undefined) Parse Fail (Undefined) Parser Crash Timeout
  104. None
  105. 45 Where to Get the Grammar From?

  106. 45 Where to Get the Grammar From? Hand-written parsers already

    encode the grammar
  107. 46 How to Extract This Grammar?

  108. 46 How to Extract This Grammar? • Inputs -> Dynamic

    Control Dependence Trees
  109. 46 How to Extract This Grammar? • Inputs -> Dynamic

    Control Dependence Trees • DCD Trees -> Context Free Grammar
  110. 47 Control Dependence Graph Statement B is control dependent on

    A if A determines whether B executes. def parse_csv(s,i): while s[i:]: if is_digit(s[i]): n,j = num(s[i:]) i = i+j else: comma(s[i]) i += 1
  111. 47 Control Dependence Graph Statement B is control dependent on

    A if A determines whether B executes. def parse_csv(s,i): while s[i:]: if is_digit(s[i]): n,j = num(s[i:]) i = i+j else: comma(s[i]) i += 1 CDG for parse_csv
  112. 47 Control Dependence Graph Statement B is control dependent on

    A if A determines whether B executes. def parse_csv(s,i): while s[i:]: if is_digit(s[i]): n,j = num(s[i:]) i = i+j else: comma(s[i]) i += 1 CDG for parse_csv while: determines whether if: executes
  113. 48 def parse_csv(s,i): while s[i:]: if is_digit(s[i]): n,j = num(s[i:])

    i = i+j else: comma(s[i]) i += 1 CDG for parse_csv Dynamic Control Dependence Tree Each statement execution is represented as a separate node
  114. 48 def parse_csv(s,i): while s[i:]: if is_digit(s[i]): n,j = num(s[i:])

    i = i+j else: comma(s[i]) i += 1 CDG for parse_csv Dynamic Control Dependence Tree Each statement execution is represented as a separate node DCD Tree for call parse_csv()
  115. 49 def parse_csv(s,i): while s[i:]: if is_digit(s[i]): n,j = num(s[i:])

    i = i+j else: comma(s[i]) i += 1 DCD Tree ~ Parse Tree •No tracking beyond input buffer •Characters are attached to nodes where they are accessed last "12," "12,"
  116. 49 def parse_csv(s,i): while s[i:]: if is_digit(s[i]): n,j = num(s[i:])

    i = i+j else: comma(s[i]) i += 1 '1' '2' ',' DCD Tree ~ Parse Tree •No tracking beyond input buffer •Characters are attached to nodes where they are accessed last "12," "12,"
  117. 50 def is_digit(i): return i in '0123456789' def parse_num(s,i): n

    = '' while s[i:] and is_digit(s[i]): n += s[i] i = i +1 return i,n def parse_paren(s, i): assert s[i] == '(' i, v = parse_expr(s, i+1) if s[i:] == '': raise Ex(s, i) assert s[i] == ')' return i+1, v def parse_expr(s, i = 0): expr, is_op = [], True while s[i:]: c = s[i] if isdigit(c): if not is_op: raise Ex(s,i) i,num = parse_num(s,i) expr.append(num) is_op = False elif c in ['+', '-', '*', '/']: if is_op: raise Ex(s,i) expr.append(c) is_op, i = True, i + 1 elif c == '(': if not is_op: raise Ex(s,i) i, cexpr = parse_paren(s, i) expr.append(cexpr) is_op = False elif c == ')': break else: raise Ex(s,i) if is_op: raise Ex(s,i) return i, expr 9+3/4 Parse tree for parse_expr('9+3/4')
  118. 50 def is_digit(i): return i in '0123456789' def parse_num(s,i): n

    = '' while s[i:] and is_digit(s[i]): n += s[i] i = i +1 return i,n def parse_paren(s, i): assert s[i] == '(' i, v = parse_expr(s, i+1) if s[i:] == '': raise Ex(s, i) assert s[i] == ')' return i+1, v def parse_expr(s, i = 0): expr, is_op = [], True while s[i:]: c = s[i] if isdigit(c): if not is_op: raise Ex(s,i) i,num = parse_num(s,i) expr.append(num) is_op = False elif c in ['+', '-', '*', '/']: if is_op: raise Ex(s,i) expr.append(c) is_op, i = True, i + 1 elif c == '(': if not is_op: raise Ex(s,i) i, cexpr = parse_paren(s, i) expr.append(cexpr) is_op = False elif c == ')': break else: raise Ex(s,i) if is_op: raise Ex(s,i) return i, expr 9+3/4 Parse tree for parse_expr('9+3/4')
  119. 51 def is_digit(i): return i in '0123456789' def parse_num(s,i): n

    = '' while s[i:] and is_digit(s[i]): n += s[i] i = i +1 return i,n def parse_paren(s, i): assert s[i] == '(' i, v = parse_expr(s, i+1) if s[i:] == '': raise Ex(s, i) assert s[i] == ')' return i+1, v def parse_expr(s, i = 0): expr, is_op = [], True while s[i:]: c = s[i] if isdigit(c): if not is_op: raise Ex(s,i) i,num = parse_num(s,i) expr.append(num) is_op = False elif c in ['+', '-', '*', '/']: if is_op: raise Ex(s,i) expr.append(c) is_op, i = True, i + 1 elif c == '(': if not is_op: raise Ex(s,i) i, cexpr = parse_paren(s, i) expr.append(cexpr) is_op = False elif c == ')': break else: raise Ex(s,i) if is_op: raise Ex(s,i) return i, expr 9+3/4 Identifying Compatible Nodes Which nodes correspond to the same nonterminal
  120. 51 def is_digit(i): return i in '0123456789' def parse_num(s,i): n

    = '' while s[i:] and is_digit(s[i]): n += s[i] i = i +1 return i,n def parse_paren(s, i): assert s[i] == '(' i, v = parse_expr(s, i+1) if s[i:] == '': raise Ex(s, i) assert s[i] == ')' return i+1, v def parse_expr(s, i = 0): expr, is_op = [], True while s[i:]: c = s[i] if isdigit(c): if not is_op: raise Ex(s,i) i,num = parse_num(s,i) expr.append(num) is_op = False elif c in ['+', '-', '*', '/']: if is_op: raise Ex(s,i) expr.append(c) is_op, i = True, i + 1 elif c == '(': if not is_op: raise Ex(s,i) i, cexpr = parse_paren(s, i) expr.append(cexpr) is_op = False elif c == ')': break else: raise Ex(s,i) if is_op: raise Ex(s,i) return i, expr 9+3/4 Identifying Compatible Nodes Which nodes correspond to the same nonterminal
  121. 52 3 * (9 + 1)

  122. 52 3 * (9 + 1)

  123. 52 (9 + 1) * 3 3 * (9 +

    1)
  124. 52 (9 + 1) * 3 3 * (9 +

    1)
  125. 52 (9 + 1) * 3 3 * (9 +

    1)
  126. 53 3 * (9 + 1)

  127. 53 3 * (9 + 1)

  128. 53 9 + 1 3 * (9 + 1)

  129. 53 9 + 1 3 * (9 + 1)

  130. 53 9 + 1 3 * (9 + 1)

  131. 54 3 * (9 + 1)

  132. 54 3 (9 + 1) * 3 * (9 +

    1)
  133. 54 3 (9 + 1) * 3 * (9 +

    1)
  134. 54 3 (9 + 1) * 3 * (9 +

    1)
  135. 55 3*(1) 1

  136. 56 3*(1) 1

  137. 56 3*(1) 1 <parse_expr> := <while 1:1> <while 1:0> <while

    1:1>
  138. 56 3*(1) 1 <parse_expr> := <while 1:1> <while 1:0> <while

    1:1> <while 1:1> <parse_expr> :=
  139. <parse_expr> := <while 1:1> <while 1:0> <while 1:1> | <while

    1:1> <while 1:0> <while 1:1> <while 1:0> <while 1:1> <while 1:0> <while 1:1> | <while 1:1> <while 1:0> <while 1:1> <while 1:0> <while 1:1> | <while 1:1> <while 1:1> := <if 1:1> <if 1:1> := <parse_num> | <parse_paren> <parse_num> := <is_digit> <is_digit> := '3' | '1' <parse_paren>:= '(' <parse_expr> ')' <while 1:0> := <if 1:0> <if 1:0> := '*' 57
  140. <parse_expr> := <while_s> <while_s> := <while_1:1> <while_1:0> <while_s> | <while_1:1>

    <parse_expr> := <while 1:1> <while 1:0> <while 1:1> | <while 1:1> <while 1:0> <while 1:1> <while 1:0> <while 1:1> <while 1:0> <while 1:1> | <while 1:1> <while 1:0> <while 1:1> <while 1:0> <while 1:1> | <while 1:1> <while 1:1> := <if 1:1> <if 1:1> := <parse_num> | <parse_paren> <parse_num> := <is_digit> <is_digit> := '3' | '1' <parse_paren>:= '(' <parse_expr> ')' <while 1:0> := <if 1:0> <if 1:0> := '*' 57
  141. 58 def is_digit(i): return i in '0123456789' def parse_num(s,i): n

    = '' while s[i:] and is_digit(s[i]): n += s[i] i = i +1 return i,n def parse_paren(s, i): assert s[i] == '(' i, v = parse_expr(s, i+1) if s[i:] == '': raise Ex(s, i) assert s[i] == ')' return i+1, v def parse_expr(s, i = 0): expr, is_op = [], True while s[i:]: c = s[i] if isdigit(c): if not is_op: raise Ex(s,i) i,num = parse_num(s,i) expr.append(num) is_op = False elif c in ['+', '-', '*', '/']: if is_op: raise Ex(s,i) expr.append(c) is_op, i = True, i + 1 elif c == '(': if not is_op: raise Ex(s,i) i, cexpr = parse_paren(s, i) expr.append(cexpr) is_op = False elif c == ')': break else: raise Ex(s,i) if is_op: raise Ex(s,i) return i, expr <START> := <parse_expr.0-0-c> <parse_expr.0-0-c> := <parse_expr.0-1-s><parse_expr.0> | <parse_expr.0> <parse_expr.0-1-s> := <parse_expr.0><parse_expr.0-2> | <parse_expr.0><parse_expr.0-2><parse_expr.0-1-s> <parse_expr.0> := '(' <parse_expr.0-0-c> ')' | <parse_num.0-1-s> <parse_expr.0-2> := '*' | '+' | '-' | '/' <parse_num.0-1-s> := <is_digit.0-0-c> | <is_digit.0-0-c><parse_num.0-1-s> <is_digit.0-0-c> : [0-9] calc.py Recovered Arithmetic Grammar
  142. 59 <START> := <parse_expr.0-0-c> <parse_expr.0-0-c> := <parse_expr.0-1-s><parse_expr.0> | <parse_expr.0> <parse_expr.0-1-s>

    := <parse_expr.0><parse_expr.0-2> | <parse_expr.0><parse_expr.0-2><parse_expr.0-1-s> <parse_expr.0> := '(' <parse_expr.0-0-c> ')' | <parse_num.0-1-s> <parse_expr.0-2> := '*' | '+' | '-' | '/' <parse_num.0-1-s> := <is_digit.0-0-c> | <is_digit.0-0-c><parse_num.0-1-s> <is_digit.0-0-c> : [0-9]
  143. 59 8.2 - 27 - -9 / +((+9 * --2

    + --+-+- ((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4) )))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * + (8 - 5 - 6)) * (-(a-+(((+(4))))) - + +4) / +(-+---((5.6 - --(3 * -1.8 * + (6 * +-(((-(-6) * ---+6)) / +--(+-+- 7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(- -2 - -++-9.0)))) / 5 * --++090 + * - +5 + 7.513)))) - (+1 / ++((-84)))))) )) * 8.2 - 27 - -9 / +((+9 * --2 + - -+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(( (+(4))))) - ++4) / +(-+---((5.6 - -- (3 * -1.8 * +(6 * +-(((-(-6) * ---+6 )) / +--(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6 .37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 <START> := <parse_expr.0-0-c> <parse_expr.0-0-c> := <parse_expr.0-1-s><parse_expr.0> | <parse_expr.0> <parse_expr.0-1-s> := <parse_expr.0><parse_expr.0-2> | <parse_expr.0><parse_expr.0-2><parse_expr.0-1-s> <parse_expr.0> := '(' <parse_expr.0-0-c> ')' | <parse_num.0-1-s> <parse_expr.0-2> := '*' | '+' | '-' | '/' <parse_num.0-1-s> := <is_digit.0-0-c> | <is_digit.0-0-c><parse_num.0-1-s> <is_digit.0-0-c> : [0-9]
  144. 60 <START> ::= <json_raw> <json_raw> ::= '"' <json_string'> | '['

    <json_list'> | '{' <json_dict'> | <json_number'> | 'true' | 'false' | 'null' <json_number'> ::= <json_number>+ | <json_number>+ 'e' <json_number>+ <json_number> ::= '+' | '-' | '.' | [0-9] | 'E' | 'e' <json_string'> ::= <json_string>* '"' <json_list'> ::= ']' | <json_raw> (','<json_raw>)* ']' | ( ',' <json_raw>)+ (',' <json_raw>)* ']' <json_dict'> ::= '}' | ( '"' <json_string'> ':' <json_raw> ',' )* '"'<json_string'> ':' <json_raw> '}' <json_string> ::= ' ' | '!' | '#' | '$' | '%' | '&' | ''' | '*' | '+' | '-' | ',' | '.' | '/' | ':' | ';' | '<' | '=' | '>' | '?' | '@' | '[' | ']' | '^' | '_', ''',| '{' | '|' | '}' | '~' | '[A-Za-z0-9]' | '\' <decode_escape> <decode_escape> ::= '"' | '/' | 'b' | 'f' | 'n' | 'r' | 't' stm.next() if expect_key: raise JSONError(E_DKEY, stm, stm.pos) if c == '}': return result expect_key = 1 continue # parse out a key/value pair elif c == '"': key = _from_json_string(stm) stm.skipspaces() c = stm.next() if c != ':': raise JSONError(E_COLON, stm, stm.pos) stm.skipspaces() val = _from_json_raw(stm) result[key] = val expect_key = 0 continue raise JSONError(E_MALF, stm, stm.pos) def _from_json_raw(stm): while True: stm.skipspaces() c = stm.peek() if c == '"': return _from_json_string(stm) elif c == '{': return _from_json_dict(stm) elif c == '[': return _from_json_list(stm) elif c == 't': return _from_json_fixed(stm, 'true', True, E_BOOL) elif c == 'f': return _from_json_fixed(stm, 'false', False, E_BOOL) elif c == 'n': return _from_json_fixed(stm, 'null', None, E_NULL) elif c in NUMSTART: return _from_json_number(stm) raise JSONError(E_MALF, stm, stm.pos) def from_json(data): stm = JSONStream(data) return _from_json_raw(stm) microjson.py Recovered JSON grammar
  145. 61 <START> ::= <json_raw> <json_raw> ::= '"' <json_string'> | '['

    <json_list'> | '{' <json_dict'> | <json_number'> | 'true' | 'false' | 'null' <json_number'> ::= <json_number>+ | <json_number>+ 'e' <json_number>+ <json_number> ::= '+' | '-' | '.' | [0-9] | 'E' | 'e' <json_string'> ::= <json_string>* '"' <json_list'> ::= ']' | <json_raw> (','<json_raw>)* ']' | ( ',' <json_raw>)+ (',' <json_raw>)* ']' <json_dict'> ::= '}' | ( '"' <json_string'> ':' <json_raw> ',' )* '"'<json_string'> ':' <json_raw> '}' <json_string> ::= ' ' | '!' | '#' | '$' | '%' | '&' | ''' | '*' | '+' | '-' | ',' | '.' | '/' | ':' | ';' | '<' | '=' | '>' | '?' | '@' | '[' | ']' | '^' | '_', ''',| '{' | '|' | '}' | '~' | '[A-Za-z0-9]' | '\' <decode_escape> <decode_escape> ::= '"' | '/' | 'b' | 'f' | 'n' | 'r' | 't' stm.next() if expect_key: raise JSONError(E_DKEY, stm, stm.pos) if c == '}': return result expect_key = 1 continue # parse out a key/value pair elif c == '"': key = _from_json_string(stm) stm.skipspaces() c = stm.next() if c != ':': raise JSONError(E_COLON, stm, stm.pos) stm.skipspaces() val = _from_json_raw(stm) result[key] = val expect_key = 0 continue raise JSONError(E_MALF, stm, stm.pos) def _from_json_raw(stm): while True: stm.skipspaces() c = stm.peek() if c == '"': return _from_json_string(stm) elif c == '{': return _from_json_dict(stm) elif c == '[': return _from_json_list(stm) elif c == 't': return _from_json_fixed(stm, 'true', True, E_BOOL) elif c == 'f': return _from_json_fixed(stm, 'false', False, E_BOOL) elif c == 'n': return _from_json_fixed(stm, 'null', None, E_NULL) elif c in NUMSTART: return _from_json_number(stm) raise JSONError(E_MALF, stm, stm.pos) def from_json(data): stm = JSONStream(data) return _from_json_raw(stm) microjson.py Recovered JSON grammar Mimid Gopinath, Mathis, and Zeller. Mining Input Grammars from Dynamic Control Flow. ESEC/FSE 2020. •Javascript •C •Lisp •JSON •URL •CGI
  146. Actual Specification

  147. Dynamic Approximation (Mimid) Actual Specification

  148. Dynamic Approximation (Mimid) Static Approximation (Static Mimid) Actual Specification

  149. Dynamic Approximation (Mimid) Static Approximation (Static Mimid) Actual Specification IOT

    & Embedded
  150. 63

  151. Generating Unbiased Samples

  152. Finding Good Samples Seed corpus?

  153. Finding Good Samples Seed corpus? (Blind spots)

  154. • Differentiate incomplete and incorrect inputs • Solve one input

    symbol at a time systematically Key Idea
  155. 67 Sample Free Generators

  156. 67 Sample Free Generators

  157. 67 Sample Free Generators

  158. 67 Sample Free Generators A

  159. 67 Sample Free Generators A A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0

  160. 67 Sample Free Generators A ( A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0

  161. 67 Sample Free Generators A ( 2 A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0

  162. 67 Sample Free Generators A ( 2 - B 9

    ) 4 ) A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0 B ∉ +,-,1,2,3,4,5,6,7,8,9,0,) ) ∉ +,-,1,2,3,4,5,6,7,8,9,0
  163. 67 Sample Free Generators A ( 2 - B 9

    ) 4 ) A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0 B ∉ +,-,1,2,3,4,5,6,7,8,9,0,) ) ∉ +,-,1,2,3,4,5,6,7,8,9,0
  164. 67 Sample Free Generators A ( 2 - B 9

    ) 4 ) A ∉ (,+,-,1,2,3,4,5,6,7,8,9,0 B ∉ +,-,1,2,3,4,5,6,7,8,9,0,) ) ∉ +,-,1,2,3,4,5,6,7,8,9,0 (2-94)
  165. Mathis, Gopinath, Mera, Kampmann, Höschele, and Zeller. Parser Directed Fuzzing.

    PLDI 2019. Mathis, Gopinath and Zeller Learning Input Tokens for Effective Fuzzing. ISSTA 2020. Gopinath, Bendrissou, Mathis, and Zeller Black-box Testing with Monotonic Prefixes. ISSRE 2021 (submitted). Sample Free Generators A ( 2 - B 9 ) 4 )
  166. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Fast Fuzzing with Grammars fuzz(expr_grammar, '<start>')
  167. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Fast Fuzzing with Grammars fuzz(expr_grammar, '<start>') def fuzz(grammar, key): return collapse_tree(gen_key(grammar, key))
  168. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Fast Fuzzing with Grammars def gen_key(grammar, key):
 if is_terminal_symbol(key):
 return key else: next_rule = random.choice(rules) return gen_rule(grammar, grammar[key][next_rule]) fuzz(expr_grammar, '<start>') def fuzz(grammar, key): return collapse_tree(gen_key(grammar, key))
  169. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Fast Fuzzing with Grammars def gen_key(grammar, key):
 if is_terminal_symbol(key):
 return key else: next_rule = random.choice(rules) return gen_rule(grammar, grammar[key][next_rule]) fuzz(expr_grammar, '<start>') def fuzz(grammar, key): return collapse_tree(gen_key(grammar, key)) def gen_rule(grammar, rule): return [gen_key(token) for token in rule]
  170. Fast Fuzzing with Grammars fuzz(expr_grammar, '<start>') def gen_key(grammar, key):
 if

    is_terminal_symbol(key):
 return key else: next_rule = random.choice(rules) return gen_rule(grammar, grammar[key][next_rule]) fuzz(expr_grammar, '<start>') def fuzz(grammar, key): return collapse_tree(gen_key(grammar, key)) def gen_rule(grammar, rule): return [gen_key(token) for token in rule]
  171. Fast Fuzzing with Grammars <start> <expr> <expr> <expr> + <number>

    <number> <integer> <integer> <digit> <digit> 1 8 fuzz(expr_grammar, '<start>') def gen_key(grammar, key):
 if is_terminal_symbol(key):
 return key else: next_rule = random.choice(rules) return gen_rule(grammar, grammar[key][next_rule]) fuzz(expr_grammar, '<start>') def fuzz(grammar, key): return collapse_tree(gen_key(grammar, key)) def gen_rule(grammar, rule): return [gen_key(token) for token in rule]
  172. Fast Fuzzing with Grammars <start> <expr> <expr> <expr> + <number>

    <number> <integer> <integer> <digit> <digit> 1 8 fuzz(expr_grammar, '<start>') def collapse(tree): key, children = tree if not children: return tree return ''.join([collapse(c) for c in children]) def gen_key(grammar, key):
 if is_terminal_symbol(key):
 return key else: next_rule = random.choice(rules) return gen_rule(grammar, grammar[key][next_rule]) fuzz(expr_grammar, '<start>') def fuzz(grammar, key): return collapse_tree(gen_key(grammar, key)) def gen_rule(grammar, rule): return [gen_key(token) for token in rule]
  173. Fast Fuzzing with Grammars <start> <expr> <expr> <expr> + <number>

    <number> <integer> <integer> <digit> <digit> 1 8 fuzz(expr_grammar, '<start>') 1 8 + "1 + 8" def collapse(tree): key, children = tree if not children: return tree return ''.join([collapse(c) for c in children]) def gen_key(grammar, key):
 if is_terminal_symbol(key):
 return key else: next_rule = random.choice(rules) return gen_rule(grammar, grammar[key][next_rule]) fuzz(expr_grammar, '<start>') def fuzz(grammar, key): return collapse_tree(gen_key(grammar, key)) def gen_rule(grammar, rule): return [gen_key(token) for token in rule]
  174. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Compiling the Grammar
  175. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Compiling the Grammar def start(): expr() def expr(): match (random() % 6): case 0: expr(); print('+'); expr() case 1: expr(); print('-'); expr() case 2: expr(); print('/'); expr() case 3: expr(); print('*'); expr() case 4: print('('); expr(); print(')') case 5: number() def number(): match (random() % 2): case 0: integer() case 1: integer(); print('.'); integer() def integer(): match (random() % 2): case 0: digit(); integer() case 1: digit() def digit(): match (random() % 10): case 0: print('0') case 1: print('1') case 2: print('2') case 3: print('3') case 4: print('4') case 5: print('5') case 6: print('6') case 7: print('7')
  176. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Compiling the Grammar def start(): expr() def expr(): match (random() % 6): case 0: expr(); print('+'); expr() case 1: expr(); print('-'); expr() case 2: expr(); print('/'); expr() case 3: expr(); print('*'); expr() case 4: print('('); expr(); print(')') case 5: number() def number(): match (random() % 2): case 0: integer() case 1: integer(); print('.'); integer() def integer(): match (random() % 2): case 0: digit(); integer() case 1: digit() def digit(): match (random() % 10): case 0: print('0') case 1: print('1') case 2: print('2') case 3: print('3') case 4: print('4') case 5: print('5') case 6: print('6') case 7: print('7')
  177. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Compiling the Grammar def start(): expr() def expr(): match (random() % 6): case 0: expr(); print('+'); expr() case 1: expr(); print('-'); expr() case 2: expr(); print('/'); expr() case 3: expr(); print('*'); expr() case 4: print('('); expr(); print(')') case 5: number() def number(): match (random() % 2): case 0: integer() case 1: integer(); print('.'); integer() def integer(): match (random() % 2): case 0: digit(); integer() case 1: digit() def digit(): match (random() % 10): case 0: print('0') case 1: print('1') case 2: print('2') case 3: print('3') case 4: print('4') case 5: print('5') case 6: print('6') case 7: print('7') def start(rops): expr(rops) def expr(rops): match (rops.next % 6): case 0: expr(rops); print('+'); e case 1: expr(rops); print('-'); e case 2: expr(rops); print('/'); e case 3: expr(rops); print('*'); e case 4: print('('); expr(rops); p case 5: number(rops) def number(rops): match (rops.next % 2): case 0: integer(rops) case 1: integer(rops); print('.') def integer(rops): match (rops.next % 2): case 0: digit(rops); integer(rops case 1: digit(rops) def digit(rops): match (rops.next % 10): case 0: print('0') case 1: print('1') case 2: print('2') case 3: print('3') case 4: print('4') case 5: print('5') case 6: print('6') case 7: print('7')
  178. <start> := <expr> <expr> := <expr> '+' <expr> | <expr>

    '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] Compiling the Grammar def start(): expr() def expr(): match (random() % 6): case 0: expr(); print('+'); expr() case 1: expr(); print('-'); expr() case 2: expr(); print('/'); expr() case 3: expr(); print('*'); expr() case 4: print('('); expr(); print(')') case 5: number() def number(): match (random() % 2): case 0: integer() case 1: integer(); print('.'); integer() def integer(): match (random() % 2): case 0: digit(); integer() case 1: digit() def digit(): match (random() % 10): case 0: print('0') case 1: print('1') case 2: print('2') case 3: print('3') case 4: print('4') case 5: print('5') case 6: print('6') case 7: print('7') def start(rops): expr(rops) def expr(rops): match (rops.next % 6): case 0: expr(rops); print('+'); e case 1: expr(rops); print('-'); e case 2: expr(rops); print('/'); e case 3: expr(rops); print('*'); e case 4: print('('); expr(rops); p case 5: number(rops) def number(rops): match (rops.next % 2): case 0: integer(rops) case 1: integer(rops); print('.') def integer(rops): match (rops.next % 2): case 0: digit(rops); integer(rops case 1: digit(rops) def digit(rops): match (rops.next % 10): case 0: print('0') case 1: print('1') case 2: print('2') case 3: print('3') case 4: print('4') case 5: print('5') case 6: print('6') case 7: print('7')
  179. Grammar Fuzzer grammar = """
 <start> := <expr> <expr> :=

    <expr> '+' <expr> | <expr> '-' <expr> | <expr> '/' <expr> | <expr> '*' <expr> | '(' <expr> ')' | <number> <number> := <integer> | <integer> '.' <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
 """ generate(grammar) Fast Fuzzers def start(): expr() def expr(): match (random() % 6): case 0: expr(); print('+'); expr() case 1: expr(); print('-'); expr() case 2: expr(); print('/'); expr() case 3: expr(); print('*'); expr() case 4: print('('); expr(); print(')') case 5: number() def number(): match (random() % 2): case 0: integer() case 1: integer(); print('.'); integer() def integer(): match (random() % 2): case 0: digit(); integer() case 1: digit() def digit(): match (random() % 10): case 0: print('0') case 1: print('1') case 2: print('2') case 3: print('3') case 4: print('4') case 5: print('5') case 6: print('6') case 7: print('7') Compiled Grammar (F1) Building Fast Fuzzers Gopinath and Zeller 2019 def start_0(rops): r = next(rops) if 0 <= r < 43: expr_0() elif 43 <= r < 85: expr_1() elif 85 <= r < 128: expr_2() elif 128 <= r < 171: expr_3() elif 171 <= r < 213: expr_4() else: expr_5() def expr_0(rops): r = next(rops) if 0 <= r < 43: expr_0() elif 43 <= r < 85: expr_1() elif 85 <= r < 128: expr_2() elif 128 <= r < 171: expr_3() elif 171 <= r < 213: expr_4() else: expr_5() print('+') r = next(rops) if 0 <= r < 43: expr_0() elif 43 <= r < 85: expr_1() elif 85 <= r < 128: expr_2() elif 128 <= r < 171: expr_3() elif 171 <= r < 213: expr_4() else: expr_5() Grammar VM (F1)
  180. 73 The Fuzzing Pipeline Program Under Test

  181. 73 The Fuzzing Pipeline Program Under Test pFuzzer Active Guidance

    F1 Grammar Fuzzer Grammar Inputs Grammar Miner Samples Active Learning
  182. 74 The Fuzzing Synergy Mimid Grammar Miner Parser Directed pFuzzer

    F1 Fuzzer VM Mathis, Gopinath, Mera, Kampmann, Höschele, and Zeller. Mining Input Grammars from Dynamic Control Flow. PLDI 2019. Mathis, Gopinath and Zeller Learning Input Tokens for Effective Fuzzing. ISSTA 2020. Gopinath, Bendrissou, Mathis, and Andreas Zeller Black-box Testing with Monotonic Prefixes. ISSTA 2021 (submitted). Gopinath and Zeller Building Fast Fuzzers 2019 (unpublished) Gopinath, Mathis, and Zeller Mining Input Grammars with Dynamic Control Flow. FSE 2020.
  183. 74 The Fuzzing Synergy Mimid Grammar Miner Parser Directed pFuzzer

    F1 Fuzzer VM Mathis, Gopinath, Mera, Kampmann, Höschele, and Zeller. Mining Input Grammars from Dynamic Control Flow. PLDI 2019. Mathis, Gopinath and Zeller Learning Input Tokens for Effective Fuzzing. ISSTA 2020. Gopinath, Bendrissou, Mathis, and Andreas Zeller Black-box Testing with Monotonic Prefixes. ISSTA 2021 (submitted). Gopinath and Zeller Building Fast Fuzzers 2019 (unpublished) Gopinath, Mathis, and Zeller Mining Input Grammars with Dynamic Control Flow. FSE 2020.
  184. 74 The Fuzzing Synergy Mimid Grammar Miner Parser Directed pFuzzer

    F1 Fuzzer VM Mathis, Gopinath, Mera, Kampmann, Höschele, and Zeller. Mining Input Grammars from Dynamic Control Flow. PLDI 2019. Mathis, Gopinath and Zeller Learning Input Tokens for Effective Fuzzing. ISSTA 2020. Gopinath, Bendrissou, Mathis, and Andreas Zeller Black-box Testing with Monotonic Prefixes. ISSTA 2021 (submitted). Gopinath and Zeller Building Fast Fuzzers 2019 (unpublished) Gopinath, Mathis, and Zeller Mining Input Grammars with Dynamic Control Flow. FSE 2020.
  185. None
  186. 76 Challenge: Multilevel Envelopes POST /InStock HTTP/1.1 Host: www.stock.org Content-Type:

    application/soap+xml; charset=utf-8 Content-Length: 312 <?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope" soap:encodingStyle="http://w3.org/2001/12/soap-encoding"> <soap:Body xmlns:m="http://www.stock.org/stock"> <m:GetStockPrice> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope>
  187. 76 Challenge: Multilevel Envelopes POST /InStock HTTP/1.1 Host: www.stock.org Content-Type:

    application/soap+xml; charset=utf-8 Content-Length: 312 <?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope" soap:encodingStyle="http://w3.org/2001/12/soap-encoding"> <soap:Body xmlns:m="http://www.stock.org/stock"> <m:GetStockPrice> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope> HTTP POST
  188. 76 Challenge: Multilevel Envelopes POST /InStock HTTP/1.1 Host: www.stock.org Content-Type:

    application/soap+xml; charset=utf-8 Content-Length: 312 <?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope" soap:encodingStyle="http://w3.org/2001/12/soap-encoding"> <soap:Body xmlns:m="http://www.stock.org/stock"> <m:GetStockPrice> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope> HTTP POST XML PAYLOAD
  189. 76 Challenge: Multilevel Envelopes POST /InStock HTTP/1.1 Host: www.stock.org Content-Type:

    application/soap+xml; charset=utf-8 Content-Length: 312 <?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope" soap:encodingStyle="http://w3.org/2001/12/soap-encoding"> <soap:Body xmlns:m="http://www.stock.org/stock"> <m:GetStockPrice> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope> HTTP POST XML PAYLOAD SOAP
  190. 76 Challenge: Multilevel Envelopes POST /InStock HTTP/1.1 Host: www.stock.org Content-Type:

    application/soap+xml; charset=utf-8 Content-Length: 312 <?xml version="1.0"?> <soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope" soap:encodingStyle="http://w3.org/2001/12/soap-encoding"> <soap:Body xmlns:m="http://www.stock.org/stock"> <m:GetStockPrice> <m:StockName>IBM</m:StockName> </m:GetStockPrice> </soap:Body> </soap:Envelope> HTTP POST XML PAYLOAD SOAP RPC Call
  191. 77 Future Challenge: Multilevel Envelopes

  192. 77 Future Challenge: Multilevel Envelopes

  193. 77 Future Challenge: Multilevel Envelopes Mimid

  194. 78 #include <stdio.h> int main() { int number1, number2, number3;

    number1 = 10; number2 = 20; number3 = sum(number1, number2); if (number3 > 100) return 0; return 1; } $ cc example.c -o example example.c 1.Syntactically correct 2.Variables declared before use 3.Use correct types 4.Statically conforming 5.Dynamically conforming 6.Model conforming Challenge: Semantic Envelopes (Mckeeman 1998) (parse) (compile) (link) (run) (synthesis)
  195. 78 #include <stdio.h> int main() { int number1, number2, number3;

    number1 = 10; number2 = 20; number3 = sum(number1, number2); if (number3 > 100) return 0; return 1; } $ cc example.c -o example example.c 1.Syntactically correct 2.Variables declared before use 3.Use correct types 4.Statically conforming 5.Dynamically conforming 6.Model conforming Challenge: Semantic Envelopes (Mckeeman 1998) (parse) (compile) (link) (run) (synthesis)
  196. 79 #include <stdio.h> int main() { int number1, number2, number3;

    number1 = 10; number2 = 20; number3 = sum(number1, number2); if (number3 > 100) return 0; return 1; } $ cc example.c -o example example.c 1.Syntactically correct 2.Variables declared before use 3.Use correct types 4.Statically conforming 5.Dynamically conforming 6.Model conforming Challenge: Semantic Envelopes (Mckeeman 1998) (parse) (compile) (link) (run) (synthesis)
  197. 80

  198. 81 Fuzzing Inputs Program Behavior

  199. 81 Fuzzing Inputs Program Behavior

  200. 81 Fuzzing Inputs Program Behavior ✔

  201. 82

  202. 83 We Found A Crash

  203. 83 We Found A Crash

  204. Why Did My Program Crash?

  205. Why Did My Program Crash? 8.2 - 27 - -9

    / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5. 6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +- -(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---( --+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +-- (+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(- -+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * - +5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +- (--2 - -++-9.0)))) / 5 * --++090 + * -+5 + 7.513) ))) - (+1 / ++((-84)))))))) * 8.2 - 27 - -9 / +(( +9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+ (((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * ( +(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6. 37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0))) ) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090
  206. Why Did My Program Crash? 8.2 - 27 - -9

    / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5. 6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +- -(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---( --+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +-- (+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(- -+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * - +5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +- (--2 - -++-9.0)))) / 5 * --++090 + * -+5 + 7.513) ))) - (+1 / ++((-84)))))))) * 8.2 - 27 - -9 / +(( +9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+ (((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * ( +(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6. 37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0))) ) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 DD Minimized Input ((4))
  207. Why Did My Program Crash? 8.2 - 27 - -9

    / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5. 6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +- -(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---( --+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +-- (+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(- -+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * - +5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +- (--2 - -++-9.0)))) / 5 * --++090 + * -+5 + 7.513) ))) - (+1 / ++((-84)))))))) * 8.2 - 27 - -9 / +(( +9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+ (((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * ( +(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6. 37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0))) ) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 DD Minimized Input ((4)) 00000 ?
  208. Why Did My Program Crash? 8.2 - 27 - -9

    / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5. 6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +- -(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---( --+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +-- (+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(- -+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * - +5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +- (--2 - -++-9.0)))) / 5 * --++090 + * -+5 + 7.513) ))) - (+1 / ++((-84)))))))) * 8.2 - 27 - -9 / +(( +9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+ (((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * ( +(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6. 37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0))) ) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 DD Minimized Input ((4)) 00000 ? ((5)) ?
  209. Why Did My Program Crash? 8.2 - 27 - -9

    / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5. 6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +- -(+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---( --+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) + 8.2 - 27 - -9 / +((+9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+(((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +-- (+-+-7 * (-0 * (+(((((2)) + 8 - 3 - ++9.0 + ---(- -+7 / (1 / +++6.37) + (1) / 482) / +++-+0)))) * - +5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +- (--2 - -++-9.0)))) / 5 * --++090 + * -+5 + 7.513) ))) - (+1 / ++((-84)))))))) * 8.2 - 27 - -9 / +(( +9 * --2 + --+-+-((-1 * +(8 - 5 - 6)) * (-(a-+ (((+(4))))) - ++4) / +(-+---((5.6 - --(3 * -1.8 * +(6 * +-(((-(-6) * ---+6)) / +--(+-+-7 * (-0 * ( +(((((2)) + 8 - 3 - ++9.0 + ---(--+7 / (1 / +++6. 37) + (1) / 482) / +++-+0)))) * -+5 + 7.513)))) - (+1 / ++((-84)))))))) * ++5 / +-(--2 - -++-9.0))) ) / 5 * --++090 ++5 / +-(--2 - -++-9.0)))) / 5 * --++090 DD Minimized Input ((4)) 00000 ? ((5)) ? (++5) ?
  210. 85 Issue 386 from Rhino var A = class extends

    (class {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure
  211. 85 Issue 386 from Rhino var A = class extends

    (class {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure
  212. 85 Issue 386 from Rhino var A = class extends

    (class {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure
  213. 85 Issue 386 from Rhino var A = class extends

    (class {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure
  214. 85 Issue 386 from Rhino var A = class extends

    (class {}){}; Issue 2937 from Closure const [y,y] = []; var {baz:{} = baz => {}} = baz => {}; Issue 385 from Rhino {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842 from Closure Delta Minimization is useful but not sufficient
  215. ( ( 4 ) )

  216. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
  217. ( ( 4 ) )

  218. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
  219. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
  220. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] ✓ Did not reproduce the failure 1 * (2 - 3)
  221. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9]
  222. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c
  223. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c
  224. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c ✓ Did not reproduce the failure 1 + 3 + 4
  225. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c c
  226. 3 * 4 <start> := <expr> <expr> := <term> '

    + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c c
  227. 3 * 4 <start> := <expr> <expr> := <term> '

    + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c c ✓ Did not reproduce the failure
  228. ( ( 4 ) ) <start> := <expr> <expr> :=

    <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c c c c c c c
  229. ( ( 1 - 2 ) ) <start> := <expr>

    <expr> := <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c c c c c c c ( ( 1 - 2 ) )
  230. ( ( 1 - 2 ) ) <start> := <expr>

    <expr> := <term> ' + ' <expr> | <term> ' - ' <expr> | <term> <term> := <factor> ' * ' <term> | <factor> ' / ' <term> | <factor> <factor> := '+' <factor> | '-' <factor> | '(' <expr> ')' | <integer> '.' <integer> | <integer> <integer>:= <digit> <integer> | <digit> <digit> := [0-9] c c c c c c c ✘ reproduced the failure ( ( 1 - 2 ) )
  231. ( ( 1 - 2 ) ) c c c

    c c c c ( ( 1 - 2 ) )
  232. ( ( 1 - 2 ) ) c c c

    c c c c ✘ ( ( 1 - 2 ) )
  233. ( ( 1 - 2 ) ) c c c

    c c c c ✘ ( ( 1 - 2 ) ) ( ( 2 * 3 + 4 ) )
  234. ( ( 1 - 2 ) ) c c c

    c c c c ✘ ( ( 1 - 2 ) ) ✘ ( ( 2 * 3 + 4 ) )
  235. ( ( 1 - 2 ) ) c c c

    c c c c ✘ ( ( 1 - 2 ) ) ✘ ( ( 2 * 3 + 4 ) ) ( ( - 2 / 1 ) )
  236. ( ( 1 - 2 ) ) c c c

    c c c c ✘ ( ( 1 - 2 ) ) ✘ ( ( 2 * 3 + 4 ) ) ✘ ( ( - 2 / 1 ) )
  237. ( ( 1 - 2 ) ) c c c

    c c c c ✘ ( ( 1 - 2 ) ) ✘ ( ( 2 * 3 + 4 ) ) ✘ ( ( - 2 / 1 ) ) ( ( 98 - 0 ) )
  238. ( ( 1 - 2 ) ) c c c

    c c c c ✘ ( ( 1 - 2 ) ) ✘ ( ( 2 * 3 + 4 ) ) ✘ ( ( - 2 / 1 ) ) ✘ ( ( 98 - 0 ) )
  239. <expr> ) ( ( ) ( ( ) 4 )

    ( ( 4 ) ) c c c c c c c A
  240. <expr> ) ( ( ) ( ( ) 4 )

    ( ( 4 ) ) c c c c c c c A
  241. ( ( 4 ) ) c c c c c

    c c A ( ( ) ) <expr> ( ( ) ) 4 Minimized Input Abstract Failure Inducing Input def check(parsed): if parsed.is_nested() and parsed.child.is_nested(): raise Exception() return input
  242. var A = class extends (class {}){}; Issue 2937 from

    Closure
  243. var A = class extends (class {}){}; Issue 2937 from

    Closure <varModifier> <Identifier> = class extends (class {}){}
  244. var {baz:{} = baz => {}} = baz => {};

    Issue 385 from Rhino
  245. var {baz:{} = baz => {}} = baz => {};

    Issue 385 from Rhino var {<$Id1>:{} = <$Id1> => {}} <variableDeclaration>;
  246. const [y,y] = []; Issue 386 from Rhino

  247. const [y,y] = []; Issue 386 from Rhino const [<$Id1>,<$Id1>]

    = []
  248. {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842

    from Closure
  249. {while ((l_0)){ if ((l_0)) {break;;var l_0; continue }0}} Issue 2842

    from Closure {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}}
  250. ( ( 4 ) ) c c c c c

    c c A ( ( ) ) <expr> ( ( ) ) 4 Minimized Input Abstract Failure Inducing Input • Effectively abstracts a minimized input • The abstraction identifies where the problem lies • Decompose complex program behaviors DDSET Gopinath, Kampmann, Havrikov, Soremekun, and Zeller. Abstracting Failure Inducing Inputs. ISSTA 2020. def check(parsed): if parsed.is_nested() and parsed.child.is_nested(): raise Exception() return input ISSTA 2020 Distinguished Award
  251. 108 <start F> := <expr F> <expr F> := <term

    F> ' + ' <expr> | <term> ' + ' <expr F> | <term F> ' - ' <expr> | <term> ' - ' <expr F> | <term F> <term F> := <factor F> ' * ' <term> | <factor> ' * ' <term F> | <factor F> ' / ' <term> | <factor> ' / ' <term F> | <factor F> <factor F> := '+' <factor F> | '-' <factor F> | '(' <expr F> ')' | '(' <expr F1> ')' <expr F1> := <term F2> <term F2> := <factor F3> <factor F3>:= '(' <expr> ')' Specialized Grammar <factor F> is ((<expr>))
  252. 108 <start F> := <expr F> <expr F> := <term

    F> ' + ' <expr> | <term> ' + ' <expr F> | <term F> ' - ' <expr> | <term> ' - ' <expr F> | <term F> <term F> := <factor F> ' * ' <term> | <factor> ' * ' <term F> | <factor F> ' / ' <term> | <factor> ' / ' <term F> | <factor F> <factor F> := '+' <factor F> | '-' <factor F> | '(' <expr F> ')' | '(' <expr F1> ')' <expr F1> := <term F2> <term F2> := <factor F3> <factor F3>:= '(' <expr> ')' ((1)) + 2 (23 * ((3)) - 34) (344- 4 + ((223))) (1) - 3 * 773 + (-22 + 1) 1798 - 889 / ((333-1)) * 2 / 3 + 1 34 + ((4)) -334 + (334 - (22) + 919 * 0 + 1 98435747+ 88 + (((0))) + (1) - 1 * 7 / 4 * 889 - 2 8 + ((8)) + --1 + 11223 / 344 - 39 + (1) - 456 + 134 / 45 437 + 8 - 1 * ((9 + ((1))) - 1 + 99111948 + 3 --1 + (112) - 2 + 445) + 0 74 + 334 + ((178 - 88 / (3393-1) * 1002 / 3 + 1+ 3439)) * 223 - 1233 + 334672 2 * ((9)) - (1798 - 889 / (333-1) * 2 / 3 + 100012 + 3434392 + 234 ----6 * 1798 - 889 / (33 778 - (((1) - 3 * 773 + (-22 + 1) * (4545) - 23 - ((2)) * 773 + (-22 + 1) / 3434 + ---1 + 1 / 34343 + 112 349 + (((1) - 3 * 3 + (-22 + 1) ((+ (-22 + 1) * (4545) - 23 - (2) * 773 + ((-22 + 1)) / 3434 + ---1 + 1 / 34343 + 1123 8 + ((8)) + --1 + / 1 - 39 + (1) - 456 + 134 / 45 ))(((1) - 2334 + ((((1)) - 3 * 773 + (-22 + 1) * (2) - 23 - (2) * 773 + (-22 + 1) / 3 74 + 3 + ((178 - 88 / (3393-1) * 1002 / 3 + 1+ 3439)) * - 1233 + 334672)) ((8 + ((8)) + --1 + / 344 - 39 + (1) - 456 + 134 / 45 ))(((1) - 3 * 773 1+ 33+ 24343433 +23343 - ((74 + 334 + ((178 - 88 / (3393-1) * 1002 / 3 + 1+ 3439)) * - 1233 + 334672)) ((8 + ((8)) + --1 + / 344 - 39 + (1) - 456 + 134 / 4 ✘ Specialized Grammar <factor F> is ((<expr>)) ✘
  253. Algebra of Grammar Specializations

  254. <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){}

    Closure 2937 <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; Rhino 385 <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Rhino 386 <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} Closure 2842 where Algebra of Grammar Specializations
  255. <JavaScript C2937 and C2842> <variableDeclarationList C2937> is <varModifier> <Identifier> =

    class extends (class {}){} Closure 2937 <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; Rhino 385 <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Rhino 386 <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} Closure 2842 where Algebra of Grammar Specializations
  256. <JavaScript C2937 and C2842> <JavaScript not(C2937 and C2842)> <variableDeclarationList C2937>

    is <varModifier> <Identifier> = class extends (class {}){} Closure 2937 <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; Rhino 385 <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Rhino 386 <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} Closure 2842 where Algebra of Grammar Specializations
  257. <JavaScript C2937 and C2842> <JavaScript not(C2937 and C2842)> <JavaScript (C2937

    or C2842) and (R385 or R386)> <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){} Closure 2937 <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; Rhino 385 <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Rhino 386 <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} Closure 2842 where Algebra of Grammar Specializations
  258. Gopinath, Nemati, Zeller. Input Algebras. ICSE 2021. Mechanized proofs are

    available Algebra of Grammar Specializations
  259. Gopinath, Nemati, Zeller. Input Algebras. ICSE 2021. <JavaScript not(C2937 or

    C2842 or R385 or R386)> where <variableDeclarationList C2937> is <varModifier> <Identifier> = class extends (class {}){} <iterationStatement C2842> is {while ((<$Id1>)){ if ((<$Id1>)) {break;;var <$Id1>; continue }0}} <variableStatement R385> is var {<$Id2>:{} = <$Id2> => {}} <variableDeclaration>; <variableDeclarationList R386> is const [<$Id3>,<$Id3>] = [] Mechanized proofs are available Algebra of Grammar Specializations
  260. None
  261. None
  262. None
  263. None
  264. None
  265. Science of Focused Fuzzing <expr E & F> where <factor

    E> is ((<expr>)) <factor F> is <term> / 0
  266. <expr E & F> where <factor E> is ((<expr>)) <factor

    F> is <term> / 0
  267. <expr E & F> where <factor E> is ((<expr>)) <factor

    F> is <term> / 0 Isolating & Decomposing Program Behaviors
  268. <expr E & F> where <factor E> is ((<expr>)) <factor

    F> is <term> / 0 Isolating & Decomposing Program Behaviors Algebra of Program Behaviors
  269. <expr E & F> where <factor E> is ((<expr>)) <factor

    F> is <term> / 0 Isolating & Decomposing Program Behaviors Algebra of Program Behaviors Science of Program Behaviors
  270. None
  271. 115 insert into tbl values (1,2,3) select b from tbl

    drop table tbl Input Behavior Program Challenge: Identify Behavior Divergence
  272. 115 insert into tbl values (1,2,3) select b from tbl

    drop table tbl update($file) read($file) rm($file) Input Behavior action='read' $action('tbl') Program assert invoked read: 'tbl.data' ✔ Challenge: Identify Behavior Divergence
  273. 115 insert into tbl values (1,2,3) select b from tbl

    drop table tbl update($file) read($file) rm($file) Input Behavior action='read' $action('tbl') Program assert invoked read: 'tbl.data' ✔ action='rm' Challenge: Identify Behavior Divergence
  274. 115 insert into tbl values (1,2,3) select b from tbl

    drop table tbl update($file) read($file) rm($file) Input Behavior action='read' $action('tbl') Program assert invoked read: 'tbl.data' ✔ action='rm' Challenge: Identify Behavior Divergence
  275. def triangle(a, b, c): if a == b: if b

    == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene Challenge: Identify Behavior Divergence
  276. def triangle(a, b, c): if a == b: if b

    == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene Challenge: Identify Behavior Divergence
  277. Challenge: Identify Behavior Divergence LockB() LockA() DoAB() UnlockB() UnlockA()

  278. Challenge: Identify Behavior Divergence LockB() LockA() DoAB() UnlockB() UnlockA()

  279. Challenge: Identify Behavior Divergence LockB() LockA() DoAB() UnlockB() UnlockA() LockB()

    LockA() DoAB() UnlockA() UnlockB()
  280. Challenge: Identify Behavior Divergence LockB() LockA() DoAB() UnlockB() UnlockA() LockB()

    LockA() DoAB() UnlockA() UnlockB() UnLockA() LockA() DoAB() LockB() UnlockB()
  281. 118 def triangle(a, b, c): if a == b: if

    b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene
  282. 118 def triangle(a, b, c): if a == b: if

    b == c: return Equilateral else: return Isosceles else: if b == c: return Isosceles else: if a == c: return Isosceles else: return Scalene
  283. 119

  284. 119 <enter> := <triangle(a,b,c)>

  285. 120 <enter> := <triangle(a,b,c)> <triangle(a,b,c)> := (a==b) <if_2 T> |

    (a!=b) <if_2 F>
  286. 121 <enter> := <triangle(a,b,c)> <triangle(a,b,c)> := (a==b) <if_2 T> |

    (a!=b) <if_2 F> <if_2 T> := (b==c) <if_3 T> | (b!=c) <if_3 F>
  287. 122 <enter> := <triangle(a,b,c)> <triangle(a,b,c)> := (a==b) <if_2 T> |

    (a!=b) <if_2 F> <if_2 T> := (b==c) <if_3 T> | (b!=c) <if_3 F> <if_3 T> := "return Equilateral" <if_3 F> := "return Isosceles"
  288. 123 <enter> := <triangle(a,b,c)> <triangle(a,b,c)> := (a==b) <if_2 T> |

    (a!=b) <if_2 F> <if_2 T> := (b==c) <if_3 T> | (b!=c) <if_3 F> <if_3 T> := "return Equilateral" <if_3 F> := "return Isosceles" <if_2 F> := (b==c) <if_8 T> | (b!=c) <if_8 F> <if_8 T> := "return Isosceles" <if_8 F> := (a==c) <if_11 T> | (a!=c) <if_11 F> <if_11 T> := "return Isosceles" <if_11 F> := "return Scalene"
  289. LockB() LockA() DoAB() UnlockB() UnlockA() Challenge: Identify Behavior Divergence "lockA"

    "lockB" <DoAB> "UnlockB" "UnlockA"
  290. None
  291. 126 The Science of Inputs Program The Science of Behaviors

  292. 126 The Science of Inputs Program The Science of Behaviors

  293. 126 The Science of Inputs Program The Science of Behaviors

  294. 126 The Science of Fuzzing The Science of Inputs Program

    The Science of Behaviors ✔
  295. None
  296. 128 Oracles for Fuzzing Focused Fuzzing Automatic Repair Fault Localization

    Beyond Syntax Grammar Mining
  297. 128 Oracles for Fuzzing Focused Fuzzing Automatic Repair Fault Localization

    Beyond Syntax Grammar Mining Onward
  298. Program Under Test pFuzzer F1 Fuzzer Inputs Grammar Miner https://rahul.gopinath.org

  299. Program Under Test pFuzzer F1 Fuzzer Inputs Grammar Miner https://rahul.gopinath.org

  300. Program Under Test pFuzzer F1 Fuzzer Inputs Grammar Miner https://rahul.gopinath.org

  301. Program Under Test pFuzzer F1 Fuzzer Inputs Grammar Miner https://rahul.gopinath.org

  302. Program Under Test pFuzzer F1 Fuzzer Inputs Grammar Miner Problem:

    How to Fuzz Parsers https://rahul.gopinath.org
  303. Program Under Test pFuzzer F1 Fuzzer Inputs Grammar Miner Problem:

    How to Fuzz Parsers https://rahul.gopinath.org Generalize
  304. Program Under Test pFuzzer F1 Fuzzer Inputs Grammar Miner Problem:

    How to Fuzz Parsers https://rahul.gopinath.org Generalize Combine
  305. Future Work: Program Under Test pFuzzer F1 Fuzzer Inputs Grammar

    Miner Problem: How to Fuzz Parsers https://rahul.gopinath.org Generalize Combine The Science of Fuzzing
  306. Future Work: Program Under Test pFuzzer F1 Fuzzer Inputs Grammar

    Miner Problem: How to Fuzz Parsers https://rahul.gopinath.org Debugging Combine The Science of Fuzzing