High Speed Bug Discovery with Fuzzing (version with presenter notes) - Dog Food Conference

How can you find hidden defects in your code without a lot of manual analysis? You’ll leave this talk knowing when to use fuzzing to test your application, which tools you should use, how to implement a fuzzer from scratch, and when other techniques are a better choice.

Transcript

1. High Speed Bug Discovery
   with Fuzzing
   Craig Stuntz ∈ Improving
   

   View Slide

2. Slides
   https://speakerdeck.com/craigstuntz
   

   View Slide

3. Fuzzing Is Bad
   …and you should feel bad about
   using it
   

   View Slide

4. How Do We Design Software Security?
   - Usability Studies (Does it work for human beings?)
   - Threat Modeling
   - Formal Verification
   - Static Analysis
   - Auditing
   - Pentesting
   - Throw Random Crap At It And See If It Dies
   

   View Slide

5. Offensive
   Software
   Offensive to me, personally
   

   View Slide

6. “
   Unfortunately, C and C++ are mostly taught the old
   way, as if programming in them isn’t like walking in
   a minefield. Nor have the books about C and C++
   caught up with the current reality. These things must
   change.
   -John Regehr
   https://blog.regehr.org/archives/1520
   Fuzzing is depressing. It Shouldn’t Be Necessary, C (and infrastructure to sustain it), Better Methods of Verification Exist, Can Miss Stuff , Worse Is Better?
   
   Sometimes when something seems weird to you it’s worth taking a deep dive,
   

   View Slide

7. “Yet, despite the crippling and obvious limitations of
   fuzzing and the virtues of symbolic execution, there
   is one jarring discord: I'm fairly certain that probably
   around 70% of all remote code execution
   vulnerabilities disclosed in the past few years trace
   back to fairly “dumb” fuzzing tools, with the pattern
   showing little change over time.
   -Michał Zalewski
   https://lcamtuf.blogspot.com/2015/02/symbolic-execution-in-vuln-research.html
   

   View Slide

8. let private bitMasks(bitIndex: int, bitsToFlip: int) =
   match bitIndex % 8, bitsToFlip with
   | 0, 1 !→ 0b00000001uy, 0b00000000uy
   | 1, 1 !→ 0b00000010uy, 0b00000000uy
   | 2, 1 !→ 0b00000100uy, 0b00000000uy
   | 3, 1 !→ 0b00001000uy, 0b00000000uy
   | 4, 1 !→ 0b00010000uy, 0b00000000uy
   | 5, 1 !→ 0b00100000uy, 0b00000000uy
   | 6, 1 !→ 0b01000000uy, 0b00000000uy
   | 7, 1 !→ 0b10000000uy, 0b00000000uy
   | 0, 2 !→ 0b00000011uy, 0b00000000uy
   | 1, 2 !→ 0b00000110uy, 0b00000000uy
   | 2, 2 !→ 0b00001100uy, 0b00000000uy
   | 3, 2 !→ 0b00011000uy, 0b00000000uy
   | 4, 2 !→ 0b00110000uy, 0b00000000uy
   | 5, 2 !→ 0b01100000uy, 0b00000000uy
   | 6, 2 !→ 0b11000000uy, 0b00000000uy
   | 7, 2 !→ 0b10000000uy, 0b00000001uy
   | 0, 4 !→ 0b00001111uy, 0b00000000uy
   | 1, 4 !→ 0b00011110uy, 0b00000000uy
   | 2, 4 !→ 0b00111100uy, 0b00000000uy
   | 3, 4 !→ 0b01111000uy, 0b00000000uy
   | 4, 4 !→ 0b11110000uy, 0b00000000uy
   | 5, 4 !→ 0b11100000uy, 0b00000001uy
   | 6, 4 !→ 0b11000000uy, 0b00000011uy
   | 7, 4 !→ 0b10000000uy, 0b00000111uy
   | bit, _ !→ failwithf "Unsupported bit %d or bitsToFlip %d" bit bitsToFlip
   so I wrote some really terrible and embarrassing F# to learn about fuzzing. A lot of it is just literally translated from C.
   

   View Slide

9. Spoilers!
   Why should I care?
   (because it’s surprisingly effective at finding bugs in software)
   What is it?
   (a simple, property-based randomized testing technique)
   When should I use it?
   (integration testing complex systems with infinite input values)
   How do I get started?
   (I’ll suggest a bunch of tools)
   Should I write my own?
   (yes, and I have stories!)
   When should I not use it?
   
   Will start very non-technical and get more technical as we go on. Light bash scripting to rewriting binary assemblies
   

   View Slide

10. Why?
    already have other testing techniques, so….
    
    Why is fuzzing interesting? Because it works!
    

    View Slide

11. 400
    Crashes,
    106 Distinct
    Security
    Bugs
    in Adobe Flash Player
    https://security.googleblog.com/2011/08/fuzzing-at-scale.html
    1 month, 2000 cores
    

    View Slide

12. 325
    C
    Compiler
    bugs
    in GCC, Clang, & Others
    https://www.flux.utah.edu/paper/yang-pldi11
    25 GCC bugs classified as release-blocking
    
    Spent less than $1000
    

    View Slide

13. “…our most
    prolific
    bug-finding
    tool.”
    Robert Guo
    http://queue.acm.org/detail.cfm?ref=rss&id=3059007
    

    View Slide

14. “fuzzers…can be
    integrated into a
    CI environment
    for permanent
    protection
    henceforth”
    Guido Vranken
    https://guidovranken.wordpress.com/2017/06/21/the-openvpn-post-audit-bug-bonanza/
    Audits are one-time, fuzzers run forever
    

    View Slide

15. DARPA
    Cyber
    Grand
    Challenge
    Mayhem, Xandra,
    Mechanical Phish, Galactica
    http://blogs.grammatech.com/the-cyber-grand-challenge
    

    View Slide

16. https://commons.wikimedia.org/wiki/File:Rabbit_american_fuzzy_lop_buck_white.jpg
    What
    is it?
    

    View Slide

17. Prevent
    Regressions
    Bug
    Discovery
    Help with
    Code Design
    Meets
    Specifications
    Fuzzing
    Integration testing
    Unit testing
    Formal verification
    Exploratory testing
    Testing goals and tools
    

    View Slide

18. How Many Cases
    Should We Test?
    One Only the Most Interesting Every Possible Case
    Unit Testing Fuzzing
    Formal
    Verification
    Formal verification example: a type signature
    

    View Slide

19. Property
    -Based
    Random
    Testing
    QuickCheck
    Chaos Monkey Fuzzing
    Scientist
    

    View Slide

20. System Under Test
    Corpus
    (Collection of Examples)
    Property
    Magic!
    

    View Slide

21. Corpus
    A few handwritten
    examples
    Fuzzing databases
    Harvest from test suites,
    defect reports
    Harvest from public
    Internet
    

    View Slide

22. System Under Test
    A function
    Entire application
    Part of OS kernel
    Many testing techniques limited to testing at a particular scope
    

    View Slide

23. Properties
    Does it crash?
    Does it hang?
    Is the output “valid”?
    Does execution trip an
    address or memory
    sanitizer?
    Does the output match
    some other system?
    

    View Slide

24. Magic
    Mutation of corpus
    Coverage guidance
    Lots of test runs
    What you’ve heard so far sounds like taking a few sample inputs and doing a boring test on the output.
    
    That’s it?
    

    View Slide

25. Possible Inputs
    Random
    Inputs
    Interesting
    Inputs
    Random
    Inputs
    with
    Profile
    Guidance
    The idea is to find the interesting (desired properties don’t hold) inputs to the system under test faster than we could find them by either exhaustive testing or purely by
    chance.
    

    View Slide

26. Getting Started with afl
    - Compile system under test with instrumentation
    - Place corpus input(s) in a folder
    - Invoke afl
    - Wait for bugs
    https://fuzzing-project.org/tutorial3.html
    What does this look like in practice?
    
    There are more complicated options, but this is how you get started
    
    Sounds kinda dumb but scales really well
    

    View Slide

27. Compile with Instrumentation
    $ ./configure CC="afl-gcc" \
    CXX="afl-g++" \
    --disable-shared; \
    make
    

    View Slide

28. Place Corpus in Folder
    $ mkdir in
    $ cd in
    $ cat > foo.json
    { "a": "bc" }
    ^D
    $ cd ..
    You probably want more than one example in the corpus, but this gets us started
    

    View Slide

29. Invoke afl
    $ afl-fuzz -i in -o out \
    my_json_parser @@
    folder containing corpus
    “@@“ means “the current test case”
    system under test
    findings go here
    This is the simple case. There are many options!
    

    View Slide

30. This might run for weeks
    
    What will I find? Crashes, hangs
    
    Can do more, with effort!
    

    View Slide

31. some_project
    ├── in
    └─┬ out
    ├── queue
    ├── crashes
    └── hangs
    findings are just new input files.
    

    View Slide

32. afl In a Nutshell
    ⃗
    Turn a few maybe uninteresting inputs into a lot of hopefully very interesting inputs, quickly
    

    View Slide

33. afl is a dumb fuzzer, meaning…
    

    View Slide

34. afl Fuzz Strategies
    Walking bit flips
    (try flipping each bit in input individually)
    Walking byte flips
    (try flipping each contiguous set of 8 bits)
    Simple arithmetic
    (increment or decrement bytes in the file by certain small values)
    Known integers (or dictionaries)
    (replace bytes with “problematic” 8, 16, and 32 bit integers like 0 and FF)
    Profile-guided stacked tweaks and test case splicing
    (magic!)
    https://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html
    Deterministic, genetic
    

    View Slide

35. Walking Bit Flip
    Original 01010101
    Flip bit 0 01010100
    Flip bit 1 01010111
    Flip bit 2 01010001
    
    Walking 2 Bit Flip
    Original 01010101
    Flip bits 0,1 01010110
    Flip bits 2,1 01010011
    Flip bits 3,2 01001101
    
    

    View Slide

36. Trace Execution Paths
    A
    B D
    C
    E
    F
    ?
    

    View Slide

37. When Should I
    Use It?
    When is it
    useful?
    https://it.wikipedia.org/wiki/File:Coniglio_ariete.JPG
    

    View Slide

38. Unit Tests Fuzzing
    Useful For
    Preventing
    Regressions, Design
    Finding New Bugs
    Tests Functions Any Level
    Test Examples Hand-selected values Corpus + Mutation
    Execution Time Milliseconds Weeks
    Magic? No Yes
    

    View Slide

39. Instrumentation
    for profile guidance
    

    View Slide

40. $ ./configure CC="afl-gcc" \
    CXX="afl-g++" \
    --disable-shared; \
    make
    You’ve seen this before; afl requires recompiling with instrumentation
    

    View Slide

41. $ pip install python-afl
    ———
    afl.init()
    

    View Slide

42. Properties
    a.k.a. Specifications
    https://lorinhochstein.wordpress.com/2014/06/04/crossing-the-river-with-tla/
    

    View Slide

43. A simple specification: Programs shouldn’t crash.
    

    View Slide

44. Sanitizers and Canaries
    
    Security cookie (/GS)
    

    View Slide

45. STJSON
    A JSON Parser in Swift 3 compliant with RFC 7159
    STJSON was written along with the article Parsing JSON is a Minefield.
    Basic usage:
    var p = STJSONParser(data: data)
    do {
    let o = p.parse()
    } catch let e {
    print(e)
    }
    Instantiation with options:
    var p = STJSON(data:data,
    maxParserDepth:1024,
    options:[.useUnicodeReplacementCharacter])
    https://github.com/nst/STJSON
    https://github.com/CraigStuntz/Fizil/tree/master/StJson
    Is there a specification for the behavior of the program?
    

    View Slide

46. Can You Fuzz the
    Domain?
    

    View Slide

47. Dumb Fuzzer
    Mangling Byte Arrays
    many interesting programs take binary input
    
    sometimes this requires less domain knowledge (not always!) or at least less custom code
    
    What about programs or functions which can’t accept binary input?
    

    View Slide

48. public byte[] ResizePng(
    byte[] image)
    {
    Great candidate for dumb fuzzing
    

    View Slide

49. public boolean SomeFunction(
    SomeEnum firstArg,
    int secondArg)
    {
    ✗
    not a good candidate for dumb fuzzing
    
    If I have a good property I can test exhaustively
    

    View Slide

50. Smart Fuzzing
    MongoDB Expression Grammar
    http://queue.acm.org/detail.cfm?ref=rss&id=3059007
    Not an end to end test — testing the internals.
    
    Don’t test stuff which takes forever
    

    View Slide

51. public Assembly Compile(
    AbstractNode syntaxTree)
    {
    Good candidate for smart fuzzing?
    

    View Slide

52. Worth the Wait?
    Results Can Take Weeks
    

    View Slide

53. Isn’t it just for Security?
    https://www.flickr.com/photos/wocintechchat/25721078480/
    Every security person knows about fuzzing. Almost no app devs do.
    
    Security people care more about bugs than app devs.
    

    View Slide

54. How do
    I get
    started?
    

    View Slide

55. How to Get Started with Fuzzing
    1. Find a program to test
    2. Find a fuzzer
    3. Find a corpus
    4. Choose a property
    5. Let it run!
    

    View Slide

56. Fuzzers
    https://commons.wikimedia.org/wiki/File:Holland_Lop_with_Broken_Orange_Coloring.jpg
    

    View Slide

57. libfuzzer
    Fuzz testing for LLVM compilers
    http://llvm.org/docs/LibFuzzer.html
    

    View Slide

58. $ cargo install cargo-fuzz
    $ cargo fuzz init
    

    View Slide

59. $ gzip -c /bin/bash > sample.gz
    $ while true
    do
    radamsa sample.gz > fuzzed.gz
    gzip -dc fuzzed.gz > /dev/null
    test $? -gt 127 && break
    done
    ← Fuzz the corpus
    ← Execute S.O.T.
    ← Check a property
    ← Repeat a lot!
    https://github.com/aoh/radamsa
    Radamsa: Un*x philosophy of “do one thing well, chain with other stuff”
    
    Works on Windows with cygwin (and maybe
    

    View Slide

60. ClusterFuzz
    Submit a fuzzer, win a bounty
    https://security.googleblog.com/2016/08/guided-in-process-fuzzing-of-chrome.html
    

    View Slide

61. afl
    

    View Slide

62. OSS-Fuzz
    Submit your project
    https://github.com/google/oss-fuzz
    1. You can use a fuzzer in your CI
    
    2. Since it’s easy to do so, Google did it for you.
    

    View Slide

63. “Project Springfield”
    https://www.microsoft.com/en-us/security-risk-detection/
    

    View Slide

64. burp, ZAP
    non-instrumented
    

    View Slide

65. Corpus
    https://en.wikipedia.org/wiki/File:Long_Room_Interior,_Trinity_College_Dublin,_Ireland_-
    _Diliff.jpg
    

    View Slide

66. “
    We didn't call it fuzzing back in the 1950s, but it was
    our standard practice to test programs by inputting
    decks of punch cards taken from the trash.
    -Gerald M. Weinberg
    http://secretsofconsulting.blogspot.com/2017/02/fuzz-testing-and-fuzz-history.html
    

    View Slide

67. View Slide

68. Fuzzing SQLite with afl
    Start with a single test case:
    create table t1(one smallint);
    insert into t1 values(1);
    select * from t1;
    Add a list of reserved words from documentation
    Then extract SQL statements from SQLite unit tests
    (550 files at around 220 bytes each)
    https://lcamtuf.blogspot.com/2015/04/finding-bugs-in-sqlite-easy-way.html
    

    View Slide

69. Properties
    

    View Slide

70. Don’t Crash
    or Hang
    

    View Slide

71. Sanitizers
    and Canaries
    https://docs.google.com/presentation/d/19OSgb1N9Ezef39Blb-5lkzycq7-tMtAvy825FofyrmY
    

    View Slide

72. Validators
    

    View Slide

73. End to End
    Input
    Output
    Same as
    Input?
    f(input)
    f-1(input)
    

    View Slide

74. Legacy Code
    https://commons.wikimedia.org/wiki/File:Blue-punch-card-front-horiz.png
    

    View Slide

75. One of The Things
    …Is Not Like the Others!
    Repeated testing with same input should yield same output
    

    View Slide

76. Should I
    write my
    own?
    

    View Slide

77. Heck Yeah!
    https://github.com/CraigStuntz/Fizil
    

    View Slide

78. Interesting Stuff I Learned While Writing a Fuzzer
    - Deceptive simplicity of fuzzing
    - F# bitwise operations
    - How to instrument .NET code
    - dnSpy is awesome
    - Same input -> Same code -> Different paths
    - Strong naming is painful
    - Unicode is also painful
    - Speed is everything
    

    View Slide

79. let jsonNetResult =
    try JsonConvert.DeserializeObject(str) |> ignore
    Success
    with
    | :? JsonReaderException as jre -> jre.Message |> Error
    | :? JsonSerializationException as jse -> jse.Message |> Error
    | :? System.FormatException as fe ->
    if fe.Message.StartsWith("Invalid hex character”) // hard coded in Json.NET
    then fe.Message |> Error
    else reraise()
    ⃪ T
    est
    ⬑ Special case error stuff
    Special test harness code specific to the system under test
    

    View Slide

80. use proc = new Process()
    proc.StartInfo.FileName <- executablePath
    inputMethod.BeforeStart proc testCase.Data
    proc.StartInfo.UseShellExecute <- false
    proc.StartInfo.RedirectStandardOutput <- true
    proc.StartInfo.RedirectStandardError <- true
    proc.StartInfo.EnvironmentVariables.Add(SharedMemory.environmentVariableName, sharedMemoryName)
    let output = new System.Text.StringBuilder()
    let err = new System.Text.StringBuilder()
    proc.OutputDataReceived.Add(fun args -> output.Append(args.Data) |> ignore)
    proc.ErrorDataReceived.Add (fun args -> err.Append(args.Data) |> ignore)
    proc.Start() |> ignore
    inputMethod.AfterStart proc testCase.Data
    proc.BeginOutputReadLine()
    proc.BeginErrorReadLine()
    proc.WaitForExit()
    let exitCode = proc.ExitCode
    let crashed = exitCode = WinApi.ClrUnhandledExceptionCode
    ⃪ Set up
    ⃪ Read results
    ⃪ Important bit
    Generic code in Fizil
    

    View Slide

81. /// An ordered list of functions to use when starting with a single piece of
    /// example data and producing new examples to try
    let private allStrategies(dictionaryValues: byte[][])= [
    bitFlip 1
    bitFlip 2
    bitFlip 4
    byteFlip 1
    byteFlip 2
    byteFlip 4
    arith8
    arith16
    arith32
    dictionary dictionaryValues
    interest8
    interest16
    ]
    

    View Slide

82. let totalBits = bytes.Length * 8
    let testCases = seq {
    for bit = 0 to totalBits - flipBits do
    let newBytes = Array.copy bytes
    let firstByte = bit / 8
    let firstByteMask, secondByteMask = bitMasks(bit, flipBits)
    let newFirstByte = bytes.[firstByte] ^^^ firstByteMask
    newBytes.[firstByte] <- newFirstByte
    let secondByte = firstByte + 1
    if secondByteMask <> 0uy && secondByte < bytes.Length
    then
    let newSecondByte = bytes.[secondByte] ^^^ secondByteMask
    newBytes.[secondByte] <- newSecondByte
    yield newBytes
    }
    Fuzz one byte →
    ^^^ means xor
    ↓
    

    View Slide

83. View Slide

84. View Slide

85. private static void F(string arg)
    {
    Console.WriteLine("f");
    Console.Error.WriteLine("Error!");
    Environment.Exit(1);
    }
    

    View Slide

86. private static void F(string arg)
    {
    instrument.Trace(29875);
    Console.WriteLine("f");
    Console.Error.WriteLine("Error!");
    Environment.Exit(1);
    }
    ← Random number
    

    View Slide

87. private static void F(string arg)
    {
    #if MANUAL_INSTRUMENTATION
    instrument.Trace(29875);
    #endif
    Console.WriteLine("f");
    Console.Error.WriteLine("Error!");
    Environment.Exit(1);
    }
    

    View Slide

88. let stringify (ob: obj) : string =
    JsonConvert.SerializeObject(ob)
    // Method: System.String\u0020Program::stringify(System.Object)
    .body stringify {
    arg_02_0 [generated]
    arg_07_0 [generated]
    nop()
    arg_02_0 = ldloc(ob)
    arg_07_0 = call(JsonConvert::SerializeObject, arg_02_0)
    ret(arg_07_0)
    }
    // Method: System.String\u0020Program::stringify(System.Object)
    .body stringify {
    arg_05_0 [generated]
    arg_0C_0 [generated]
    arg_11_0 [generated]
    arg_05_0 = ldc.i4(23831)
    call(Instrument::Trace, arg_05_0)
    nop()
    arg_0C_0 = ldloc(ob)
    arg_11_0 = call(JsonConvert::SerializeObject, arg_0C_0)
    ret(arg_11_0)
    }
    

    View Slide

89. View Slide

90. let private insertTraceInstruction(ilProcessor: ILProcessor, before: Instruction, state) =
    let compileTimeRandom = state.Random.Next(0, UInt16.MaxValue |> Convert.ToInt32)
    let ldArg = ilProcessor.Create(OpCodes.Ldc_I4, compileTimeRandom)
    let callTrace = ilProcessor.Create(OpCodes.Call, state.Trace)
    ilProcessor.InsertBefore(before, ldArg)
    ilProcessor.InsertAfter (ldArg, callTrace)
    This margin is too narrow to contain a try/finally example, so see:
    https://goo.gl/W4y7JH
    Inserting the IL instructions I needed was fairly easy. Here is the important bit of the code which does it. How did I learn how to write this? I instrumented a small program
    “manually” by writing the instrumentation code myself, and then decompiled that program to figure out which IL instructions I needed. Inserting them with Mono.Cecil is
    just a few lines of code.
    
    try/finally is much, much harder. I won’t even try to walk you through it here. Look at the GitHub repo if you want to see how it’s done.
    

    View Slide

91. View Slide

92. http://www.json.org/
    I need a way to determine if Json.NET is parsing the JSON correctly. So I thought I should write a JSON validator to check its behavior. Fortunately, there’s a standard!
    
    “Probably the boldest design decision I made was to not put a version number on JSON so there is no mechanism for revising it. We are stuck with JSON: whatever it is
    in its current form, that’s it.” -Crockford
    

    View Slide

93. https://tools.ietf.org/html/rfc4627
    So, naturally, there’s also another JSON standard
    

    View Slide

94. http://www.ecma-international.org/ecma-262/5.1/#sec-15.12
    And another JSON standard
    

    View Slide

95. http://www.ecma-international.org/publications/standards/Ecma-404.htm
    And another JSON standard
    

    View Slide

96. https://tools.ietf.org/html/rfc7158
    And another JSON standard
    

    View Slide

97. https://tools.ietf.org/html/rfc7159
    And another JSON standard. And no, they don’t all agree on everything, nor is there a single, “latest” version. Despite this multitude of standards, there are still edge
    cases intentionally delegated to the implementer — what we would call “undefined behavior” in C.
    

    View Slide

98. https://github.com/nst/STJSON
    I was going to write my own validator, but…
    
    Nicolas Seriot wrote a validator called STJSON which attempts to synthesize these as much as possible.
    

    View Slide

99. https://github.com/CraigStuntz/Fizil/blob/master/StJson/StJsonParser.fs
    Swift doesn’t readily compile to Windows, but if you squint hard enough it kind of looks like F#, so I ported the code and used it to validate Json.NET's behavior.
    

    View Slide

100. { "a" : "bc" }
     

     View Slide

101. View Slide

102. View Slide

103. Standard Accepts, Json.NET Rejects
     Value
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     88888888888888888888888888888888888888888888888888
     Standard Says No limit
     Json.NET MaximumJavascriptIntegerCharacterLength = 380;
     Things JSON.NET fails on that the standard accepts
     

     View Slide

104. Standard Rejects, Json.NET Accepts
     Value [,,,]
     Standard Says
     A JSON value MUST be an object, array, number, or string, or one
     of
     the following three literal names:
     false null true
     Json.NET [null, null, null, null]
     Things JSON.NET succeeds on that the standard rejects
     

     View Slide

105. Strong naming was a consistent pain for me.
     
     I’m altering the binaries of assemblies, and part of the point of strong naming is to stop you from doing just that, so naturally if the assembly is strongly named it can’t be
     loaded when I’m finished.
     

     View Slide

106. let private removeStrongName (assemblyDefinition : AssemblyDefinition) =
     let name = assemblyDefinition.Name;
     name.HasPublicKey <- false;
     name.PublicKey <- Array.empty;
     assemblyDefinition.Modules |> Seq.iter (
     fun moduleDefinition ->
     moduleDefinition.Attributes <-
     moduleDefinition.Attributes &&& ~~~ModuleAttributes.StrongNameSigned)
     let aptca = assemblyDefinition.CustomAttributes.FirstOrDefault(
     fun attr -> attr.AttributeType.FullName
     = typeof.FullName)
     assemblyDefinition.CustomAttributes.Remove aptca |> ignore
     assembly.MainModule.AssemblyReferences
     |> Seq.filter (fun reference -> Set.contains reference.Name assembliesToInstrument)
     |> Seq.iter (fun reference ->
     reference.PublicKeyToken <- null
     )
     So I need to remove the strong name from any assembly I fuzz, but I also need to remove the PublicKeyToken from any other assembly which references it. Doing this in
     Mono.Cecil is not well-documented, and after quite a bit of time spent in GitHub issues and trial and error I figured out that it takes 5 distinct steps to do this.
     

     View Slide

107. “
     “If marked BeforeFieldInit then the type’s initializer
     method is executed at, or sometime before, first
     access to any static field defined for that type.”
     -ECMA-335, Common Language
     Infrastructure (CLI), Partition I
     

     View Slide

108. Unicode
     Original JSON
     { "a": "bc" }
     ASCII Bytes
     7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D
     UTF-8 with Byte Order Mark
     EF BB BF 7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D
     UTF-16 BE with BOM
     FE FF 00 7B 00 20 00 22 00 61 00 22 00 20 00 3A 00 20 00 22
     00 62 00 63 00 22 00 20 00 7D
     

     View Slide

109. View Slide

110. Resources
     Awesome Fuzzing: A Curated List of Fuzzing Resources
     https://github.com/secfigo/Awesome-Fuzzing
     OWASP Fuzzing Page
     https://www.owasp.org/index.php/Fuzzing
     MongoDB’s JavaScript Fuzzer
     http://queue.acm.org/detail.cfm?ref=rss&id=3059007
     afl technical details
     http://lcamtuf.coredump.cx/afl/technical_details.txt
     afl Help Email List
     afl-users@googlegroups.com
     Fizil
     https://github.com/CraigStuntz/Fizil
     WTF, ACM?
     

     View Slide

111. Thank You!
     - Michał Zalewski, for afl documentation
     - Rehearsal audiences, employees of
     - Dynamit
     - Improving
     - Ineffable Solutions
     

     View Slide

112. Craig Stuntz
     craig.stuntz@improving.com
     www.craigstuntz.com
     @craigstuntz
     http://www.meetup.com/Papers-We-Love-Columbus/
     

     View Slide