Upgrade to Pro — share decks privately, control downloads, hide ads and more …

High Speed Bug Discovery with Fuzzing (version with presenter notes) - Dog Food Conference

High Speed Bug Discovery with Fuzzing (version with presenter notes) - Dog Food Conference

How can you find hidden defects in your code without a lot of manual analysis? You’ll leave this talk knowing when to use fuzzing to test your application, which tools you should use, how to implement a fuzzer from scratch, and when other techniques are a better choice.

Craig Stuntz

October 05, 2017
Tweet

More Decks by Craig Stuntz

Other Decks in Programming

Transcript

  1. High Speed Bug Discovery
    with Fuzzing
    Craig Stuntz ∈ Improving

    View Slide

  2. Slides
    https://speakerdeck.com/craigstuntz

    View Slide

  3. Fuzzing Is Bad
    …and you should feel bad about
    using it

    View Slide

  4. How Do We Design Software Security?
    - Usability Studies (Does it work for human beings?)
    - Threat Modeling
    - Formal Verification
    - Static Analysis
    - Auditing
    - Pentesting
    - Throw Random Crap At It And See If It Dies

    View Slide

  5. Offensive
    Software
    Offensive to me, personally

    View Slide


  6. Unfortunately, C and C++ are mostly taught the old
    way, as if programming in them isn’t like walking in
    a minefield. Nor have the books about C and C++
    caught up with the current reality. These things must
    change.
    -John Regehr
    https://blog.regehr.org/archives/1520
    Fuzzing is depressing. It Shouldn’t Be Necessary, C (and infrastructure to sustain it), Better Methods of Verification Exist, Can Miss Stuff , Worse Is Better?

    Sometimes when something seems weird to you it’s worth taking a deep dive,

    View Slide

  7. “Yet, despite the crippling and obvious limitations of
    fuzzing and the virtues of symbolic execution, there
    is one jarring discord: I'm fairly certain that probably
    around 70% of all remote code execution
    vulnerabilities disclosed in the past few years trace
    back to fairly “dumb” fuzzing tools, with the pattern
    showing little change over time.
    -Michał Zalewski
    https://lcamtuf.blogspot.com/2015/02/symbolic-execution-in-vuln-research.html

    View Slide

  8. let private bitMasks(bitIndex: int, bitsToFlip: int) =
    match bitIndex % 8, bitsToFlip with
    | 0, 1 !→ 0b00000001uy, 0b00000000uy
    | 1, 1 !→ 0b00000010uy, 0b00000000uy
    | 2, 1 !→ 0b00000100uy, 0b00000000uy
    | 3, 1 !→ 0b00001000uy, 0b00000000uy
    | 4, 1 !→ 0b00010000uy, 0b00000000uy
    | 5, 1 !→ 0b00100000uy, 0b00000000uy
    | 6, 1 !→ 0b01000000uy, 0b00000000uy
    | 7, 1 !→ 0b10000000uy, 0b00000000uy
    | 0, 2 !→ 0b00000011uy, 0b00000000uy
    | 1, 2 !→ 0b00000110uy, 0b00000000uy
    | 2, 2 !→ 0b00001100uy, 0b00000000uy
    | 3, 2 !→ 0b00011000uy, 0b00000000uy
    | 4, 2 !→ 0b00110000uy, 0b00000000uy
    | 5, 2 !→ 0b01100000uy, 0b00000000uy
    | 6, 2 !→ 0b11000000uy, 0b00000000uy
    | 7, 2 !→ 0b10000000uy, 0b00000001uy
    | 0, 4 !→ 0b00001111uy, 0b00000000uy
    | 1, 4 !→ 0b00011110uy, 0b00000000uy
    | 2, 4 !→ 0b00111100uy, 0b00000000uy
    | 3, 4 !→ 0b01111000uy, 0b00000000uy
    | 4, 4 !→ 0b11110000uy, 0b00000000uy
    | 5, 4 !→ 0b11100000uy, 0b00000001uy
    | 6, 4 !→ 0b11000000uy, 0b00000011uy
    | 7, 4 !→ 0b10000000uy, 0b00000111uy
    | bit, _ !→ failwithf "Unsupported bit %d or bitsToFlip %d" bit bitsToFlip
    so I wrote some really terrible and embarrassing F# to learn about fuzzing. A lot of it is just literally translated from C.

    View Slide

  9. Spoilers!
    Why should I care?
    (because it’s surprisingly effective at finding bugs in software)
    What is it?
    (a simple, property-based randomized testing technique)
    When should I use it?
    (integration testing complex systems with infinite input values)
    How do I get started?
    (I’ll suggest a bunch of tools)
    Should I write my own?
    (yes, and I have stories!)
    When should I not use it?

    Will start very non-technical and get more technical as we go on. Light bash scripting to rewriting binary assemblies

    View Slide

  10. Why?
    already have other testing techniques, so….

    Why is fuzzing interesting? Because it works!

    View Slide

  11. 400
    Crashes,
    106 Distinct
    Security
    Bugs
    in Adobe Flash Player
    https://security.googleblog.com/2011/08/fuzzing-at-scale.html
    1 month, 2000 cores

    View Slide

  12. 325
    C
    Compiler
    bugs
    in GCC, Clang, & Others
    https://www.flux.utah.edu/paper/yang-pldi11
    25 GCC bugs classified as release-blocking

    Spent less than $1000

    View Slide

  13. “…our most
    prolific
    bug-finding
    tool.”
    Robert Guo
    http://queue.acm.org/detail.cfm?ref=rss&id=3059007

    View Slide

  14. “fuzzers…can be
    integrated into a
    CI environment
    for permanent
    protection
    henceforth”
    Guido Vranken
    https://guidovranken.wordpress.com/2017/06/21/the-openvpn-post-audit-bug-bonanza/
    Audits are one-time, fuzzers run forever

    View Slide

  15. DARPA
    Cyber
    Grand
    Challenge
    Mayhem, Xandra,
    Mechanical Phish, Galactica
    http://blogs.grammatech.com/the-cyber-grand-challenge

    View Slide

  16. https://commons.wikimedia.org/wiki/File:Rabbit_american_fuzzy_lop_buck_white.jpg
    What
    is it?

    View Slide

  17. Prevent
    Regressions
    Bug
    Discovery
    Help with
    Code Design
    Meets
    Specifications
    Fuzzing
    Integration testing
    Unit testing
    Formal verification
    Exploratory testing
    Testing goals and tools

    View Slide

  18. How Many Cases
    Should We Test?
    One Only the Most Interesting Every Possible Case
    Unit Testing Fuzzing
    Formal
    Verification
    Formal verification example: a type signature

    View Slide

  19. Property
    -Based
    Random
    Testing
    QuickCheck
    Chaos Monkey Fuzzing
    Scientist

    View Slide

  20. System Under Test
    Corpus
    (Collection of Examples)
    Property
    Magic!

    View Slide

  21. Corpus
    A few handwritten
    examples
    Fuzzing databases
    Harvest from test suites,
    defect reports
    Harvest from public
    Internet

    View Slide

  22. System Under Test
    A function
    Entire application
    Part of OS kernel
    Many testing techniques limited to testing at a particular scope

    View Slide

  23. Properties
    Does it crash?
    Does it hang?
    Is the output “valid”?
    Does execution trip an
    address or memory
    sanitizer?
    Does the output match
    some other system?

    View Slide

  24. Magic
    Mutation of corpus
    Coverage guidance
    Lots of test runs
    What you’ve heard so far sounds like taking a few sample inputs and doing a boring test on the output.

    That’s it?

    View Slide

  25. Possible Inputs
    Random
    Inputs
    Interesting
    Inputs
    Random
    Inputs
    with
    Profile
    Guidance
    The idea is to find the interesting (desired properties don’t hold) inputs to the system under test faster than we could find them by either exhaustive testing or purely by
    chance.

    View Slide

  26. Getting Started with afl
    - Compile system under test with instrumentation
    - Place corpus input(s) in a folder
    - Invoke afl
    - Wait for bugs
    https://fuzzing-project.org/tutorial3.html
    What does this look like in practice?

    There are more complicated options, but this is how you get started

    Sounds kinda dumb but scales really well

    View Slide

  27. Compile with Instrumentation
    $ ./configure CC="afl-gcc" \
    CXX="afl-g++" \
    --disable-shared; \
    make

    View Slide

  28. Place Corpus in Folder
    $ mkdir in
    $ cd in
    $ cat > foo.json
    { "a": "bc" }
    ^D
    $ cd ..
    You probably want more than one example in the corpus, but this gets us started

    View Slide

  29. Invoke afl
    $ afl-fuzz -i in -o out \
    my_json_parser @@
    folder containing corpus
    “@@“ means “the current test case”
    system under test
    findings go here
    This is the simple case. There are many options!

    View Slide

  30. This might run for weeks

    What will I find? Crashes, hangs

    Can do more, with effort!

    View Slide

  31. some_project
    ├── in
    └─┬ out
    ├── queue
    ├── crashes
    └── hangs
    findings are just new input files.

    View Slide

  32. afl In a Nutshell

    Turn a few maybe uninteresting inputs into a lot of hopefully very interesting inputs, quickly

    View Slide

  33. afl is a dumb fuzzer, meaning…

    View Slide

  34. afl Fuzz Strategies
    Walking bit flips
    (try flipping each bit in input individually)
    Walking byte flips
    (try flipping each contiguous set of 8 bits)
    Simple arithmetic
    (increment or decrement bytes in the file by certain small values)
    Known integers (or dictionaries)
    (replace bytes with “problematic” 8, 16, and 32 bit integers like 0 and FF)
    Profile-guided stacked tweaks and test case splicing
    (magic!)
    https://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html
    Deterministic, genetic

    View Slide

  35. Walking Bit Flip
    Original 01010101
    Flip bit 0 01010100
    Flip bit 1 01010111
    Flip bit 2 01010001

    Walking 2 Bit Flip
    Original 01010101
    Flip bits 0,1 01010110
    Flip bits 2,1 01010011
    Flip bits 3,2 01001101

    View Slide

  36. Trace Execution Paths
    A
    B D
    C
    E
    F
    ?

    View Slide

  37. When Should I
    Use It?
    When is it
    useful?
    https://it.wikipedia.org/wiki/File:Coniglio_ariete.JPG

    View Slide

  38. Unit Tests Fuzzing
    Useful For
    Preventing
    Regressions, Design
    Finding New Bugs
    Tests Functions Any Level
    Test Examples Hand-selected values Corpus + Mutation
    Execution Time Milliseconds Weeks
    Magic? No Yes

    View Slide

  39. Instrumentation
    for profile guidance

    View Slide

  40. $ ./configure CC="afl-gcc" \
    CXX="afl-g++" \
    --disable-shared; \
    make
    You’ve seen this before; afl requires recompiling with instrumentation

    View Slide

  41. $ pip install python-afl
    ———
    afl.init()

    View Slide

  42. Properties
    a.k.a. Specifications
    https://lorinhochstein.wordpress.com/2014/06/04/crossing-the-river-with-tla/

    View Slide

  43. A simple specification: Programs shouldn’t crash.

    View Slide

  44. Sanitizers and Canaries

    Security cookie (/GS)

    View Slide

  45. STJSON
    A JSON Parser in Swift 3 compliant with RFC 7159
    STJSON was written along with the article Parsing JSON is a Minefield.
    Basic usage:
    var p = STJSONParser(data: data)
    do {
    let o = p.parse()
    } catch let e {
    print(e)
    }
    Instantiation with options:
    var p = STJSON(data:data,
    maxParserDepth:1024,
    options:[.useUnicodeReplacementCharacter])
    https://github.com/nst/STJSON
    https://github.com/CraigStuntz/Fizil/tree/master/StJson
    Is there a specification for the behavior of the program?

    View Slide

  46. Can You Fuzz the
    Domain?

    View Slide

  47. Dumb Fuzzer
    Mangling Byte Arrays
    many interesting programs take binary input

    sometimes this requires less domain knowledge (not always!) or at least less custom code

    What about programs or functions which can’t accept binary input?

    View Slide

  48. public byte[] ResizePng(
    byte[] image)
    {
    Great candidate for dumb fuzzing

    View Slide

  49. public boolean SomeFunction(
    SomeEnum firstArg,
    int secondArg)
    {

    not a good candidate for dumb fuzzing

    If I have a good property I can test exhaustively

    View Slide

  50. Smart Fuzzing
    MongoDB Expression Grammar
    http://queue.acm.org/detail.cfm?ref=rss&id=3059007
    Not an end to end test — testing the internals.

    Don’t test stuff which takes forever

    View Slide

  51. public Assembly Compile(
    AbstractNode syntaxTree)
    {
    Good candidate for smart fuzzing?

    View Slide

  52. Worth the Wait?
    Results Can Take Weeks

    View Slide

  53. Isn’t it just for Security?
    https://www.flickr.com/photos/wocintechchat/25721078480/
    Every security person knows about fuzzing. Almost no app devs do.

    Security people care more about bugs than app devs.

    View Slide

  54. How do
    I get
    started?

    View Slide

  55. How to Get Started with Fuzzing
    1. Find a program to test
    2. Find a fuzzer
    3. Find a corpus
    4. Choose a property
    5. Let it run!

    View Slide

  56. Fuzzers
    https://commons.wikimedia.org/wiki/File:Holland_Lop_with_Broken_Orange_Coloring.jpg

    View Slide

  57. libfuzzer
    Fuzz testing for LLVM compilers
    http://llvm.org/docs/LibFuzzer.html

    View Slide

  58. $ cargo install cargo-fuzz
    $ cargo fuzz init

    View Slide

  59. $ gzip -c /bin/bash > sample.gz
    $ while true
    do
    radamsa sample.gz > fuzzed.gz
    gzip -dc fuzzed.gz > /dev/null
    test $? -gt 127 && break
    done
    ← Fuzz the corpus
    ← Execute S.O.T.
    ← Check a property
    ← Repeat a lot!
    https://github.com/aoh/radamsa
    Radamsa: Un*x philosophy of “do one thing well, chain with other stuff”

    Works on Windows with cygwin (and maybe

    View Slide

  60. ClusterFuzz
    Submit a fuzzer, win a bounty
    https://security.googleblog.com/2016/08/guided-in-process-fuzzing-of-chrome.html

    View Slide

  61. afl

    View Slide

  62. OSS-Fuzz
    Submit your project
    https://github.com/google/oss-fuzz
    1. You can use a fuzzer in your CI

    2. Since it’s easy to do so, Google did it for you.

    View Slide

  63. “Project Springfield”
    https://www.microsoft.com/en-us/security-risk-detection/

    View Slide

  64. burp, ZAP
    non-instrumented

    View Slide

  65. Corpus
    https://en.wikipedia.org/wiki/File:Long_Room_Interior,_Trinity_College_Dublin,_Ireland_-
    _Diliff.jpg

    View Slide


  66. We didn't call it fuzzing back in the 1950s, but it was
    our standard practice to test programs by inputting
    decks of punch cards taken from the trash.
    -Gerald M. Weinberg
    http://secretsofconsulting.blogspot.com/2017/02/fuzz-testing-and-fuzz-history.html

    View Slide

  67. View Slide

  68. Fuzzing SQLite with afl
    Start with a single test case:
    create table t1(one smallint);
    insert into t1 values(1);
    select * from t1;
    Add a list of reserved words from documentation
    Then extract SQL statements from SQLite unit tests
    (550 files at around 220 bytes each)
    https://lcamtuf.blogspot.com/2015/04/finding-bugs-in-sqlite-easy-way.html

    View Slide

  69. Properties

    View Slide

  70. Don’t Crash
    or Hang

    View Slide

  71. Sanitizers
    and Canaries
    https://docs.google.com/presentation/d/19OSgb1N9Ezef39Blb-5lkzycq7-tMtAvy825FofyrmY

    View Slide

  72. Validators

    View Slide

  73. End to End
    Input
    Output
    Same as
    Input?
    f(input)
    f-1(input)

    View Slide

  74. Legacy Code
    https://commons.wikimedia.org/wiki/File:Blue-punch-card-front-horiz.png

    View Slide

  75. One of The Things
    …Is Not Like the Others!
    Repeated testing with same input should yield same output

    View Slide

  76. Should I
    write my
    own?

    View Slide

  77. Heck Yeah!
    https://github.com/CraigStuntz/Fizil

    View Slide

  78. Interesting Stuff I Learned While Writing a Fuzzer
    - Deceptive simplicity of fuzzing
    - F# bitwise operations
    - How to instrument .NET code
    - dnSpy is awesome
    - Same input -> Same code -> Different paths
    - Strong naming is painful
    - Unicode is also painful
    - Speed is everything

    View Slide

  79. let jsonNetResult =
    try JsonConvert.DeserializeObject(str) |> ignore
    Success
    with
    | :? JsonReaderException as jre -> jre.Message |> Error
    | :? JsonSerializationException as jse -> jse.Message |> Error
    | :? System.FormatException as fe ->
    if fe.Message.StartsWith("Invalid hex character”) // hard coded in Json.NET
    then fe.Message |> Error
    else reraise()
    ⃪ T
    est
    ⬑ Special case error stuff
    Special test harness code specific to the system under test

    View Slide

  80. use proc = new Process()
    proc.StartInfo.FileName <- executablePath
    inputMethod.BeforeStart proc testCase.Data
    proc.StartInfo.UseShellExecute <- false
    proc.StartInfo.RedirectStandardOutput <- true
    proc.StartInfo.RedirectStandardError <- true
    proc.StartInfo.EnvironmentVariables.Add(SharedMemory.environmentVariableName, sharedMemoryName)
    let output = new System.Text.StringBuilder()
    let err = new System.Text.StringBuilder()
    proc.OutputDataReceived.Add(fun args -> output.Append(args.Data) |> ignore)
    proc.ErrorDataReceived.Add (fun args -> err.Append(args.Data) |> ignore)
    proc.Start() |> ignore
    inputMethod.AfterStart proc testCase.Data
    proc.BeginOutputReadLine()
    proc.BeginErrorReadLine()
    proc.WaitForExit()
    let exitCode = proc.ExitCode
    let crashed = exitCode = WinApi.ClrUnhandledExceptionCode
    ⃪ Set up
    ⃪ Read results
    ⃪ Important bit
    Generic code in Fizil

    View Slide

  81. /// An ordered list of functions to use when starting with a single piece of
    /// example data and producing new examples to try
    let private allStrategies(dictionaryValues: byte[][])= [
    bitFlip 1
    bitFlip 2
    bitFlip 4
    byteFlip 1
    byteFlip 2
    byteFlip 4
    arith8
    arith16
    arith32
    dictionary dictionaryValues
    interest8
    interest16
    ]

    View Slide

  82. let totalBits = bytes.Length * 8
    let testCases = seq {
    for bit = 0 to totalBits - flipBits do
    let newBytes = Array.copy bytes
    let firstByte = bit / 8
    let firstByteMask, secondByteMask = bitMasks(bit, flipBits)
    let newFirstByte = bytes.[firstByte] ^^^ firstByteMask
    newBytes.[firstByte] <- newFirstByte
    let secondByte = firstByte + 1
    if secondByteMask <> 0uy && secondByte < bytes.Length
    then
    let newSecondByte = bytes.[secondByte] ^^^ secondByteMask
    newBytes.[secondByte] <- newSecondByte
    yield newBytes
    }
    Fuzz one byte →
    ^^^ means xor

    View Slide

  83. View Slide

  84. View Slide

  85. private static void F(string arg)
    {
    Console.WriteLine("f");
    Console.Error.WriteLine("Error!");
    Environment.Exit(1);
    }

    View Slide

  86. private static void F(string arg)
    {
    instrument.Trace(29875);
    Console.WriteLine("f");
    Console.Error.WriteLine("Error!");
    Environment.Exit(1);
    }
    ← Random number

    View Slide

  87. private static void F(string arg)
    {
    #if MANUAL_INSTRUMENTATION
    instrument.Trace(29875);
    #endif
    Console.WriteLine("f");
    Console.Error.WriteLine("Error!");
    Environment.Exit(1);
    }

    View Slide

  88. let stringify (ob: obj) : string =
    JsonConvert.SerializeObject(ob)
    // Method: System.String\u0020Program::stringify(System.Object)
    .body stringify {
    arg_02_0 [generated]
    arg_07_0 [generated]
    nop()
    arg_02_0 = ldloc(ob)
    arg_07_0 = call(JsonConvert::SerializeObject, arg_02_0)
    ret(arg_07_0)
    }
    // Method: System.String\u0020Program::stringify(System.Object)
    .body stringify {
    arg_05_0 [generated]
    arg_0C_0 [generated]
    arg_11_0 [generated]
    arg_05_0 = ldc.i4(23831)
    call(Instrument::Trace, arg_05_0)
    nop()
    arg_0C_0 = ldloc(ob)
    arg_11_0 = call(JsonConvert::SerializeObject, arg_0C_0)
    ret(arg_11_0)
    }

    View Slide

  89. View Slide

  90. let private insertTraceInstruction(ilProcessor: ILProcessor, before: Instruction, state) =
    let compileTimeRandom = state.Random.Next(0, UInt16.MaxValue |> Convert.ToInt32)
    let ldArg = ilProcessor.Create(OpCodes.Ldc_I4, compileTimeRandom)
    let callTrace = ilProcessor.Create(OpCodes.Call, state.Trace)
    ilProcessor.InsertBefore(before, ldArg)
    ilProcessor.InsertAfter (ldArg, callTrace)
    This margin is too narrow to contain a try/finally example, so see:
    https://goo.gl/W4y7JH
    Inserting the IL instructions I needed was fairly easy. Here is the important bit of the code which does it. How did I learn how to write this? I instrumented a small program
    “manually” by writing the instrumentation code myself, and then decompiled that program to figure out which IL instructions I needed. Inserting them with Mono.Cecil is
    just a few lines of code.

    try/finally is much, much harder. I won’t even try to walk you through it here. Look at the GitHub repo if you want to see how it’s done.

    View Slide

  91. View Slide

  92. http://www.json.org/
    I need a way to determine if Json.NET is parsing the JSON correctly. So I thought I should write a JSON validator to check its behavior. Fortunately, there’s a standard!

    “Probably the boldest design decision I made was to not put a version number on JSON so there is no mechanism for revising it. We are stuck with JSON: whatever it is
    in its current form, that’s it.” -Crockford

    View Slide

  93. https://tools.ietf.org/html/rfc4627
    So, naturally, there’s also another JSON standard

    View Slide

  94. http://www.ecma-international.org/ecma-262/5.1/#sec-15.12
    And another JSON standard

    View Slide

  95. http://www.ecma-international.org/publications/standards/Ecma-404.htm
    And another JSON standard

    View Slide

  96. https://tools.ietf.org/html/rfc7158
    And another JSON standard

    View Slide

  97. https://tools.ietf.org/html/rfc7159
    And another JSON standard. And no, they don’t all agree on everything, nor is there a single, “latest” version. Despite this multitude of standards, there are still edge
    cases intentionally delegated to the implementer — what we would call “undefined behavior” in C.

    View Slide

  98. https://github.com/nst/STJSON
    I was going to write my own validator, but…

    Nicolas Seriot wrote a validator called STJSON which attempts to synthesize these as much as possible.

    View Slide

  99. https://github.com/CraigStuntz/Fizil/blob/master/StJson/StJsonParser.fs
    Swift doesn’t readily compile to Windows, but if you squint hard enough it kind of looks like F#, so I ported the code and used it to validate Json.NET's behavior.

    View Slide

  100. { "a" : "bc" }

    View Slide

  101. View Slide

  102. View Slide

  103. Standard Accepts, Json.NET Rejects
    Value
    88888888888888888888888888888888888888888888888888
    88888888888888888888888888888888888888888888888888
    88888888888888888888888888888888888888888888888888
    88888888888888888888888888888888888888888888888888
    88888888888888888888888888888888888888888888888888
    Standard Says No limit
    Json.NET MaximumJavascriptIntegerCharacterLength = 380;
    Things JSON.NET fails on that the standard accepts

    View Slide

  104. Standard Rejects, Json.NET Accepts
    Value [,,,]
    Standard Says
    A JSON value MUST be an object, array, number, or string, or one
    of
    the following three literal names:
    false null true
    Json.NET [null, null, null, null]
    Things JSON.NET succeeds on that the standard rejects

    View Slide

  105. Strong naming was a consistent pain for me.

    I’m altering the binaries of assemblies, and part of the point of strong naming is to stop you from doing just that, so naturally if the assembly is strongly named it can’t be
    loaded when I’m finished.

    View Slide

  106. let private removeStrongName (assemblyDefinition : AssemblyDefinition) =
    let name = assemblyDefinition.Name;
    name.HasPublicKey <- false;
    name.PublicKey <- Array.empty;
    assemblyDefinition.Modules |> Seq.iter (
    fun moduleDefinition ->
    moduleDefinition.Attributes <-
    moduleDefinition.Attributes &&& ~~~ModuleAttributes.StrongNameSigned)
    let aptca = assemblyDefinition.CustomAttributes.FirstOrDefault(
    fun attr -> attr.AttributeType.FullName
    = typeof.FullName)
    assemblyDefinition.CustomAttributes.Remove aptca |> ignore
    assembly.MainModule.AssemblyReferences
    |> Seq.filter (fun reference -> Set.contains reference.Name assembliesToInstrument)
    |> Seq.iter (fun reference ->
    reference.PublicKeyToken <- null
    )
    So I need to remove the strong name from any assembly I fuzz, but I also need to remove the PublicKeyToken from any other assembly which references it. Doing this in
    Mono.Cecil is not well-documented, and after quite a bit of time spent in GitHub issues and trial and error I figured out that it takes 5 distinct steps to do this.

    View Slide


  107. “If marked BeforeFieldInit then the type’s initializer
    method is executed at, or sometime before, first
    access to any static field defined for that type.”
    -ECMA-335, Common Language
    Infrastructure (CLI), Partition I

    View Slide

  108. Unicode
    Original JSON
    { "a": "bc" }
    ASCII Bytes
    7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D
    UTF-8 with Byte Order Mark
    EF BB BF 7B 20 22 61 22 20 3A 20 22 62 63 22 20 7D
    UTF-16 BE with BOM
    FE FF 00 7B 00 20 00 22 00 61 00 22 00 20 00 3A 00 20 00 22
    00 62 00 63 00 22 00 20 00 7D

    View Slide

  109. View Slide

  110. Resources
    Awesome Fuzzing: A Curated List of Fuzzing Resources
    https://github.com/secfigo/Awesome-Fuzzing
    OWASP Fuzzing Page
    https://www.owasp.org/index.php/Fuzzing
    MongoDB’s JavaScript Fuzzer
    http://queue.acm.org/detail.cfm?ref=rss&id=3059007
    afl technical details
    http://lcamtuf.coredump.cx/afl/technical_details.txt
    afl Help Email List
    [email protected]
    Fizil
    https://github.com/CraigStuntz/Fizil
    WTF, ACM?

    View Slide

  111. Thank You!
    - Michał Zalewski, for afl documentation
    - Rehearsal audiences, employees of
    - Dynamit
    - Improving
    - Ineffable Solutions

    View Slide

  112. Craig Stuntz
    [email protected]
    www.craigstuntz.com
    @craigstuntz
    http://www.meetup.com/Papers-We-Love-Columbus/

    View Slide