$30 off During Our Annual Pro Sale. View Details »

Introduction to Fuzzing

Ren Kimura
February 13, 2020

Introduction to Fuzzing

Basic Fuzzing Training by Ren Kimura CEO of Ricerca Security, Inc.

Ren Kimura

February 13, 2020
Tweet

More Decks by Ren Kimura

Other Decks in Programming

Transcript

  1. Introduction to Fuzzing
    Ren Kimura
    [email protected]
    https://ricsec.co.jp

    View Slide

  2. Ren Kimura
    2
    CVE-2019-14247, CVE-2019-14248, CVE-2019-14249,
    CVE-2019-14250, CVE-2019-16161, CVE-2019-16162,
    CVE-2019-16163, CVE-2019-16164, CVE-2019-16165,
    CVE-2019-16166, CVE-2019-16167, CVE-2019-19725
    Ricerca Security, Inc. CEO
    twitter: @RKX1209
    mail: [email protected]

    View Slide

  3. 3
    Background
    In 2016, DARPA hosted fully automated hacking challenge
    Software components become more larger and complicated.
    → Many people try to automate analyzing process.
    In 2016, DARPA hosted Cyber Grand Challenge (CGC)
    → Almost all winners used Fuzzing technique in the vulnerability detection process.

    View Slide

  4. 4
    Vulnerability
    Finding
    Triage
    Evaluation
    Exploit
    Generation
    “Fuzzing”, “Static Analysis”
    “Symbolic Execution”
    “Triage”, “Exploitability”
    “Bug Reproduction”
    “Automatic Exploit Generation”
    (AEG)
    Crash Inputs Triaged / POC
    Exploit Code
    How to automate Hacking?

    View Slide

  5. Mutation based Black box Fuzzing
    Vulnerability Finding

    View Slide

  6. 6
    They randomly generates large amount of inputs and execute program with it.
    Black box Fuzzing (Synopsys defensics, zzuf)
    “a”
    “kqeqert”
    “G\x13\x02”
    “iohbpofi9qnpiof”
    “3i129074g”
    They don’t observe program
    behavior, learn nothing.
    They continue dumb generation forever

    View Slide

  7. 7
    Fuzzer
    Target Program
    (PUT)
    Initial Seed
    Mutator
    Black box Fuzzing
    They mutates initial seed, generate inputs and execute program with it.

    View Slide

  8. 8
    Initial Seed
    ・・・
    Mutation
    They mutate initial seed to generate new random inputs
    Input Generation (defensics, zzuf)
    Initial.png, Initial.jpeg...

    View Slide

  9. 9
    Bitflip mutation
    Flip n-th bit of input. Many fuzzers (AFL, zzuf..) use it.
    0 0 1 0 1 0 0 1
    0 0 1 0 1 1 0 1
    BitFlip

    View Slide

  10. 10
    Byteflip mutation
    0 0 1 0 1 0 0 1
    0 0 1 0 1 1 0 1
    Byteflip
    1 1 0 1 0 1 1 0
    Flip n-th byte of input. Many fuzzers (AFL, zzuf..) use it.

    View Slide

  11. 11
    Arithmetic mutation
    Add, Subtract, Multiply or Divide crazy integer on n-th byte
    0 0 1 0 1 0 0 1
    0 0 1 0 1 1 0 1
    1 1 0 1 0 1 1 0
    Arithmetic operation +-*/ 256, U32_MAX, U32_MIN

    View Slide

  12. 12
    Insert/Delete mutation
    Insert extra bytes at n-th offset. Delete n bytes subset of input
    0 0 1 0 1 0 0 1
    0 0 1 0 1 0 0 1
    Insert/Delete
    0 0 1 0 1 1 0 1
    1 1 0 1 0 1 1 0

    View Slide

  13. 13
    Hands-on 1 (zzuf)
    “Using zzuf directly“ https://fuzzing-project.org/tutorial1.html
    zzuf -s 0:1000000 -c -C 0 -q -T 3 objdump -x win9x.exe
    zzuf -s -T
    sudo apt-get install zzuf
    Try other combination, like readelf -a /bin/ls, file, ...

    View Slide

  14. Mutation based Grey box Fuzzing
    Vulnerability Finding

    View Slide

  15. 15
    Fuzzer
    Target Program
    (PUT)
    Initial Seed
    Mutator
    Feedback
    Grey box Fuzzing
    They generates large amount of inputs in the smart way with feedback.

    View Slide

  16. 16
    How does fuzzer get feedback?
    Fuzzer
    Target Program
    (PUT)
    Initial Seed
    Mutator
    Feedback
    They get some kind of feedback from program execution.

    View Slide

  17. 17
    Feedback mechanism (edge coverage)
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    2536
    124
    6210
    8147
    297
    4010
    Instrument unique random numbers
    to every basic blocks by compiler.

    View Slide

  18. 18
    Feedback mechanism (edge coverage)
    if (input[0] == ‘G’)
    if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    2536
    124
    297
    4010
    Calculate hash keys based on
    random numbers on program path.
    Map hash keys to memory
    key = nextBB ^ prevBB >> 1

    View Slide

  19. 19
    https://tunnelshade.in/blog/2018/01/afl-internals-compile-time-instrumentation/
    Hands-on 2 (afl-gcc on binutils)
    cd AFL && make
    git clone https://github.com/google/AFL
    wget https://ftp.gnu.org/gnu/binutils/binutils-2.26.tar.gz
    tar xf binutils-2.26.tar.gz
    cd binutils-2.26
    CC=/path/to/AFL/afl-gcc ./configure --disable-werror
    make

    View Slide

  20. 20
    https://tunnelshade.in/blog/2018/01/afl-internals-compile-time-instrumentation/
    Hands-on 2 (afl-gcc on binutils)
    gdb -q binutils-2.26/binutils/nm-new
    (gdb) disas main
    Dump of assembler code for function main:
    0x0000000000031ba0 <+0>: lea -0x98(%rsp),%rsp
    0x0000000000031ba8 <+8>: mov %rdx,(%rsp)
    0x0000000000031bac <+12>: mov %rcx,0x8(%rsp)
    0x0000000000031bb1 <+17>: mov %rax,0x10(%rsp)
    0x0000000000031bb6 <+22>: mov $0xab10,%rcx
    0x0000000000031bbd <+29>: callq 0x3caf8 <__afl_maybe_log>

    View Slide

  21. 21
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    2536
    124
    6210
    8147
    297
    4010
    Feedback mechanism (edge coverage)
    If we only have a binary executable...
    Instrument unique random numbers
    to every basic blocks by emulator.

    View Slide

  22. 22
    Black box vs Grey box Fuzzer
    Unconditional Branch
    Conditional Branch
    Vunlerable
    Control Flow Graph(CFG) of target program
    “GET”
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))

    View Slide

  23. 23
    Black-box Fuzzer (zzuf)
    Initial Seed
    “a”
    “GET”
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    ・・・・・
    “kq” “\xfeXp\x2a”
    Mutation

    View Slide

  24. 24
    Black-box Fuzzer (zzuf)
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    “a”
    ・・・
    “kq” “G” “G”
    Generated “G” from initial seed.
    Initial Seed
    Mutation
    “GET”

    View Slide

  25. 25
    Black-box Fuzzer (zzuf)
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    “a”
    ・・・
    “kq” “GE”
    “GE”
    Initial Seed
    Mutation
    Generated “GE” from initial seed.
    “GET”

    View Slide

  26. 26
    Black-box Fuzzer (zzuf)
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    “a”
    ・・・
    “kq”
    “GET”
    “GET”
    Initial Seed
    Mutation
    Generated “GET” from initial seed.

    View Slide

  27. 27
    Initial Seed
    “a”
    ・・・
    “kq”
    Mutation
    “GET”
    Coverage information + Genetic Algorithm (GA)
    Initial Seed
    “a”
    Mutation
    “G”
    “GE”
    “GET”
    Generation
    Selection and Mutation based on Fitness Function
    Grey-box Fuzzer (AFL, libfuzzer)
    Mutation
    Mutation

    View Slide

  28. 28
    Mutator
    Fuzzer
    Target Program
    (PUT)
    Initial Seed
    Mutator
    Feedback
    They generate inputs by mutating initial seed or parent seeds.
    Initial Seed
    “a”
    Mutation
    “G”
    “GE”
    “GET”
    Mutation
    Mutation

    View Slide

  29. 29
    Initial Seed
    “a” “G”
    “GE”
    “GET”
    Fitness Function
    Seed Scheduler (Selection)
    Mutation
    If generated input leads new
    program coverage, F(input) = 1.
    Which inputs should be mutated?
    Generate next inputs by mutatting
    parent inputs .
    Grey-box Fuzzer (AFL, libfuzzer)
    Coverage information + Genetic Algorithm (GA)
    Generation
    Mutation
    Mutation
    Mutation

    View Slide

  30. 30
    “GET<\x00”
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    “GET”
    “NULL” (Initial Seed)
    “G” “HTTP” “501”
    “GE”
    “GET”
    “GET<\x00”
    “HTTP1.x”
    “a”
    “GE”
    “G”
    Many state-of-art fuzzers use genetic
    algorithm with coverage information.
    Grey-box Fuzzer (AFL, libfuzzer)
    Coverage information + Genetic Algorithm (GA)

    View Slide

  31. 31
    Hands-on 3 (AFL)
    https://github.com/google/AFL
    ./afl-fuzz -i -t -m
    -o -- @@
    cd AFL && make
    git clone https://github.com/google/AFL
    ls /queue # Show all saved seeds
    ls /crashes # Show crash inputs

    View Slide

  32. 32
    Hands-on 3 (AFL example, nm)
    https://github.com/google/AFL
    ./afl-fuzz -i initial/ -t 10000 -m 1024
    -o output -- binutils-2.26/binutils/nm-new @@
    cp /bin/ls initial/
    mkdir initial
    ls /queue # Show all saved seeds
    ls /crashes # Show crash inputs

    View Slide

  33. Generation based Fuzzing
    Vulnerability Finding

    View Slide

  34. 34
    Highly structured inputs
    Fuzzer
    JavaScript
    HTML
    Firefox, Chrome
    (Target Program)

    View Slide

  35. 35
    Rule
    ・・・
    Generation
    They generate large amount of inputs from rule
    Generation based Fuzzing
    Fileformat (PIT)
    Grammar

    View Slide

  36. 36
    Fuzzer
    Target Program
    (PUT)
    Rule
    Generator
    Generation based Fuzzing
    They generate inputs and execute program with it.

    View Slide

  37. 37
    Hands-on 4 (PEACH)
    Read: https://github.com/MozillaSecurity/peach
    Run PEACH on Firefox
    ./peach.py -pit Pits///.xml -target Pits/Targets/firefox.xml
    -run Browser

    View Slide

  38. 38
    They infer data dependency between API calls and generate valid stub code.
    XNU kernel fuzzing with API Inferring
    HyungSeok Han IMF: Inferred Model-based Fuzzer [CCS17]

    View Slide

  39. White box Fuzzing
    Vulnerability Finding

    View Slide

  40. 40
    Fuzzer
    Target Program
    (PUT)
    Initial Seed
    Executor
    SMT solver
    White box Fuzzing
    They generate inputs by solving constraints, using SMT solver.
    Patrice Godefroid Automated Whitebox Fuzz Testing [NDSS08]

    View Slide

  41. 41
    How to work?
    Conditional Branch
    Unconditional Branch
    Vunlerable
    Control Flow Graph(CFG) of target program
    “GET<\x00”
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (input[1] == ‘E’)
    if (input[2] == ‘T’) if (isinteger(input))
    if (!strcmp(input, “HTTP”))

    View Slide

  42. 42
    White box Fuzzer (SAGE) Initial seed
    “GET<\x00”
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[0] == ‘G’)
    if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    “a”
    (1) Execute program with initial input “a”
    (2) Build constraints from trace
    (input[0] != “G”) & (strcmp(input, HTTP)) &
    (!isinteger(input))
    (3) Negate one of the constraints
    (input[0] == “G”) & (strcmp(input, HTTP)) &
    (!isinteger(input))
    (4) Solve constraints by SMT solver
    → next input “G”

    View Slide

  43. 43
    “GET<\x00”
    “HTTP”
    “501”
    “HTTP1.?
    if (input[1] == ‘E’)
    if (input[2] == ‘T’)
    “G”
    if (input[0] == ‘G’)
    Next seed
    “G”
    (1) Execute program with initial input “G”
    (2) Build constraints from trace
    (input[0] == “G”) & (input[1] != ‘E’) &
    (input[2] != “T”)
    (3) Negate one of the constraints
    (input[0] == “G”) & (input[1] == ‘E’) &
    (input[2] != “T”)
    (4) Solve constraints by SMT solver
    → next input “GE”
    White box Fuzzer (SAGE)

    View Slide

  44. 44
    “GET<\x00”
    “HTTP”
    “501”
    “HTTP1.?
    if (input[1] == ‘E’)
    if (input[2] == ‘T’)
    “GE”
    if (input[0] == ‘G’)
    Next seed
    “GE”
    (1) Execute program with initial input “GE”
    (2) Build constraints from trace
    (input[0] == “G”) & (input[1] != ‘E’) &
    (input[2] != “T”)
    (3) Negate one of the constraints
    (input[0] == “G”) & (input[1] == ‘E’) &
    (input[2] == “T”)
    (4) Solve constraints by SMT solver
    → next input “GET”
    White box Fuzzer (SAGE)

    View Slide

  45. 45
    “GET<\x00”
    “HTTP”
    “501”
    “HTTP1.?”
    if (input[1] == ‘E’)
    if (input[2] == ‘T’)
    “GET”
    if (input[0] == ‘G’)
    Next seed
    “GET”
    Congratulation!
    We found 3 program path lead by
    “G”, “GE” and “GET” inputs.
    White box Fuzzer (SAGE)
    It’s also called
    “Dynamic Symbolic Execution” (DSE)

    View Slide

  46. 46
    White box vs Grey box Fuzzer
    White box fuzzer entails significant computational cost (Build constraints, SMT solve)
    { C1 & C2 & C3 & C4 & C5 …
    C100 & C101 & C102 …
    ….
    C1000 && C1001 && C1002
    ….
    C2360 & C2361 & C2362
    …. }
    C1
    C2
    C3
    C2360
    C2361
    Thousand of Constraints….
    SMT query (over 2000 queries)
    Performance
    Overhead

    View Slide

  47. 47
    White box vs Grey box Fuzzer
    Grey box fuzzer is hard to overcome the long magic number comparison.
    if (input == 0xdeadbeefcafebabe) {
    crash();
    }
    Grey box way
    White box way
    Mutation
    “ 0x0” “0xdeadbeefcafebabe ”
    P(crash) = 1/(2^64)
    SMT solve
    “ 0x0” “0xdeadbeefcafebabe ”

    View Slide

  48. 48
    Hybrid Fuzzing (Driller)
    Control Flow Graph(CFG) of target program
    if (input == 0xdeadbeefcafebabe) if (input[0] == ‘A’’)
    if (input[2] < 10)
    Nick Stephen Driller: Augmenting Fuzzing Through Selective Symbolic Execution [NDSS16]
    Grey-box fuzzing
    Dynamic Symbolic Execution (DSE)

    View Slide

  49. Feedback Mechanism
    Vulnerability Finding

    View Slide

  50. 50
    Edge Coverage by Compiler, Emulator/Intel PT (AFL/kAFL)
    if (input[0] == ‘G’)
    if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    2536
    124
    297
    4010
    Calculate hash keys based on
    random numbers on program path.
    Sergej Schumilo kAFL: Hardware-Assisted Feedback Fuzzing for OS Kernels[UESNIX17]

    View Slide

  51. 51
    Build CFG from binary
    by Static Analysis
    Sanjay Rawat VUzzer: Application-aware Evolutionary Fuzzing [NDSS 17]
    Markov model on static analysis (VUzzer)
    if (input[0] == ‘G’)
    if (isinteger(input))
    if (!strcmp(input, “HTTP”))
    1.0
    0.5 0.5
    0.5 0.5
    0.5
    0.5
    0.5
    1.0
    1.0
    0.5
    0.5
    0.5
    0.5
    P(path) = 1.0*0.5*0.5*0.5*0.5 = 0.0625
    F(path) = 1/P(path) = 1/0.0625 = 16
    Binary executable format
    Markov Model

    View Slide

  52. 52
    Dynamic Binary Rewriting (Chizpurfle, RetroWrite)
    Stalker server rewrites the code block
    to instrument additional instructions,
    reveal basic block coverage.
    Antonio Chizpurfle: A Gray-Box Android Fuzzer for Vendor Service Customizations [ISSRE17]
    Sushant Dinesh RetroWrite: Statically Instrumenting COTS Binaries for Fuzzing and Sanitization[S&P20]
    They use frida,
    rewrite Android system services

    View Slide

  53. State-of-art Fuzzing
    Vulnerability Finding

    View Slide

  54. 54
    Fuzzer
    Target Program
    (PUT)
    Rule
    Mutator
    Feedback
    Smart Grey box Fuzzing (AFLSmart, Nautilus)
    They generates large amount of inputs based on Rule with feedback.

    View Slide

  55. 55
    Smart mutation with rule
    Mutate inputs with knowledge about grammar or file format.
    Semantics Mutation
    int f(int arg) { return g(arg); }
    short f(int arg) { return h(arg); }
    Grammar File (EBNF)
    Van-Thuan Pham Smart Greybox Fuzzing [TSE20]
    File Format (PIT)
    Cornelius Aschermann NAUTILUS: Fishing for Deep Bugs with Grammars [NDSS19]

    View Slide

  56. 56
    Hands-on 6 (AFLSmart)
    Read: https://github.com/aflsmart/aflsmart
    Find Crash input of WavPack by AFLSmart
    ./afl-fuzz -h -i -o -w peach -g
    -x -- @@
    ls /queue # Show all saved seeds
    ls /crashes # Show crash inputs

    View Slide

  57. 57
    Group work (Bug hunting in Real World)
    Choose your favorite real world programs.
    Try to find bug or vulnerabilities from them.
    You can use any fuzzers.
    You should try many (program, fuzzer) combinations!

    View Slide

  58. 58
    Group work (Bug hunting in Real World)
    afl-qemu (AFL against binary) https://github.com/google/AFL
    VUzzer (Fuzzing with tant analysis) https://github.com/vusec/vuzzer
    kAFL (Fuzzing linux kernel) https://github.com/RUB-SysSec/kAFL
    Nautilus (Grammar fuzzing) https://github.com/RUB-SysSec/nautilus
    Driller (Hybrid fuzzing) https://blog.grimm-co.com/post/guided-fuzzing-with-driller/

    View Slide

  59. https://ricsec.co.jp

    View Slide