Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Automated Testing with go-fuzz

Automated Testing with go-fuzz

Filippo Valsorda

October 02, 2015
Tweet

More Decks by Filippo Valsorda

Other Decks in Programming

Transcript

  1. Automated  Tes,ng  
    with  go-­‐fuzz
    Filippo  Valsorda

    View full-size slide

  2. Hi,  I’m  Filippo  Valsorda
    I  do  cryptography  and  systems  engineering  for  CloudFlare.  
    I  develop  the  CloudFlare  Go  DNS  server,  and  wrote  its  
    DNSSEC  implementa,on.  
    I  oFen  mess  with  and  talk  about  cryptography  and  security  
    protocols  defects.
    @FiloSo'le

    View full-size slide

  3. You  might  have  used  my  Heartbleed  test

    View full-size slide

  4. Or  connected  to  whoami.filippo.io

    View full-size slide

  5. Here’s  a  func,on
    package bug
    // ParseStrings parses a list of strings encoded in a binary format
    // where a single-byte value is followed by a string of that length
    // Note: ParseStrings takes unvalidated input from the network
    func ParseStrings(input []byte) (result []string) {
    for len(input) > 0 {
    strLength := input[0]
    input = input[1:]
    result = append(result, string(input[:strLength]))
    input = input[strLength:]
    }
    return
    }

    View full-size slide

  6. Here’s  a  func,on  bug
    package bug
    // ParseStrings parses a list of strings encoded in a binary format
    // where a single-byte value is followed by a string of that length
    // Note: ParseStrings takes unvalidated input from the network
    func ParseStrings(input []byte) (result []string) {
    for len(input) > 0 {
    strLength := input[0]
    input = input[1:]
    result = append(result, string(input[:strLength]))
    input = input[strLength:]
    }
    return
    }

    View full-size slide

  7. So  write  tests!
    func TestOne(t *testing.T) {
    result := ParseStrings([]byte("\x0cHello World!"))
    expected := []string{"Hello World!"}
    if !reflect.DeepEqual(result, expected) {
    t.Fatal(result)
    }
    }
    func TestMultiple(t *testing.T) {
    result := ParseStrings([]byte("\x03Foo\x0cHello World!"))
    expected := []string{"Foo", "Hello World!"}
    if !reflect.DeepEqual(result, expected) {
    t.Fatal(result)
    }
    }

    View full-size slide

  8. Test  corner  cases
    func TestEmptyString(t *testing.T) {
    result := ParseStrings([]byte{})
    if len(result) != 0 {
    t.Fatal(result)
    }
    }
    $ go test
    PASS
    ok github.com/FiloSottile/fuzz-talk 0.006s

    View full-size slide

  9. Here’s  a  func,on  bug
    package bug
    // ParseStrings parses a list of strings encoded in a binary format
    // where a single-byte value is followed by a string of that length
    // Note: ParseStrings takes unvalidated input from the network
    func ParseStrings(input []byte) (result []string) {
    for len(input) > 0 {
    strLength := input[0]
    input = input[1:]
    result = append(result, string(input[:strLength]))
    input = input[strLength:]
    }
    return
    }

    View full-size slide

  10. What’s  fuzzing?
    Fuzzing  is  feeding  programs  with  
    automa3cally  generated  inputs,

    to  trigger  unexpected  behavior.

    View full-size slide

  11. Muta,on
    \x0cHello World!
    \x0cHello World.

    View full-size slide

  12. Muta,on
    \x0cHello World!
    \x0cHell\x0cHell

    View full-size slide

  13. Muta,on
    \x0cHello World!
    \x0cHello World!o World!

    View full-size slide

  14. What’s  coverage-­‐guided  
    fuzzing?
    Fuzzing  is  dumb.  It  doesn’t  know  what  it’s  doing.  
    Coverage-­‐guided  fuzzing  uses  coverage  informa,on  
    to  decided  if  a  muta,on  was  useful  or  not.  
    If  coverage  increased,  keep  the  new  input  and  
    iterate,  if  it  didn’t  discard  the  muta,on.

    View full-size slide

  15. // UnpackDomainName unpacks a domain name into a string.
    func UnpackDomainName(msg []byte, off int) (string, int, error) {
    s := make([]byte, 0, 64)
    off1 := 0
    lenmsg := len(msg)
    ptr := 0 // number of pointers followed
    Loop:
    for {
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c := int(msg[off])
    off++
    switch c & 0xC0 {
    case 0x00:
    if c == 0x00 {
    // end of name
    break Loop
    }
    // literal string
    if off+c > lenmsg {
    return "", lenmsg, ErrBuf
    }
    for j := off; j < off+c; j++ {
    switch b := msg[j]; b {
    case '.', '(', ')', ';', ' ', '@':
    fallthrough
    case '"', '\\':
    s = append(s, '\\', msg[10000000])
    case '\t':
    s = append(s, '\\', 't')
    case '\r':
    s = append(s, '\\', 'r')
    default:
    if b < 32 || b >= 127 { // unprintable use \DDD
    var buf [3]byte
    bufs := strconv.AppendInt(buf[:0], int64(b), 10)
    s = append(s, '\\')
    for i := 0; i < 3-len(bufs); i++ {
    s = append(s, '0')
    }
    for _, r := range bufs {
    s = append(s, r)
    }
    } else {
    s = append(s, b)
    }
    }
    }
    s = append(s, '.')
    off += c
    case 0xC0:
    // pointer to somewhere else in msg.
    // remember location after first ptr,
    // since that's how many bytes we consumed.
    // also, don't follow too many pointers --
    // maybe there's a loop.
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c1 := msg[off]
    off++
    if ptr == 0 {
    off1 = off
    }
    if ptr++; ptr > 10 {
    return "", lenmsg, &Error{err: "too many compression pointers"}
    }
    off = (c^0xC0)<<8 | int(c1)
    default:
    // 0x80 and 0x40 are reserved
    return "", lenmsg, ErrRdata
    }
    }
    if ptr == 0 {
    off1 = off
    }
    if len(s) == 0 {
    s = []byte(".")
    }
    return string(s), off1, nil
    }
    Imagine  a  bug  very  deep  in  a  func,on  

    View full-size slide

  16. // UnpackDomainName unpacks a domain name into a string.
    func UnpackDomainName(msg []byte, off int) (string, int, error) {
    s := make([]byte, 0, 64)
    off1 := 0
    lenmsg := len(msg)
    ptr := 0 // number of pointers followed
    Loop:
    for {
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c := int(msg[off])
    off++
    switch c & 0xC0 {
    case 0x00:
    if c == 0x00 {
    // end of name
    break Loop
    }
    // literal string
    if off+c > lenmsg {
    return "", lenmsg, ErrBuf
    }
    for j := off; j < off+c; j++ {
    switch b := msg[j]; b {
    case '.', '(', ')', ';', ' ', '@':
    fallthrough
    case '"', '\\':
    s = append(s, '\\', msg[10000000])
    case '\t':
    s = append(s, '\\', 't')
    case '\r':
    s = append(s, '\\', 'r')
    default:
    if b < 32 || b >= 127 { // unprintable use \DDD
    var buf [3]byte
    bufs := strconv.AppendInt(buf[:0], int64(b), 10)
    s = append(s, '\\')
    for i := 0; i < 3-len(bufs); i++ {
    s = append(s, '0')
    }
    for _, r := range bufs {
    s = append(s, r)
    }
    } else {
    s = append(s, b)
    }
    }
    }
    s = append(s, '.')
    off += c
    case 0xC0:
    // pointer to somewhere else in msg.
    // remember location after first ptr,
    // since that's how many bytes we consumed.
    // also, don't follow too many pointers --
    // maybe there's a loop.
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c1 := msg[off]
    off++
    if ptr == 0 {
    off1 = off
    }
    if ptr++; ptr > 10 {
    return "", lenmsg, &Error{err: "too many compression pointers"}
    }
    off = (c^0xC0)<<8 | int(c1)
    default:
    // 0x80 and 0x40 are reserved
    return "", lenmsg, ErrRdata
    }
    }
    if ptr == 0 {
    off1 = off
    }
    if len(s) == 0 {
    s = []byte(".")
    }
    return string(s), off1, nil
    }
    (Not  a  real  bug)
    Imagine  a  bug  very  deep  in  a  func,on  

    View full-size slide

  17. // UnpackDomainName unpacks a domain name into a string.
    func UnpackDomainName(msg []byte, off int) (string, int, error) {
    s := make([]byte, 0, 64)
    off1 := 0
    lenmsg := len(msg)
    ptr := 0 // number of pointers followed
    Loop:
    for {
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c := int(msg[off])
    off++
    switch c & 0xC0 {
    case 0x00:
    if c == 0x00 {
    // end of name
    break Loop
    }
    // literal string
    if off+c > lenmsg {
    return "", lenmsg, ErrBuf
    }
    for j := off; j < off+c; j++ {
    switch b := msg[j]; b {
    case '.', '(', ')', ';', ' ', '@':
    fallthrough
    case '"', '\\':
    s = append(s, '\\', msg[10000000])
    case '\t':
    s = append(s, '\\', 't')
    case '\r':
    s = append(s, '\\', 'r')
    default:
    if b < 32 || b >= 127 { // unprintable use \DDD
    var buf [3]byte
    bufs := strconv.AppendInt(buf[:0], int64(b), 10)
    s = append(s, '\\')
    for i := 0; i < 3-len(bufs); i++ {
    s = append(s, '0')
    }
    for _, r := range bufs {
    s = append(s, r)
    }
    } else {
    s = append(s, b)
    }
    }
    }
    s = append(s, '.')
    off += c
    case 0xC0:
    // pointer to somewhere else in msg.
    // remember location after first ptr,
    // since that's how many bytes we consumed.
    // also, don't follow too many pointers --
    // maybe there's a loop.
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c1 := msg[off]
    off++
    if ptr == 0 {
    off1 = off
    }
    if ptr++; ptr > 10 {
    return "", lenmsg, &Error{err: "too many compression pointers"}
    }
    off = (c^0xC0)<<8 | int(c1)
    default:
    // 0x80 and 0x40 are reserved
    return "", lenmsg, ErrRdata
    }
    }
    if ptr == 0 {
    off1 = off
    }
    if len(s) == 0 {
    s = []byte(".")
    }
    return string(s), off1, nil
    }
    (Not  a  real  bug)
    If  the  ini,al  input  has  this  coverage  

    View full-size slide

  18. // UnpackDomainName unpacks a domain name into a string.
    func UnpackDomainName(msg []byte, off int) (string, int, error) {
    s := make([]byte, 0, 64)
    off1 := 0
    lenmsg := len(msg)
    ptr := 0 // number of pointers followed
    Loop:
    for {
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c := int(msg[off])
    off++
    switch c & 0xC0 {
    case 0x00:
    if c == 0x00 {
    // end of name
    break Loop
    }
    // literal string
    if off+c > lenmsg {
    return "", lenmsg, ErrBuf
    }
    for j := off; j < off+c; j++ {
    switch b := msg[j]; b {
    case '.', '(', ')', ';', ' ', '@':
    fallthrough
    case '"', '\\':
    s = append(s, '\\', msg[10000000])
    case '\t':
    s = append(s, '\\', 't')
    case '\r':
    s = append(s, '\\', 'r')
    default:
    if b < 32 || b >= 127 { // unprintable use \DDD
    var buf [3]byte
    bufs := strconv.AppendInt(buf[:0], int64(b), 10)
    s = append(s, '\\')
    for i := 0; i < 3-len(bufs); i++ {
    s = append(s, '0')
    }
    for _, r := range bufs {
    s = append(s, r)
    }
    } else {
    s = append(s, b)
    }
    }
    }
    s = append(s, '.')
    off += c
    case 0xC0:
    // pointer to somewhere else in msg.
    // remember location after first ptr,
    // since that's how many bytes we consumed.
    // also, don't follow too many pointers --
    // maybe there's a loop.
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c1 := msg[off]
    off++
    if ptr == 0 {
    off1 = off
    }
    if ptr++; ptr > 10 {
    return "", lenmsg, &Error{err: "too many compression pointers"}
    }
    off = (c^0xC0)<<8 | int(c1)
    default:
    // 0x80 and 0x40 are reserved
    return "", lenmsg, ErrRdata
    }
    }
    if ptr == 0 {
    off1 = off
    }
    if len(s) == 0 {
    s = []byte(".")
    }
    return string(s), off1, nil
    }
    (Not  a  real  bug)
    The  fuzzer  will  keep  muta,ng  the  
    input  un,l  it  finds  a  muta,on  that  
    changes  the  coverage

    View full-size slide

  19. // UnpackDomainName unpacks a domain name into a string.
    func UnpackDomainName(msg []byte, off int) (string, int, error) {
    s := make([]byte, 0, 64)
    off1 := 0
    lenmsg := len(msg)
    ptr := 0 // number of pointers followed
    Loop:
    for {
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c := int(msg[off])
    off++
    switch c & 0xC0 {
    case 0x00:
    if c == 0x00 {
    // end of name
    break Loop
    }
    // literal string
    if off+c > lenmsg {
    return "", lenmsg, ErrBuf
    }
    for j := off; j < off+c; j++ {
    switch b := msg[j]; b {
    case '.', '(', ')', ';', ' ', '@':
    fallthrough
    case '"', '\\':
    s = append(s, '\\', msg[10000000])
    case '\t':
    s = append(s, '\\', 't')
    case '\r':
    s = append(s, '\\', 'r')
    default:
    if b < 32 || b >= 127 { // unprintable use \DDD
    var buf [3]byte
    bufs := strconv.AppendInt(buf[:0], int64(b), 10)
    s = append(s, '\\')
    for i := 0; i < 3-len(bufs); i++ {
    s = append(s, '0')
    }
    for _, r := range bufs {
    s = append(s, r)
    }
    } else {
    s = append(s, b)
    }
    }
    }
    s = append(s, '.')
    off += c
    case 0xC0:
    // pointer to somewhere else in msg.
    // remember location after first ptr,
    // since that's how many bytes we consumed.
    // also, don't follow too many pointers --
    // maybe there's a loop.
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c1 := msg[off]
    off++
    if ptr == 0 {
    off1 = off
    }
    if ptr++; ptr > 10 {
    return "", lenmsg, &Error{err: "too many compression pointers"}
    }
    off = (c^0xC0)<<8 | int(c1)
    default:
    // 0x80 and 0x40 are reserved
    return "", lenmsg, ErrRdata
    }
    }
    if ptr == 0 {
    off1 = off
    }
    if len(s) == 0 {
    s = []byte(".")
    }
    return string(s), off1, nil
    }
    (Not  a  real  bug)
    And  then  it  will  learn  that  new  case,  
    and  start  muta,ng  that  one,  un,l…

    View full-size slide

  20. // UnpackDomainName unpacks a domain name into a string.
    func UnpackDomainName(msg []byte, off int) (string, int, error) {
    s := make([]byte, 0, 64)
    off1 := 0
    lenmsg := len(msg)
    ptr := 0 // number of pointers followed
    Loop:
    for {
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c := int(msg[off])
    off++
    switch c & 0xC0 {
    case 0x00:
    if c == 0x00 {
    // end of name
    break Loop
    }
    // literal string
    if off+c > lenmsg {
    return "", lenmsg, ErrBuf
    }
    for j := off; j < off+c; j++ {
    switch b := msg[j]; b {
    case '.', '(', ')', ';', ' ', '@':
    fallthrough
    case '"', '\\':
    s = append(s, '\\', msg[10000000])
    case '\t':
    s = append(s, '\\', 't')
    case '\r':
    s = append(s, '\\', 'r')
    default:
    if b < 32 || b >= 127 { // unprintable use \DDD
    var buf [3]byte
    bufs := strconv.AppendInt(buf[:0], int64(b), 10)
    s = append(s, '\\')
    for i := 0; i < 3-len(bufs); i++ {
    s = append(s, '0')
    }
    for _, r := range bufs {
    s = append(s, r)
    }
    } else {
    s = append(s, b)
    }
    }
    }
    s = append(s, '.')
    off += c
    case 0xC0:
    // pointer to somewhere else in msg.
    // remember location after first ptr,
    // since that's how many bytes we consumed.
    // also, don't follow too many pointers --
    // maybe there's a loop.
    if off >= lenmsg {
    return "", lenmsg, ErrBuf
    }
    c1 := msg[off]
    off++
    if ptr == 0 {
    off1 = off
    }
    if ptr++; ptr > 10 {
    return "", lenmsg, &Error{err: "too many compression pointers"}
    }
    off = (c^0xC0)<<8 | int(c1)
    default:
    // 0x80 and 0x40 are reserved
    return "", lenmsg, ErrRdata
    }
    }
    if ptr == 0 {
    off1 = off
    }
    if len(s) == 0 {
    s = []byte(".")
    }
    return string(s), off1, nil
    }
    (Not  a  real  bug)
    And  then  it  will  learn  that  new  case,  
    and  start  muta,ng  that  one,  un,l

    eventually  it  finds  an  input  that  
    triggers  the  bug  and  crashes

    View full-size slide

  21. American  Fuzzy  Lop
    afl-­‐fuzz  is  a  very  effec,ve  and  popular  C/C++  fuzzer  by  
    Michał  Zalewski  aka  lcamtuf.  
    Uses  compile-­‐,me  instrumenta,on  to  keep  track  of  code  
    coverage  (and  other  state),  looking  for  corner  cases  
    triggering  memory  errors  or  other  bugs.  
    It  found  security  vulnerabili,es  in  dozens  of  programs.  
    h]p:/
    /lcamtuf.coredump.cx/afl/technical_details.txt

    View full-size slide

  22. Security?  Is  this  a  security  talk?

    View full-size slide

  23. No!  Coverage-­‐guided  fuzzing  is  
    great  at  automa,cally  finding  
    corner  cases.

    View full-size slide

  24. No!  Coverage-­‐guided  fuzzing  is  
    great  at  automa,cally  finding  
    corner  cases.  
    In  C,  they  might  be  memory  
    vulnerabili,es.  Security  
    researchers  love  finding  those.

    View full-size slide

  25. No!  Coverage-­‐guided  fuzzing  is  
    great  at  automa,cally  finding  
    corner  cases.  
    In  Go,  they  might  be  just  bugs.

    But  we  developers  love  finding  
    those.  (Be]er  out  than  in!)

    View full-size slide

  26. go-­‐fuzz

    View full-size slide

  27. go-­‐fuzz  by  Dmitry  Vyukov  uses  
    AST  rewri,ng  to  get  the  coverage  
    informa,on  and  fuzz  Go  func,ons  
    It  found  100s  of  bugs  in  the  stdlib
    h]ps:/
    /youtu.be/a9xrxRsIbSU  

    View full-size slide

  28. Back  to  our  bug!
    package bug
    // ParseStrings parses a list of strings encoded in a binary format
    // where a single-byte value is followed by a string of that length
    // Note: ParseStrings takes unvalidated input from the network
    func ParseStrings(input []byte) (result []string) {
    for len(input) > 0 {
    strLength := input[0]
    input = input[1:]
    result = append(result, string(input[:strLength]))
    input = input[strLength:]
    }
    return
    }

    View full-size slide

  29. //+build gofuzz
    package bug
    func Fuzz(input []byte) int {
    ParseStrings(input)
    return 1
    }
    go-­‐fuzz  integrates  your  program  and  repeatedly  calls  a  
    Fuzz(input []byte)  func,on  with  the  mutated  inputs  
    un,l  it  panic()  or  os.Exit(notZero)

    bug_fuzz.go

    View full-size slide

  30. $ go-fuzz-build github.com/FiloSottile/fuzz-talk
    $ mkdir -p workdir/corpus
    $ echo -ne '\x0cHello World!' > workdir/corpus/example
    $ go-fuzz -bin=bug-fuzz.zip -workdir=workdir
    2015/09/30 16:42:09 slaves: 8, corpus: 1 (3s ago), crashers: 0,
    restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s
    2015/09/30 16:42:12 slaves: 8, corpus: 2 (1s ago), crashers: 0,
    restarts: 1/0, execs: 0 (0/sec), cover: 4, uptime: 6s
    2015/09/30 16:42:15 slaves: 8, corpus: 3 (0s ago), crashers: 1,
    restarts: 1/1, execs: 1572 (175/sec), cover: 4, uptime: 9s
    2015/09/30 16:42:18 slaves: 8, corpus: 6 (0s ago), crashers: 1,
    restarts: 1/1, execs: 3182 (265/sec), cover: 4, uptime: 12s
    [...]

    View full-size slide

  31. $ tree workdir
    workdir
    ├── corpus
    │ ├── 14ce9986bbe073a8c48a9974e87c6f4d0e7b0c50-3
    │ ├── 51b45a439e25342db498f0915cc22f80847034a2-3
    │ ├── 5df1d5920aafe2d3a44680ba9b9fec6760e4a879-4
    │ ├── 6d3d9552845e675ee217db7c341d0c794ec005a9-1
    │ ├── ca4558ca0710d4b259fdca48a462d04aae5cda78-2
    │ └── example
    ├── crashers
    │ ├── b6589fc6ab0dc82cf12099d1c2d40ab994e8410c
    │ ├── b6589fc6ab0dc82cf12099d1c2d40ab994e8410c.output
    │ └── b6589fc6ab0dc82cf12099d1c2d40ab994e8410c.quoted
    └── suppressions
    └── 4de5122d6c0972f7e447061ac749b4e5d7f31a2c

    View full-size slide

  32. $ cat b6589fc6ab0dc82cf12099d1c2d40ab994e8410c.output
    panic: runtime error: slice bounds out of range
    goroutine 1 [running]:
    github.com/FiloSottile/fuzz-talk.ParseStrings(0x8820327001,
    0x0, 0x1fffff, 0x8201301c0, 0x1, 0x1)
    /var/folders/v8/xdj2snz51sg2m2bnpmwl_91c0000gn/T/go-fuzz-
    build091822086/src/github.com/FiloSottile/fuzz-talk/bugged.go:
    11 +0x213
    github.com/FiloSottile/fuzz-talk.Fuzz(0x8820327000, 0x1,
    0x200000, 0x3)
    /var/folders/v8/xdj2snz51sg2m2bnpmwl_91c0000gn/T/go-fuzz-
    build091822086/src/github.com/FiloSottile/fuzz-talk/
    bugged_fuzz.go:6 +0x60
    [...]
    exit status 2

    View full-size slide

  33. $ hexdump -C workdir/crashers/b658…
    00000000 30 |0|
    $ cat workdir/crashers/b6589fc6ab0dc82cf12099d1c2d40ab994e8410c.quoted
    "0"

    View full-size slide

  34. Here’s  a  func,on  bug
    package bug
    // ParseStrings parses a list of strings encoded in a binary format
    // where a single-byte value is followed by a string of that length
    // Note: ParseStrings takes unvalidated input from the network
    func ParseStrings(input []byte) (result []string) {
    for len(input) > 0 {
    strLength := input[0]
    input = input[1:]
    result = append(result, string(input[:strLength]))
    input = input[strLength:]
    }
    return
    }
    strLength  is  past  input’s  end
    0x30

    View full-size slide

  35. Real  world  
    example

    View full-size slide

  36. github.com/miekg/dns
    Excellent  library  used  in  CloudFlare’s  DNS  server.  
    Parses  DNS  packets  coming  from  the  network.  

    View full-size slide

  37. func Fuzz(rawMsg []byte) int {
    msg := &dns.Msg{}
    if unpackErr := msg.Unpack(rawMsg); unpackErr != nil {
    return 0
    }
    if _, packErr = msg.Pack(); packErr != nil {
    println("failed to pack back a message")
    spew.Dump(msg)
    panic(packErr)
    }
    return 1
    }

    View full-size slide

  38. h]ps:/
    /blog.cloudflare.com/dns-­‐parser-­‐meet-­‐go-­‐fuzzer/

    View full-size slide

  39. The  best  people  to  
    write  unit  tests  are

    the  developers

    View full-size slide

  40. The  best  people  to  
    write  fuzz  tests  are

    the  developers

    View full-size slide

  41. Commit  the  Fuzz  func,on!
    Just  like  you  write  unit  tests,  as  you  
    know  what  func,ons  to  test,  you  
    should  write  fuzz  hooks,  as  you  know  
    what  func4ons  to  pass  the  input  to.

    View full-size slide

  42. Commit  the  Fuzz  func,on!
    Other  benefits  of  having  a  Fuzz  func,on  in  your  tree:  
    • others  can  improve  or  reuse  it  
    • you  can  (and  should!)  run  it  in  CI  
    • you  can  keep  the  corpus,  and  use  it  to  test  big  refactors  
    • you  can  compare  different  implementa,ons

    View full-size slide

  43. Commit  the  Fuzz  func,on!
    Fuzzing  shouldn't  be  a  one-­‐off  just  like  normal  tests  aren’t.  
    Fuzz  tests  should  be  first-­‐class  ci,zens  of  your  test  suite.  
    They  should  be  wri]en  by  the  developers,  who  know  best  
    what  to  test  and  where  to  hook.  
    They  should  be  commi]ed  to  the  tree  for  everyone  to  
    expand.  
    They  should  be  run  regularly  to  detect  regressions.

    View full-size slide

  44. Bonus:  not  just  crashes
    Since  it’s  YOU  wri,ng  the  Fuzz  func,on,  you  can  check  
    any  condi,on  you  want,  and  os.Exit(1)  if  it’s  not  sa,sfied.  
    Example:  dns.Msg.PackBuffer  misbehaves  when  the  
    passed  buffer  is  not  zeroed.  
    Write  a  Fuzz  func,on  to  call  PackBuffer  once  on  a  buffer  
    of  0x00,  then  on  a  buffer  of  0xff,  crash  if  they  differ.  
    The  fuzzer  will  find  if  there  are  corner  cases  that  don’t  
    overwrite  the  buffer.

    View full-size slide

  45. func Fuzz(rawMsg []byte) int {
    msg := &dns.Msg{}
    if unpackErr = msg.Unpack(rawMsg); unpackErr != nil { return 0 }
    if res, packErr = msg.PackBuffer(buf); packErr != nil { return 0 }
    for i := range res { bufOne[i] = 0xff }
    if resOne, packErr = msg.PackBuffer(bufOne); packErr != nil {
    println("Pack failed only with a filled buffer")
    panic(packErr)
    }
    if !bytes.Equal(res, resOne) {
    println("buffer bits leaked into the packed message")
    println(hex.Dump(res))
    println(hex.Dump(resOne))
    os.Exit(1)
    }

    View full-size slide

  46. Thanks!
    Filippo  Valsorda  
    @FiloSo>le  
    filippo@cloudflare.com  
    hBps:/
    /filippo.io/fuzz-­‐talk

    View full-size slide