Getting started with fuzzing

Getting started with fuzzing, Go meetup in Prague #7, 31.5.2022

Fuzzing is a type of automated testing that continuously manipulates inputs to a program to find bugs. Since it can reach edge cases that humans often miss, fuzz testing can be particularly valuable for finding security exploits and vulnerabilities.

Transcript

1. G tt ng ta te w th
   f zz ng
   L di la P sk ve , P od ct oa d
   L v P c, P b - @a - m #7, 3 .5.2 1
   

   View Slide

2. W at s uz in ?
   L v P c, P b - @a - m #7, 3 .5.2 2
   

   View Slide

3. 4 g .d /d /f /
   L v P c, P b - @a - m #7, 3 .5.2 3
   

   View Slide

4. R qu re en s (05/2022)
   - go 1.18+
   - AMD64 and ARM64 architectures only
   L v P c, P b - @a - m #7, 3 .5.2 4
   

   View Slide

5. T pe a gu en s
   4 T f a t a l h f n t :
   string, []byte
   int, int8, int16, int32/rune, int64
   uint, uint8/byte, uint16, uint32, uint64
   float32, float64
   bool
   4 h ://g .c /t s/f - a l t
   c s , i c
   L v P c, P b - @a - m #7, 3 .5.2 5
   

   View Slide

6. W y o ou an t u e
   f zz ng?
   L v P c, P b - @a - m #7, 3 .5.2 6
   

   View Slide

7. O S-F zz
   C l , O -F 3 s s C/C++, R , G , P
   a J /J c . O l e s e V
   m r t . O -F s s f x _6 n
   i b .
   4 A J 2 , O -F a f e 3 ,0
   b 5 e s p s.
   3 h ://g .c /g /o -f
   L v P c, P b - @a - m #7, 3 .5.2 7
   

   View Slide

8. G uz
   W s b a t , n G , z (r y
   m ) m o e c e g
   p s. u p 1 p e 2 A .
   t , f 8 p s u c e g
   i f o -s s e.
   — L S
   1 W e F ? D n C e g i M R i
   L v P c, P b - @a - m #7, 3 .5.2 8
   

   View Slide

9. T pe o B gs
   4 C , T s n H
   4 M C i e k o o n l
   4 R C o
   4 E v R e C t
   4 U e e o
   4 U n B r
   T r g a z t a e d r n c : u a t s n c ,
   c m , m o . s s h a t t g s k e 5 % l
   C 2
   2 h ://b .f z.i /w -i -f -t /
   L v P c, P b - @a - m #7, 3 .5.2 9
   

   View Slide

10. W at ho ld F zz?
    I p e, t b i a m h r o :
    4 P y e A
    4 E d s e (p a c n t o r n s )
    4 C r & e i c
    4 E l i c D e n C u n
    4 M , a i n o o a t
    4 I i C S
    4 P & m l o t , v n a p l
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

11. F zz ut ri l
    4 h ://g .d /d /t l/f
    4 F g y
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

12. func Reverse(s string) string {
    b := []byte(s)
    for i, j := 0, len(b)-1; i < len(b)/2; i, j = i+1, j-1 {
    b[i], b[j] = b[j], b[i]
    }
    return string(b)
    }
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

13. go run main.go
    original: "The quick brown fox jumped over the lazy dog"
    reversed: "god yzal eht revo depmuj xof nworb kciuq ehT"
    reversed again: "The quick brown fox jumped over the lazy dog"
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

14. func TestReverse(t *testing.T) {
    testcases := []struct {
    in, want string
    }{
    {"Hello, world", "dlrow ,olleH"},
    {" ", " "},
    {"!12345", "54321!"},
    }
    for _, tc := range testcases {
    rev := Reverse(tc.in)
    if rev != tc.want {
    t.Errorf("Reverse: %q, want %q", rev, tc.want)
    }
    }
    }
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

15. $ go test
    PASS
    ok github.com/abtris/golang-meetup-7-fuzz-demo/fuzz 0.095s
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

16. func FuzzReverse(f *testing.F) {
    testcases := []string{"Hello, world", " ", "!12345"}
    for _, tc := range testcases {
    f.Add(tc) // Use f.Add to provide a seed corpus
    }
    f.Fuzz(func(t *testing.T, orig string) {
    rev := Reverse(orig)
    doubleRev := Reverse(rev)
    if orig != doubleRev {
    t.Errorf("Before: %q, after: %q", orig, doubleRev)
    }
    if utf8.ValidString(orig) && !utf8.ValidString(rev) {
    t.Errorf("Reverse produced invalid UTF-8 string %q", rev)
    }
    })
    }
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

17. $ go test -fuzz=Fuzz
    fuzz: elapsed: 0s, gathering baseline coverage: 0/47 completed
    fuzz: minimizing 43-byte failing input file
    fuzz: elapsed: 0s, gathering baseline coverage: 8/47 completed
    --- FAIL: FuzzReverse (0.03s)
    --- FAIL: FuzzReverse (0.00s)
    reverse_test.go:36: Reverse produced invalid UTF-8 string "\x81\xcc"
    Failing input written to testdata/fuzz/FuzzReverse/287eb9f2ef9b6220c30be65d3f291f4816b296dbd9a859172e648d6cce979cbb
    To re-run:
    go test -run=FuzzReverse/287eb9f2ef9b6220c30be65d3f291f4816b296dbd9a859172e648d6cce979cbb
    FAIL
    exit status 1
    FAIL github.com/abtris/golang-meetup-7-fuzz-demo/fuzz 0.250s
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

18. W er i p ob em?
    reverse_test.go:36: Reverse produced invalid UTF-8 string "\x81\xcc"
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

19. S ri gs, b te , r ne a d ha ac er i
    G
    V r r w s c s e y v
    U -8-e r .
    — utf8.ValidString
    L v P c, P b - @a - m #7, 3 .5.2 1
    

    View Slide

20. L t's eb g
    func FuzzReverse(f *testing.F) {
    testcases := []string{"Hello, world", " ", "!12345"}
    for _, tc := range testcases {
    f.Add(tc) // Use f.Add to provide a seed corpus
    }
    f.Fuzz(func(t *testing.T, orig string) {
    rev := Reverse(orig)
    doubleRev := Reverse(rev)
    t.Logf("Number of runes: orig=%d, rev=%d, doubleRev=%d", utf8.RuneCountInString(orig), utf8.RuneCountInString(rev), utf8.RuneCountInString(doubleRev))
    if orig != doubleRev {
    t.Errorf("Before: %q, after: %q", orig, doubleRev)
    }
    if utf8.ValidString(orig) && !utf8.ValidString(rev) {
    t.Errorf("Reverse produced invalid UTF-8 string %q", rev)
    }
    })
    }
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

21. L t's eb g
    $ go test -fuzz=Fuzz
    fuzz: elapsed: 0s, gathering baseline coverage: 0/49 completed
    failure while testing seed corpus entry: FuzzReverse/287eb9f2ef9b6220c30be65d3f291f4816b296dbd9a859172e648d6cce979cbb
    fuzz: elapsed: 0s, gathering baseline coverage: 3/49 completed
    --- FAIL: FuzzReverse (0.01s)
    --- FAIL: FuzzReverse (0.00s)
    reverse_test.go:32: Number of runes: orig=1, rev=2, doubleRev=1
    reverse_test.go:38: Reverse produced invalid UTF-8 string "\x81\xcc"
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

22. F x he od
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

23. func Reverse(s string) string {
    b := []byte(s)
    for i, j := 0, len(b)-1; i < len(b)/2; i, j = i+1, j-1 {
    b[i], b[j] = b[j], b[i]
    }
    return string(b)
    }
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

24. func Reverse(s string) string {
    b := []rune(s)
    for i, j := 0, len(b)-1; i < len(b)/2; i, j = i+1, j-1 {
    b[i], b[j] = b[j], b[i]
    }
    return string(b)
    }
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

25. $ go test
    PASS
    ok github.com/abtris/golang-meetup-7-fuzz-demo/fuzz 0.098s
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

26. $ go test -fuzz=Fuzz
    fuzz: elapsed: 0s, gathering baseline coverage: 0/48 completed
    fuzz: minimizing 34-byte failing input file
    fuzz: elapsed: 0s, gathering baseline coverage: 6/48 completed
    --- FAIL: FuzzReverse (0.03s)
    --- FAIL: FuzzReverse (0.00s)
    reverse_test.go:33: Before: "\xe9", after: "�"
    Failing input written to testdata/fuzz/FuzzReverse/a3cbbeabbd96e2e2c6720a6a50279ffed09b613fd8449f6ce983d151e266243c
    To re-run:
    go test -run=FuzzReverse/a3cbbeabbd96e2e2c6720a6a50279ffed09b613fd8449f6ce983d151e266243c
    FAIL
    exit status 1
    FAIL github.com/abtris/golang-meetup-7-fuzz-demo/fuzz 0.186s
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

27. F x he od a ai
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

28. func Reverse(s string) (string, error) {
    if !utf8.ValidString(s) {
    return s, errors.New("input is not valid UTF-8")
    }
    r := []rune(s)
    for i, j := 0, len(r)-1; i < len(r)/2; i, j = i+1, j-1 {
    r[i], r[j] = r[j], r[i]
    }
    return string(r), nil
    }
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

29. $ go test -fuzz=Fuzz -fuzztime 30s
    fuzz: elapsed: 0s, gathering baseline coverage: 0/72 completed
    fuzz: elapsed: 0s, gathering baseline coverage: 72/72 completed, now fuzzing with 10 workers
    fuzz: elapsed: 3s, execs: 1605672 (535192/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 6s, execs: 3229185 (541141/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 9s, execs: 4764939 (511904/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 12s, execs: 6384862 (540026/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 15s, execs: 8009255 (541405/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 18s, execs: 9605174 (531767/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 21s, execs: 11224373 (539649/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 24s, execs: 12834356 (536849/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 27s, execs: 14441640 (535511/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 30s, execs: 16023442 (527570/sec), new interesting: 0 (total: 72)
    fuzz: elapsed: 30s, execs: 16023442 (0/sec), new interesting: 0 (total: 72)
    PASS
    ok github.com/abtris/golang-meetup-7-fuzz-demo 30.346s
    L v P c, P b - @a - m #7, 3 .5.2 2
    

    View Slide

30. H w e an un hi i C ?
    L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

31. G th b ct on
    L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

32. name: Go
    on: [push, pull_request]
    jobs:
    build:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - name: Set up Go
    uses: actions/setup-go@v3
    with:
    go-version-file: "go.mod"
    - name: Build
    run: go build -v ./...
    - name: Test
    run: go test -v ./...
    - name: Fuzz test
    run: go test -v ./... -fuzz=Fuzz -fuzztime 30s
    L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

33. G th b ct on
    4 u t
    4 p l $G O
    4 h C
    4 c a t
    L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

34. F zz uz (b ta)
    4 b !!!
    4 d e V
    4 y a u o n o n
    (n l n )
    L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

35. L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

36. L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

37. L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

38. L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

39. F Q
    L v P c, P b - @a - m #7, 3 .5.2 3
    

    View Slide

40. R fe en es
    4 T A e M E S e D e
    A e , P e s o A U n
    C e t (N E !)
    4 F T o D e
    4 C l f
    L v P c, P b - @a - m #7, 3 .5.2 4
    

    View Slide