Continuous Go Fuzzing - Yevgeny Pats - Fuzzit

Fuzzing or fuzz testing is an automated software testing technique that involves providing semi-random data as input to test programs. Fuzzing helps with security, stability and performance.
Integrating fuzz testing into the development workflow and CI is a great addition to code quality but also involves new challenges due the special nature of fuzzing.
We will share our experience of running continuous fuzzing for both open-source and close-source Golang aprojects. We will discuss challenges, possible solutions and other best-practices in continuous fuzzing.

Transcript

1. (Continuous) Automated Go Fuzz Testing Yevgeny Pats @fuzzitdev January 2020, GoDays Berlin
2. A bit about me • @yevgenypats • Founder & CEO at @fuzzitdev • Security Researcher • Serial entrepreneur • Cyber Security @ IDF
3. Agenda • What is fuzzing? What to Fuzz? • Types of Fuzzers • go-fuzz • Continuous Fuzzing • Trophies (Case studies) • What Fuzzing is not? • The Future • Q&A
4. What is Fuzzing / Fuzz Testing • Fuzzing - providing semi-random input in automated way to a program in order to uncover bugs and crashes. • Is it only to find security/memory corruption vulnerabilities? Because go is a safe language…. • In safe languages fuzzing helps find a LOT of bugs and improve stability and code coverage. Logic bugs and DoS with security impact as well. • Run millions of unit-tests without writing them.
5. What to Fuzz? • Mostly intended for programs that parse complex input: ◦ compression/decompression libraries ◦ network parsers - DNS/HTTPS/etc… ◦ deserialization ◦ Crypto ◦ Media formats (JPEG/MP3/...) ◦ Database queries • Must have for anything that consumes and parse input from the outside world - i.e untrusted input.
6. Quick history & Types of fuzzers • Traditional/Random • Coverage-Guided ◦ AFL ◦ libFuzzer ◦ go-fuzz (Initially developed by Dmitry Vyukov) ◦ pythonfuzz, jsfuzz, javafuzz (ported by me:))
7. ParseComplex Example func ParseComplex(data [] byte) bool { if len(data) == 5 { if data[0] == 'F' && data[1] == 'U' && data[ 2] == 'Z' && data[ 3] == 'Z' && data[ 4] == 'I' && data[ 5] == 'T' { return true } } return false }
8. Random vs Coverage guided fuzzing Algorithm // pseudo code for { Generate random input Execute input } // pseudo code Instrument program for code coverage for { Choose random input from corpus Mutate input Execute input and collect coverage If new coverage/paths are hit add it to corpus }
9. Random Vs Coverage Guided Fuzzing import ( "github.com/google/gofuzz" "testing" ) func TestParseComplex(t *testing.T) { f := fuzz.New() var inputString string for { f.Fuzz(&inputString) ParseComplex([]byte(inputString)) } } // go test package parser func Fuzz(data []byte) int { ParseComplex(data) return 0 } // go get -u github.com/dvyukov/go-fuzz/go-fuzz // github.com/dvyukov/go-fuzz/go-fuzz-build // go-fuzz-build // go-fuzz Time till Crash: ~Never, stopped after 1 hour. Time till Crash: ~2 sec
10. Demo
11. Data generation “Sdlkfgnjk12 iv7$” “Laksjdh2345 24י4ךל3יכגחלדשך ” “as(*&^&^%*&^%” (The testcases that are saved in the corpus) “” “FAAAA” “FUAAA” “FUZAA” “FUZZA” “FUZZI” - Crash ??
12. Property based fuzz testing // +build gofuzz package parser func Fuzz(data []byte) int { if MyDecode(MyEncode(data)) != data { log.Fatalf(“MyDecode(MyEncode(%v)) != %v”, data, data) } return 0 }
13. Trophies
14. Mutations • Bit-flipping, Byte-flipping, arithmetics, • Sonar
15. Solutions and What Fuzzing is not • Doesn’t replace unit-tests, integration tests. • Secure design, threat modeling & attack surface reduction ◦ Sandbox ◦ Thread modeling ◦ Up-to-date third-party-libraries • As the developer you are responsible for writing the fuzz tests just as you write the unit-tests for your code. You are the best person to understand which parts of the code need to be fuzzed.
16. Continuous Fuzzing • Running a fuzzer once is nice and it will probably find bugs. • Just like unit-tests, you want to run the fuzzers every time you push new code. • Unlike unit-tests which are quick (usually), fuzzing can run indefinitely. • How long should we fuzz? What version should we fuzz?
17. Continuous Fuzzing Workflow // Pseudo code // Fuzzing workflow for { Push new code to master/dev Build the fuzzers in the CI and upload to a server where you will run them. The fuzzer will run either until it finds a crash or until a new version of the fuzzer is uploaded Corpus is saved between runs } // Regression workflow for { Open a Pull-Request Download the corpus Run the fuzzers through all the files available in the corpus (quick) - Free unit-tests! }
18. Continuous Trophies
19. Future • Structure Aware Fuzzing - https://github.com/dvyukov/go-fuzz/issues/65 • Fuzzing as part of go standard library
20. Q & A Follow me on @fuzzitdev @yevgenypats Thx!