Fuzz Testing and go-fuzz

Transcript

1. Fuzz Testing and go- fuzz
2. Testing • Unit Test • Integration Test
3. Hard-to-test • Combination • Uncontrolled Input • hard to define "Corner cases"
4. Randomized Test?
5. Parsing email address Any string that doesn't contains @ will be ignored. func parseAddress(address string) { if (!address.contains("@")) return .... }
6. Fuzzing Feeding programs with automatically generated inputs to trigger unexpected behaviour.
7. Coverage-guided Fuzzing assume we have a huge function func parseAddress(address string) { // ----------------- switch .... { case : // ----------------- // ----------------- if (...) { // ----------------- // ----------------- } case : if (...) { // ----------------- // ----------------- } case : // ----------------- // ----------------- } }
8. Coverage-guided Fuzzing First input func parseAddress(address string) { // ***************** switch .... { case : // ----------------- // ----------------- if (...) { // ----------------- // ----------------- } case : if (...) { // ----------------- // ----------------- } case : // ***************** // ***************** } }
9. Coverage-guided Fuzzing Any input that changed the coverage is an effective input func parseAddress(address string) { // ***************** switch .... { case : // ***************** // ***************** if (...) { // ----------------- // ----------------- } case : if (...) { // ----------------- // ----------------- } case : // ----------------- // ----------------- } }
10. American Fuzz Lop
11. American Fuzz Lop American Fuzzy Lop is a brute-force fuzzer coupled with an exceedingly simple but rock-solid instrumentation-guided genetic algorithm. It uses a modified form of edge coverage to effortlessly pick up subtle, local-scale changes to program control flow.
12. go-fuzz
13. Trophy ... * 50 pages
14. Setup project for go-fuzz Use AST-rewrite to get coverage information $ go get github.com/dvyukov/go-fuzz/go-fuzz $ go get github.com/dvyukov/go-fuzz/go-fuzz-build
15. Write the fuzz function // +build gofuzz // application-level fuzzing func Fuzz(data []byte) int { img, err := png.Decode(bytes.NewReader(data)) if err != nil { if img != nil { panic("img != nil on error") } return 0 } var w bytes.Buffer err = png.Encode(&w, img) if err != nil { panic(err) } return 1 }
16. Build fuzzer // put initial corpus to go-fuzz/examples/png/corpus $ go-fuzz-build github.com/dvyukov/go-fuzz/examples/png // generate png-fuzz.zip
17. Run the test $ go-fuzz -bin=./png-fuzz.zip -workdir=examples/png $ tree examples/png examples/png/ !"" corpus # !"" 00184ecf083019781fa3cd954f07ae5f6f8996c5-4 # !"" 00694592b23b147b3ed48fdd58ad93190495c0e1-6 # !"" e1ffccce440e7d27f9f8f4f21b57e1092d5701bc-13 # !"" f1c9f52119ce4f4086ce39c50c84c88373284bb9-9 # !"" f4b5fde0975f447920100b63ca8faa811cd084e5-10 # !"" f4fcdc199b808050a943d900e04e5507d8ccc0f1-7 # $"" f84b0521ed4ee32fcc6f87f1af486efab81986cb-13 # $"" ... !"" crashers $"" suppressions
18. Examine the output [~/projects/fuzz-test] $ go-fuzz -bin=./png-fuzz.zip -workdir=examples/png 2017/04/17 00:10:44 slaves: 4, corpus: 19 (0s ago), crashers: 0, restarts: 1/0, execs: 0 (0/sec), cover: 0, uptime: 3s 2017/04/17 00:10:47 slaves: 4, corpus: 20 (2s ago), crashers: 0, restarts: 1/3370, execs: 10110 (1681/sec), cover: 173, uptime: 6s 2017/04/17 00:10:50 slaves: 4, corpus: 20 (5s ago), crashers: 0, restarts: 1/4501, execs: 54021 (5964/sec), cover: 173, uptime: 9s ... • slave: concurrent test count • corpus: generated corpus • crashers: corpus which crash the program • restarts: restart rate (due to crashes) • execs: total number of execution • cover: coverage bits
19. • When should I run fuzz test?: CI • Fuzz test is only for security issue?: NO • How do I know which corpus crashed my program?: quoted input • Who should write Fuzz test?: You!
20. Thank you! @devpoga poga.po@gmail.com