How Go calculates code coverage David Chou @ Golang Taiwan CC-BY-SA-3.0-TW 

@ Umbo Computer Vision 回家吃⾃⼰🏠 @ Golang Taiwan Co-organizer Software engineer, DevOps, and Gopher david74.chou @ gmail david74.chou @ facebook david74chou @ telegram Blog: https://blog.david74.dev 

Go's coverage-based fuzzing 

“ Fuzzing is the process of sending intentionally invalid data to a product in the hopes of triggering an error. - H.D. Moore What is fuzzing test? 

No content

Fuzzing test Start from a set of initial inputs Continuously manipulate inputs Semi-random input from various mutation Discover new code coverage based on instrumentation 

Go's official fuzzing solution Official proposal [ ] Coverage-based fuzzing Write fuzz function just like test function func FuzzFoo(f *testing.F) Integrate with go command go test -fuzz Plan to land in 1.18 link 

Code coverage in Go go test -cover go test -fuzz 

A simple example func CountAverage(num []byte) int { sum := byte(0) for _, v := range num { sum += v } return int(sum) / len(num) } 1 2 3 4 5 6 7 

func TestCountAverage(t *testing.T) { tests := []struct { name string num []byte want int }{ { num: []byte{1, 2, 3, 4, 5}, want: 3, }, } for _, tt := range tests { t.Run(tt.name, func(t *testing.T) { got := CountAverage(tt.num) assert.EqualValues(t, tt.want, got) }) } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 $ go test -cover PASS coverage: 100.0% of statements Go test coverage 

Coverage visualization $ go test -coverprofile=coverage.out $ go tool cover -html=coverage.out 

Go tool cover $ go tool cover Usage of 'go tool cover': Given a coverage profile produced by 'go test': go test -coverprofile=c.out Open a web browser displaying annotated source code: go tool cover -html=c.out Write out an HTML file instead of launching a web browser: go tool cover -html=c.out -o coverage.html Display coverage percentages to stdout for each function: go tool cover -func=c.out Finally, to generate modified source code with coverage annotations (what go test -cover does): go tool cover -mode=set -var=CoverageVariableName program.go 

package go_fuzzing_playground func CountAverage(num []byte) int {GoCover.Count[0] = 1; sum := byte(0) for _, v := range num {GoCover.Count[2] = 1; sum += v } GoCover.Count[1] = 1;return int(sum) / len(num) } var GoCover = struct { Count [3]uint32 Pos [3 * 3]uint32 NumStmt [3]uint16 } { Pos: [3 * 3]uint32{ 3, 5, 0x180023, // [0] 8, 8, 0x1c0002, // [1] 5, 7, 0x30018, // [2] }, NumStmt: [3]uint16{ 2, // 0 1, // 1 1, // 2 }, } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 $ go tool cover -mode=set ./count_average.go {StartLine, EndLine, ColumnInfo} [ ] link Number of statements 

package go_fuzzing_playground func CountAverage(num []byte) int {GoCover.Count[0] = 1; sum := byte(0) for _, v := range num {GoCover.Count[2] = 1; sum += v } GoCover.Count[1] = 1;return int(sum) / len(num) } var GoCover = struct { Count [3]uint32 Pos [3 * 3]uint32 NumStmt [3]uint16 } { Pos: [3 * 3]uint32{ 3, 5, 0x180023, // [0] 8, 8, 0x1c0002, // [1] 5, 7, 0x30018, // [2] }, NumStmt: [3]uint16{ 2, // 0 1, // 1 1, // 2 }, } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 block0 block1 block2 

$ go test ./ -coverprofile=coverage.out && cat coverage.out ok 0.003s coverage: 100.0% of statements mode: set count_average.go:3.35,5.24 2 1 count_average.go:8.2, 8.28 1 1 count_average.go:5.24,7.3 1 1 1 2 3 4 5 6 $ go test ./ -coverprofile=coverage.out && cat coverage.out Coverage format: name.go: Line.Column, Line.Column NumStmt Count [ ] 100% = (2x1 + 1x1 + 1x1) / (2 + 1 + 1) link 

Go test coverage Source to source transform Add instrument code before compiling Basic block coverage 

Basic block v.s. Branch coverage block0 block1 block2 block0 block1 block2 Basic block: 100% Branch coverage: 67% 

package go_fuzzing_playground func CountAverage(num []byte) int {GoCover.Count[0] = 1; sum := byte(0) for _, v := range num {GoCover.Count[2] = 1; sum += v } GoCover.Count[1] = 1;return int(sum) / len(num) } 1 2 3 4 5 6 7 8 9 block0 block1 block2 

How "go test -fuzz" works 

Go fuzz coverage 

Go fuzz coverage Compiler instrumentation Add instrument code during compiling [ ] and [ ] by mdempsky 1 2 

Compiler instrumentation // edge inserts coverage instrumentation for libfuzzer. func (o *orderState) edge() { // Create a new uint8 counter to be allocated in section // __libfuzzer_extra_counters. counter := staticinit.StaticName(types.Types[types.TUINT8]) counter.SetLibfuzzerExtraCounter(true) // counter += 1 incr := ir.NewAssignOpStmt(base.Pos, ir.OADD, counter, ir.NewInt(1)) o.append(incr) } 1 2 3 4 5 6 7 8 9 10 11 edge() inserts coverage instrumentation 

func (o *orderState) stmt(n ir.Node) { switch n.Op() { ... case ir.OFOR: edge() case ir.OIF: edge() case ir.ORANGE: edge() case ir.OSELECT: edge() case ir.OSWITCH: edge() case OANDAND, OOROR: edge() ... } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 compiler adds edge() into each edge 

// _counters and _ecounters mark the start and end, respectively, of where // the 8-bit coverage counters reside in memory. They're known to cmd/link, // which specially assigns their addresses for this purpose. var _counters, _ecounters [0]byte func coverage() []byte { addr := unsafe.Pointer(&_counters) size := uintptr(unsafe.Pointer(&_ecounters)) - uintptr(addr) var res []byte *(*unsafeheader.Slice)(unsafe.Pointer(&res)) = unsafeheader.Slice{ Data: addr, Len: int(size), Cap: int(size), } return res } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 coverage() returns the coverage counters 

Go fuzz coverage Still using basic block, but this could be improved in the future Compared to source-to-source, it's much easier to implement branch coverage with compiler instrumentation go-fuzz uses s2s and has lots of corner cases: miscompile, crash, invalid codes. E.g., [ ], [ ] 1 2 

$ objdump go-fuzzing.test -h go-fuzzing.test: file format elf64-x86-64 Sections: Idx Name Size VMA LMA File off Algn 0 .text 0027890e 0000000000401000 0000000000401000 00001000 2**5 CONTENTS, ALLOC, LOAD, READONLY, CODE ... 19 .data 0000a550 0000000000883da0 0000000000883da0 00483da0 2**5 CONTENTS, ALLOC, LOAD, DATA 20 .bss 00031708 000000000088e300 000000000088e300 0048e300 2**5 ALLOC 21 .noptrbss 00006fc0 00000000008bfa20 00000000008bfa20 004bfa20 2**5 ALLOC 22 __libfuzzer_extra_counters 000052a4 00000000008c69e0 00000000008c69e0 004c69e0 2**0 ALLOC 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Bonus: libfuzzer_extra_counters 

libfuzzer is a well-known LLVM fuzzy engine It could take coverage number from libfuzzer_extra_counters variable It means Go could use external fuzzy engine [ ] link $ go build -gcflags=all=-d=fuzzing -buildmode=c-archive -o pngfuzz.a . $ clang -o png.fuzzer pngfuzz.a -fsanitize=fuzzer 1 2 Bonus: libfuzzer_extra_counters 

Compiler instrumentation Source to source transform Go test coverage Add instrument code before compiling Basic block coverage Difficult to adopt branch coverage Go fuzz coverage Add instrument code during compiling Basic block coverage Easier to adopt branch coverage Could expose code coverage during execution 

Amazon CTO Dr. Werner Vogels