Fuzzy generics

Transcript

1. Fuzzy generics Several months of using 1.18 features Alexey Palazhchenko FOSDEM 2022-02-05
3. Why Go 1.18?
4. Why Go 1.18? Generics
5. Why Go 1.18? Generics Fuzzing
6. Why Go 1.18? Generics Fuzzing
7. Generics
8. Generics • Generics do not work there; let's go back to interfaces
9. Generics • Generics do not work there; let's go back to interfaces • Let me extract a small example for a talk
10. Generics • Generics do not work there; let's go back to interfaces • Let me extract a small example for a talk • Hm, that's not that bad
11. BSON Document
12. BSON Document • Ordered map
13. BSON Document • Ordered map • Keys – string, values – any BSON value, including other documents
14. BSON Document • Ordered map • Keys – string, values – any BSON value, including other documents • The central data structure in FerretDB
15. Interfaces type BSONType interface { bsontype() // sealed } type Int int func (Int) bsontype() {} type String string func (String) bsontype() {}
16. Interfaces type BSONType interface { bsontype() // sealed } type Int int func (Int) bsontype() {} type String string func (String) bsontype() {} type Document struct { m map[string]BSONType keys []string } func (*Document) bsontype() {}
17. Interfaces func (d *Document) Set(key String, value BSONType) { if _, ok := d.m[key]; !ok { d.keys = append(d.keys, key) } d.m[key] = value }
18. Interfaces d.Set("foo", Int(42))
19. Interfaces d.Set("foo", Int(42)) d.Set("qux", nil)
20. Generics type BSONType interface { int | string | *Document }
21. Generics type BSONType interface { int | string | *Document } type Document struct { m map[string]any keys []string }
22. Generics func (d *Document) Set[T BSONType](key string, value T) { if _, ok := d.m[key]; !ok { d.keys = append(d.keys, key) } d.m[key] = value }
23. Generics func (d *Document) Set[T BSONType](key string, value T) { if _, ok := d.m[key]; !ok { d.keys = append(d.keys, key) } d.m[key] = value } syntax error: method must have no type parameters
24. Generics func DocumentSet[T BSONType](d *Document, key string, value T) { if _, ok := d.m[key]; !ok { d.keys = append(d.keys, key) } d.m[key] = value }
25. Generics DocumentSet(d, "foo", 42)
26. Generics DocumentSet(d, "foo", 42) // DocumentSet(d, "qux", nil)
27. Interfaces func MakeDocument(pairs ...BSONType) (*Document, error) { l := len(pairs) if l%2 != 0 { return nil, fmt.Errorf("%d arguments", l)
28. Interfaces
29. Interfaces MakeDocument( String("foo"), Int(42), String("bar"), String("baz"), )
30. Interfaces MakeDocument( String("foo"), Int(42), String("bar"), String("baz"), ) MakeDocument(String("invalid number of arguments"))
31. Interfaces MakeDocument( String("foo"), Int(42), String("bar"), String("baz"), ) MakeDocument(String("invalid number of arguments")) MakeDocument(Int(42), String("invalid key type"))
32. Generics
33. Generics func MakeDocument[T BSONType](key string, value T) *Document
34. Generics func MakeDocument[T BSONType](key string, value T) *Document func MakeDocument2[T1, T2 BSONType](key1 string, value1 T1, key2 string, value2 T2) *Document
35. Generics func MakeDocument[T BSONType](key string, value T) *Document func MakeDocument2[T1, T2 BSONType](key1 string, value1 T1, key2 string, value2 T2) *Document func MakeDocument3[T1, T2, T3 BSONType](key1 string, value1 T1, key2 string, value2 T2, key3 string, value3 T3) *Document
36. Interfaces
37. Interfaces func (d *Document) Get(key String) BSONType { return d.m[key] }
38. Interfaces func (d *Document) Get(key String) BSONType { return d.m[key] } assert.Equal(t, Int(42), d.Get("foo")) assert.Equal(t, nil, d.Get("baz"))
39. Generics
40. Generics func DocumentGet[T BSONType](d *Document, key string) T { v, _ := d.m[key].(T) return v }
41. Generics func DocumentGet[T BSONType](d *Document, key string) T { v, _ := d.m[key].(T) return v } assert.Equal(t, 42, DocumentGet[int](d, "foo")) assert.Equal(t, "", DocumentGet[string](d, "foo"))
42. Generics vs Interfaces
43. Generics vs Interfaces • Better in some aspects
44. Generics vs Interfaces • Better in some aspects • Worse in others
45. Generics vs Interfaces • Better in some aspects • Worse in others
46. Fuzzing
47. Fuzzing • testing/quick on steroids
48. Fuzzing • testing/quick on steroids • MongoDB binary protocol parsing packages
49. Fuzzing • testing/quick on steroids • MongoDB binary protocol parsing packages • FerretDB had fuzzing tests even before unit tests
50. Table-driven tests type testCase struct { name string b []byte v bsontype err error }
51. Table-driven tests type testCase struct { name string b []byte v bsontype err error } • Unmarshal b to (v2, err2) • compare with v or err • Marshal v2 to b2 • compare with b
52. Table-driven tests func TestDocument(t *testing.T) { for _, tc := range testCases { tc := tc t.Run(tc.name, func(t *testing.T) {
53. Fuzzing func FuzzDocument(f *testing.F) { for _, tc := range testCases { f.Add(tc.b) } f.Fuzz(func(t *testing.T, b []byte) {
54. When b2 != b
55. When b2 != b • b might be larger than needed
56. When b2 != b • b might be larger than needed • The unread portion of b should be removed
57. When b2 != b • b might be larger than needed • The unread portion of b should be removed • Some bytes might be insigni fi cant
58. When b2 != b • b might be larger than needed • The unread portion of b should be removed • Some bytes might be insigni fi cant • Use canonization (json.Compact, etc)
59. Fuzzing issues
60. Fuzzing issues • No subtests (testing.F.Run)
61. Fuzzing issues • No subtests (testing.F.Run) • Unnamed seed values
62. Fuzzing issues • No subtests (testing.F.Run) • Unnamed seed values • Hanging detection is unreliable with unset GOMAXPROCS
63. Fuzzing
64. Fuzzing • Found a few bugs in `go test -fuzz`
65. Fuzzing • Found a few bugs in `go test -fuzz` • Found many many bugs in FerretDB
66. Fuzzing • Found a few bugs in `go test -fuzz` • Found many many bugs in FerretDB • De fi nitely use it for parsing
67. Links • https://github.com/AlekSi/ generics-vs-interfaces • https://github.com/akutz/
 go-generics-the-hard-way • https://go.dev/doc/tutorial/ • https://go.dev/doc/fuzz/ • https://www.ferretdb.io • https://github.com/FerretDB ⭐ • https://github.com/AlekSi • @paaleksey