Tweet
Tweet

Slide 1

Slide 1 text

Inception Go programs that generate Go code

Slide 2

Slide 2 text

I really like Go…

Slide 3

Slide 3 text

it is very simple

Slide 4

Slide 4 text

and very powerful

Slide 5

Slide 5 text

it has great tooling

Slide 6

Slide 6 text

and the stdlib is amazing

Slide 7

Slide 7 text

but…

Slide 8

Slide 8 text

Who likes boilerplate?

Slide 9

Slide 9 text

this talk

Slide 10

Slide 10 text

My experience

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

reducing boilerplate

Slide 14

Slide 14 text

Go programs that: 1. Parse data to generate Go code 2. Modify Go code 3. Parse Go code to generate Go code

Slide 15

Slide 15 text

1. Parsing data to generate Go code

Slide 16

Slide 16 text

structured data => repetitive code

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

No content

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

No content

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

$ schematic platform-api.json > heroku/heroku.go

Slide 24

Slide 24 text

import "text/template"

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

2. Modifying Go code

Slide 28

Slide 28 text

a big part of Go's brilliant tooling

Slide 29

Slide 29 text

$ go fmt

Slide 30

Slide 30 text

package main import "fmt" func main( ) { fmt. Println ( "Hello world")}

Slide 31

Slide 31 text

package main import "fmt" func main() { fmt.Println("Hello world") }

Slide 32

Slide 32 text

golang.org/x/tools/cmd/goimports

Slide 33

Slide 33 text

package main func main() { fmt.Println("Hello world") }

Slide 34

Slide 34 text

package main import "fmt" func main() { fmt.Println("Hello world") }

Slide 35

Slide 35 text

golang.org/x/tools/cmd/gorename github.com/sqs/goreturns ...

Slide 36

Slide 36 text

Read code > Modify code > Replace code

Slide 37

Slide 37 text

powered by the stdlib

Slide 38

Slide 38 text

Read code > Modify code > Replace code

Slide 39

Slide 39 text

import "go/build" // gathers information about Go packages

Slide 40

Slide 40 text

import "go/scanner" import "go/parser" // read and parse Go source code

Slide 41

Slide 41 text

Read code > Modify code > Replace code

Slide 42

Slide 42 text

import "go/ast" // declares types used to represent the syntax tree

Slide 43

Slide 43 text

import "go/format" // "go fmt" go code, including AST

Slide 44

Slide 44 text

Read code > Modify code > Replace code

Slide 45

Slide 45 text

import "go/printer" // generating code from AST nodes

Slide 46

Slide 46 text

Read code > Generate AST > Modify AST > Replace code

Slide 47

Slide 47 text

3. Parse Go code to generate Go code

Slide 48

Slide 48 text

your code => boilerplate code

Slide 49

Slide 49 text

No content

Slide 50

Slide 50 text

Read code > Generate AST > Analyze AST > Generate Code

Slide 51

Slide 51 text

generating mocks

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

package main import ( "testing" "github.com/stretchr/testify/mock" ) type downcaser interface { Downcase(string) (string, error) } func TestMock(t *testing.T) { m := &mockDowncaser{} m.On("Downcase", "FOO").Return("foo", nil) m.Downcase("FOO") m.AssertNumberOfCalls(t, "Downcase", 1) }

Slide 54

Slide 54 text

type mockDowncaser struct { mock.Mock } func (m *mockDowncaser) Downcase(a0 string) (string, error) { ret := m.Called(a0) return ret.Get(0).(string), ret.Error(1) }

Slide 55

Slide 55 text

type downcaser interface { Downcase(string) (string, error) } func (m *mockDowncaser) Downcase(a0 string) (string, error) { ret := m.Called(a0) return ret.Get(0).(string), ret.Error(1) }

Slide 56

Slide 56 text

No content

Slide 57

Slide 57 text

$ mockery -inpkg -testonly -name=downcaser Generating mock for: downcaser

Slide 58

Slide 58 text

mock_downcaser_test.go package main import "github.com/stretchr/testify/mock" type mockDowncaser struct { mock.Mock } func (_m *mockDowncaser) Downcase(_a0 string) (string, error) { ret := _m.Called(_a0) var r0 string if rf, ok := ret.Get(0).(func(string) string); ok { r0 = rf(_a0) } else { r0 = ret.Get(0).(string) } var r1 error if rf, ok := ret.Get(1).(func(string) error); ok { r1 = rf(_a0) } else { r1 = ret.Error(1) } return r0, r1 }

Slide 59

Slide 59 text

how do you document the steps to generate that code?

Slide 60

Slide 60 text

$ go generate

Slide 61

Slide 61 text

package main import ( "testing" ) type downcaser interface { Downcase(string) (string, error) } //go:generate mockery -inpkg -testonly -name=downcaser func TestMock(t *testing.T) { m := &mockDowncaser{} m.On("Downcase", "FOO").Return("foo", nil) m.Downcase("FOO") m.AssertNumberOfCalls(t, "Downcase", 1) }

Slide 62

Slide 62 text

$ go test # github.com/ernesto-jimenez/test ./main_test.go:14: undefined: mockDowncaser FAIL github.com/ernesto-jimenez/test [build failed] $ go generate Generating mock for: downcaser $ go test PASS ok github.com/ernesto-jimenez/test 0.011s

Slide 63

Slide 63 text

Read code > Generate AST > Analyze AST > Generate Code

Slide 64

Slide 64 text

No content

Slide 65

Slide 65 text

Yeah, this is a known issue. I'm not sure how deal with it because it requires being able to parse entire packages to find the other interfaces.

Slide 66

Slide 66 text

go/ast provides very raw data

Slide 67

Slide 67 text

go/ast go/types

Slide 68

Slide 68 text

import "go/types" // types and algorithms for type-checking of Go packages

Slide 69

Slide 69 text

Read code > Generate AST > Analyze AST > Generate Code

Slide 70

Slide 70 text

generating any mock

Slide 71

Slide 71 text

github.com/ernesto-jimenez/gogen/cmd/goautomock

Slide 72

Slide 72 text

less code and less issues

Slide 73

Slide 73 text

$ goautomock -o=- io.Reader /* * CODE GENERATED AUTOMATICALLY WITH github.com/ernesto-jimenez/gogen/automock * THIS FILE SHOULD NOT BE EDITED BY HAND */ package app import ( "fmt" mock "github.com/stretchr/testify/mock" ) // ReaderMock mock type ReaderMock struct { mock.Mock } // Read mocked method func (m *ReaderMock) Read(p0 []byte) (int, error) { // […]

Slide 74

Slide 74 text

No content

Slide 75

Slide 75 text

tens of assertion helpers

Slide 76

Slide 76 text

two functions per assertion func Panics(t TestingT, fn func(), msgAndArgs ...interface{}) bool { if funcDidPanic, _ := didPanic(f); !funcDidPanic { return Fail(t, fmt.Sprintf("func %#v should panic", f), msgAndArgs...) } return true } func (a *Assertions) Panics(fn func(), msgAndArgs ...interface{}) bool { return Panics(a.t, expected, actual, msgAndArgs…) }

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

two extra functions per assertion func Panics(t TestingT, fn func(), msgAndArgs ...interface{}) bool { if !assert.Panics(t, fn, msgAndArgs...) { t.FailNow() } } func (a *Assertions) Panics(fn func(), msgAndArgs ...interface{}) bool { Panics(a.t, fn, msgAndArgs...) }

Slide 79

Slide 79 text

3 boilerplate functions per each assertion

Slide 80

Slide 80 text

No content

Slide 81

Slide 81 text

//go:generate go run ../_codegen/main.go -output-package=assert -template=assertion_forward.go.tmpl //go:generate go run ../_codegen/main.go -output-package=require -template=require_forward.go.tmpl //go:generate go run ../_codegen/main.go -output-package=require -template=require.go.tmpl

Slide 82

Slide 82 text

{{.Comment}} func (a *Assertions) {{.DocInfo.Name}}({{.Params}}) bool { return {{.DocInfo.Name}}(a.t, {{.ForwardedParams}}) }

Slide 83

Slide 83 text

go/types go/doc

Slide 84

Slide 84 text

codegen: 239 lines of code removed: 839 lines of code

Slide 85

Slide 85 text

net/http func Get(url string) (resp *Response, err error) { return DefaultClient.Get(url) }

Slide 86

Slide 86 text

var DefaultClient = New() func New() *Client { return &Client{} } func (c *Client) Do(interface{}) error { // […] } // Do calls `DefaultClient.Do` func Do(v interface{}) error { return DefaultClient.Do(v) }

Slide 87

Slide 87 text

No content

Slide 88

Slide 88 text

var DefaultClient = New() //go:generate goexportdefault DefaultClient func New() *Client { return &Client{} } func (c *Client) Do(interface{}) error { // […] }

Slide 89

Slide 89 text

$ go generate Generating funcs for DefaultClient

Slide 90

Slide 90 text

/* * CODE GENERATED AUTOMATICALLY WITH goexportdefault * THIS FILE MUST NOT BE EDITED BY HAND * * Install goexportdefault with: * go get github.com/ernesto-jimenez/gogen/cmd/goexportdefault */ package example import () // Do is a wrapper around DefaultClient.Do func Do(p0 interface{}) error { return DefaultClient.Do(p0) }

Slide 91

Slide 91 text

avoid goimports on generated code

Slide 92

Slide 92 text

Generating decoders

Slide 93

Slide 93 text

WebSocket > map[string]interface{} > Check m[“type”] > Create struct

Slide 94

Slide 94 text

No content

Slide 95

Slide 95 text

import “reflect”

Slide 96

Slide 96 text

BenchmarkMapstructure 100.000 17.493 ns/op 2.825 B/op 61 allocs/op

Slide 97

Slide 97 text

if v, ok := m["name"].(string); ok { s.Name = v } else if v, exists := m["name"]; exists && v != nil { return fmt.Errorf("expected field name to be string but got %T", m["name"]) }

Slide 98

Slide 98 text

No content

Slide 99

Slide 99 text

//go:generate gounmarshalmap -o user_unmarshalmap.go User

Slide 100

Slide 100 text

$ cat user_unmarshalmap.go| wc -l 218

Slide 101

Slide 101 text

user.UnmarshalMap(m)

Slide 102

Slide 102 text

BenchmarkMapstructure 100.000 17.493 ns/op 2.825 B/op 61 allocs/op BenchmarkUnmarshalmap 5.000.000 320 ns/op 0 B/op 0 allocs/op

Slide 103

Slide 103 text

What if we add a new field to the struct?

Slide 104

Slide 104 text

Read code > Analyze code > Generate code and tests

Slide 105

Slide 105 text

$ go test ./unmarshalmap/testpkg --- FAIL: TestNestedUnmarshalMap (0.00s) test.go:35: UnmarshalMap() method from *testpkg.Nested out of date. regenerate the code FAIL FAIL github.com/ernesto-jimenez/gogen/unmarshalmap/testpkg 0.010s $ go generate ./unmarshalmap/testpkg Generating func (*Nested) UnmarshalMap(map[string]interface{}) error $ go test ./unmarshalmap/testpkg ok github.com/ernesto-jimenez/gogen/unmarshalmap/testpkg 0.009s

Slide 106

Slide 106 text

one more thing…

Slide 107

Slide 107 text

No content

Slide 108

Slide 108 text

4. Go DSLs (Domain-Specific Languages)

Slide 109

Slide 109 text

http://goa.design

Slide 110

Slide 110 text

$ tree goa-adder . └── design └── design.go 1 directory, 1 file

Slide 111

Slide 111 text

package design import ( . "github.com/goadesign/goa/design" . "github.com/goadesign/goa/design/apidsl" ) var _ = API("adder", func() { Title("The adder API") Description("A teaser for goa") Host("localhost:8080") Scheme("http") }) var _ = Resource("operands", func() { Action("add", func() { Routing(GET("add/:left/:right")) Description("add returns the sum of the left and right parameters in the response body") Params(func() { Param("left", Integer, "Left operand") Param("right", Integer, "Right operand") }) Response(OK, "plain/text") }) })

Slide 112

Slide 112 text

$ goagen bootstrap -d goa-adder/design

Slide 113

Slide 113 text

$ tree goa-adder . ├── app │ ├── contexts.go │ ├── controllers.go │ ├── hrefs.go │ ├── media_types.go │ ├── test │ │ └── operands_testing.go │ └── user_types.go ├── client │ ├── client.go │ ├── media_types.go │ ├── operands.go │ └── user_types.go ├── design │ └── design.go ├── main.go ├── operands.go ├── swagger │ ├── swagger.json │ └── swagger.yaml └── tool ├── adder-cli │ └── main.go └── cli └── commands.go 8 directories, 17 files

Slide 114

Slide 114 text

func main() { // Create service service := goa.New("adder") // Mount middleware service.Use(middleware.RequestID()) service.Use(middleware.LogRequest(true)) service.Use(middleware.ErrorHandler(service, true)) service.Use(middleware.Recover()) // Mount "operands" controller c := NewOperandsController(service) app.MountOperandsController(service, c) // Start service if err := service.ListenAndServe(":8080"); err != nil { service.LogError("startup", "err", err) } }

Slide 115

Slide 115 text

// OperandsController implements the operands resource. type OperandsController struct { *goa.Controller } // Add runs the add action. func (c *OperandsController) Add(ctx *app.AddOperandsContext) error { // OperandsController_Add: start_implement // Put your logic here // OperandsController_Add: end_implement return nil }

Slide 116

Slide 116 text

// Add runs the add action. func (c *OperandsController) Add(ctx *app.AddOperandsContext) error { sum := ctx.Left + ctx.Right return ctx.OK([]byte(strconv.Itoa(sum))) }

Slide 117

Slide 117 text

$ goa-adder [INFO] mount ctrl=Operands action=Add route=GET /add/:left/:right [INFO] listen transport=http addr=:8080 [INFO] started req_id=iaRusv3Joj-1 GET=/add/10/23 from=::1 ctrl=OperandsController action=Add [INFO] params req_id=iaRusv3Joj-1 right=23 left=10 [INFO] completed req_id=iaRusv3Joj-1 status=200 bytes=2 time=316.398µs ------------------------------------------------ $ adder-cli add operands --left 10 --right 23 [INFO] started id=ACowX1Kj GET=http://localhost:8080/add/10/23 [INFO] completed id=ACowX1Kj status=200 time=5.981623ms 33

Slide 118

Slide 118 text

How does goa do it?

Slide 119

Slide 119 text

package design import ( . "github.com/goadesign/goa/design" . "github.com/goadesign/goa/design/apidsl" ) var _ = API("adder", func() { Title("The adder API") Description("A teaser for goa") Host("localhost:8080") Scheme("http") }) var _ = Resource("operands", func() { Action("add", func() { Routing(GET("add/:left/:right")) Description("add returns the sum of the left and right parameters in the response body") Params(func() { Param("left", Integer, "Left operand") Param("right", Integer, "Right operand") }) Response(OK, "plain/text") }) })

Slide 120

Slide 120 text

Magic

Slide 121

Slide 121 text

No content

Slide 122

Slide 122 text

design code > temp program > generated code

Slide 123

Slide 123 text

package main import ( "fmt" "strings" "github.com/goadesign/goa/dslengine" "github.com/goadesign/goa/goagen/gen_app" _ "github.com/ernesto-jimenez/golang-uk/goa-adder/design" ) func main() { // Check if there were errors while running the first DSL pass dslengine.FailOnError(dslengine.Errors) // Now run the secondary DSLs dslengine.FailOnError(dslengine.Run()) files, err := genapp.Generate() dslengine.FailOnError(err) // We're done fmt.Println(strings.Join(files, "\n")) }

Slide 124

Slide 124 text

Wrapping up

Slide 125

Slide 125 text

you can reduce boilerplate

Slide 126

Slide 126 text

tools already available to reduce boilerplate

Slide 127

Slide 127 text

stdlib has great packages to build your own tools

Slide 128

Slide 128 text

many open source projects to learn from

Slide 129

Slide 129 text

Just remember…

Slide 130

Slide 130 text

No content

Slide 131

Slide 131 text

Questions? @ernesto_jimenez [email protected]