GoSF Oct. '17 - Code Generation

Transcript

1. CODE GENERATION TEJAS MANOHAR
2. CODE GENERATION $ whoami ▸ Tejas Manohar ▸ tejasmanohar on GitHub, Twitter, etc. ▸ Software Engineer at Segment
3. CODE GENERATION WHAT'S THIS TALK ABOUT? ▸ Why generate code? ▸ How does code generation work? ▸ How do I generate code?
4. CODE GENERATION
5. CODE GENERATION AT SEGMENT, WE USE RPC ▸ RPC allows you to call func()'s across a network ▸ Your RPC protocol abstracts how ▸ Go has a pluggable implementation in net/rpc
6. CODE GENERATION DEFINE A METHOD ON THE SERVER... func (s *Service) DoThing(t Task) (Result, error) { // ... }
7. CODE GENERATION THEN, JUST CALL IT FROM THE CLIENT... result, err := DoSomething(Task{}) if err != nil { // ... } // use result
8. CODE GENERATION
9. CODE GENERATION SADLY, THE DEFAULT CLIENT LOOKS LIKE THIS... ▸ Method name as a string ▸ Arguments as an interface{} ▸ Response as an interface{} that must be a pointer err := client.Call("Service.Method", Req{}, &Res{})
10. CODE GENERATION WHAT'S WRONG? ▸ interface{}?!? ▸ Types are great! ▸ Method names as strings? rpc.Call("MySevice.SaveReport", ..., ...)
11. CODE GENERATION STATUS QUO: WRITE WRAPPERS BY HAND func (c *Client) Create(req *CreateRequest) (*CreateResponse, error) { res := &CreateResponse{} err := c.rpc.Call("Reports.Create", req, res) return res, err } func (c *Client) Update(req *UpdateRequest) (*UpdateResponse, error) { res := &UpdateResponse{} err := c.rpc.Call("Reports.Update", req, res) return res, err } func (c *Client) Delete(req *DeleteRequest) (*DeleteResponse, error) { res := &DeleteResponse{} err := c.rpc.Call("Reports.Delete", req, res) return res, err }
12. CODE GENERATION SOUNDS EASY! WHAT'S WRONG? ▸ Error-prone ▸ Bikeshedding ▸ Tedious ▸ Programmers tend to get bored easily :')
13. CODE GENERATION HOW I FEEL WHEN WRITING CLIENTS
14. CODE GENERATION HOW I REALLY WANNA FEEL
15. CODE GENERATION INTRODUCING GLUE... $ glue -name=Service -service=Math
16. CODE GENERATION THE END RESULT $ glue -name=Service -service=Math -stdout package client import "github.com/segmentio/glue/example/stl/math" type Math struct { RPC Client } func (c *Math) Sum(args math.SumArg) (*math.SumReply, error) { reply := new(math.SumReply) err := c.RPC.Call("Math.Sum", args, reply) return reply, err }
17. CODE GENERATION HOW DOES GO SEE FIBONACCI? func fib(n int) int { if n == 0 { return 0 } else if n == 1 { return 1 } else { return fib(n-1) + fib(n-2) } }
18. CODE GENERATION FIRST, COMES THE LEXER ▸ Lexer converts source code to "tokens" ▸ "tokens" are series of meaningful characters ▸ Disregards whitespace and fluff
19. CODE GENERATION FIRST, COMES THE LEXER
20. CODE GENERATION THEN, COMES THE PARSER ▸ Parser translates tokens to a syntax tree ▸ "Abstract Syntax Tree", abbrev. AST ▸ AST > Tokens > Code ▸ AST is a logical representation ("abstract") ▸ Disregards syntax
21. CODE GENERATION ABSTRACT SYNTAX TREE
22. CODE GENERATION ABSTRACT SYNTAX TREE
23. CODE GENERATION ABSTRACT SYNTAX TREE
24. CODE GENERATION
25. CODE GENERATION GO MAKES PARSING CODE REALLY EASY ▸ go/build returns metadata about source code ▸ go/scanner and go/parser parse code to AST ▸ go/ast helps you walk and modify AST ▸ go/types makes type-checking easy ▸ x/tools/go/loader makes loading code easy
26. CODE GENERATION FIRST, LOAD THE CODE import "golang.org/x/tools/go/loader" var conf loader.Config conf.Import("github.com/my/rpc/service") prgm, err := conf.Load()
27. CODE GENERATION SO MUCH INFORMATION!
28. CODE GENERATION SO MUCH INFORMATION!
29. CODE GENERATION TOO MUCH INFORMATION! $ glue ./example/stl/math/ | wc -l 5848
30. CODE GENERATION GO/AST HAS A LOT OF TYPES
31. CODE GENERATION EACH TYPE HAS A HIERARCHY
32. CODE GENERATION
33. CODE GENERATION EVERYTHING IS A NODE f, ok := node.(*ast.FuncDecl)
34. CODE GENERATION WRITE A VISITOR
35. CODE GENERATION
36. CODE GENERATION WRITE A VISITOR func (v *Visitor) Visit(node ast.Node) ast.Visitor { switch n := node.(type) { case nil: return nil case *ast.TypeSpec: p.visitType(n) } return p }
37. CODE GENERATION "WALK" THROUGH THE CODE func (v *Visitor) Walk() { for _, package := range prgm.InitialPackages() { for _, file := range p.pkg.Files { ast.Walk(v, file) } } }
38. CODE GENERATION
39. CODE GENERATION FIND WHAT YOU NEED namedType := p.pkg.Info.ObjectOf(ts.Name).Type().(*types.Named) for i := 0; i < namedType.NumMethods(); i++ { method := namedType.Method(i) if p.provider.IsSuitableMethod(method) { recv := namedType.Obj().Name() p.methods[recv] = append(p.methods[recv], method) } }
40. CODE GENERATION
41. CODE GENERATION EXTRACT WHAT YOU WANT func (p *Provider) GetArgType(f *types.Func) types.Type { signature := f.Type().(*types.Signature) params := signature.Params() arg := params.At(0) return internal.Dereference(arg.Type()) } func (t *Arith) Multiply(args *Args, reply *int) error { *reply = args.A * args.B return nil }
42. CODE GENERATION GENERATE OUTPUT USING TEXT/TEMPLATE type {{ .Service }} struct { RPC client.Client } {{ range .Methods }} func (c *{{ $.Service }}) {{ .Name }}(args {{ .ArgType }}) (*{{ .ReplyType }}, error) { reply := new({{ .ReplyType }}) err := c.RPC.Call("{{ $.Service }}.{{ .Name }}", args, reply) return reply, err } {{ end }}
43. CODE GENERATION OR, USE GO/FORMAT
44. CODE GENERATION CHECK IT OUT @ GITHUB.COM/SEGMENTIO/GLUE!
45. POSTGRES CONCURRENCY Thank you ▸ Segment is hiring ▸ Let's talk Go! ▸ tejas@segment.com ▸ me@tejas.io ▸ tejasmanohar @ everywhere