Tweet
Tweet

Slide 1

Slide 1 text

How to write your own Go tool Fatih Arslan @DigitalOcean GothamGo - 2016

Slide 2

Slide 2 text

One year ago, I gave a talk about the state of Go tools

Slide 3

Slide 3 text

I wanted to write a tool myself But where do I begin?

Slide 4

Slide 4 text

After reading lots of documentation, code, etc... I wrote a tool called "motion"

Slide 5

Slide 5 text

Some of features in motion: • Jump to function or type declaration • Jump to next or previous function declaration • Select function content (delete/copy/change...) • ...

Slide 6

Slide 6 text

Jump to function declarations

Slide 7

Slide 7 text

Let’s start from the beginning ...

Slide 8

Slide 8 text

The Lexer & Parser Family

Slide 9

Slide 9 text

go/token go/scanner go/parser go/ast go/printer

Slide 10

Slide 10 text

go/token Package token defines constants representing the lexical tokens of the Go programming language and basic operations on tokens (printing, predicates).

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

token.VAR token.IDENT token.IDENT token.ASSIGN token.INT token.ADD token.INT var sum int = 3 + 2

Slide 15

Slide 15 text

No content

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

go/scanner Package scanner implements a scanner for Go source text. It takes a []byte as source which can then be tokenized through repeated calls to the Scan method.

Slide 18

Slide 18 text

var sum int = 3 + 2 Run through a scanner go/scanner token.VAR token.IDENT token.EOF . . .

Slide 19

Slide 19 text

How do we scan?

Slide 20

Slide 20 text

func main() { src := []byte(`var sum int = 3 + 2`) // Initialize the scanner. var s scanner.Scanner fset := token.NewFileSet() file := fset.AddFile("", fset.Base(), len(src)) s.Init(file, src, nil, scanner.ScanComments) // run the scanner for { pos, tok, lit := s.Scan() if tok == token.EOF { break } fmt.Printf("%s\t%s\t%q\n", fset.Position(pos), tok, lit) } }

Slide 21

Slide 21 text

func main() { src := []byte(`var sum int = 3 + 2`) // Initialize the scanner. var s scanner.Scanner fset := token.NewFileSet() file := fset.AddFile("", fset.Base(), len(src)) s.Init(file, src, nil, scanner.ScanComments) // run the scanner for { pos, tok, lit := s.Scan() if tok == token.EOF { break } fmt.Printf("%s\t%s\t%q\n", fset.Position(pos), tok, lit) } }

Slide 22

Slide 22 text

func main() { src := []byte(`var sum int = 3 + 2`) // Initialize the scanner. var s scanner.Scanner fset := token.NewFileSet() file := fset.AddFile("", fset.Base(), len(src)) s.Init(file, src, nil, scanner.ScanComments) // run the scanner for { pos, tok, lit := s.Scan() if tok == token.EOF { break } fmt.Printf("%s\t%s\t%q\n", fset.Position(pos), tok, lit) } }

Slide 23

Slide 23 text

var sum int = 3 + 2 go/scanner

Slide 24

Slide 24 text

var sum int = 3 + 2 Pos Token Literal 1:1 VAR "var" 1:5 IDENT "sum" 1:9 IDENT "int" 1:13 ASSIGN "=" 1:15 INT "3" 1:19 INT "2" 1:17 ADD "+" 1:20 EOF "" go/scanner

Slide 25

Slide 25 text

go/ast Package ast declares the types used to represent syntax trees for Go packages.

Slide 26

Slide 26 text

ast.Node

Slide 27

Slide 27 text

3 token.INT token.ADD token.INT + 2

Slide 28

Slide 28 text

*ast.BinaryExpr

Slide 29

Slide 29 text

*ast.BinaryExpr token.INT token.ADD token.INT

Slide 30

Slide 30 text

3 token.INT token.ADD token.INT + 2 *ast.BinaryExpr

Slide 31

Slide 31 text

3 + 2 *ast.BinaryExpr

Slide 32

Slide 32 text

3 + 2 *ast.BinaryExpr *ast.BasicLit 3

Slide 33

Slide 33 text

3 + 2 *ast.BinaryExpr *ast.BasicLit 2

Slide 34

Slide 34 text

3 + 2 *ast.BinaryExpr *ast.BasicLit *ast.BasicLit 3 2 +

Slide 35

Slide 35 text

3 + (7 - 5) *ast.BinaryExpr *ast.BasicLit 3 + *ast.BinaryExpr *ast.BasicLit *ast.BasicLit 5 - 7

Slide 36

Slide 36 text

3 + (7 - 5) *ast.BinaryExpr *ast.BasicLit 3 + *ast.BinaryExpr *ast.BasicLit *ast.BasicLit 5 - 7

Slide 37

Slide 37 text

var sum int = 3 + 2

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

var sum int = 3 + 2 *ast.ValueSpec

Slide 40

Slide 40 text

var sum int = 3 + 2 *ast.ValueSpec *ast.Ident sum var

Slide 41

Slide 41 text

var sum int = 3 + 2 *ast.ValueSpec *ast.Ident int var

Slide 42

Slide 42 text

var sum int = 3 + 2 *ast.ValueSpec *ast.BinaryExpr *ast.BasicLit *ast.BasicLit 3 2 + var

Slide 43

Slide 43 text

var sum int = 3 + 2 *ast.ValueSpec *ast.Ident *ast.Ident *ast.BinaryExpr *ast.BasicLit *ast.BasicLit sum int 3 2 + var =

Slide 44

Slide 44 text

go/parser Package parser implements a parser for Go source files. Input may be provided in a variety of forms; the output is an abstract syntax tree (AST) representing the Go source.

Slide 45

Slide 45 text

var sum int = 3 + 2 go/parser *ast.ValueSpec *ast.Ident *ast.Ident *ast.BinaryExpr *ast.BasicLit *ast.BasicLit

Slide 46

Slide 46 text

var sum int = 3 + 2 go/parser *ast.ValueSpec *ast.Ident *ast.Ident *ast.BinaryExpr *ast.BasicLit *ast.BasicLit go/scanner go/token

Slide 47

Slide 47 text

How do we parse?

Slide 48

Slide 48 text

func main() { src := []byte(`package main var sum int = 3 + 2`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Fprint(os.Stdout, fset, node, nil) }

Slide 49

Slide 49 text

func main() { src := []byte(`package main var sum int = 3 + 2`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Fprint(os.Stdout, fset, node, nil) }

Slide 50

Slide 50 text

func main() { src := []byte(`package main var sum int = 3 + 2`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Fprint(os.Stdout, fset, node, nil) }

Slide 51

Slide 51 text

No content

Slide 52

Slide 52 text

How do we traverse the tree?

Slide 53

Slide 53 text

Let us find a line and column number package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) }

Slide 54

Slide 54 text

package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } *ast.BinaryExpr Let us find a line and column number

Slide 55

Slide 55 text

src := []byte(`package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) }`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } // start searching for the main() function declaration for _, decl := range node.Decls { ...

Slide 56

Slide 56 text

src := []byte(`package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) }`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } // start searching for the main() function declaration for _, decl := range node.Decls { ...

Slide 57

Slide 57 text

src := []byte(`package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) }`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } // start searching for the main() function declaration for _, decl := range node.Decls { ... package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } 1 2 3 4 5 6 7 8

Slide 58

Slide 58 text

for _, decl := range node.Decls { // search for main function fn, ok := decl.(*ast.FuncDecl) if !ok { continue } // inside the main function body for _, stmt := range fn.Body.List { // search through statements for a declaration... declStmt, ok := stmt.(*ast.DeclStmt) if !ok { continue } // continue with declStmt.Decl // ... } } package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } 1 2 3 4 5 6 7 8

Slide 59

Slide 59 text

for _, decl := range node.Decls { // search for main function fn, ok := decl.(*ast.FuncDecl) if !ok { continue } // inside the main function body for _, stmt := range fn.Body.List { // search through statements for a declaration... declStmt, ok := stmt.(*ast.DeclStmt) if !ok { continue } // continue with declStmt.Decl // ... } } package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } 1 2 3 4 5 6 7 8

Slide 60

Slide 60 text

// continue with declStmt.Decl genDecl, ok := declStmt.Decl.(*ast.GenDecl) if !ok { continue } // declarations can have multiple specs, // search for a valuespec for _, spec := range genDecl.Specs { valueSpec, ok := spec.(*ast.ValueSpec) if !ok { continue } // continue with valueSpec.Values // ... } package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } 1 2 3 4 5 6 7 8

Slide 61

Slide 61 text

// continue with declStmt.Decl genDecl, ok := declStmt.Decl.(*ast.GenDecl) if !ok { continue } // declarations can have multiple specs, // search for a valuespec for _, spec := range genDecl.Specs { valueSpec, ok := spec.(*ast.ValueSpec) if !ok { continue } // continue with valueSpec.Values // ... } package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } 1 2 3 4 5 6 7 8

Slide 62

Slide 62 text

// continue with valueSpec.Values for _, expr := range valueSpec.Values { // search for a binary expr binaryExpr, ok := expr.(*ast.BinaryExpr) if !ok { continue } // found it! fmt.Printf("Found binary expression at: %d:%d\n", fset.Position(binaryExpr.Pos()).Line, fset.Position(binaryExpr.Pos()).Column, ) } package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } 1 2 3 4 5 6 7 8

Slide 63

Slide 63 text

// continue with valueSpec.Values for _, expr := range valueSpec.Values { // search for a binary expr binaryExpr, ok := expr.(*ast.BinaryExpr) if !ok { continue } // found it! fmt.Printf("Found binary expression at: %d:%d\n", fset.Position(binaryExpr.Pos()).Line, fset.Position(binaryExpr.Pos()).Column, ) } package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } 1 2 3 4 5 6 7 8

Slide 64

Slide 64 text

// continue with valueSpec.Values for _, expr := range valueSpec.Values { // search for a binary expr binaryExpr, ok := expr.(*ast.BinaryExpr) if !ok { continue } // found it! fmt.Printf("Found binary expression at: %d:%d\n", fset.Position(binaryExpr.Pos()).Line, fset.Position(binaryExpr.Pos()).Column, ) } package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } Output: Found binary expression at: 6:17 1 2 3 4 5 6 7 8

Slide 65

Slide 65 text

This is a tedious process

Slide 66

Slide 66 text

Is there an easier way?

Slide 67

Slide 67 text

Is there an easier way? Yes!

Slide 68

Slide 68 text

ast.Inspect()

Slide 69

Slide 69 text

visitorFunc := func(n ast.Node) bool { binaryExpr, ok := n.(*ast.BinaryExpr) if !ok { return true } fmt.Printf("Found binary expression at: %d:%d\n", fset.Position(binaryExpr.Pos()).Line, fset.Position(binaryExpr.Pos()).Column, ) return true } // walk trough all nodes and run the // given function for each node ast.Inspect(node, visitorFunc)

Slide 70

Slide 70 text

visitorFunc := func(n ast.Node) bool { binaryExpr, ok := n.(*ast.BinaryExpr) if !ok { return true } fmt.Printf("Found binary expression at: %d:%d\n", fset.Position(binaryExpr.Pos()).Line, fset.Position(binaryExpr.Pos()).Column, ) return true } // walk trough all nodes and run the // given function for each node ast.Inspect(node, visitorFunc) Output: Found binary expression at: 6:17

Slide 71

Slide 71 text

ast.Inspect() for the President!

Slide 72

Slide 72 text

go/printer Package printer implements printing of AST nodes.

Slide 73

Slide 73 text

go/ast go/printer package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) }

Slide 74

Slide 74 text

How do we print?

Slide 75

Slide 75 text

func main() { src := []byte(`package main var sum int = 3 + 2`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } printer.Fprint(os.Stdout, fset, node) }

Slide 76

Slide 76 text

func main() { src := []byte(`package main var sum int = 3 + 2`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } printer.Fprint(os.Stdout, fset, node) }

Slide 77

Slide 77 text

func main() { src := []byte(`package main var sum int = 3 + 2`) fset := token.NewFileSet() node, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } printer.Fprint(os.Stdout, fset, node) } Output: package main var sum int = 3 + 2

Slide 78

Slide 78 text

Back to the motion tool

Slide 79

Slide 79 text

Basic rules for a simple tool 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 80

Slide 80 text

Case study: motion 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 81

Slide 81 text

func NewParser(opts *ParserOptions) (*Parser, error) { // ... switch { case opts.File != "": p.file, err = parser.ParseFile(fset, opts.File, nil, mode) case opts.Dir != "": p.pkgs, err = parser.ParseDir(fset, opts.Dir, nil, mode) case opts.Src != nil: p.file, err = parser.ParseFile(fset, "src.go", opts.Src, mode) default: return nil, errors.New("file, src or dir is not specified") } return p, nil }

Slide 82

Slide 82 text

func NewParser(opts *ParserOptions) (*Parser, error) { // ... switch { case opts.File != "": p.file, err = parser.ParseFile(fset, opts.File, nil, mode) case opts.Dir != "": p.pkgs, err = parser.ParseDir(fset, opts.Dir, nil, mode) case opts.Src != nil: p.file, err = parser.ParseFile(fset, "src.go", opts.Src, mode) default: return nil, errors.New("file, src or dir is not specified") } return p, nil }

Slide 83

Slide 83 text

func NewParser(opts *ParserOptions) (*Parser, error) { // ... switch { case opts.File != "": p.file, err = parser.ParseFile(fset, opts.File, nil, mode) case opts.Dir != "": p.pkgs, err = parser.ParseDir(fset, opts.Dir, nil, mode) case opts.Src != nil: p.file, err = parser.ParseFile(fset, "src.go", opts.Src, mode) default: return nil, errors.New("file, src or dir is not specified") } return p, nil }

Slide 84

Slide 84 text

func NewParser(opts *ParserOptions) (*Parser, error) { // ... switch { case opts.File != "": p.file, err = parser.ParseFile(fset, opts.File, nil, mode) case opts.Dir != "": p.pkgs, err = parser.ParseDir(fset, opts.Dir, nil, mode) case opts.Src != nil: p.file, err = parser.ParseFile(fset, "src.go", opts.Src, mode) default: return nil, errors.New("file, src or dir is not specified") } return p, nil }

Slide 85

Slide 85 text

Case study: motion 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 86

Slide 86 text

func (p *Parser) collectFunctions() []ast.Node { var funcs []*ast.FuncDecl visitorFunc := func(n ast.Node) bool { fn, ok := n.(*ast.FuncDecl) if !ok { return true } funcs = append(funcs, fn) return true } ast.Inspect(p.file, visitorFunc) return funcs }

Slide 87

Slide 87 text

func (p *Parser) collectFunctions() []ast.Node { var funcs []*ast.FuncDecl visitorFunc := func(n ast.Node) bool { fn, ok := n.(*ast.FuncDecl) if !ok { return true } funcs = append(funcs, fn) return true } ast.Inspect(p.file, visitorFunc) return funcs }

Slide 88

Slide 88 text

func (p *Parser) collectFunctions() []ast.Node { var funcs []*ast.FuncDecl visitorFunc := func(n ast.Node) bool { fn, ok := n.(*ast.FuncDecl) if !ok { return true } funcs = append(funcs, fn) return true } ast.Inspect(p.file, visitorFunc) return funcs }

Slide 89

Slide 89 text

func (p *Parser) collectFunctions() []ast.Node { var funcs []*ast.FuncDecl visitorFunc := func(n ast.Node) bool { fn, ok := n.(*ast.FuncDecl) if !ok { return true } funcs = append(funcs, fn) return true } ast.Inspect(p.file, visitorFunc) return funcs }

Slide 90

Slide 90 text

func (p *Parser) collectFunctions() []ast.Node { var funcs []*ast.FuncDecl visitorFunc := func(n ast.Node) bool { fn, ok := n.(*ast.FuncDecl) if !ok { return true } funcs = append(funcs, fn) return true } ast.Inspect(p.file, visitorFunc) return funcs }

Slide 91

Slide 91 text

Case study: motion 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 92

Slide 92 text

func formatFunctions(funcs []*ast.FuncDecl) string { out := new(bytes.Buffer) for _, fn := range funcs { fnPos := p.fset.Position(fn.Type.Func) // foo.go:line:column filePos := fmt.Sprintf("%s:%d:%d", fnPos.Filename, fnPos.Line, fnPos.Column) // foo.go:line:column | func name | func signature() fmt.Fprintf(out, "%s | %s | %s\n", filePos, fn.Name.Name, funcSignature(fn)) } return out.String() }

Slide 93

Slide 93 text

func formatFunctions(funcs []*ast.FuncDecl) string { out := new(bytes.Buffer) for _, fn := range funcs { fnPos := p.fset.Position(fn.Type.Func) // foo.go:line:column filePos := fmt.Sprintf("%s:%d:%d", fnPos.Filename, fnPos.Line, fnPos.Column) // foo.go:line:column | func name | func signature() fmt.Fprintf(out, "%s | %s | %s\n", filePos, fn.Name.Name, funcSignature(fn)) } return out.String() }

Slide 94

Slide 94 text

func formatFunctions(funcs []*ast.FuncDecl) string { out := new(bytes.Buffer) for _, fn := range funcs { fnPos := p.fset.Position(fn.Type.Func) // foo.go:line:column filePos := fmt.Sprintf("%s:%d:%d", fnPos.Filename, fnPos.Line, fnPos.Column) // foo.go:line:column | func name | func signature() fmt.Fprintf(out, "%s | %s | %s\n", filePos, fn.Name.Name, funcSignature(fn)) } return out.String() }

Slide 95

Slide 95 text

func formatFunctions(funcs []*ast.FuncDecl) string { out := new(bytes.Buffer) for _, fn := range funcs { fnPos := p.fset.Position(fn.Type.Func) // foo.go:line:column filePos := fmt.Sprintf("%s:%d:%d", fnPos.Filename, fnPos.Line, fnPos.Column) // foo.go:line:column | func name | func signature() fmt.Fprintf(out, "%s | %s | %s\n", filePos, fn.Name.Name, funcSignature(fn)) } return out.String() }

Slide 96

Slide 96 text

Case study: motion 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 97

Slide 97 text

func main() { flagFile = flag.String("file", "", "Filename to be parsed") flagMode = flag.String("mode", "", "Running mode") flag.Parse() opts := &ParserOptions{File: *flagFile} parser, err := NewParser(opts) if err != nil { panic(err) } if *flagMode == "funcs" { funcs := parser.collectFunctions() out := formatFunctions(funcs) fmt.Println(out) } // ... }

Slide 98

Slide 98 text

func main() { flagFile = flag.String("file", "", "Filename to be parsed") flagMode = flag.String("mode", "", "Running mode") flag.Parse() opts := &ParserOptions{File: *flagFile} parser, err := NewParser(opts) if err != nil { panic(err) } if *flagMode == "funcs" { funcs := parser.collectFunctions() out := formatFunctions(funcs) fmt.Println(out) } // ... }

Slide 99

Slide 99 text

func main() { flagFile = flag.String("file", "", "Filename to be parsed") flagMode = flag.String("mode", "", "Running mode") flag.Parse() opts := &ParserOptions{File: *flagFile} parser, err := NewParser(opts) if err != nil { panic(err) } if *flagMode == "funcs" { funcs := parser.collectFunctions() out := formatFunctions(funcs) fmt.Println(out) } // ... }

Slide 100

Slide 100 text

func main() { flagFile = flag.String("file", "", "Filename to be parsed") flagMode = flag.String("mode", "", "Running mode") flag.Parse() opts := &ParserOptions{File: *flagFile} parser, err := NewParser(opts) if err != nil { panic(err) } if *flagMode == "funcs" { funcs := parser.collectFunctions() out := formatFunctions(funcs) fmt.Println(out) } // ... }

Slide 101

Slide 101 text

$ cat main.go package main import "fmt" func main() { a := 5 incr := func(x int) int { fmt.Printf("incremeting %d by one\n", x) return x + 1 } b := incr(a) fmt.Println(b, check(b)) } // check checks if the given integer // is smaller than 10 func check(a int) bool { return a < 10 } // decr decrements the given integer func decr(x int) int { return x - 1 }

Slide 102

Slide 102 text

$ motion -mode funcs -file main.go main.go:5:1 | main | func main() main.go:19:1 | check | func check(a int) bool main.go:24:1 | decr | func decr(x int) int

Slide 103

Slide 103 text

No content

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

Another case study gofmt

Slide 106

Slide 106 text

Case study: gofmt 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 107

Slide 107 text

Case study: gofmt package main import "fmt" func main(){ var sum int = 3+2 fmt.Println( sum ) }

Slide 108

Slide 108 text

Case study: gofmt go/parser package main import "fmt" func main(){ var sum int = 3+2 fmt.Println( sum ) }

Slide 109

Slide 109 text

Case study: gofmt go/parser go/ast package main import "fmt" func main(){ var sum int = 3+2 fmt.Println( sum ) }

Slide 110

Slide 110 text

func processFile(filename string, in io.Reader, out io.Writer, stdin bool) error { // ... src, err := ioutil.ReadAll(in) if err != nil { return err } file, sourceAdj, indentAdj, err := parse(fileSet, filename, src, stdin) if err != nil { return err } // ...

Slide 111

Slide 111 text

func processFile(filename string, in io.Reader, out io.Writer, stdin bool) error { // ... src, err := ioutil.ReadAll(in) if err != nil { return err } file, sourceAdj, indentAdj, err := parse(fileSet, filename, src, stdin) if err != nil { return err } // ...

Slide 112

Slide 112 text

Case study: gofmt 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 113

Slide 113 text

Case study: gofmt go/parser go/ast sort simplify rewrite package main import "fmt" func main(){ var sum int = 3+2 fmt.Println( sum ) }

Slide 114

Slide 114 text

func processFile(filename string, in io.Reader, out io.Writer, stdin bool) error { // ... if rewrite != nil { if sourceAdj == nil { file = rewrite(file) } else { fmt.Fprintf(os.Stderr, "warning: rewrite ignored for incomplete programs\n") } } ast.SortImports(fileSet, file) if *simplifyAST { simplify(file) } // ...

Slide 115

Slide 115 text

Case study: gofmt 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 116

Slide 116 text

Case study: gofmt go/parser go/ast sort simplify rewrite go/printer package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } package main import "fmt" func main(){ var sum int = 3+2 fmt.Println( sum ) }

Slide 117

Slide 117 text

func processFile(filename string, in io.Reader, out io.Writer, stdin bool) error { // ... res, err := format(fileSet, file, sourceAdj, indentAdj, src, printer.Config{Mode: printerMode, Tabwidth: tabWidth}) if err != nil { return err } if !*list && !*write && !*doDiff { _, err = out.Write(res) } // ...

Slide 118

Slide 118 text

Case study: gofmt 1. Read (go/parser) 2. Inspect (go/ast) 3. Write (go/printer) or custom format (JSON, etc...) 4. Create a CLI

Slide 119

Slide 119 text

func main() { // ... for i := 0; i < flag.NArg(); i++ { path := flag.Arg(i) switch dir, err := os.Stat(path); { case err != nil: report(err) case dir.IsDir(): walkDir(path) default: if err := processFile(path, nil, os.Stdout, false); err != nil { report(err) } } } // ... }

Slide 120

Slide 120 text

$ cat main.go package main import "fmt" func main() { var sum int = 3+2 fmt.Println( sum ) }

Slide 121

Slide 121 text

$ cat main.go package main import "fmt" func main() { var sum int = 3+2 fmt.Println( sum ) } $ gofmt main.go package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) }

Slide 122

Slide 122 text

Recap

Slide 123

Slide 123 text

package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } go/parser go/ast go/printer package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) }

Slide 124

Slide 124 text

package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } go/ast package main import "fmt" func main() { var sum int = 3 + 2 fmt.Println(sum) } go/printer go/parser go/scanner go/token go/parser

Slide 125

Slide 125 text

Next steps

Slide 126

Slide 126 text

Source: https://talks.golang.org/2014/static-analysis.slide#8

Slide 127

Slide 127 text

Thanks Fatih Arslan fatih fatih fatih@digitalocean.com