Static Analysis in Go

Talk given at Singapore Gophers meetup on 2014-05-21. Slides and source code at https://bitbucket.org/keyist/static_analysis_in_go-slides

Transcript

1. Static Analysis in Go Tim Goh Nitrous.IO Static Analysis in Go 1 of 29
2. Static analysis Analyze software Without executing it Not limited to statically-typed languages! Static Analysis in Go 2 of 29
3. Static analysis for dummies for _, file := range files { if lineCount(file) > 10000 { fmt.Printf("%s is too big, split it up.", file) } } Static Analysis in Go 3 of 29
4. What can you find in automated fashion? repeated definitions repeated lines unreachable code complexity heuristics Static Analysis in Go 4 of 29
5. Some languages are harder to analyze class Zombie def undead_method # some code end end Zombie.new.send(%w(undead method).join('_')) Can't just check for calls to the exact name 'undead_method' Static Analysis in Go 5 of 29
6. What's out there? Static Analysis in Go 6 of 29
7. Coverity Static Analysis in Go 7 of 29
8. CodeClimate Static Analysis in Go 8 of 29
9. go tools go fmt go vet go fix `go vet` can find unreachable code, amongst other things. Audience participation: in Go, what do you look for to determine unreachable? Static Analysis in Go 9 of 29
10. Abstract Syntax Trees Corresponds to while b ≠ 0 if a > b a := a − b else b := b − a return a Static Analysis in Go 10 of 29
11. language spec vs ast comparison EBNF: IfStmt = "if" [ SimpleStmt ";" ] Expression Block [ "else" ( IfStmt | Block ) ] . go/ast definition: type IfStmt struct { If token.Pos // position of "if" keyword Init Stmt // initialization statement; or nil Cond Expr // condition Body *BlockStmt Else Stmt // else branch; or nil } Static Analysis in Go 11 of 29
12. AST printing // from http://golang.org/pkg/go/ast/#example_Print package main import ( "go/ast" "go/parser" "go/token" ) func main() { src := ` package main; func main() { println("Hello, World!") } ` fset := token.NewFileSet() f, err := parser.ParseFile(fset, "", src, 0) if err != nil { panic(err) } ast.Print(fset, f) } Audience participation part 2: how many lines will be printed? Static Analysis in Go 12 of 29
13. AST printing 0 *ast.File { 1 . Package: 1:2 2 . Name: *ast.Ident { 3 . . NamePos: 1:10 4 . . Name: "main" 5 . } 6 . Decls: []ast.Decl (len = 1) { 7 . . 0: *ast.FuncDecl { 8 . . . Name: *ast.Ident { 9 . . . . NamePos: 1:21 10 . . . . Name: "main" 11 . . . . Obj: *ast.Object { 12 . . . . . Kind: func 13 . . . . . Name: "main" 14 . . . . . Decl: *(obj @ 7) 15 . . . . } 16 . . . } 17 . . . Type: *ast.FuncType { 18 . . . . Func: 1:16 19 . . . . Params: *ast.FieldList { Static Analysis in Go 13 of 29
14. AST printing 20 . . . . . Opening: 1:25 21 . . . . . Closing: 1:26 22 . . . . } 23 . . . } 24 . . . Body: *ast.BlockStmt { 25 . . . . Lbrace: 1:28 26 . . . . List: []ast.Stmt (len = 1) { 27 . . . . . 0: *ast.ExprStmt { 28 . . . . . . X: *ast.CallExpr { 29 . . . . . . . Fun: *ast.Ident { 30 . . . . . . . . NamePos: 1:30 31 . . . . . . . . Name: "println" 32 . . . . . . . } 33 . . . . . . . Lparen: 1:37 34 . . . . . . . Args: []ast.Expr (len = 1) { 35 . . . . . . . . 0: *ast.BasicLit { 36 . . . . . . . . . ValuePos: 1:38 37 . . . . . . . . . Kind: STRING 38 . . . . . . . . . Value: "\"Hello, World!\"" 39 . . . . . . . . } Static Analysis in Go 14 of 29
15. AST printing 40 . . . . . . . } 41 . . . . . . . Ellipsis: - 42 . . . . . . . Rparen: 1:53 43 . . . . . . } 44 . . . . . } 45 . . . . } 46 . . . . Rbrace: 1:55 47 . . . } 48 . . } 49 . } 50 . Scope: *ast.Scope { 51 . . Objects: map[string]*ast.Object (len = 1) { 52 . . . "main": *(obj @ 11) 53 . . } 54 . } 55 . Unresolved: []*ast.Ident (len = 1) { 56 . . 0: *(obj @ 29) 57 . } 58 } Static Analysis in Go 15 of 29
16. Just reading is boring. Let's write Static Analysis in Go 16 of 29
17. Javify type FuncDeclVisitor struct{} func (v *FuncDeclVisitor) Visit(node ast.Node) ast.Visitor { switch t := node.(type) { case *ast.FuncDecl: t.Name = ast.NewIdent(fmt.Sprintf("Abstract%sFactory", strings.Title(t.Name.Name))) return nil } return v } func main() { fset := token.NewFileSet() file, err := parser.ParseFile(fset, os.Args[1], nil, 0) if err != nil { panic(err) } ast.Walk(new(FuncDeclVisitor), file) printer.Fprint(os.Stdout, fset, file) } Static Analysis in Go 17 of 29
18. Javify type FuncDeclVisitor struct{} func (v *FuncDeclVisitor) Visit(node ast.Node) ast.Visitor { switch t := node.(type) { case *ast.FuncDecl: t.Name = ast.NewIdent(fmt.Sprintf("Abstract%sFactory", strings.Title(t.Name.Name))) return nil } return v } func main() { fset := token.NewFileSet() file, err := parser.ParseFile(fset, os.Args[1], nil, 0) if err != nil { panic(err) } ast.Walk(new(FuncDeclVisitor), file) printer.Fprint(os.Stdout, fset, file) } Static Analysis in Go 18 of 29
19. Javify on Javify func (v *FuncDeclVisitor) AbstractVisitFactory(node ast.Node) ast.Visitor { switch t := node.(type) { case *ast.FuncDecl: t.Name = ast.NewIdent(fmt.Sprintf("Abstract%sFactory", strings.Title(t.Name.Name))) return nil } return v } func AbstractMainFactory() { fset := token.NewFileSet() file, err := parser.ParseFile(fset, os.Args[1], nil, 0) if err != nil { panic(err) } ast.Walk(new(FuncDeclVisitor), file) printer.Fprint(os.Stdout, fset, file) } Static Analysis in Go 19 of 29
20. Javify**3 func (v *FuncDeclVisitor) AbstractAbstractVisitFactoryFactory(node ast.Node) ast.Visitor { switch t := node.(type) { case *ast.FuncDecl: t.Name = ast.NewIdent(fmt.Sprintf("Abstract%sFactory", strings.Title(t.Name.Name))) return nil } return v } func AbstractAbstractMainFactoryFactory() { fset := token.NewFileSet() file, err := parser.ParseFile(fset, os.Args[1], nil, 0) if err != nil { panic(err) } ast.Walk(new(FuncDeclVisitor), file) printer.Fprint(os.Stdout, fset, file) } Static Analysis in Go 20 of 29
21. Real-World applications transform code before tests instrument code for profiling preprocessor macros Static Analysis in Go 21 of 29
22. go vet deadcode Static Analysis in Go 22 of 29
23. go vet deadcode code.google.com/p/go/source/browse/cmd/vet/deadcode.go?repo=tools (https://code.google.com/p/go/source/browse/cmd/vet/deadcode.go?repo=tools) // findDead walks the statement looking for dead code. // If d.reachable is false on entry, stmt itself is dead. // When findDead returns, d.reachable tells whether the // statement following stmt is reachable. func (d *deadState) findDead(stmt ast.Stmt) { // ...elided... case *ast.ExprStmt: // Call to panic? call, ok := x.X.(*ast.CallExpr) if ok { name, ok := call.Fun.(*ast.Ident) if ok && name.Name == "panic" && name.Obj == nil { d.reachable = false } } Static Analysis in Go 23 of 29
24. go-faster github.com/bouk/go-faster (https://github.com/bouk/go-faster) Transforms package main func a(c int) int { c += 1 return c } func main() { println(a(2)) } Static Analysis in Go 24 of 29
25. go-faster Into package main func a(c int) chan int { result := make(chan int) go func() { result <- func() int { c += 1 return c }() }() return result } func main() { println(<-a(2)) } Static Analysis in Go 25 of 29
26. Go/ast Rider My semantic diff tool with a really dumb name bitbucket.org/keyist/goastrider (http://bitbucket.org/keyist/goastrider) Static Analysis in Go 26 of 29
27. Use cases has the interface of a dependency changed? have I broken my public API? Static Analysis in Go 27 of 29
28. Fun Take-Home Exercises find uses of keywords as struct field names find identifiers more than 20 characters long insert logging of "Now running FuncName" before each function find functions FuncName without a corresponding TestFuncName function Static Analysis in Go 28 of 29
29. Thank you Tim Goh Nitrous.IO @keyist (http://twitter.com/keyist) Static Analysis in Go 29 of 29