Diff'ing Go library API changes

Introducing https://github.com/motemen/gompatible

Transcript

1. Diff’ing Go library

   API changes
   Go Conference 2015 Winter

   @motemen, motemen.github.io
   

   View Slide

2. About me
   - @motemen on GitHub/Twitter/Hatena
   - Chief engineer at Hatena
   - Loves: Go, git
   - Wrote: ghq, gore and more
   

   View Slide

3. Diff’ing Go library API changes
   - Introduction of a tool recently I’ve written: gompat
   - Motivation, overview & Implementation
   

   View Slide

4. Motivation
   

   View Slide

5. “go get -u” and pray
   - Go packages can be obtained as “latest”
   - Dependent libraries may introduce breaking
   changes without your notice
   - Could use “vendoring” or “revision locking”
   

   View Slide

6. How can we:
   - Know dependent libraries’ API changes?
   - Know to what version to update them?
   - Know if we are introducing breaking changes

   to our package?
   

   View Slide

7. API changes overview
   

   View Slide

8. API entities
   - Package-level and published:
   • Functions and methods (func)
   • Types (type)
   • Variables (var), Constants (const)
   - May be changed by the package update
   

   View Slide

9. Changes: the obvious
   = Unchanged
   + Added
   - Deleted
   

   View Slide

10. ! Breaking
    - Named API entities with their interface changed
    • Users have to modify API usages
    - … except for the “compatible” case (later)
    

    View Slide

11. Breaking: struct type example
    - Struct field deleted
    - Struct field type changed
    // before
    type BreakingT struct {
    Foo string
    Bar bool
    }
    // after
    type BreakingT struct {
    Bar int
    }
    

    View Slide

12. Breaking: function example
    - Parameter types, number changed
    - Return value types, number changed
    // before
    func BreakingF(n, m int) int
    // after
    func BreakingF(b bool) (int, error)
    

    View Slide

13. * Compatible
    - API signatures changed
    - Users don’t need their API usages
    

    View Slide

14. Compatible: function example
    - Function changed to variadic
    // before
    func CompatibleF(n, m int)
    // after
    func CompatibleF(n, m int, opts ...string)
    

    View Slide

15. Compatible: struct type example
    - Public field added
    * Breaking for T{x, y, z}
    initializers
    // before
    type CompatibleT struct {
    Foo int
    }
    // after
    type CompatibleT struct {
    Foo int
    Bar bool
    }
    

    View Slide

16. Compatible: other cases
    - Value changed from const to var
    - Function parameter type became more general
    • eg. *bytes.Buffer to io.Reader
    

    View Slide

17. API Changes
    
    = Unchanged
    + Added
    * Compatible
    
    ! Breaking
    - Deleted
    

    View Slide

18. Enter Gompatible
    

    View Slide

19. ! motemen/gompatible
    - Compares API of a package under two revisions
    - Produces change kinds per API entities
    • Added / Removed / Breaking / Compatible
    

    View Slide

20. cmd/gompat
    - Main entrypoint of gompatible
    - Usage:
    • gompat rev..rev package
    

    View Slide

21. Inside Gompatible
    

    View Slide

22. Gompatible tasks
    - Mostly achieved by std packages:
    • Load packages at revisions
    • Load API signatures from packages
    • Compare type information
    

    View Slide

23. go/types
    - Formerly x/tools/go/types
    - Generates type information from source ASTs
    - Entrypoint: *types.Package
    • pkg.Scope().Lookup(“main”) → *types.Func
    • fun.Type() → *types.Signature
    • sig.Params(), sig.Results() → *types.Tuple
    

    View Slide

24. go/types
    - Can query on types
    • types.Implements(Type, *Interface)
    • types.AssignableTo(Type, Type)
    

    View Slide

25. go/types
    - It’s a lot of work (and maybe chore) to:
    • Convert import path to source code directory
    • Filtering out e.g. *_test.go or *_windows.go’s
    • Parse source files and combine them as package
    • Then process typings by go/types
    

    View Slide

26. x/tools/go/loader
    - Package path and filenames to *loader.Program
    • Syntactically parsed and typed packages
    - Built for command line tools
    • loader.Config.FromArgs([]string, bool)
    • Users: x/tools/cmd/…
    - (Can see loader.FromArgsUsage inside oracle -help)
    

    View Slide

27. x/tools/go/loader: Usage
    // set up configuration
    conf := &loader.Config{
    Build: &build.Context{…},
    ParserMode: parser.ParseComments,
    TypeChecker: types.Config{…},
    }
    // specify package path & files
    conf.CreateFromFilenames(path, files...)
    // load the program
    program, _ := conf.Load()
    // access packages
    for _, pkg := range program.Created {...}
    

    View Slide

28. go/build
    - Used by loader.Config
    - Handles build information of Go source code files
    - Know target files under some GOOS, -tag, etc.
    • Convert import path to source code directory
    • Filtering out e.g. *_test.go or *_windows.go’s
    

    View Slide

29. go/build
    - Entrypoint: *build.Context
    • Set GOOS or BuildTags and call ImportDir()
    • Can override ReadDir, OpenFile
    ‣ Can Build source code on virtual filesystems
    

    View Slide

30. ! sourcegraph/go-vcs
    - Virtual source tree at revisions on git/mercurial
    - Implements x/tools/godoc/vfs.FileSystem
    • Open(string) (ReadSeekCloser, error)
    • ReadDir(string) ([]os.FileInfo, error)
    

    View Slide

31. Inside gompatible
    - Assign sourcegraph/go-vcs's implementation to
    go/build.Context
    - Use go/loader with the build context to obtain type
    information of two packages
    - Compare APIs using compatibility rules
    

    View Slide

32. TODO
    - Support for:
    • Vendoring
    • Mercurial (hg)
    • More complicated changes
    - Web interface
    

    View Slide

33. Diff’ing Go library API changes
    https://motemen.github.io/
    

    View Slide