$30 off During Our Annual Pro Sale. View Details »

Diff'ing Go library API changes

motemen
December 06, 2015

Diff'ing Go library API changes

motemen

December 06, 2015
Tweet

More Decks by motemen

Other Decks in Programming

Transcript

  1. Diff’ing Go library

    API changes
    Go Conference 2015 Winter

    @motemen, motemen.github.io

    View Slide

  2. About me
    - @motemen on GitHub/Twitter/Hatena
    - Chief engineer at Hatena
    - Loves: Go, git
    - Wrote: ghq, gore and more

    View Slide

  3. Diff’ing Go library API changes
    - Introduction of a tool recently I’ve written: gompat
    - Motivation, overview & Implementation

    View Slide

  4. Motivation

    View Slide

  5. “go get -u” and pray
    - Go packages can be obtained as “latest”
    - Dependent libraries may introduce breaking
    changes without your notice
    - Could use “vendoring” or “revision locking”

    View Slide

  6. How can we:
    - Know dependent libraries’ API changes?
    - Know to what version to update them?
    - Know if we are introducing breaking changes

    to our package?

    View Slide

  7. API changes overview

    View Slide

  8. API entities
    - Package-level and published:
    • Functions and methods (func)
    • Types (type)
    • Variables (var), Constants (const)
    - May be changed by the package update

    View Slide

  9. Changes: the obvious
    = Unchanged
    + Added
    - Deleted

    View Slide

  10. ! Breaking
    - Named API entities with their interface changed
    • Users have to modify API usages
    - … except for the “compatible” case (later)

    View Slide

  11. Breaking: struct type example
    - Struct field deleted
    - Struct field type changed
    // before
    type BreakingT struct {
    Foo string
    Bar bool
    }
    // after
    type BreakingT struct {
    Bar int
    }

    View Slide

  12. Breaking: function example
    - Parameter types, number changed
    - Return value types, number changed
    // before
    func BreakingF(n, m int) int
    // after
    func BreakingF(b bool) (int, error)

    View Slide

  13. * Compatible
    - API signatures changed
    - Users don’t need their API usages

    View Slide

  14. Compatible: function example
    - Function changed to variadic
    // before
    func CompatibleF(n, m int)
    // after
    func CompatibleF(n, m int, opts ...string)

    View Slide

  15. Compatible: struct type example
    - Public field added
    * Breaking for T{x, y, z}
    initializers
    // before
    type CompatibleT struct {
    Foo int
    }
    // after
    type CompatibleT struct {
    Foo int
    Bar bool
    }

    View Slide

  16. Compatible: other cases
    - Value changed from const to var
    - Function parameter type became more general
    • eg. *bytes.Buffer to io.Reader

    View Slide

  17. API Changes

    = Unchanged
    + Added
    * Compatible

    ! Breaking
    - Deleted

    View Slide

  18. Enter Gompatible

    View Slide

  19. ! motemen/gompatible
    - Compares API of a package under two revisions
    - Produces change kinds per API entities
    • Added / Removed / Breaking / Compatible

    View Slide

  20. cmd/gompat
    - Main entrypoint of gompatible
    - Usage:
    • gompat rev..rev package

    View Slide

  21. Inside Gompatible

    View Slide

  22. Gompatible tasks
    - Mostly achieved by std packages:
    • Load packages at revisions
    • Load API signatures from packages
    • Compare type information

    View Slide

  23. go/types
    - Formerly x/tools/go/types
    - Generates type information from source ASTs
    - Entrypoint: *types.Package
    • pkg.Scope().Lookup(“main”) → *types.Func
    • fun.Type() → *types.Signature
    • sig.Params(), sig.Results() → *types.Tuple

    View Slide

  24. go/types
    - Can query on types
    • types.Implements(Type, *Interface)
    • types.AssignableTo(Type, Type)

    View Slide

  25. go/types
    - It’s a lot of work (and maybe chore) to:
    • Convert import path to source code directory
    • Filtering out e.g. *_test.go or *_windows.go’s
    • Parse source files and combine them as package
    • Then process typings by go/types

    View Slide

  26. x/tools/go/loader
    - Package path and filenames to *loader.Program
    • Syntactically parsed and typed packages
    - Built for command line tools
    • loader.Config.FromArgs([]string, bool)
    • Users: x/tools/cmd/…
    - (Can see loader.FromArgsUsage inside oracle -help)

    View Slide

  27. x/tools/go/loader: Usage
    // set up configuration
    conf := &loader.Config{
    Build: &build.Context{…},
    ParserMode: parser.ParseComments,
    TypeChecker: types.Config{…},
    }
    // specify package path & files
    conf.CreateFromFilenames(path, files...)
    // load the program
    program, _ := conf.Load()
    // access packages
    for _, pkg := range program.Created {...}

    View Slide

  28. go/build
    - Used by loader.Config
    - Handles build information of Go source code files
    - Know target files under some GOOS, -tag, etc.
    • Convert import path to source code directory
    • Filtering out e.g. *_test.go or *_windows.go’s

    View Slide

  29. go/build
    - Entrypoint: *build.Context
    • Set GOOS or BuildTags and call ImportDir()
    • Can override ReadDir, OpenFile
    ‣ Can Build source code on virtual filesystems

    View Slide

  30. ! sourcegraph/go-vcs
    - Virtual source tree at revisions on git/mercurial
    - Implements x/tools/godoc/vfs.FileSystem
    • Open(string) (ReadSeekCloser, error)
    • ReadDir(string) ([]os.FileInfo, error)

    View Slide

  31. Inside gompatible
    - Assign sourcegraph/go-vcs's implementation to
    go/build.Context
    - Use go/loader with the build context to obtain type
    information of two packages
    - Compare APIs using compatibility rules

    View Slide

  32. TODO
    - Support for:
    • Vendoring
    • Mercurial (hg)
    • More complicated changes
    - Web interface

    View Slide

  33. Diff’ing Go library API changes
    https://motemen.github.io/

    View Slide