How to manage tool dependencies in Go

How to manage tool dependencies in Go

talked on golang.tokyo #20
https://golangtokyo.connpass.com/event/111077/

talked about izumin5210/gex https://github.com/izumin5210/gex

9eed44f137609e6ce3b6f1e14f80b9e1?s=128

Masayuki Izumi

December 18, 2018
Tweet

Transcript

  1. How to manage tool dependencies golang.tokyo #20 - @izumi5210

  2. @izumin5210 Application Engineer, Wantedly People Wantedly, Inc.

  3. tool dependencies ? Executable tools that installed
 via `go get`

    e.g.mockgen, protoc-gen-grpc-gateway, ...
  4. Why should we manage versions of tools?

  5. Why should we manage versions of tools? ⬇ Behaviors of

    tools may change when developers install them via `go get` (e.g. an output file format of code generator)
  6. mockgen@v1.0.1 mockgen@v1.1.0 mockgen@v1.2.0 mockgen@bd3c8e8 mockgen@v1.1.1 mockgen@v1.0.0 mockgen@ddf6e33 Why should we

    manage versions of tools? ⬇ Behaviors of tools may change when developers install them via `go get` (e.g. an output file format of code generator)
  7. Mock implementation that I wanna create Mocks that updated unexpectedly

    Because I use mockgen@v1.2.0 but they are generated by mockgen@v1.0.1 If you do not manage tool dependencies ...
  8. How about existing package managers? (e.g. dep ,go mod) `go

    build` is the out of scope of their responsibility dep document says* if you need the tool to be installed you still run the following (manually or from a `Makefile`) after each `dep ensure`
 ```
 cd vendor/pkg/to/install
 go install .
 ``` * https://golang.github.io/dep/docs/Gopkg.toml.html#required
  9. How about existing package managers? (e.g. dep ,go mod) #

    Makefile DEP_COMMANDS := \ vendor/github.com/golang/mock/mockgen .PHONY: dep dep: Gopkg.toml Gopkg.lock @dep ensure -v -vendor-only pkgs="$(DEP_COMMANDS)"; \ for pkg in $$pkgs; do \ cd $$pkg; \ GOBIN="$(shell pwd)/bin" go install .; \ cd -; \ done # Gopkg.toml required = [ "github.com/golang/mock/mockgen", ]
  10. https://github.com/golang/go/issues/25922 cmd/go: clarify best practice for tool dependencies #25922 ·

    golang/go
  11. cmd/go: clarify best practice for tool dependencies #25922 · golang/go

    // tools.go // +build tools package tools import ( _ "golang.org/x/tools/cmd/stringer" ) // go.mod module example.com/hello require golang.org/x/tools v0.0.0-20180813205110-a434f64ace81 // indirect go get export GOBIN=$(pwd)/bin go install golang.org/x/tools/cmd/stringer
  12. cmd/go: clarify best practice for tool dependencies #25922 · golang/go

    Other 3rd-party tools (e.g. virtualgo, retool, ...) need to
 introduce new management mechanisms "tools.go" method can use existing dependency management mechanisms (e.g. dep, go mod)
  13. cmd/go: clarify best practice for tool dependencies #25922 · golang/go

    // tools.go // +build tools package tools import ( _ "golang.org/x/tools/cmd/stringer" ) // go.mod module example.com/hello require golang.org/x/tools v0.0.0-20180813205110-a434f64ace81 // indirect go get export GOBIN=$(pwd)/bin go install golang.org/x/tools/cmd/stringer Best practice?
  14. cmd/go: clarify best practice for tool dependencies #25922 · golang/go

    // tools.go // +build tools package tools import ( _ "golang.org/x/tools/cmd/stringer" ) // go.mod module example.com/hello require golang.org/x/tools v0.0.0-20180813205110-a434f64ace81 // indirect go get export GOBIN=$(pwd)/bin go install golang.org/x/tools/cmd/stringer We want to also automate this step!
  15. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen

    $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text...
  16. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen

    $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... Install mockgen via gex - gex manages dependencies version using `dep` or `go mod` - So `gex --add ...` runs `go get ...` or `dep ensure -add ...` internally - Build mockgen and output to `$PWD/bin`
  17. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen

    $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... `tools.go` will generate by `gex`
  18. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen

    $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... `go.mod` will modify because `gex` run `go get`
  19. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen

    $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... Run versioned mockgen - Execute `$PWD/bin/mockgen` - If `$PWD/bin/mockgen` does not exist, run `go build -o mockgen ...` automatically
  20. gex: manage tools with `tools.go` github.com/izumin5210/gex Does not introduce new

    mechanism to manage tool deps. - `gex` is very thin wrapper command of `dep` or `go mod` - For example, if `go mod` was to support to manage tools officially,
 we would be able to move from `gex` easily Only 2 commands that you use: `--add` and `--build` (I believe) tools management mechanism will be introduced
 into `golang/go` officially - When the time coms, we graduate from`gex`
  21. How to manage tool dependencies golang.tokyo #20 - @izumi5210 Let's

    manage tools with `gex` efficiently, and happy hacking with Go!
  22. https://www.wantedly.com/projects/268180