How to manage tool dependencies in Go

talked on golang.tokyo #20
https://golangtokyo.connpass.com/event/111077/

talked about izumin5210/gex https://github.com/izumin5210/gex

Transcript

1. How to manage tool dependencies golang.tokyo #20 - @izumi5210
2. @izumin5210 Application Engineer, Wantedly People Wantedly, Inc.
3. tool dependencies ? Executable tools that installed
 via `go get` e.g.mockgen, protoc-gen-grpc-gateway, ...
4. Why should we manage versions of tools?
5. Why should we manage versions of tools? ⬇ Behaviors of tools may change when developers install them via `go get` (e.g. an output file format of code generator)
6. mockgen@v1.0.1 mockgen@v1.1.0 mockgen@v1.2.0 mockgen@bd3c8e8 mockgen@v1.1.1 mockgen@v1.0.0 mockgen@ddf6e33 Why should we manage versions of tools? ⬇ Behaviors of tools may change when developers install them via `go get` (e.g. an output file format of code generator)
7. Mock implementation that I wanna create Mocks that updated unexpectedly Because I use mockgen@v1.2.0 but they are generated by mockgen@v1.0.1 If you do not manage tool dependencies ...
8. How about existing package managers? (e.g. dep ,go mod) `go build` is the out of scope of their responsibility dep document says* if you need the tool to be installed you still run the following (manually or from a `Makefile`) after each `dep ensure`
 ```
 cd vendor/pkg/to/install
 go install .
 ``` * https://golang.github.io/dep/docs/Gopkg.toml.html#required
9. How about existing package managers? (e.g. dep ,go mod) # Makefile DEP_COMMANDS := \ vendor/github.com/golang/mock/mockgen .PHONY: dep dep: Gopkg.toml Gopkg.lock @dep ensure -v -vendor-only pkgs="$(DEP_COMMANDS)"; \ for pkg in $$pkgs; do \ cd $$pkg; \ GOBIN="$(shell pwd)/bin" go install .; \ cd -; \ done # Gopkg.toml required = [ "github.com/golang/mock/mockgen", ]
10. https://github.com/golang/go/issues/25922 cmd/go: clarify best practice for tool dependencies #25922 · golang/go
11. cmd/go: clarify best practice for tool dependencies #25922 · golang/go // tools.go // +build tools package tools import ( _ "golang.org/x/tools/cmd/stringer" ) // go.mod module example.com/hello require golang.org/x/tools v0.0.0-20180813205110-a434f64ace81 // indirect go get export GOBIN=$(pwd)/bin go install golang.org/x/tools/cmd/stringer
12. cmd/go: clarify best practice for tool dependencies #25922 · golang/go Other 3rd-party tools (e.g. virtualgo, retool, ...) need to
 introduce new management mechanisms "tools.go" method can use existing dependency management mechanisms (e.g. dep, go mod)
13. cmd/go: clarify best practice for tool dependencies #25922 · golang/go // tools.go // +build tools package tools import ( _ "golang.org/x/tools/cmd/stringer" ) // go.mod module example.com/hello require golang.org/x/tools v0.0.0-20180813205110-a434f64ace81 // indirect go get export GOBIN=$(pwd)/bin go install golang.org/x/tools/cmd/stringer Best practice?
14. cmd/go: clarify best practice for tool dependencies #25922 · golang/go // tools.go // +build tools package tools import ( _ "golang.org/x/tools/cmd/stringer" ) // go.mod module example.com/hello require golang.org/x/tools v0.0.0-20180813205110-a434f64ace81 // indirect go get export GOBIN=$(pwd)/bin go install golang.org/x/tools/cmd/stringer We want to also automate this step!
15. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text...
16. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... Install mockgen via gex - gex manages dependencies version using `dep` or `go mod` - So `gex --add ...` runs `go get ...` or `dep ensure -add ...` internally - Build mockgen and output to `$PWD/bin`
17. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... `tools.go` will generate by `gex`
18. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... `go.mod` will modify because `gex` run `go get`
19. gex: manage tools with `tools.go` github.com/izumin5210/gex $ gex --add github.com/golang/mock/mockgen $ cat tools.go // Code generated by github.com/izumin5210/gex. DO NOT EDIT. // +build tools package tools // tool dependencies import ( _ "github.com/golang/mock/mockgen" ) $ cat go.mod | grep mock github.com/golang/mock v1.1.1 // indirect $ gex mockgen # prints mockgen's help text... Run versioned mockgen - Execute `$PWD/bin/mockgen` - If `$PWD/bin/mockgen` does not exist, run `go build -o mockgen ...` automatically
20. gex: manage tools with `tools.go` github.com/izumin5210/gex Does not introduce new mechanism to manage tool deps. - `gex` is very thin wrapper command of `dep` or `go mod` - For example, if `go mod` was to support to manage tools officially,
 we would be able to move from `gex` easily Only 2 commands that you use: `--add` and `--build` (I believe) tools management mechanism will be introduced
 into `golang/go` officially - When the time coms, we graduate from`gex`
21. How to manage tool dependencies golang.tokyo #20 - @izumi5210 Let's manage tools with `gex` efficiently, and happy hacking with Go!
22. https://www.wantedly.com/projects/268180