The usage and dependency resolving mechanism of go module

Things about Go module and Semantic Versioning.

Transcript

1. The usage and dependency resolving mechanism of Go module Jacky Hung@Emotibot Technologies Ltd. DigitalOcean Hsichu x Golang TW Meetup / Sept. 7, 2019
2. Outline ▷ History of Go package management ▷ Problems with Vendoring ▷ Go module ▷ Semantic Import Versioning (SIV) ▷ Module-aware go commands ▷ Dependency resolving mechanism
3. History of Go Package Management
4. History of Go Package Management Vendor • 2015: vendor added, off by default • 2016: vendor on by default • 2017: vendor always on Timeline • Before: GOPATH + go get • 2013: Godep • 2014: glide, gopkg.in • 2015: govendor • 2018: vgo/go module Tools • Godep: originally about reproducibility, restore/save your GOPATH • gopkgin.in: putting version information in the URL • gom, glide: modeled after tools in other languages • the vendor directory • go module with MVS https://about.sourcegraph.com/go/the-new-era-of-go-package-management
5. Manual: GOPATH + go get ▷ Set GOPATH ▷ go get all packages one by one... Orz
6. Problems with go get ▷ go get doesn’t upgrade indirect packages if that already in local. ▷ got get -u upgrades all indirect packages to the latest version. ▷ That may broke go build and cause your program unexpected behavior! https://research.swtch.com/vgo-mvs
7. Dependency Management Tools for Go
8. godep ▷ $YOUR_PROJ/Godeps/_workspace/ as $GOPATH ▷ Godeps.json ▷ $YOUR_PROJ/vendor/ (for new version of godep) save list and copy dependencies into Godeps go run the go tool with saved dependencies get download and install packages with specified dependencies path print GOPATH for dependency code restore check out listed dependency versions in GOPATH update update selected packages or the go version diff shows the diff between current and previously saved set of dependencies version show version info
9. gopkg.in import "gopkg.in/pkg.v3" → github.com/go-pkg/pkg (branch/tag v3, v3.N, or v3.N.M) import "gopkg.in/user/pkg.v3" → github.com/user/pkg (branch/tag v3, v3.N, or v3.N.M) For clarity, assuming a repository containing the following tags or branches: • v1 • v2.0 • v2.0.3 • v2.1.2 • v3 • v3.0 The following selectors would be resolved as indicated: • pkg.v1 → tag or branch v1 • pkg.v2 → tag or branch v2.1.2 • pkg.v3 → tag or branch v3.0 http://labix.org/gopkg.in
10. glide ▷ Store packages in $YOUR_PROJ/_vendor/ ▷ glide.yaml ▷ glide.lock Older versions use _vendor/ https://github.com/Masterminds/glide
11. gb ▷ Save all packages in $YOUR_PROJ/vendor/src/ ▷ 'gb build' instead of 'go build' https://tabalt.net/blog/golang-package-dependency-management-tool-gb/
12. Vendoring ▷ A GOPATH-like directory for individual projects ▷ go build will search packages in vendor directory before the real GOPATH. ▷ Go 1.5 introduced as experiment ▷ Go 1.6 default on ▷ Go 1.7 always on https://blog.wu-boy.com/2016/05/package-management-for-golang-glide/
13. Vendoring based tools ▷ govendor ▷ gsv ▷ gom ▷ dep -> official? ▷ ... https://github.com/golang/go/wiki/PackageManagementTools
14. Problems with Vendoring ▷ Must be in GOPATH ▷ Nested vendor directories ▷ No versioning ▷ Duplicate code and wasting disk space ▷ ...
15. Debate about dep and vgo (go module) ▷ https://news.ycombinator.com/item?id=17628311 ▷ https://www.reddit.com/r/golang/comments/9274zi/ ▷ https://sdboyer.io/blog/vgo-and-dep/ ▷ https://zhuanlan.zhihu.com/p/41627929
16. Go module
17. vgo/go module ▷ Try to reproduce the original build of packages ▷ Based on Semantic Versioning (semver) ▷ vgo was merged into Go 1.11 ▷ high-fidelity builds ▷ low-fidelity builds ▷ go.mod ▷ go.sum https://research.swtch.com/vgo-mvs https://research.swtch.com/vgo-tour
18. Semantic Versioning https://research.swtch.com/vgo-import
19. go.mod
20. go.sum
21. Basic usage of go module ▷ go mod init $MODULE_NAME (module name e.g. myexample/gomod) ▷ go build ▷ go list -m all ▷ go tidy ▷ go get -u ▷ go get -u=patch ▷ go get github.com/user/testmod@v1.0.1
22. Commands of go module download download modules to local cache edit edit go.mod from tools or scripts graph print module requirement graph init initialize new module in current directory tidy add missing and remove unused modules vendor make vendored copy of dependencies verify verify dependencies have expected content why explain why packages or modules are needed
23. Vendoring with go module ▷ $ go mod vendor ◦ Store all dependency modules to vendor/ ▷ $ go build -mod vendor ◦ Build with vendor/ ◦ It’s no more default behavior of go build https://roberto.selbach.ca/intro-to-go-modules/
24. What is '/ / indirect' in the go.mod? ▷ It’s indirect dependent module ▷ It only shows when you change the default version of the module like upgrade or downgrade ▷ Override go.mod settings of indirect modules
25. Remove all versions of a module ▷ go get myexample/gomod@none
26. Clean downloaded modules ▷ go clean --modcache
27. Import local modules
28. Old git version problem Go module needs some feature that old git versions (before v1.9.1) don’t have. https://pureage.info/post/bad-feeling-about-go-cmd/
29. module-aware go commands ▷ get ◦ src/cmd/go/internal/get/get.go ◦ src/cmd/go/internal/modget/get.go ◦ $GOPATH/src => $GOPATH/pkg/mod ▷ run ▷ build ▷ test
30. Semantic Import Versioning
31. Semantic Import Versioning (SIV) ▷ Different major versions may not compatible ▷ Treat different major versions of same module as different modules ▷ They can be imported at the same time ▷ import "github.com/russross/blackfriday" => v1.x.x ▷ import blackfridayV2 "github.com/russross/blackfriday/v2" => v2.x.x ▷ import blackfridayV3 "github.com/russross/blackfriday/v3" => v3.x.x https://research.swtch.com/vgo-import
32. Using Different Major Versions in go.mod module mod require ( github.com/russross/blackfriday v1.0.1 github.com/russross/blackfriday/v2 v2.2.0 github.com/russross/blackfriday/v3 v3.1.1 ) https://roberto.selbach.ca/intro-to-go-modules/
33. Default Major Version ▷ Select latest version of v0 or v1 when there is no go.mod or no require in go.mod
34. Dependency Resolving Mechanism
35. Minimal Version Selection max of the minimums https://research.swtch.com/vgo-mvs https://medium.com/@aman.sardana/d5f96b490774
36. Upgrade All Modules go get -u
37. Upgrade One Module go get -u pkg@newver go get -u pkg@latest e.g. C 1.2 => C 1.3
38. Downgrade One Module go get -u pkg@oldver e.g. D 1.4 => D 1.2
39. Q & A