Go modules

Transcript

1. Go modules
   WARSAW, MAY 22 2019
   Oleg Kovalov
   Allegro
   Twitter:
   oleg_kovalov
   Github: cristaloleg
   

   View Slide

2. Me
   - Gopher for ~4 years
   - Open-source contributor
   - Engineer at allegro.pl core team
   Twitter: @oleg_kovalov
   Github: @cristaloleg
   

   View Slide

3. Long story short
   Image credit: ©Istockphoto/JamesBrey
   time
   

   View Slide

4. - GOPATH
   Pre-modules era (2012-2018)
   

   View Slide

5. - GOPATH
   - godep
   - gopkg.in
   - glide
   Pre-modules era (2012-2018)
   

   View Slide

6. - GOPATH
   - godep
   - gopkg.in
   - glide
   - vendor dir
   Pre-modules era (2012-2018)
   

   View Slide

7. - GOPATH
   - godep
   - gopkg.in
   - glide
   - vendor dir
   - dep
   Pre-modules era (2012-2018)
   

   View Slide

8. - GOPATH
   - godep
   - gopkg.in
   - glide
   - vendor dir
   - dep
   - ...
   Pre-modules era (2012-2018)
   

   View Slide

9. Go modules
   

   View Slide

10. - Modules
    Go modules
    

    View Slide

11. - Modules
    - Semantic versioning
    Go modules
    

    View Slide

12. - Modules
    - Semantic versioning
    - Import compatibility rule
    Go modules
    

    View Slide

13. - Modules
    - Semantic versioning
    - Import compatibility rule
    - go.mod
    Go modules
    

    View Slide

14. - A module is a collection of Go packages that
    are versioned as a single unit
    What is a module?
    

    View Slide

15. - A module is a collection of Go packages that
    are versioned as a single unit
    - (often) a single version-control repository is a
    single module
    What is a module?
    

    View Slide

16. What is a module?
    - A module is a tree/directory of Go source files
    with a go.mod file in the root directory
    

    View Slide

17. What is a module?
    - A module is a tree/directory of Go source files
    with a go.mod file in the root directory
    - Can live outside of a GOPATH
    

    View Slide

18. Semantic versioning (http://semver.org)
    

    View Slide

19. If an old and a new package have the same
    import path, the new package must be
    backwards-compatible with the old package.
    Import compatibility rule
    

    View Slide

20. GO111MODULE=auto go build ./…
    GO111MODULE
    

    View Slide

21. GO111MODULE=auto go build ./…
    - Inside GOPATH defaults to old 1.10 behavior
    (ignoring modules)
    GO111MODULE
    

    View Slide

22. GO111MODULE
    GO111MODULE=auto go build ./…
    - Inside GOPATH defaults to old 1.10 behavior
    (ignoring modules)
    - Outside GOPATH while inside a file tree with a
    go.mod = defaults to modules behavior
    

    View Slide

23. - auto(default)
    - on
    - off
    GO111MODULE
    

    View Slide

24. GO111MODULE
    - auto(default)
    - on
    - off
    $ export GO111MODULE=on
    $ go build ./...
    

    View Slide

25. Changes to go get
    

    View Slide

26. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    Changes to go get
    

    View Slide

27. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/mux@v1.6.2
    Changes to go get
    

    View Slide

28. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/mux@v1.6.2
    # records v1.6.2
    $ go get github.com/gorilla/mux@e3702bed2
    Changes to go get
    

    View Slide

29. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/mux@v1.6.2
    # records v1.6.2
    $ go get github.com/gorilla/mux@e3702bed2
    # records v0.0.0-20180517173623-c85619274f5d
    $ go get github.com/gorilla/mux@c856192
    Changes to go get
    

    View Slide

30. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/mux@v1.6.2
    # records v1.6.2
    $ go get github.com/gorilla/mux@e3702bed2
    # records v0.0.0-20180517173623-c85619274f5d
    $ go get github.com/gorilla/mux@c856192
    # records current meaning of master
    $ go get github.com/gorilla/mux@master
    Changes to go get
    

    View Slide

31. $ mkdir /mygo/hello
    $ cd /mygo/hello
    $ go mod init github.com/golang/hello
    go: creating new go.mod: module github.com/golang/hello
    $ ls
    go.mod
    $ cat go.mod
    module github.com/golang/hello
    Let’s start
    

    View Slide

32. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    Convert to modules
    

    View Slide

33. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    $ go mod init
    go: creating new go.mod: module github.com/go-gitea/gitea
    go: copying requirements from Gopkg.lock
    Convert to modules
    

    View Slide

34. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    $ go mod init
    go: creating new go.mod: module github.com/go-gitea/gitea
    go: copying requirements from Gopkg.lock
    $ go mod tidy
    go: extracting github.com/PuerkitoBio/goquery v1.5.0
    go: extracting github.com/blevesearch/bleve v0.7.0
    go: finding github.com/go-macaron/session/redis latest
    go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
    go: finding golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f
    go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
    go: finding golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
    go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
    go: downloading github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434
    Convert to modules
    

    View Slide

35. How to read go.mod
    

    View Slide

36. module github.com/hashicorp/consul
    ...
    How to read go.mod
    

    View Slide

37. module github.com/hashicorp/consul
    go 1.12
    ...
    How to read go.mod
    

    View Slide

38. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    ...
    How to read go.mod
    

    View Slide

39. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    ...
    How to read go.mod
    

    View Slide

40. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    ...
    How to read go.mod
    

    View Slide

41. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    ...
    How to read go.mod
    

    View Slide

42. How to read go.mod
    module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    replace github.com/hashicorp/consul/api => ./api
    replace github.com/hashicorp/consul/sdk => ./sdk
    ...
    

    View Slide

43. How to read go.mod
    module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    replace github.com/hashicorp/consul/api => ./api
    replace github.com/hashicorp/consul/sdk => ./sdk
    exclude github.com/not-a-hacker/supergeil v1270.0.1
    

    View Slide

44. How to read go.sum
    

    View Slide

45. How to read go.sum
    

    View Slide

46. cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
    cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
    How to read go.sum
    

    View Slide

47. cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
    cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible
    h1:gr1qKY/Ll72VjFTZmaBwRK1yQHAxCnV25ekOKroc9ws=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible/go.mod
    h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
    How to read go.sum
    

    View Slide

48. cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
    cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible
    h1:gr1qKY/Ll72VjFTZmaBwRK1yQHAxCnV25ekOKroc9ws=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible/go.mod
    h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
    // you might want to use go mod verify
    How to read go.sum
    

    View Slide

49. Usage:
    go mod [arguments]
    The commands are:
    download download modules to local cache
    edit edit go.mod from tools or scripts
    fix make go.mod semantically consistent
    graph print module requirement graph
    init initialize new module in current directory
    tidy add missing and remove unused modules
    vendor make vendored copy of dependencies
    verify verify dependencies have expected content
    why explain why packages or modules are needed
    go mod commands
    

    View Slide

50. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    go mod why
    

    View Slide

51. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    # set a env var and check why do we use this dependency
    $ export GO111MODULE=on
    go mod why
    

    View Slide

52. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    # set a env var and check why do we use this dependency
    $ export GO111MODULE=on
    $ go mod why github.com/RoaringBitmap/roaring
    # github.com/RoaringBitmap/roaring
    code.gitea.io/gitea/modules/indexer
    github.com/blevesearch/bleve
    github.com/blevesearch/bleve/index/scorch
    github.com/RoaringBitmap/roaring
    go mod why
    

    View Slide

53. FROM golang
    ENV GO111MODULE=on
    Dockerfile
    

    View Slide

54. FROM golang
    ENV GO111MODULE=on
    WORKDIR /app
    COPY go.mod .
    COPY go.sum .
    Dockerfile
    

    View Slide

55. FROM golang
    ENV GO111MODULE=on
    WORKDIR /app
    COPY go.mod .
    COPY go.sum .
    RUN go mod download
    COPY . .
    Dockerfile
    

    View Slide

56. Dockerfile
    FROM golang
    ENV GO111MODULE=on
    WORKDIR /app
    COPY go.mod .
    COPY go.sum .
    RUN go mod download
    COPY . .
    RUN CGO_ENABLED=0 GOOS=linux go build -a -o app.exec ./...
    EXPOSE 8000
    ENTRYPOINT ["./app.exec"]
    

    View Slide

57. Use local changes
    

    View Slide

58. Use local changes
    - Сhange in go.mod manually
    

    View Slide

59. Use local changes
    - Сhange in go.mod manually
    - go mod edit -replace ‘foo.com/bar=../bar’
    

    View Slide

60. Use local changes
    - Сhange in go.mod manually
    - go mod edit -replace ‘foo.com/bar=../bar’
    - or use awesome gohack tool by Roger Peppe
    

    View Slide

61. # install
    $ go get github.com/rogpeppe/gohack
    gohack by Roger Peppe
    

    View Slide

62. # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add: replace example.com/foo/bar /home/rog/gohack/example.com/foo/bar
    gohack by Roger Peppe
    

    View Slide

63. # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add: replace example.com/foo/bar /home/rog/gohack/example.com/foo/bar
    $ cat go.mod
    replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar
    gohack by Roger Peppe
    

    View Slide

64. gohack by Roger Peppe
    # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add: replace example.com/foo/bar /home/rog/gohack/example.com/foo/bar
    $ cat go.mod
    replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar
    # to remove specific replace:
    $ gohack undo example.com/foo/bar
    # to remove all replaces:
    $ gohack undo
    

    View Slide

65. Go Proxy
    

    View Slide

66. Go Proxy
    https://proxy.golang.org
    https://github.com/gomods/athens
    - export GOPROXY=my.proxy.com
    - go get github.com/foo/bar
    

    View Slide

67. Future
    

    View Slide

68. - Enabled by default
    Future
    

    View Slide

69. - Enabled by default
    - More robust ecosystem
    Future
    

    View Slide

70. - Enabled by default
    - More robust ecosystem
    - More secure
    Future
    

    View Slide

71. - Enabled by default
    - More robust ecosystem
    - More secure
    - More tools understand modules
    Future
    

    View Slide

72. - Enabled by default
    - More robust ecosystem
    - More secure
    - More tools understand modules
    - No vendor (perhaps?)
    Future
    

    View Slide

73. - Enabled by default
    - More robust ecosystem
    - More secure
    - More tools understand modules
    - No vendor (perhaps?)
    - Less “but it works on my machine!”
    Future
    

    View Slide

74. Why package management is hard?
    

    View Slide

75. That’s all folks
    Thank you
    Questions?
    Twitter: @oleg_kovalov
    Github: @cristaloleg
    

    View Slide