Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Go modules

Go modules

Oleg Kovalov

May 22, 2019
Tweet

More Decks by Oleg Kovalov

Other Decks in Programming

Transcript

  1. Go modules
    WARSAW, MAY 22 2019
    Oleg Kovalov
    Allegro
    Twitter:
    oleg_kovalov
    Github: cristaloleg

    View Slide

  2. Me
    - Gopher for ~4 years
    - Open-source contributor
    - Engineer at allegro.pl core team
    Twitter: @oleg_kovalov
    Github: @cristaloleg

    View Slide

  3. Long story short
    Image credit: ©Istockphoto/JamesBrey
    time

    View Slide

  4. - GOPATH
    Pre-modules era (2012-2018)

    View Slide

  5. - GOPATH
    - godep
    - gopkg.in
    - glide
    Pre-modules era (2012-2018)

    View Slide

  6. - GOPATH
    - godep
    - gopkg.in
    - glide
    - vendor dir
    Pre-modules era (2012-2018)

    View Slide

  7. - GOPATH
    - godep
    - gopkg.in
    - glide
    - vendor dir
    - dep
    Pre-modules era (2012-2018)

    View Slide

  8. - GOPATH
    - godep
    - gopkg.in
    - glide
    - vendor dir
    - dep
    - ...
    Pre-modules era (2012-2018)

    View Slide

  9. Go modules

    View Slide

  10. - Modules
    Go modules

    View Slide

  11. - Modules
    - Semantic versioning
    Go modules

    View Slide

  12. - Modules
    - Semantic versioning
    - Import compatibility rule
    Go modules

    View Slide

  13. - Modules
    - Semantic versioning
    - Import compatibility rule
    - go.mod
    Go modules

    View Slide

  14. - A module is a collection of Go packages that
    are versioned as a single unit
    What is a module?

    View Slide

  15. - A module is a collection of Go packages that
    are versioned as a single unit
    - (often) a single version-control repository is a
    single module
    What is a module?

    View Slide

  16. What is a module?
    - A module is a tree/directory of Go source files
    with a go.mod file in the root directory

    View Slide

  17. What is a module?
    - A module is a tree/directory of Go source files
    with a go.mod file in the root directory
    - Can live outside of a GOPATH

    View Slide

  18. Semantic versioning (http://semver.org)

    View Slide

  19. If an old and a new package have the same
    import path, the new package must be
    backwards-compatible with the old package.
    Import compatibility rule

    View Slide

  20. GO111MODULE=auto go build ./…
    GO111MODULE

    View Slide

  21. GO111MODULE=auto go build ./…
    - Inside GOPATH defaults to old 1.10 behavior
    (ignoring modules)
    GO111MODULE

    View Slide

  22. GO111MODULE
    GO111MODULE=auto go build ./…
    - Inside GOPATH defaults to old 1.10 behavior
    (ignoring modules)
    - Outside GOPATH while inside a file tree with a
    go.mod = defaults to modules behavior

    View Slide

  23. - auto(default)
    - on
    - off
    GO111MODULE

    View Slide

  24. GO111MODULE
    - auto(default)
    - on
    - off
    $ export GO111MODULE=on
    $ go build ./...

    View Slide

  25. Changes to go get

    View Slide

  26. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    Changes to go get

    View Slide

  27. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/[email protected]
    Changes to go get

    View Slide

  28. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/[email protected]
    # records v1.6.2
    $ go get github.com/gorilla/mux@e3702bed2
    Changes to go get

    View Slide

  29. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/[email protected]
    # records v1.6.2
    $ go get github.com/gorilla/mux@e3702bed2
    # records v0.0.0-20180517173623-c85619274f5d
    $ go get github.com/gorilla/mux@c856192
    Changes to go get

    View Slide

  30. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/[email protected]
    # records v1.6.2
    $ go get github.com/gorilla/mux@e3702bed2
    # records v0.0.0-20180517173623-c85619274f5d
    $ go get github.com/gorilla/mux@c856192
    # records current meaning of master
    $ go get github.com/gorilla/mux@master
    Changes to go get

    View Slide

  31. $ mkdir /mygo/hello
    $ cd /mygo/hello
    $ go mod init github.com/golang/hello
    go: creating new go.mod: module github.com/golang/hello
    $ ls
    go.mod
    $ cat go.mod
    module github.com/golang/hello
    Let’s start

    View Slide

  32. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    Convert to modules

    View Slide

  33. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    $ go mod init
    go: creating new go.mod: module github.com/go-gitea/gitea
    go: copying requirements from Gopkg.lock
    Convert to modules

    View Slide

  34. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    $ go mod init
    go: creating new go.mod: module github.com/go-gitea/gitea
    go: copying requirements from Gopkg.lock
    $ go mod tidy
    go: extracting github.com/PuerkitoBio/goquery v1.5.0
    go: extracting github.com/blevesearch/bleve v0.7.0
    go: finding github.com/go-macaron/session/redis latest
    go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
    go: finding golang.org/x/lint v0.0.0-20190301231843-5614ed5bae6f
    go: finding github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b
    go: finding golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
    go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
    go: downloading github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434
    Convert to modules

    View Slide

  35. How to read go.mod

    View Slide

  36. module github.com/hashicorp/consul
    ...
    How to read go.mod

    View Slide

  37. module github.com/hashicorp/consul
    go 1.12
    ...
    How to read go.mod

    View Slide

  38. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    ...
    How to read go.mod

    View Slide

  39. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    ...
    How to read go.mod

    View Slide

  40. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    ...
    How to read go.mod

    View Slide

  41. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    ...
    How to read go.mod

    View Slide

  42. How to read go.mod
    module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    replace github.com/hashicorp/consul/api => ./api
    replace github.com/hashicorp/consul/sdk => ./sdk
    ...

    View Slide

  43. How to read go.mod
    module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    github.com/DataDog/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    replace github.com/hashicorp/consul/api => ./api
    replace github.com/hashicorp/consul/sdk => ./sdk
    exclude github.com/not-a-hacker/supergeil v1270.0.1

    View Slide

  44. How to read go.sum

    View Slide

  45. How to read go.sum

    View Slide

  46. cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
    cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
    How to read go.sum

    View Slide

  47. cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
    cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible
    h1:gr1qKY/Ll72VjFTZmaBwRK1yQHAxCnV25ekOKroc9ws=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible/go.mod
    h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
    How to read go.sum

    View Slide

  48. cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
    cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible
    h1:gr1qKY/Ll72VjFTZmaBwRK1yQHAxCnV25ekOKroc9ws=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible/go.mod
    h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
    // you might want to use go mod verify
    How to read go.sum

    View Slide

  49. Usage:
    go mod [arguments]
    The commands are:
    download download modules to local cache
    edit edit go.mod from tools or scripts
    fix make go.mod semantically consistent
    graph print module requirement graph
    init initialize new module in current directory
    tidy add missing and remove unused modules
    vendor make vendored copy of dependencies
    verify verify dependencies have expected content
    why explain why packages or modules are needed
    go mod commands

    View Slide

  50. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    go mod why

    View Slide

  51. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    # set a env var and check why do we use this dependency
    $ export GO111MODULE=on
    go mod why

    View Slide

  52. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    # set a env var and check why do we use this dependency
    $ export GO111MODULE=on
    $ go mod why github.com/RoaringBitmap/roaring
    # github.com/RoaringBitmap/roaring
    code.gitea.io/gitea/modules/indexer
    github.com/blevesearch/bleve
    github.com/blevesearch/bleve/index/scorch
    github.com/RoaringBitmap/roaring
    go mod why

    View Slide

  53. FROM golang
    ENV GO111MODULE=on
    Dockerfile

    View Slide

  54. FROM golang
    ENV GO111MODULE=on
    WORKDIR /app
    COPY go.mod .
    COPY go.sum .
    Dockerfile

    View Slide

  55. FROM golang
    ENV GO111MODULE=on
    WORKDIR /app
    COPY go.mod .
    COPY go.sum .
    RUN go mod download
    COPY . .
    Dockerfile

    View Slide

  56. Dockerfile
    FROM golang
    ENV GO111MODULE=on
    WORKDIR /app
    COPY go.mod .
    COPY go.sum .
    RUN go mod download
    COPY . .
    RUN CGO_ENABLED=0 GOOS=linux go build -a -o app.exec ./...
    EXPOSE 8000
    ENTRYPOINT ["./app.exec"]

    View Slide

  57. Use local changes

    View Slide

  58. Use local changes
    - Сhange in go.mod manually

    View Slide

  59. Use local changes
    - Сhange in go.mod manually
    - go mod edit -replace ‘foo.com/bar=../bar’

    View Slide

  60. Use local changes
    - Сhange in go.mod manually
    - go mod edit -replace ‘foo.com/bar=../bar’
    - or use awesome gohack tool by Roger Peppe

    View Slide

  61. # install
    $ go get github.com/rogpeppe/gohack
    gohack by Roger Peppe

    View Slide

  62. # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add: replace example.com/foo/bar /home/rog/gohack/example.com/foo/bar
    gohack by Roger Peppe

    View Slide

  63. # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add: replace example.com/foo/bar /home/rog/gohack/example.com/foo/bar
    $ cat go.mod
    replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar
    gohack by Roger Peppe

    View Slide

  64. gohack by Roger Peppe
    # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add: replace example.com/foo/bar /home/rog/gohack/example.com/foo/bar
    $ cat go.mod
    replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar
    # to remove specific replace:
    $ gohack undo example.com/foo/bar
    # to remove all replaces:
    $ gohack undo

    View Slide

  65. Go Proxy

    View Slide

  66. Go Proxy
    https://proxy.golang.org
    https://github.com/gomods/athens
    - export GOPROXY=my.proxy.com
    - go get github.com/foo/bar

    View Slide

  67. Future

    View Slide

  68. - Enabled by default
    Future

    View Slide

  69. - Enabled by default
    - More robust ecosystem
    Future

    View Slide

  70. - Enabled by default
    - More robust ecosystem
    - More secure
    Future

    View Slide

  71. - Enabled by default
    - More robust ecosystem
    - More secure
    - More tools understand modules
    Future

    View Slide

  72. - Enabled by default
    - More robust ecosystem
    - More secure
    - More tools understand modules
    - No vendor (perhaps?)
    Future

    View Slide

  73. - Enabled by default
    - More robust ecosystem
    - More secure
    - More tools understand modules
    - No vendor (perhaps?)
    - Less “but it works on my machine!”
    Future

    View Slide

  74. Why package management is hard?

    View Slide

  75. That’s all folks
    Thank you
    Questions?
    Twitter: @oleg_kovalov
    Github: @cristaloleg

    View Slide