Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Packages & Modules - GopherCon Poland 2019

Packages & Modules - GopherCon Poland 2019

Oleg Kovalov

June 27, 2019
Tweet

More Decks by Oleg Kovalov

Other Decks in Programming

Transcript

  1. 6

  2. 8 - Gopher for ~4 years - Open-source contributor -

    Go-critic - static analysis tool - Go-advice - list of Go tips @ GoGoConf 2018
  3. 9 - Gopher for ~4 years - Open-source contributor -

    Go-critic - static analysis tool - Go-advice - list of Go tips @ GoGoConf 2018 - Engineer at allegro.pl core team
  4. - Gopher for ~4 years - Open-source contributor - Go-critic

    - static analysis tool - Go-advice - list of Go tips @ GoGoConf 2018 - Engineer at allegro.pl core team Website: https://olegk.dev Twitter: @oleg_kovalov Github: @cristaloleg 10 Twitter & slides
  5. 12

  6. - Short and clear - May be abbreviated when the

    abbreviation is familiar - Abbrv name is easy to understand 15
  7. - Short and clear - May be abbreviated when the

    abbreviation is familiar - Abbrv name is easy to understand See standard packages for good examples - io (input and output) 16
  8. - Short and clear - May be abbreviated when the

    abbreviation is familiar - Abbrv name is easy to understand See standard packages for good examples - io (input and output) - strconv (string conversion) 17
  9. - Short and clear - May be abbreviated when the

    abbreviation is familiar - Abbrv name is easy to understand See standard packages for good examples - io (input and output) - strconv (string conversion) - syscall (system call) 18
  10. 19

  11. - If abbreviating is unclear - If name is ambiguous

    - snake_case or camelCase - psdb (Postgres? Parallel search?) 23
  12. - If abbreviating is unclear - If name is ambiguous

    - snake_case or camelCase - psdb (Postgres? Parallel search?) - srv (service? server? surviving?) 24
  13. - If abbreviating is unclear - If name is ambiguous

    - snake_case or camelCase - psdb (Postgres? Parallel search?) - srv (service? server? surviving?) - buf (buffer? buffer what? WoW buff?) 25
  14. - Avoid giving a package a name that is commonly

    used in client code - buf is a good variable name for a buffer - the buffered I/O package is called bufio (buffer + io) 27
  15. - Avoid giving a package a name that is commonly

    used in client code - buf is a good variable name for a buffer - the buffered I/O package is called bufio (buffer + io) - client is a good name for...client - but not a package name 28
  16. - Avoid giving a package a name that is commonly

    used in client code - buf is a good variable name for a buffer - the buffered I/O package is called bufio (buffer + io) - client is a good name for...client - but not a package name - imagine a replacement for it 29 c ? clnt ? cli ? clientHTTP ?
  17. 30

  18. These std packages are a good names thiefs: - path

    - path/filepath - net/url And you cannot use your variable url easily, oh :( 31
  19. 32 These std packages are a good names thiefs: -

    path - path/filepath - net/url And you cannot use your variable url easily, oh :( But there are good examples: - strings - bytes - net/urls instead of net/url looks good enough
  20. Most of the projects in Go are flat and this

    is fine - Try to keep it clean github.com/foo/bar/pl/goava/src/main/code/go/validation 34
  21. Most of the projects in Go are flat and this

    is fine - Try to keep it clean github.com/foo/bar/pl/goava/src/main/code/go/validation - Avoid do src/ or pkg/ (in most cases) gitlab.com/foo/bar/pkg/process 35
  22. Write shy code - modules that don’t reveal anything unnecessary

    to other modules and that don’t rely on other modules' implementations. --- Dave Thomas (with Andy Hunt, he co-authored The Pragmatic Programmer) 36
  23. Write shy code - modules that don’t reveal anything unnecessary

    to other modules and that don’t rely on other modules' implementations. --- Dave Thomas (with Andy Hunt, he co-authored The Pragmatic Programmer) And shy also means: 1 package = 1 thing 37
  24. 38

  25. 39

  26. 40

  27. 41

  28. 42

  29. 43

  30. 44

  31. 47 - Organize by functional responsibility - Don’t put all

    the type into models package - Keeping them in one gives nothing
  32. 48 - Organize by functional responsibility - Don’t put all

    the type into models package - Keeping them in one gives nothing - In the end this will be a spaghetti code
  33. 49 - Organize by functional responsibility - Don’t put all

    the type into models package - Keeping them in one gives nothing - In the end this will be a spaghetti code - (The same applies to the interfaces, implementations and so on)
  34. 50 . ├── app ├── cmd │ └── <project-name> ├──

    blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  35. 51 . ├── app ├── cmd │ └── <project-name> ├──

    blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  36. 52 . ├── app ├── cmd │ └── <project-name> ├──

    blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  37. 53 mock.NewBlog(...) . ├── app ├── cmd │ └── <project-name>

    ├── blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  38. 54 . ├── app ├── cmd │ └── <project-name> ├──

    blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  39. 55 . ├── app ├── cmd │ └── <project-name> ├──

    blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  40. 56 . ├── app ├── cmd │ └── <project-name> ├──

    blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  41. pkg, internal, misc or util Whatever works for your team

    and you 57 . ├── app ├── cmd │ └── <project-name> ├── blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  42. 58 . ├── app ├── cmd │ └── <project-name> ├──

    blog ├── comment ├── mock ├── pkg │ ├── superpay │ └── validation ├── store │ └── postgres ├── user └── <project-name>.go
  43. 59

  44. 66 - util.URLError - common.SanitizeMessage - base.EnsureAbsolutePath - helpers.IsEmail type

    URLError = app.URLError var isEmail = validation.IsEmail These packages say nothing!
  45. - Use multiple files - Do not separate structs and

    their methods (not a C/C++) - It’s okay to store few structs in one file (it’s not Java after all) - And keep test files next to code store ├── postgres │ ├── postgres.go │ ├── user.go │ └── user_test.go └── redis ├── redis.go └── comments.go 67
  46. - Use multiple files - Do not separate structs and

    their methods (not a C/C++) - It’s okay to store few structs in one file (it’s not Java after all) - And keep test files next to code store ├── postgres │ ├── postgres.go │ ├── user.go │ └── user_test.go └── redis ├── redis.go └── comments.go // type Comment and Comment.HasImage method 68
  47. - Use multiple files - Do not separate structs and

    their methods (not a C/C++) - It’s okay to store few structs in one file (it’s not Java after all) - And keep test files next to code store ├── postgres │ ├── postgres.go │ ├── user.go // type User and type Credentials │ └── user_test.go └── redis ├── redis.go └── comments.go 69
  48. - Use multiple files - Do not separate structs and

    their methods (not a C/C++) - It’s okay to store few structs in one file (it’s not Java after all) - And keep test files next to code store ├── postgres │ ├── postgres.go │ ├── user.go │ └── user_test.go └── redis ├── redis.go └── comments.go 70
  49. 71 postgres ├── postgres.go ├── user.go ├── stats.go └── user_test.go

    redis ├── redis.go ├── cache.go └── comments.go
  50. 72 postgres ├── postgres.go ├── user.go ├── stats.go └── user_test.go

    redis ├── redis.go ├── cache.go └── comments.go cache ├── redis │ └── redis.go store ├── postgres │ ├── postgres.go │ ├── user.go │ └── user_test.go └── redis ├── redis.go └── comments.go
  51. 73

  52. - Main packages are not importable - Don’t export from

    main, it’s useless - main package is hard to test (see above) 76
  53. 78 package main func init() { config.Load() } func main()

    { os.Exit(realMain()) } func realMain() int { log.SetOutput(ioutil.Discard) // ... }
  54. 80

  55. - 1 init() per package (pretty please) - Do not

    depend on init’s call order 82
  56. - 1 init() per package (pretty please) - Do not

    depend on init’s call order - No heavy tasks 83
  57. - 1 init() per package (pretty please) - Do not

    depend on init’s call order - No heavy tasks - Preferably no I/O, especially blocking 84
  58. - 1 init() per package (pretty please) - Do not

    depend on init’s call order - No heavy tasks - Preferably no I/O, especially blocking - Better to not have it at all 85
  59. # Python is quite flexible # try: import foo except:

    print(‘good try lol’) // Go isn’t so… // package foo func init() { iCanPanicAndYouCanDoNothing() } 87
  60. // but wait, it can be flexible... package foo func

    Init() { iCanPanicButOoooohhYouCanCatchMe() } 88
  61. // but wait, it can be flexible... package foo func

    Init() { iCanPanicButOoooohhYouCanCatchMe() } ----------- package bar func ... { defer func() { if r := recover(); r != nil { // process a panic } }() foo.Init() } 89
  62. package foo func (s *Service) Process() { // let’s start

    a sweet goroutine go func() { iCanPanicAndYouCanDoNothing() }() } 90
  63. package foo func (s *Service) Process() { // let’s start

    a sweet goroutine go func() { iCanPanicAndYouCanDoNothing() }() } ----------- package bar func ... { // but internal goroutine // isn’t accessible here :( s.Process() } 91
  64. package foo func (s *Service) Process() { go func() {

    defer func() { // recovery process }() iCanPanicAndAuthorWillRecover() }() } 92
  65. package foo func (s *Service) Process() { iCanPanicButItDoesntMatter() } -----------

    package bar func ... { go func() { defer func() { // user defined recovery process }() s.Process() }() } 94
  66. 95

  67. - Exported package variable is mutable by everyone - You

    have no power to control other packages 97
  68. - Exported package variable is mutable by everyone - You

    have no power to control other packages - Tight coupling makes code less flexible 98
  69. - Exported package variable is mutable by everyone - You

    have no power to control other packages - Tight coupling makes code less flexible - You cannot mock a package! 99
  70. - Exported package variable is mutable by everyone - You

    have no power to control other packages - Tight coupling makes code less flexible - You cannot mock a package! - Logger, metrics? - mmmmaybe 100
  71. - Exported package variable is mutable by everyone - You

    have no power to control other packages - Tight coupling makes code less flexible - You cannot mock a package! - Logger, metrics? - mmmmaybe - Config, flags? - better no 101
  72. - Exported package variable is mutable by everyone - You

    have no power to control other packages - Tight coupling makes code less flexible - You cannot mock a package! - Logger, metrics? - mmmmaybe - Config, flags? - better no - DB driver? - NONONO 102
  73. - Exported package variable is mutable by everyone - You

    have no power to control other packages - Tight coupling makes code less flexible - You cannot mock a package! - Logger, metrics? - mmmmaybe - Config, flags? - better no - DB driver? - NONONO 103 by the way unexported variables are also bad :(
  74. 104

  75. - Should be simply a container for consts, types, funcs

    - Zero variables is brilliant (both exported and unexported) 106
  76. - Should be simply a container for consts, types, funcs

    - Zero variables is brilliant (both exported and unexported) - Documentation and tests :) 107
  77. 108

  78. - log.Printf A safe action, might be noisy, but harmless

    - log.Panicf Not idiomatic, but in an edge case might be fine 110
  79. 111 - log.Printf A safe action, might be noisy, but

    harmless - log.Panicf Not idiomatic, but in an edge case might be fine - log.Fatalf Evil has no boundaries, omit it, no one can survive os.Exit(1) (even defer)
  80. 112

  81. - doc.go is a README of your package - Run

    godoc locally to see how it looks 114
  82. 115 - doc.go is a README of your package -

    Run godoc locally to see how it looks - Also add package documentation // Package bytes implements functions for the manipulation of byte slices. // It is analogous to the facilities of the strings package. package bytes
  83. 117

  84. Be conservative in what you send, be liberal in what

    you accept (с) Postel’s Law, Wikipedia 118
  85. Be conservative in what you send, be liberal in what

    you accept (с) Postel’s Law, Wikipedia TCP architecture: 119
  86. Be conservative in what you send, be liberal in what

    you accept (с) Postel’s Law, Wikipedia TCP architecture: - We don’t care about how packets go through network 120
  87. Be conservative in what you send, be liberal in what

    you accept (с) Postel’s Law, Wikipedia TCP architecture: - We don’t care about how packets go through network - but we expect them in a well defined order to process 121
  88. func FetchUsers(db Store) ([]*User, error) { ... } - We

    don’t care what is inside: Postgres, Redis, plain file or a mock 123
  89. func FetchUsers(db Store) ([]*User, error) { ... } - We

    don’t care what is inside: Postgres, Redis, plain file or a mock - Until it has desired API for us 124
  90. func FetchUsers(db Store) ([]*User, error) { ... } - We

    don’t care what is inside: Postgres, Redis, plain file or a mock - Until it has desired API for us - But we know for sure that we need a specific struct User 125
  91. func FetchUsers(db Store) ([]*User, error) { ... } - We

    don’t care what is inside: Postgres, Redis, plain file or a mock - Until it has desired API for us - But we know for sure that we need a specific struct User Tip: return error as interface, not as a struct 126
  92. 127

  93. - There is the “Java-style” interface declaration - Define an

    interface - Create a type that implements it 130
  94. 131 - There is the “Java-style” interface declaration - Define

    an interface - Create a type that implements it - Often you have one interface and one implementation
  95. - There is the “Java-style” interface declaration - Define an

    interface - Create a type that implements it - Often you have one interface and one implementation That’s not a “Go style” ʕ◔ϖ◔ʔ 132
  96. 133

  97. - Go interfaces are in the package that uses it

    - And this makes Go code easy to extend 135
  98. 138

  99. 145

  100. - A module is a collection of Go packages that

    are versioned as a single unit 147
  101. - A module is a collection of Go packages that

    are versioned as a single unit - A module is a tree/directory of Go source files with a go.mod file in the root ├── cmd │ └── main.go ├── consum │ ├── consum.go │ └── handlers.go ├── produce │ ├── handlers.go │ ├── handlers_test.go │ └── processing.go ├── go.mod └── go.sum 148
  102. - A module is a collection of Go packages that

    are versioned as a single unit - A module is a tree/directory of Go source files with a go.mod file in the root ├── cmd │ └── main.go ├── consum │ ├── consum.go │ └── handlers.go ├── produce │ ├── handlers.go │ ├── handlers_test.go │ └── processing.go ├── go.mod └── go.sum 149 Can leave outside of GOPATH
  103. 150

  104. 151 - If an old and a new package have

    the same import path - the new package must be backwards-compatible with the old package
  105. 152 - If an old and a new package have

    the same import path - the new package must be backwards-compatible with the old package import ( "github.com/golang/hello/hey" // v0 or v1 "github.com/golang/hello/v2/hey" )
  106. 154 Usage: go mod <command> [arguments] The commands are: download

    download modules to local cache edit edit go.mod from tools or scripts graph print module requirement graph init initialize new module in current directory tidy add missing and remove unused modules vendor make vendored copy of dependencies verify verify dependencies have expected content why explain why packages or modules are needed
  107. $ mkdir /mygo/hello $ cd /mygo/hello $ go mod init

    github.com/golang/hello go: creating new go.mod: module github.com/golang/hello $ ls go.mod 156
  108. 157 $ mkdir /mygo/hello $ cd /mygo/hello $ go mod

    init github.com/golang/hello go: creating new go.mod: module github.com/golang/hello $ ls go.mod $ cat go.mod module github.com/golang/hello
  109. $ git clone https://github.com/go-gitea/gitea Cloning into gitea... $ cd gitea

    $ go mod init go: creating new go.mod: module github.com/go-gitea/gitea go: copying requirements from Gopkg.lock 159
  110. 160 $ git clone https://github.com/go-gitea/gitea Cloning into gitea... $ cd

    gitea $ go mod init go: creating new go.mod: module github.com/go-gitea/gitea go: copying requirements from Gopkg.lock $ go mod tidy go: extracting github.com/PuerkitoBio/goquery v1.5.0 go: extracting github.com/blevesearch/bleve v0.7.0 go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57 go: finding golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961 go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c go: downloading github.com/facebookgo/grace v0.0.0-20180706040059-75cf19382434 ...
  111. 164 module github.com/hashicorp/consul go 1.12 require ( github.com/NYTimes/gziphandler v1.0.1 <...>/datadog-go

    v0.0.0-20160329135253-cc2f4770f4d6 // indirect github.com/go-redis/redis v6.15.2+incompatible ... ) replace github.com/hashicorp/consul/api => ./api // can be fork or branch
  112. 165 module github.com/hashicorp/consul go 1.12 require ( github.com/NYTimes/gziphandler v1.0.1 <...>/datadog-go

    v0.0.0-20160329135253-cc2f4770f4d6 // indirect github.com/go-redis/redis v6.15.2+incompatible ... ) replace github.com/hashicorp/consul/api => ./api // can be fork or branch exclude github.com/not-a-hacker/supergeil v1270.0.1
  113. 166

  114. 167

  115. 169

  116. # same (@latest is default for 'go get') $ go

    get github.com/gorilla/mux@latest 170
  117. # same (@latest is default for 'go get') $ go

    get github.com/gorilla/mux@latest # records v1.6.2 $ go get github.com/gorilla/[email protected] $ go get github.com/gorilla/mux@e3702bed2 171
  118. # same (@latest is default for 'go get') $ go

    get github.com/gorilla/mux@latest # records v1.6.2 $ go get github.com/gorilla/[email protected] $ go get github.com/gorilla/mux@e3702bed2 # records v0.0.0-20180517173623-c85619274f5d $ go get github.com/gorilla/mux@c856192 172
  119. 173 # same (@latest is default for 'go get') $

    go get github.com/gorilla/mux@latest # records v1.6.2 $ go get github.com/gorilla/[email protected] $ go get github.com/gorilla/mux@e3702bed2 # records v0.0.0-20180517173623-c85619274f5d $ go get github.com/gorilla/mux@c856192 # records current meaning of master $ go get github.com/gorilla/mux@master
  120. # go and get a repo $ go get github.com/go-gitea/gitea

    $ cd ~/go/src/github.com/go-gitea/gitea 174
  121. # go and get a repo $ go get github.com/go-gitea/gitea

    $ cd ~/go/src/github.com/go-gitea/gitea # set a env var and check why do we use this dependency $ export GO111MODULE=on $ go mod why github.com/RoaringBitmap/roaring 175
  122. 176 # go and get a repo $ go get

    github.com/go-gitea/gitea $ cd ~/go/src/github.com/go-gitea/gitea # set a env var and check why do we use this dependency $ export GO111MODULE=on $ go mod why github.com/RoaringBitmap/roaring # github.com/RoaringBitmap/roaring code.gitea.io/gitea/modules/indexer github.com/blevesearch/bleve github.com/blevesearch/bleve/index/scorch github.com/RoaringBitmap/roaring
  123. # install $ go get github.com/rogpeppe/gohack $ gohack get example.com/foo/bar

    # will clone repo into $HOME/gohack/example.com/foo/bar # will add replace statement $ cat go.mod replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar 178
  124. 179 # install $ go get github.com/rogpeppe/gohack $ gohack get

    example.com/foo/bar # will clone repo into $HOME/gohack/example.com/foo/bar # will add replace statement $ cat go.mod replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar # to remove specific replace: $ gohack undo example.com/foo/bar # to remove all replaces: $ gohack undo
  125. 180

  126. 182

  127. 184

  128. - Usability ~ Reusability - Make application team-friendly - Keep

    library easy to use Animated-emoji-colorful output in app - cool, whatever 188
  129. - Usability ~ Reusability - Make application team-friendly - Keep

    library easy to use Animated-emoji-colorful output in app - cool, whatever Math library with zero allocation HTTP router - thanks, but no 189
  130. 191

  131. - Simple and shy packages - Clear naming is important

    - No global state at all - Interfaces are for abstraction 195
  132. 196 - Simple and shy packages - Clear naming is

    important - No global state at all - Interfaces are for abstraction - Modules == Go ecosystem 2.0
  133. 197 - Simple and shy packages - Clear naming is

    important - No global state at all - Interfaces are for abstraction - Modules == Go ecosystem 2.0 - App != Lib
  134. - Ben Johnson @benbjohnson - Dave Cheney @davecheney - Jaana

    B. Dogan @rakyll - Go team ʕ◔ϖ◔ʔ - all the friends \o/ 199