Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Packages & Modules

Packages & Modules

Go packages & modules. Best practices. How to organise things. How to use new shiny modules.

Oleg Kovalov

June 14, 2019
Tweet

More Decks by Oleg Kovalov

Other Decks in Programming

Transcript

  1. Packages & Modules
    Oleg Kovalov

    View Slide

  2. Agenda

    View Slide

  3. In this talk
    3
    - About me
    - Packages
    - Modules
    - Summary
    - Q&A

    View Slide

  4. In this talk + extra
    4
    - About me
    - Packages
    - Modules
    - Summary
    - Q&A Interfaces

    View Slide

  5. About me

    View Slide

  6. I don’t have a Soundcloud...
    6

    View Slide

  7. - Gopher for ~4 years
    I don’t have a Soundcloud...but
    7

    View Slide

  8. I don’t have a Soundcloud...but
    8
    - Gopher for ~4 years
    - Open-source contributor
    - Go-critic - static analysis tool
    - Go-advice - list of Go tips @ GoGoConf 2018

    View Slide

  9. I don’t have a Soundcloud...but
    9
    - Gopher for ~4 years
    - Open-source contributor
    - Go-critic - static analysis tool
    - Go-advice - list of Go tips @ GoGoConf 2018
    - Engineer at allegro.pl core team

    View Slide

  10. - Gopher for ~4 years
    - Open-source contributor
    - Go-critic - static analysis tool
    - Go-advice - list of Go tips @ GoGoConf 2018
    - Engineer at allegro.pl core team
    Website: https://olegk.dev
    Twitter: @oleg_kovalov
    Github: @cristaloleg
    I don’t have a Soundcloud...but
    10
    Twitter & slides

    View Slide

  11. Packages

    View Slide

  12. Naming
    12

    View Slide

  13. - Short and clear
    Naming
    13

    View Slide

  14. - Short and clear
    - May be abbreviated when the abbreviation is familiar
    Naming
    14

    View Slide

  15. - Short and clear
    - May be abbreviated when the abbreviation is familiar
    - Abbrv name is easy to understand
    Naming
    15

    View Slide

  16. Naming
    - Short and clear
    - May be abbreviated when the abbreviation is familiar
    - Abbrv name is easy to understand
    See standard packages for good examples
    - io (input and output)
    16

    View Slide

  17. - Short and clear
    - May be abbreviated when the abbreviation is familiar
    - Abbrv name is easy to understand
    See standard packages for good examples
    - io (input and output)
    - strconv (string conversion)
    Naming
    17

    View Slide

  18. - Short and clear
    - May be abbreviated when the abbreviation is familiar
    - Abbrv name is easy to understand
    See standard packages for good examples
    - io (input and output)
    - strconv (string conversion)
    - syscall (system call)
    Naming
    18

    View Slide

  19. Naming don’ts
    19

    View Slide

  20. - If abbreviating is unclear
    Naming don’ts
    20

    View Slide

  21. - If abbreviating is unclear
    - If name is ambiguous
    Naming don’ts
    21

    View Slide

  22. - If abbreviating is unclear
    - If name is ambiguous
    - snake_case or camelCase
    Naming don’ts
    22

    View Slide

  23. Naming don’ts
    - If abbreviating is unclear
    - If name is ambiguous
    - snake_case or camelCase
    - psdb (Postgres? Parallel search?)
    23

    View Slide

  24. - If abbreviating is unclear
    - If name is ambiguous
    - snake_case or camelCase
    - psdb (Postgres? Parallel search?)
    - srv (service? server? surviving?)
    Naming don’ts
    24

    View Slide

  25. - If abbreviating is unclear
    - If name is ambiguous
    - snake_case or camelCase
    - psdb (Postgres? Parallel search?)
    - srv (service? server? surviving?)
    - buf (buffer? buffer what? WoW buff?)
    Naming don’ts
    25

    View Slide

  26. - Avoid giving a package a name that is commonly used in client code
    Don’t steal good names
    26

    View Slide

  27. - Avoid giving a package a name that is commonly used in client code
    - buf is a good variable name for a buffer
    - the buffered I/O package is called bufio (buffer + io)
    Don’t steal good names
    27

    View Slide

  28. - Avoid giving a package a name that is commonly used in client code
    - buf is a good variable name for a buffer
    - the buffered I/O package is called bufio (buffer + io)
    - client is a good name for...client
    - but not a package name
    Don’t steal good names
    28

    View Slide

  29. - Avoid giving a package a name that is commonly used in client code
    - buf is a good variable name for a buffer
    - the buffered I/O package is called bufio (buffer + io)
    - client is a good name for...client
    - but not a package name
    - imagine a replacement for it
    Don’t steal good names
    29
    c ? clnt ? cli ?
    clientHTTP ?

    View Slide

  30. ...and few from std packages
    30

    View Slide

  31. These std packages are a good names thiefs:
    - path
    - path/filepath
    - net/url
    And you cannot use your variable url easily, oh :(
    ...and few from std packages
    31

    View Slide

  32. ...and few from std packages
    32
    These std packages are a good names thiefs:
    - path
    - path/filepath
    - net/url
    And you cannot use your variable url easily, oh :(
    But there are good examples:
    - strings
    - bytes
    - net/urls instead of net/url looks good enough

    View Slide

  33. Most of the projects in Go are flat and this is fine
    Try to keep clean import paths
    33

    View Slide

  34. Most of the projects in Go are flat and this is fine
    - Try to keep it clean
    github.com/foo/bar/pl/goava/src/main/code/go/validation
    Try to keep clean import paths
    34

    View Slide

  35. Most of the projects in Go are flat and this is fine
    - Try to keep it clean
    github.com/foo/bar/pl/goava/src/main/code/go/validation
    - Avoid do src/ or pkg/ (in most cases)
    gitlab.com/foo/bar/pkg/process
    Try to keep clean import paths
    35

    View Slide

  36. Write shy code - modules that don’t reveal anything unnecessary to other
    modules and that don’t rely on other modules' implementations.
    --- Dave Thomas (with Andy Hunt, he co-authored The Pragmatic Programmer)
    Shy packages
    36

    View Slide

  37. Write shy code - modules that don’t reveal anything unnecessary to other
    modules and that don’t rely on other modules' implementations.
    --- Dave Thomas (with Andy Hunt, he co-authored The Pragmatic Programmer)
    And shy also means: 1 package = 1 thing
    Shy packages
    37

    View Slide

  38. Let’s put all structs into model(s) package
    38

    View Slide

  39. Nice
    39

    View Slide

  40. Yeah!
    40

    View Slide

  41. Oh wait...
    41

    View Slide

  42. ...
    42

    View Slide

  43. PLZ STAHP
    43

    View Slide

  44. Models package is a weak idea
    44

    View Slide

  45. - Organize by functional responsibility
    Models package is a weak idea, so
    45

    View Slide

  46. - Organize by functional responsibility
    - Don’t put all the type into models package
    Models package is a weak idea, so
    46

    View Slide

  47. Models package is a weak idea, so
    47
    - Organize by functional responsibility
    - Don’t put all the type into models package
    - Keeping them in one gives nothing

    View Slide

  48. Models package is a weak idea, so
    48
    - Organize by functional responsibility
    - Don’t put all the type into models package
    - Keeping them in one gives nothing
    - In the end this will be a spaghetti code

    View Slide

  49. Models package is a weak idea, so
    49
    - Organize by functional responsibility
    - Don’t put all the type into models package
    - Keeping them in one gives nothing
    - In the end this will be a spaghetti code
    - (The same applies to the interfaces, implementations and so on)

    View Slide

  50. One of the possible structures
    50
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  51. One of the possible structures
    51
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  52. One of the possible structures
    52
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  53. One of the possible structures
    53
    mock.NewBlog(...)
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  54. One of the possible structures
    54
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  55. One of the possible structures
    55
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  56. One of the possible structures
    56
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  57. pkg, internal,
    misc or util
    Whatever works for
    your team and you
    One of the possible structures
    57
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  58. One of the possible structures
    58
    .
    ├── app
    ├── cmd
    │ └──
    ├── blog
    ├── comment
    ├── mock
    ├── pkg
    │ ├── superpay
    │ └── validation
    ├── store
    │ └── postgres
    ├── user
    └── .go

    View Slide

  59. Avoid util/common/base/helpers packages
    59

    View Slide

  60. - util.URLError
    Avoid util/common/base/helpers packages
    60

    View Slide

  61. - util.URLError
    - common.SanitizeMessage
    Avoid util/common/base/helpers packages
    61

    View Slide

  62. - util.URLError
    - common.SanitizeMessage
    - base.EnsureAbsolutePath
    Avoid util/common/base/helpers packages
    62

    View Slide

  63. - util.URLError
    - common.SanitizeMessage
    - base.EnsureAbsolutePath
    - helpers.IsEmail
    Avoid util/common/base/helpers packages
    63

    View Slide

  64. - util.URLError
    - common.SanitizeMessage
    - base.EnsureAbsolutePath
    - helpers.IsEmail
    Avoid util/common/base/helpers packages
    64
    These packages say nothing!

    View Slide

  65. Avoid util/common/base/helpers packages
    65
    - util.URLError
    - common.SanitizeMessage
    - base.EnsureAbsolutePath
    - helpers.IsEmail
    type URLError = app.URLError
    These packages say nothing!

    View Slide

  66. Avoid util/common/base/helpers packages
    66
    - util.URLError
    - common.SanitizeMessage
    - base.EnsureAbsolutePath
    - helpers.IsEmail
    type URLError = app.URLError
    var isEmail = validation.IsEmail
    These packages say nothing!

    View Slide

  67. - Use multiple files
    - Do not separate structs and their methods (not a C/C++)
    - It’s okay to store few structs in one file (it’s not Java after all)
    - And keep test files next to code
    store
    ├── postgres
    │ ├── postgres.go
    │ ├── user.go
    │ └── user_test.go
    └── redis
    ├── redis.go
    └── comments.go
    Inside the package
    67

    View Slide

  68. - Use multiple files
    - Do not separate structs and their methods (not a C/C++)
    - It’s okay to store few structs in one file (it’s not Java after all)
    - And keep test files next to code
    store
    ├── postgres
    │ ├── postgres.go
    │ ├── user.go
    │ └── user_test.go
    └── redis
    ├── redis.go
    └── comments.go // type Comment and Comment.HasImage method
    Inside the package
    68

    View Slide

  69. - Use multiple files
    - Do not separate structs and their methods (not a C/C++)
    - It’s okay to store few structs in one file (it’s not Java after all)
    - And keep test files next to code
    store
    ├── postgres
    │ ├── postgres.go
    │ ├── user.go // type User and type Credentials
    │ └── user_test.go
    └── redis
    ├── redis.go
    └── comments.go
    Inside the package
    69

    View Slide

  70. - Use multiple files
    - Do not separate structs and their methods (not a C/C++)
    - It’s okay to store few structs in one file (it’s not Java after all)
    - And keep test files next to code
    store
    ├── postgres
    │ ├── postgres.go
    │ ├── user.go
    │ └── user_test.go
    └── redis
    ├── redis.go
    └── comments.go
    Inside the package
    70

    View Slide

  71. Is this storage? Stats? Cache? Benchmark suites?
    71
    postgres
    ├── postgres.go
    ├── user.go
    ├── stats.go
    └── user_test.go
    redis
    ├── redis.go
    ├── cache.go
    └── comments.go

    View Slide

  72. Is this storage? Stats? Cache? Benchmark suites?
    72
    postgres
    ├── postgres.go
    ├── user.go
    ├── stats.go
    └── user_test.go
    redis
    ├── redis.go
    ├── cache.go
    └── comments.go
    cache
    ├── redis
    │ └── redis.go
    store
    ├── postgres
    │ ├── postgres.go
    │ ├── user.go
    │ └── user_test.go
    └── redis
    ├── redis.go
    └── comments.go

    View Slide

  73. What about main package?
    73

    View Slide

  74. - Main packages are not importable
    What about main package?
    74

    View Slide

  75. - Main packages are not importable
    - Don’t export from main, it’s useless
    What about main package?
    75

    View Slide

  76. - Main packages are not importable
    - Don’t export from main, it’s useless
    - main package is hard to test (see above)
    What about main package?
    76

    View Slide

  77. Keep your main simple and testable
    77
    package main
    func init() {
    config.Load()
    }

    View Slide

  78. Keep your main simple and testable
    78
    package main
    func init() {
    config.Load()
    }
    func main() {
    os.Exit(realMain())
    }
    func realMain() int {
    log.SetOutput(ioutil.Discard)
    // ...
    }

    View Slide

  79. package main
    func main() {
    log.Fatal(app.Run())
    }
    Keep your main simpler
    79

    View Slide

  80. Keep your init() light
    80

    View Slide

  81. - 1 init() per package (pretty please)
    Keep your init() light
    81

    View Slide

  82. - 1 init() per package (pretty please)
    - Do not depend on init’s call order
    Keep your init() light
    82

    View Slide

  83. - 1 init() per package (pretty please)
    - Do not depend on init’s call order
    - No heavy tasks
    Keep your init() light
    83

    View Slide

  84. - 1 init() per package (pretty please)
    - Do not depend on init’s call order
    - No heavy tasks
    - Preferably no I/O, especially blocking
    Keep your init() light
    84

    View Slide

  85. - 1 init() per package (pretty please)
    - Do not depend on init’s call order
    - No heavy tasks
    - Preferably no I/O, especially blocking
    - Better to not have it at all
    Keep your init() light
    85

    View Slide

  86. # Python is quite flexible
    #
    try:
    import foo
    except:
    print(‘good try lol’)
    panic in init()
    86

    View Slide

  87. # Python is quite flexible
    #
    try:
    import foo
    except:
    print(‘good try lol’)
    // Go isn’t so…
    //
    package foo
    func init() {
    iCanPanicAndYouCanDoNothing()
    }
    panic in init()
    87

    View Slide

  88. // but wait, it can be flexible...
    package foo
    func Init() {
    iCanPanicButOoooohhYouCanCatchMe()
    }
    panic in init()
    88

    View Slide

  89. panic in init()
    // but wait, it can be flexible...
    package foo
    func Init() {
    iCanPanicButOoooohhYouCanCatchMe()
    }
    -----------
    package bar
    func ... {
    defer func() {
    if r := recover(); r != nil {
    // process a panic
    }
    }()
    foo.Init()
    } 89

    View Slide

  90. package foo
    func (s *Service) Process() {
    // let’s start a sweet goroutine
    go func() {
    iCanPanicAndYouCanDoNothing()
    }()
    }
    I heard you like goroutines
    90

    View Slide

  91. package foo
    func (s *Service) Process() {
    // let’s start a sweet goroutine
    go func() {
    iCanPanicAndYouCanDoNothing()
    }()
    }
    -----------
    package bar
    func ... {
    // but internal goroutine
    // isn’t accessible here :(
    s.Process()
    }
    I heard you like goroutines
    91

    View Slide

  92. package foo
    func (s *Service) Process() {
    go func() {
    defer func() {
    // recovery process
    }()
    iCanPanicAndAuthorWillRecover()
    }()
    }
    I heard you like goroutines
    92

    View Slide

  93. package foo
    func (s *Service) Process() {
    iCanPanicButItDoesntMatter()
    }
    I heard you like goroutines
    93

    View Slide

  94. package foo
    func (s *Service) Process() {
    iCanPanicButItDoesntMatter()
    }
    -----------
    package bar
    func ... {
    go func() {
    defer func() {
    // user defined recovery process
    }()
    s.Process()
    }()
    }
    I heard you like goroutines
    94

    View Slide

  95. Global state
    95

    View Slide

  96. - Exported package variable is mutable by everyone
    Global state
    96

    View Slide

  97. - Exported package variable is mutable by everyone
    - You have no power to control other packages
    Global state
    97

    View Slide

  98. - Exported package variable is mutable by everyone
    - You have no power to control other packages
    - Tight coupling makes code less flexible
    Global state
    98

    View Slide

  99. - Exported package variable is mutable by everyone
    - You have no power to control other packages
    - Tight coupling makes code less flexible
    - You cannot mock a package!
    Global state
    99

    View Slide

  100. - Exported package variable is mutable by everyone
    - You have no power to control other packages
    - Tight coupling makes code less flexible
    - You cannot mock a package!
    - Logger, metrics? - mmmmaybe
    Global state
    100

    View Slide

  101. - Exported package variable is mutable by everyone
    - You have no power to control other packages
    - Tight coupling makes code less flexible
    - You cannot mock a package!
    - Logger, metrics? - mmmmaybe
    - Config, flags? - better no
    Global state
    101

    View Slide

  102. - Exported package variable is mutable by everyone
    - You have no power to control other packages
    - Tight coupling makes code less flexible
    - You cannot mock a package!
    - Logger, metrics? - mmmmaybe
    - Config, flags? - better no
    - DB driver? - NONONO
    Global state
    102

    View Slide

  103. - Exported package variable is mutable by everyone
    - You have no power to control other packages
    - Tight coupling makes code less flexible
    - You cannot mock a package!
    - Logger, metrics? - mmmmaybe
    - Config, flags? - better no
    - DB driver? - NONONO
    Global state
    103
    by the way unexported variables
    are also bad :(

    View Slide

  104. Good package == stateless container
    104

    View Slide

  105. - Should be simply a container for consts, types, funcs
    Good package == stateless container
    105

    View Slide

  106. - Should be simply a container for consts, types, funcs
    - Zero variables is brilliant (both exported and unexported)
    Good package == stateless container
    106

    View Slide

  107. - Should be simply a container for consts, types, funcs
    - Zero variables is brilliant (both exported and unexported)
    - Documentation and tests :)
    Good package == stateless container
    107

    View Slide

  108. Logging is cool
    108

    View Slide

  109. - log.Printf
    A safe action, might be noisy, but harmless
    Logging is cool, but...
    109

    View Slide

  110. - log.Printf
    A safe action, might be noisy, but harmless
    - log.Panicf
    Not idiomatic, but in an edge case might be fine
    Logging is cool, but...
    110

    View Slide

  111. Logging is cool, but...
    111
    - log.Printf
    A safe action, might be noisy, but harmless
    - log.Panicf
    Not idiomatic, but in an edge case might be fine
    - log.Fatalf
    Evil has no boundaries, omit it, no one can survive os.Exit(1) (even defer)

    View Slide

  112. Forgotten doc.go
    112

    View Slide

  113. - doc.go is a README of your package
    Forgotten doc.go
    113

    View Slide

  114. - doc.go is a README of your package
    - Run godoc locally to see how it looks
    Forgotten doc.go
    114

    View Slide

  115. Forgotten doc.go
    115
    - doc.go is a README of your package
    - Run godoc locally to see how it looks
    - Also add package documentation
    // Package bytes implements functions for the manipulation of byte slices.
    // It is analogous to the facilities of the strings package.
    package bytes

    View Slide

  116. Interfaces

    View Slide

  117. Accept interfaces, return structs
    117

    View Slide

  118. Be conservative in what you send, be liberal in what you accept
    (с) Postel’s Law, Wikipedia
    Accept interfaces, return structs
    118

    View Slide

  119. Be conservative in what you send, be liberal in what you accept
    (с) Postel’s Law, Wikipedia
    TCP architecture:
    Accept interfaces, return structs
    119

    View Slide

  120. Accept interfaces, return structs
    Be conservative in what you send, be liberal in what you accept
    (с) Postel’s Law, Wikipedia
    TCP architecture:
    - We don’t care about how packets go through network
    120

    View Slide

  121. Be conservative in what you send, be liberal in what you accept
    (с) Postel’s Law, Wikipedia
    TCP architecture:
    - We don’t care about how packets go through network
    - but we expect them in a well defined order to process
    Accept interfaces, return structs
    121

    View Slide

  122. func FetchUsers(db Store) ([]*User, error) { ... }
    Accept interfaces, return structs
    122

    View Slide

  123. func FetchUsers(db Store) ([]*User, error) { ... }
    - We don’t care what is inside: Postgres, Redis, plain file or a mock
    Accept interfaces, return structs
    123

    View Slide

  124. func FetchUsers(db Store) ([]*User, error) { ... }
    - We don’t care what is inside: Postgres, Redis, plain file or a mock
    - Until it has desired API for us
    Accept interfaces, return structs
    124

    View Slide

  125. func FetchUsers(db Store) ([]*User, error) { ... }
    - We don’t care what is inside: Postgres, Redis, plain file or a mock
    - Until it has desired API for us
    - But we know for sure that we need a specific struct User
    Accept interfaces, return structs
    125

    View Slide

  126. func FetchUsers(db Store) ([]*User, error) { ... }
    - We don’t care what is inside: Postgres, Redis, plain file or a mock
    - Until it has desired API for us
    - But we know for sure that we need a specific struct User
    Tip: return error as interface, not as a struct
    Accept interfaces, return structs
    126

    View Slide

  127. Where to declare interfaces?
    127

    View Slide

  128. - There is the “Java-style” interface declaration
    Where to declare interfaces?
    128

    View Slide

  129. - There is the “Java-style” interface declaration
    - Define an interface
    Where to declare interfaces?
    129

    View Slide

  130. - There is the “Java-style” interface declaration
    - Define an interface
    - Create a type that implements it
    Where to declare interfaces?
    130

    View Slide

  131. Where to declare interfaces?
    131
    - There is the “Java-style” interface declaration
    - Define an interface
    - Create a type that implements it
    - Often you have one interface and one implementation

    View Slide

  132. Where to declare interfaces?
    - There is the “Java-style” interface declaration
    - Define an interface
    - Create a type that implements it
    - Often you have one interface and one implementation
    That’s not a “Go style”
    ʕ◔ϖ◔ʔ
    132

    View Slide

  133. Where to declare Go interfaces?
    133

    View Slide

  134. - Go interfaces are in the package that uses it
    Where to declare Go interfaces?
    134

    View Slide

  135. - Go interfaces are in the package that uses it
    - And this makes Go code easy to extend
    Where to declare Go interfaces?
    135

    View Slide

  136. Modules

    View Slide

  137. Long story short
    time
    137
    time
    Image credit: ©Istockphoto/JamesBrey

    View Slide

  138. Pre-modules era
    138

    View Slide

  139. - GOPATH
    Pre-modules era (2012-...) Go 1.0
    139

    View Slide

  140. - GOPATH
    - godep
    Pre-modules era (2013-...)
    140

    View Slide

  141. - GOPATH
    - godep
    - gopkg.in
    Pre-modules era (2014-...)
    141

    View Slide

  142. - GOPATH
    - godep
    - gopkg.in
    - glide
    Pre-modules era (2014-...)
    142

    View Slide

  143. - GOPATH
    - godep
    - gopkg.in
    - glide
    - vendor dir
    Pre-modules era (2015-...) Go 1.5
    143

    View Slide

  144. - GOPATH
    - godep
    - gopkg.in
    - glide
    - vendor dir
    - dep
    Pre-modules era (2017-...)
    144

    View Slide

  145. Go modules (2018)
    145

    View Slide

  146. Go modules
    146
    - Modules
    - Semantic versioning
    - Import compatibility rule
    - go.mod

    View Slide

  147. - A module is a collection of Go packages that are versioned as a single unit
    What is a module?
    147

    View Slide

  148. - A module is a collection of Go packages that are versioned as a single unit
    - A module is a tree/directory of Go source files with a go.mod file in the root
    ├── cmd
    │ └── main.go
    ├── consum
    │ ├── consum.go
    │ └── handlers.go
    ├── produce
    │ ├── handlers.go
    │ ├── handlers_test.go
    │ └── processing.go
    ├── go.mod
    └── go.sum
    What is a module?
    148

    View Slide

  149. - A module is a collection of Go packages that are versioned as a single unit
    - A module is a tree/directory of Go source files with a go.mod file in the root
    ├── cmd
    │ └── main.go
    ├── consum
    │ ├── consum.go
    │ └── handlers.go
    ├── produce
    │ ├── handlers.go
    │ ├── handlers_test.go
    │ └── processing.go
    ├── go.mod
    └── go.sum
    What is a module?
    149
    Can leave
    outside of
    GOPATH

    View Slide

  150. Semantic versioning (https://semver.org)
    150

    View Slide

  151. Import compatibility rule
    151
    - If an old and a new package have the same import path
    - the new package must be backwards-compatible with the old package

    View Slide

  152. Import compatibility rule
    152
    - If an old and a new package have the same import path
    - the new package must be backwards-compatible with the old package
    import (
    "github.com/golang/hello/hey" // v0 or v1
    "github.com/golang/hello/v2/hey"
    )

    View Slide

  153. How to enable modules?
    153
    - auto(default)
    - on
    - off
    $ export GO111MODULE=on
    $ go build ./...

    View Slide

  154. go mod commands
    154
    Usage:
    go mod [arguments]
    The commands are:
    download download modules to local cache
    edit edit go.mod from tools or scripts
    graph print module requirement graph
    init initialize new module in current directory
    tidy add missing and remove unused modules
    vendor make vendored copy of dependencies
    verify verify dependencies have expected content
    why explain why packages or modules are needed

    View Slide

  155. $ mkdir /mygo/hello
    $ cd /mygo/hello
    Let’s start with modules
    155

    View Slide

  156. Let’s start with modules
    $ mkdir /mygo/hello
    $ cd /mygo/hello
    $ go mod init github.com/golang/hello
    go: creating new go.mod: module github.com/golang/hello
    $ ls
    go.mod
    156

    View Slide

  157. Let’s start with modules
    157
    $ mkdir /mygo/hello
    $ cd /mygo/hello
    $ go mod init github.com/golang/hello
    go: creating new go.mod: module github.com/golang/hello
    $ ls
    go.mod
    $ cat go.mod
    module github.com/golang/hello

    View Slide

  158. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    Convert to modules
    158

    View Slide

  159. $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    $ go mod init
    go: creating new go.mod: module github.com/go-gitea/gitea
    go: copying requirements from Gopkg.lock
    Convert to modules
    159

    View Slide

  160. Convert to modules
    160
    $ git clone https://github.com/go-gitea/gitea
    Cloning into gitea...
    $ cd gitea
    $ go mod init
    go: creating new go.mod: module github.com/go-gitea/gitea
    go: copying requirements from Gopkg.lock
    $ go mod tidy
    go: extracting github.com/PuerkitoBio/goquery v1.5.0
    go: extracting github.com/blevesearch/bleve v0.7.0
    go: finding github.com/google/pprof v0.0.0-20181206194817-3ea8567a2e57
    go: finding golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961
    go: finding golang.org/x/time v0.0.0-20181108054448-85acf8d2951c
    go: downloading github.com/facebookgo/grace
    v0.0.0-20180706040059-75cf19382434
    ...

    View Slide

  161. How to read go.mod?
    module github.com/hashicorp/consul
    go 1.12
    ...
    161

    View Slide

  162. How to read go.mod?
    module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    <...>/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    162

    View Slide

  163. module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    <...>/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    How to read go.mod?
    163

    View Slide

  164. How to read go.mod?
    164
    module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    <...>/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    replace github.com/hashicorp/consul/api => ./api // can be fork or branch

    View Slide

  165. How to read go.mod?
    165
    module github.com/hashicorp/consul
    go 1.12
    require (
    github.com/NYTimes/gziphandler v1.0.1
    <...>/datadog-go v0.0.0-20160329135253-cc2f4770f4d6 // indirect
    github.com/go-redis/redis v6.15.2+incompatible
    ...
    )
    replace github.com/hashicorp/consul/api => ./api // can be fork or branch
    exclude github.com/not-a-hacker/supergeil v1270.0.1

    View Slide

  166. How to read go.sum?
    166

    View Slide

  167. How to read go.sum?
    167

    View Slide

  168. How to read go.sum?
    168
    cloud.google.com/go v0.26.0 h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=
    cloud.google.com/go v0.26.0/go.mod
    h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible
    h1:gr1qKY/Ll72VjFTZmaBwRK1yQHAxCnV25ekOKroc9ws=
    github.com/Azure/azure-sdk-for-go v16.0.0+incompatible/go.mod
    h1:9XXNKU+eRnpl9moKnB4QOLf1HestfXbmab5FXxiDBjc=
    // you might want to use go mod verify

    View Slide

  169. go get understands versioning
    169

    View Slide

  170. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    go get understands versioning
    170

    View Slide

  171. # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/[email protected]
    $ go get github.com/gorilla/mux@e3702bed2
    go get understands versioning
    171

    View Slide

  172. go get understands versioning
    # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/[email protected]
    $ go get github.com/gorilla/mux@e3702bed2
    # records v0.0.0-20180517173623-c85619274f5d
    $ go get github.com/gorilla/mux@c856192
    172

    View Slide

  173. go get understands versioning
    173
    # same (@latest is default for 'go get')
    $ go get github.com/gorilla/mux@latest
    # records v1.6.2
    $ go get github.com/gorilla/[email protected]
    $ go get github.com/gorilla/mux@e3702bed2
    # records v0.0.0-20180517173623-c85619274f5d
    $ go get github.com/gorilla/mux@c856192
    # records current meaning of master
    $ go get github.com/gorilla/mux@master

    View Slide

  174. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    go mod why
    174

    View Slide

  175. # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    # set a env var and check why do we use this dependency
    $ export GO111MODULE=on
    $ go mod why github.com/RoaringBitmap/roaring
    go mod why
    175

    View Slide

  176. go mod why
    176
    # go and get a repo
    $ go get github.com/go-gitea/gitea
    $ cd ~/go/src/github.com/go-gitea/gitea
    # set a env var and check why do we use this dependency
    $ export GO111MODULE=on
    $ go mod why github.com/RoaringBitmap/roaring
    # github.com/RoaringBitmap/roaring
    code.gitea.io/gitea/modules/indexer
    github.com/blevesearch/bleve
    github.com/blevesearch/bleve/index/scorch
    github.com/RoaringBitmap/roaring

    View Slide

  177. # install
    $ go get github.com/rogpeppe/gohack
    gohack by Roger Peppe
    177

    View Slide

  178. # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add replace statement
    $ cat go.mod
    replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar
    gohack by Roger Peppe
    178

    View Slide

  179. gohack by Roger Peppe
    179
    # install
    $ go get github.com/rogpeppe/gohack
    $ gohack get example.com/foo/bar
    # will clone repo into $HOME/gohack/example.com/foo/bar
    # will add replace statement
    $ cat go.mod
    replace example.com/foo/bar => /home/rog/gohack/example.com/foo/bar
    # to remove specific replace:
    $ gohack undo example.com/foo/bar
    # to remove all replaces:
    $ gohack undo

    View Slide

  180. GOPROXY
    180

    View Slide

  181. GOPROXY
    181
    - https://proxy.golang.org
    - https://github.com/gomods/athens
    - https://github.com/goproxy/goproxy
    - export GOPROXY=my.proxy.com
    - go get github.com/foo/bar

    View Slide

  182. Why package management is hard?
    182

    View Slide

  183. *Remark

    View Slide

  184. Application != Library
    184

    View Slide

  185. - Usability ~ Reusability
    Application != Library
    185

    View Slide

  186. - Usability ~ Reusability
    - Make application team-friendly
    Application != Library
    186

    View Slide

  187. - Usability ~ Reusability
    - Make application team-friendly
    - Keep library easy to use
    Application != Library
    187

    View Slide

  188. - Usability ~ Reusability
    - Make application team-friendly
    - Keep library easy to use
    Animated-emoji-colorful output in app - cool, whatever
    Application != Library
    188

    View Slide

  189. - Usability ~ Reusability
    - Make application team-friendly
    - Keep library easy to use
    Animated-emoji-colorful output in app - cool, whatever
    Math library with zero allocation HTTP router - thanks, but no
    Application != Library
    189

    View Slide

  190. Summary

    View Slide

  191. Summary
    191

    View Slide

  192. - Simple and shy packages
    Summary
    192

    View Slide

  193. - Simple and shy packages
    - Clear naming is important
    Summary
    193

    View Slide

  194. - Simple and shy packages
    - Clear naming is important
    - No global state at all
    Summary
    194

    View Slide

  195. Summary
    - Simple and shy packages
    - Clear naming is important
    - No global state at all
    - Interfaces are for abstraction
    195

    View Slide

  196. Summary
    196
    - Simple and shy packages
    - Clear naming is important
    - No global state at all
    - Interfaces are for abstraction
    - Modules == Go ecosystem 2.0

    View Slide

  197. Summary
    197
    - Simple and shy packages
    - Clear naming is important
    - No global state at all
    - Interfaces are for abstraction
    - Modules == Go ecosystem 2.0
    - App != Lib

    View Slide

  198. Q&A

    View Slide

  199. - Ben Johnson @benbjohnson
    - Dave Cheney @davecheney
    - Jaana B. Dogan @rakyll
    - Go team ʕ◔ϖ◔ʔ
    - all the friends \o/
    Thanks
    199

    View Slide

  200. Thank You

    View Slide