impguard - protect your project structure

Transcript

1. Impguard protect your project structure Warsaw, November 2020 Oleg Kovalov
2. • Gopher for ~5 years • Open source contributor • Engineer at GoGoApps Hi, i’m Oleg olegk.dev @oleg_kovalov @cristaloleg
3. At GoGoApps we’re...
4. At GoGoApps we’re... • (re)building a product for a big company™
5. At GoGoApps we’re... • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic
6. At GoGoApps we’re... • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be
7. At GoGoApps we’re... • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!
8. At GoGoApps we’re... But there is one thing is running in my mind for a long time… • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!
9. Define me a problem first
10. Define me a problem first • linters give you suggestions on code level • mighty golangci-lint is The Beast
11. Define me a problem first • linters give you suggestions on code level • mighty golangci-lint is The Beast • but let’s look level above
12. Define me a problem first • linters give you suggestions on code level • mighty golangci-lint is The Beast • but let’s look level above • how your packages are organized?
13. Setup, goals, solution
14. Setup, goals, solution • you’ve a monorepo • or just a bunch of repos
15. Setup, goals, solution • you’ve a monorepo • or just a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something
16. Setup, goals, solution • you’ve a monorepo • or just a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something • custom tooling solves this!
17. Example
18. Let’s talk about imports
19. Let’s talk about imports • core idea of (re)using others code • staying on the shoulders of giants as Newton said
20. Let’s talk about imports • core idea of (re)using others code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed
21. Let’s talk about imports • core idea of (re)using others code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed • but nothing is free • others code needs to be verified and used carefully • you can rely on any code and use it everywhere • but that might be a ticking bomb (LeftPad-like problem)
22. When in doubts - see golang/go
23. When in doubts - see golang/go • Go doesn’t allow import cycles - that’s cool • simpler compilation routine and code organization
24. When in doubts - see golang/go • Go doesn’t allow import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb
25. When in doubts - see golang/go • Go doesn’t allow import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb • Stdlib has a test for imports https://github.com/golang/go/blob/release-branch.go1.14/src/go/build/deps_test.go https://github.com/golang/go/blob/release-branch.go1.15/src/go/build/deps_test.go
28. What does it mean?
29. What does it mean? • in other words you’ve a rigid code structure • you cannot import something without an explicit change
30. What does it mean? • in other words you’ve a rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages
31. What does it mean? • in other words you’ve a rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages • what we can do? • check Github!
32. What do we’ve right now?
33. What do we’ve right now? • `$ go mod why` • out of the box solution
34. What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint)
35. What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file)
36. What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies
37. What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies • … (sorry, I cannot copy-paste all the Github ):
38. go mod why
39. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole tailscale.com/wgengine/router github.com/go-ole/go-ole
40. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole tailscale.com/wgengine/router github.com/go-ole/go-ole > go mod why github.com/RoaringBitmap/roaring # github.com/RoaringBitmap/roaring code.gitea.io/gitea/modules/indexer/code github.com/blevesearch/bleve github.com/blevesearch/bleve/index/scorch github.com/RoaringBitmap/roaring
41. tailscale/depaware
42. divan/depscheck
43. What is not ok? (only for me?)
44. What is not ok? (only for me?) • solving author’s problem • let’s look at a broader problem
45. What is not ok? (only for me?) • solving author’s problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list
46. What is not ok? (only for me?) • solving author’s problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list • already feature complete • boooooooo, we need a room for the improvements!
47. Welcome impguard
48. Welcome impguard • read as „import guard”
49. Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed
50. Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me!
51. Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found
52. Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found yet?…
53. How does it work?
54. How does it work? • go list -json all | impguard project.yaml
55. How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things
56. How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic
57. How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file
58. How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file • simple, isn’t ?
59. go list gitea ❯ time go list -json all > 1.txt go list -json all > 1.txt 3.07s user 2.63s system 376% cpu 1.516 total ❯ du -sh 1.txt 4.0M 1.txt
60. Show me examples!
61. Show me examples! "*": - name: must have doc.go checks: type: has_file pattern: doc.go
62. Show me examples! "*": - name: must have doc.go checks: type: has_file pattern: doc.go - name: explicit runtime is forbidden checks: type: has_import pattern: runtime exclude: github.com/company/runtimeutil
63. Show me more examples! github.com/company: - name: don't use bad UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid
64. Show me more examples! github.com/company: - name: don't use bad UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid - name: no vendors, just modules checks: type: no_dir pattern: vendor
65. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md
66. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md github.com/company/project/services/*: - name: services should be independent checks: - type: no_import pattern: github.com/company/project/services/*
67. Where is the code Lebowski?
68. Where is the code Lebowski? • good question
69. Where is the code Lebowski? • good question • will be available • https://github.com/cristaloleg/impguard
70. Where is the code Lebowski? • good question • will be available • https://github.com/cristaloleg/impguard • feel free to share your ideas!
71. Where is the code Lebowski? • good question • will be available • https://github.com/cristaloleg/impguard • feel free to share your ideas! • no, really
72. Thanks
73. Thanks • Yegor Myskin
74. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan Storozhuk • Alexey Palazhchenko
75. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi
76. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi • you and Golang Poland <3
77. Also see • https://github.com/go-critic/go-critic • https://github.com/quasilyte/go-ruleguard • https://github.com/cristalhq
78. ruleguard
79. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent"
80. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`).
81. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`)
82. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`) m.Match(`!($x != $y)`).Suggest(`$x == $y`) m.Match(`!($x == $y)`).Suggest(`$x != $y`) }
83. Thank you Questions? @oleg_kovalov @cristaloleg