impguard - protect your project structure

Transcript

1. Impguard protect your project structure Warsaw, November 2020 Oleg Kovalov

2. • Gopher for ~5 years • Open source contributor •

   Engineer at GoGoApps Hi, i’m Oleg olegk.dev @oleg_kovalov @cristaloleg

3. At GoGoApps we’re...

4. At GoGoApps we’re... • (re)building a product for a big

   company™

5. At GoGoApps we’re... • (re)building a product for a big

   company™ • with a lot of microservices • and a complicated business logic

6. At GoGoApps we’re... • (re)building a product for a big

   company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be

7. At GoGoApps we’re... • (re)building a product for a big

   company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!

8. At GoGoApps we’re... But there is one thing is running

   in my mind for a long time… • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!

9. Define me a problem first

10. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast

11. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast • but let’s look level above

12. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast • but let’s look level above • how your packages are organized?

13. Setup, goals, solution

14. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos

15. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something

16. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something • custom tooling solves this!

17. Example

18. Let’s talk about imports

19. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said

20. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed

21. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed • but nothing is free • others code needs to be verified and used carefully • you can rely on any code and use it everywhere • but that might be a ticking bomb (LeftPad-like problem)

22. When in doubts - see golang/go

23. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization

24. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb

25. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb • Stdlib has a test for imports https://github.com/golang/go/blob/release-branch.go1.14/src/go/build/deps_test.go https://github.com/golang/go/blob/release-branch.go1.15/src/go/build/deps_test.go
26. None
27. None

28. What does it mean?

29. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change

30. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages

31. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages • what we can do? • check Github!

32. What do we’ve right now?

33. What do we’ve right now? • `$ go mod why`

    • out of the box solution

34. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint)

35. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file)

36. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies

37. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies • … (sorry, I cannot copy-paste all the Github ):

38. go mod why

39. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole

    tailscale.com/wgengine/router github.com/go-ole/go-ole

40. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole

    tailscale.com/wgengine/router github.com/go-ole/go-ole > go mod why github.com/RoaringBitmap/roaring # github.com/RoaringBitmap/roaring code.gitea.io/gitea/modules/indexer/code github.com/blevesearch/bleve github.com/blevesearch/bleve/index/scorch github.com/RoaringBitmap/roaring

41. tailscale/depaware

42. divan/depscheck

43. What is not ok? (only for me?)

44. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem

45. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list

46. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list • already feature complete • boooooooo, we need a room for the improvements!

47. Welcome impguard

48. Welcome impguard • read as „import guard”

49. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed

50. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me!

51. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found

52. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found yet?…

53. How does it work?

54. How does it work? • go list -json all |

    impguard project.yaml

55. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things

56. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic

57. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file

58. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file • simple, isn’t ?

59. go list gitea ❯ time go list -json all >

    1.txt go list -json all > 1.txt 3.07s user 2.63s system 376% cpu 1.516 total ❯ du -sh 1.txt 4.0M 1.txt

60. Show me examples!

61. Show me examples! "*": - name: must have doc.go checks:

    type: has_file pattern: doc.go

62. Show me examples! "*": - name: must have doc.go checks:

    type: has_file pattern: doc.go - name: explicit runtime is forbidden checks: type: has_import pattern: runtime exclude: github.com/company/runtimeutil

63. Show me more examples! github.com/company: - name: don't use bad

    UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid

64. Show me more examples! github.com/company: - name: don't use bad

    UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid - name: no vendors, just modules checks: type: no_dir pattern: vendor

65. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages

    for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md

66. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages

    for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md github.com/company/project/services/*: - name: services should be independent checks: - type: no_import pattern: github.com/company/project/services/*

67. Where is the code Lebowski?

68. Where is the code Lebowski? • good question

69. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard

70. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard • feel free to share your ideas!

71. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard • feel free to share your ideas! • no, really

72. Thanks

73. Thanks • Yegor Myskin

74. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko

75. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi

76. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi • you and Golang Poland <3

77. Also see • https://github.com/go-critic/go-critic • https://github.com/quasilyte/go-ruleguard • https://github.com/cristalhq

78. ruleguard

79. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent"

80. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`).

81. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`)

82. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`) m.Match(`!($x != $y)`).Suggest(`$x == $y`) m.Match(`!($x == $y)`).Suggest(`$x != $y`) }

83. Thank you Questions? @oleg_kovalov @cristaloleg