Upgrade to Pro — share decks privately, control downloads, hide ads and more …

impguard - protect your project structure

64a4ba69d50590e592cd8e572454daa8?s=47 Oleg
November 26, 2020

impguard - protect your project structure

64a4ba69d50590e592cd8e572454daa8?s=128

Oleg

November 26, 2020
Tweet

Transcript

  1. Impguard protect your project structure Warsaw, November 2020 Oleg Kovalov

  2. • Gopher for ~5 years • Open source contributor •

    Engineer at GoGoApps Hi, i’m Oleg olegk.dev @oleg_kovalov @cristaloleg
  3. At GoGoApps we’re...

  4. At GoGoApps we’re... • (re)building a product for a big

    company™
  5. At GoGoApps we’re... • (re)building a product for a big

    company™ • with a lot of microservices • and a complicated business logic
  6. At GoGoApps we’re... • (re)building a product for a big

    company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be
  7. At GoGoApps we’re... • (re)building a product for a big

    company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!
  8. At GoGoApps we’re... But there is one thing is running

    in my mind for a long time… • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!
  9. Define me a problem first

  10. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast
  11. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast • but let’s look level above
  12. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast • but let’s look level above • how your packages are organized?
  13. Setup, goals, solution

  14. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos
  15. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something
  16. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something • custom tooling solves this!
  17. Example

  18. Let’s talk about imports

  19. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said
  20. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed
  21. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed • but nothing is free • others code needs to be verified and used carefully • you can rely on any code and use it everywhere • but that might be a ticking bomb (LeftPad-like problem)
  22. When in doubts - see golang/go

  23. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization
  24. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb
  25. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb • Stdlib has a test for imports https://github.com/golang/go/blob/release-branch.go1.14/src/go/build/deps_test.go https://github.com/golang/go/blob/release-branch.go1.15/src/go/build/deps_test.go
  26. None
  27. None
  28. What does it mean?

  29. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change
  30. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages
  31. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages • what we can do? • check Github!
  32. What do we’ve right now?

  33. What do we’ve right now? • `$ go mod why`

    • out of the box solution
  34. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint)
  35. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file)
  36. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies
  37. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies • … (sorry, I cannot copy-paste all the Github ):
  38. go mod why

  39. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole

    tailscale.com/wgengine/router github.com/go-ole/go-ole
  40. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole

    tailscale.com/wgengine/router github.com/go-ole/go-ole > go mod why github.com/RoaringBitmap/roaring # github.com/RoaringBitmap/roaring code.gitea.io/gitea/modules/indexer/code github.com/blevesearch/bleve github.com/blevesearch/bleve/index/scorch github.com/RoaringBitmap/roaring
  41. tailscale/depaware

  42. divan/depscheck

  43. What is not ok? (only for me?)

  44. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem
  45. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list
  46. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list • already feature complete • boooooooo, we need a room for the improvements!
  47. Welcome impguard

  48. Welcome impguard • read as „import guard”

  49. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed
  50. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me!
  51. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found
  52. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found yet?…
  53. How does it work?

  54. How does it work? • go list -json all |

    impguard project.yaml
  55. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things
  56. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic
  57. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file
  58. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file • simple, isn’t ?
  59. go list gitea ❯ time go list -json all >

    1.txt go list -json all > 1.txt 3.07s user 2.63s system 376% cpu 1.516 total ❯ du -sh 1.txt 4.0M 1.txt
  60. Show me examples!

  61. Show me examples! "*": - name: must have doc.go checks:

    type: has_file pattern: doc.go
  62. Show me examples! "*": - name: must have doc.go checks:

    type: has_file pattern: doc.go - name: explicit runtime is forbidden checks: type: has_import pattern: runtime exclude: github.com/company/runtimeutil
  63. Show me more examples! github.com/company: - name: don't use bad

    UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid
  64. Show me more examples! github.com/company: - name: don't use bad

    UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid - name: no vendors, just modules checks: type: no_dir pattern: vendor
  65. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages

    for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md
  66. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages

    for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md github.com/company/project/services/*: - name: services should be independent checks: - type: no_import pattern: github.com/company/project/services/*
  67. Where is the code Lebowski?

  68. Where is the code Lebowski? • good question

  69. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard
  70. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard • feel free to share your ideas!
  71. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard • feel free to share your ideas! • no, really
  72. Thanks

  73. Thanks • Yegor Myskin

  74. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko
  75. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi
  76. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi • you and Golang Poland <3
  77. Also see • https://github.com/go-critic/go-critic • https://github.com/quasilyte/go-ruleguard • https://github.com/cristalhq

  78. ruleguard

  79. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent"

  80. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`).
  81. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`)
  82. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`) m.Match(`!($x != $y)`).Suggest(`$x == $y`) m.Match(`!($x == $y)`).Suggest(`$x != $y`) }
  83. Thank you Questions? @oleg_kovalov @cristaloleg