Tweet
Tweet

Slide 1

Slide 1 text

Impguard protect your project structure Warsaw, November 2020 Oleg Kovalov

Slide 2

Slide 2 text

• Gopher for ~5 years • Open source contributor • Engineer at GoGoApps Hi, i’m Oleg olegk.dev @oleg_kovalov @cristaloleg

Slide 3

Slide 3 text

At GoGoApps we’re...

Slide 4

Slide 4 text

At GoGoApps we’re... • (re)building a product for a big company™

Slide 5

Slide 5 text

At GoGoApps we’re... • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic

Slide 6

Slide 6 text

At GoGoApps we’re... • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be

Slide 7

Slide 7 text

At GoGoApps we’re... • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!

Slide 8

Slide 8 text

At GoGoApps we’re... But there is one thing is running in my mind for a long time… • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!

Slide 9

Slide 9 text

Define me a problem first

Slide 10

Slide 10 text

Define me a problem first • linters give you suggestions on code level • mighty golangci-lint is The Beast

Slide 11

Slide 11 text

Define me a problem first • linters give you suggestions on code level • mighty golangci-lint is The Beast • but let’s look level above

Slide 12

Slide 12 text

Define me a problem first • linters give you suggestions on code level • mighty golangci-lint is The Beast • but let’s look level above • how your packages are organized?

Slide 13

Slide 13 text

Setup, goals, solution

Slide 14

Slide 14 text

Setup, goals, solution • you’ve a monorepo • or just a bunch of repos

Slide 15

Slide 15 text

Setup, goals, solution • you’ve a monorepo • or just a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something

Slide 16

Slide 16 text

Setup, goals, solution • you’ve a monorepo • or just a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something • custom tooling solves this!

Slide 17

Slide 17 text

Example

Slide 18

Slide 18 text

Let’s talk about imports

Slide 19

Slide 19 text

Let’s talk about imports • core idea of (re)using others code • staying on the shoulders of giants as Newton said

Slide 20

Slide 20 text

Let’s talk about imports • core idea of (re)using others code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed

Slide 21

Slide 21 text

Let’s talk about imports • core idea of (re)using others code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed • but nothing is free • others code needs to be verified and used carefully • you can rely on any code and use it everywhere • but that might be a ticking bomb (LeftPad-like problem)

Slide 22

Slide 22 text

When in doubts - see golang/go

Slide 23

Slide 23 text

When in doubts - see golang/go • Go doesn’t allow import cycles - that’s cool • simpler compilation routine and code organization

Slide 24

Slide 24 text

When in doubts - see golang/go • Go doesn’t allow import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb

Slide 25

Slide 25 text

When in doubts - see golang/go • Go doesn’t allow import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb • Stdlib has a test for imports https://github.com/golang/go/blob/release-branch.go1.14/src/go/build/deps_test.go https://github.com/golang/go/blob/release-branch.go1.15/src/go/build/deps_test.go

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

No content

Slide 28

Slide 28 text

What does it mean?

Slide 29

Slide 29 text

What does it mean? • in other words you’ve a rigid code structure • you cannot import something without an explicit change

Slide 30

Slide 30 text

What does it mean? • in other words you’ve a rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages

Slide 31

Slide 31 text

What does it mean? • in other words you’ve a rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages • what we can do? • check Github!

Slide 32

Slide 32 text

What do we’ve right now?

Slide 33

Slide 33 text

What do we’ve right now? • `$ go mod why` • out of the box solution

Slide 34

Slide 34 text

What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint)

Slide 35

Slide 35 text

What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file)

Slide 36

Slide 36 text

What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies

Slide 37

Slide 37 text

What do we’ve right now? • `$ go mod why` • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies • … (sorry, I cannot copy-paste all the Github ):

Slide 38

Slide 38 text

go mod why

Slide 39

Slide 39 text

go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole tailscale.com/wgengine/router github.com/go-ole/go-ole

Slide 40

Slide 40 text

go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole tailscale.com/wgengine/router github.com/go-ole/go-ole > go mod why github.com/RoaringBitmap/roaring # github.com/RoaringBitmap/roaring code.gitea.io/gitea/modules/indexer/code github.com/blevesearch/bleve github.com/blevesearch/bleve/index/scorch github.com/RoaringBitmap/roaring

Slide 41

Slide 41 text

tailscale/depaware

Slide 42

Slide 42 text

divan/depscheck

Slide 43

Slide 43 text

What is not ok? (only for me?)

Slide 44

Slide 44 text

What is not ok? (only for me?) • solving author’s problem • let’s look at a broader problem

Slide 45

Slide 45 text

What is not ok? (only for me?) • solving author’s problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list

Slide 46

Slide 46 text

What is not ok? (only for me?) • solving author’s problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list • already feature complete • boooooooo, we need a room for the improvements!

Slide 47

Slide 47 text

Welcome impguard

Slide 48

Slide 48 text

Welcome impguard • read as „import guard”

Slide 49

Slide 49 text

Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed

Slide 50

Slide 50 text

Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me!

Slide 51

Slide 51 text

Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found

Slide 52

Slide 52 text

Welcome impguard • read as „import guard” • helps to verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found yet?…

Slide 53

Slide 53 text

How does it work?

Slide 54

Slide 54 text

How does it work? • go list -json all | impguard project.yaml

Slide 55

Slide 55 text

How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things

Slide 56

Slide 56 text

How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic

Slide 57

Slide 57 text

How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file

Slide 58

Slide 58 text

How does it work? • go list -json all | impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file • simple, isn’t ?

Slide 59

Slide 59 text

go list gitea ❯ time go list -json all > 1.txt go list -json all > 1.txt 3.07s user 2.63s system 376% cpu 1.516 total ❯ du -sh 1.txt 4.0M 1.txt

Slide 60

Slide 60 text

Show me examples!

Slide 61

Slide 61 text

Show me examples! "*": - name: must have doc.go checks: type: has_file pattern: doc.go

Slide 62

Slide 62 text

Show me examples! "*": - name: must have doc.go checks: type: has_file pattern: doc.go - name: explicit runtime is forbidden checks: type: has_import pattern: runtime exclude: github.com/company/runtimeutil

Slide 63

Slide 63 text

Show me more examples! github.com/company: - name: don't use bad UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid

Slide 64

Slide 64 text

Show me more examples! github.com/company: - name: don't use bad UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid - name: no vendors, just modules checks: type: no_dir pattern: vendor

Slide 65

Slide 65 text

Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md

Slide 66

Slide 66 text

Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md github.com/company/project/services/*: - name: services should be independent checks: - type: no_import pattern: github.com/company/project/services/*

Slide 67

Slide 67 text

Where is the code Lebowski?

Slide 68

Slide 68 text

Where is the code Lebowski? • good question

Slide 69

Slide 69 text

Where is the code Lebowski? • good question • will be available • https://github.com/cristaloleg/impguard

Slide 70

Slide 70 text

Where is the code Lebowski? • good question • will be available • https://github.com/cristaloleg/impguard • feel free to share your ideas!

Slide 71

Slide 71 text

Where is the code Lebowski? • good question • will be available • https://github.com/cristaloleg/impguard • feel free to share your ideas! • no, really

Slide 72

Slide 72 text

Thanks

Slide 73

Slide 73 text

Thanks • Yegor Myskin

Slide 74

Slide 74 text

Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan Storozhuk • Alexey Palazhchenko

Slide 75

Slide 75 text

Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi

Slide 76

Slide 76 text

Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi • you and Golang Poland <3

Slide 77

Slide 77 text

Also see • https://github.com/go-critic/go-critic • https://github.com/quasilyte/go-ruleguard • https://github.com/cristalhq

Slide 78

Slide 78 text

ruleguard

Slide 79

Slide 79 text

ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent"

Slide 80

Slide 80 text

ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`).

Slide 81

Slide 81 text

ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`)

Slide 82

Slide 82 text

ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`) m.Match(`!($x != $y)`).Suggest(`$x == $y`) m.Match(`!($x == $y)`).Suggest(`$x != $y`) }

Slide 83

Slide 83 text

Thank you Questions? @oleg_kovalov @cristaloleg