Fully automated release processes for Go projects

Go projects need to rely on an automated process that ushers changes through all aspects of a delivery pipeline. A typical pipeline involves resolving packages, compilation, testing, generation of code coverage, static code analysis, binary creation, and the actual release of the binaries.

This session discusses automation aspects and tools helpful for streamlining Go projects. First, you’ll get to know individual tools and why they are so useful. Then you’ll tie it all together to form a simple delivery pipeline for Go projects powered by Travis CI or Jenkins. You will leave the session with a recipe for implementing a sensible, yet simple end-to-end automation approach for your own Go projects to ensure reliable and reproducible builds.

Transcript

1. Fully automated release processes for projects Benjamin Muschko
2. AUTOMATED ASCENT bmuschko bmuschko bmuschko.com About the speaker automatedascent.com
3. Who’s using the language Go?
4. Typical development tasks Package Management Source Code Analysis Testing Building Binaries Publishing Binaries CI/CD Pipeline
5. Do I need a build tool?
6. Go Modules The de facto packager manager
7. High-level usage Enabled via env. variable GO111MODULE Check in generated go.mod, go.sum Direct integra5on with go get Will try to download from central proxy first
8. SemanFc versioning 2 . 13 . 5 hBps:/ /semver.org/ Major Minor Patch (Breaking) (Feature) (Fix)
9. Dependency declaraFon module github.com/bmuschko/letsgopher require ( github.com/Flaque/filet v0.0.0-20190209224823-fc4d33cfcf93 github.com/blang/semver v3.5.1+incompatible github.com/ghodss/yaml v1.0.0 github.com/gosuri/uitable v0.0.0-20160404203958-36ee7e946282 github.com/inconshreveable/mousetrap v1.0.0 // indirect github.com/kr/text v0.1.0 github.com/mattn/go-runewidth v0.0.4 // indirect github.com/mitchellh/go-homedir v1.0.0 github.com/spf13/afero v1.2.2 // indirect github.com/spf13/cobra v0.0.3 github.com/spf13/pflag v1.0.3 // indirect github.com/stretchr/testify v1.3.0 gopkg.in/AlecAivazis/survey.v1 v1.8.2 gopkg.in/yaml.v2 v2.2.2 // indirect ) go 1.13
10. Building blocks go.mod go.sum vendor Optional ⚙ Go runtime External packages
11. Testing is essential for fast feedback
12. Picking a test framework Standard tesFng package is built-in Eliminate asser5on logic with TesFfy Ginkgo supports wri5ng BDD-style tests hBps:/ /bmuschko.com/blog/go-tes5ng-frameworks/
13. Capturing code coverage metrics $ go test ./... -coverprofile=coverage.txt -covermode=count ? github.com/bmuschko/letsgopher [no test files] ok github.com/bmuschko/letsgopher/cmd 0.047s↵ coverage: 72.1% of statements ok github.com/bmuschko/letsgopher/template/archive 0.028s↵ coverage: 69.6% of statements ok github.com/bmuschko/letsgopher/template/config 0.022s↵ coverage: 94.1% of statements Vendor directory excluded with Go 1.9+ Write metrics to coverage.txt for processing
14. Rendering HTML report $ go tool cover -html=coverage.txt
15. Codecov coverage visualizaFon $ bash <(curl -s https://codecov.io/bash) hBps:/ /codecov.io
16. Demo
17. Code quality needs to be checked continuously
18. Typical concerns Enforcing coding style convenFons Detec5ng common mistakes Finding unused code Calcula5ng cyclomaFc complexiFes
19. Using GolangCI-Lint Aggregator for mul5ple tools Parallel execuFon of quality tools Free for Open Source projects hBps:/ /github.com/golangci/golangci-lint
20. Example output $ golangci-lint run
 cmd/install.go:40:13: Error return value of `errors.New` is not↵ checked (errcheck)
 errors.New("Currently templates can only be installed from↵ a Git repository")
 ^
 templ/gen.go:22:15: Error return value of `utils.CopyDir` is not↵ checked (errcheck)
 utils.CopyDir(templatePath, targetPath)
 ^
 cmd/install.go:38:11: composites: `github.com/bmuschko/↵ lets-gopher-exercise/remote.GitRepo` composite literal uses↵ unkeyed fields (govet)
 repo = &remote.GitRepo{repoUrl, templ.TemplateDir}
 ^
21. Demo
22. Cross-compile binaries for target platforms
23. Assembling binaries Executables for plaSorms & architectures Op5mized for performance & includes deps go build + GOOS and GOARCH env vars
24. OpFons for creaFng binaries Write a script to automate CLI invoca5ons Gox: Parallelize builds for mul5ple plaSorms GoReleaser: Binary crea5on and releasing
25. Using GoReleaser Defini5on in .goreleaser.yml Combina5on of OS and architecture Configurable binary crea5on and archives hBps:/ /goreleaser.com
26. Building binaries $ goreleaser init • Generating .goreleaser.yml file • config created; please edit accordingly to your needs↵ file=.goreleaser.yml $ goreleaser --snapshot --skip-publish --rm-dist • BUILDING BINARIES • building binary=dist/darwin_386 • building binary=dist/linux_386 • building binary=dist/linux_amd64 • building binary=dist/darwin_amd64 • ARCHIVES • creating archive=dist/_v0.0.0-next_Darwin_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Linux_i386.tar.gz • creating archive=dist/_v0.0.0-next_Linux_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Darwin_i386.tar.gz
27. InjecFng version into applicaFon builds:
 - ldflags: -s -w -X↵ main.version={{.Version}} .goreleaser.yml package main
 
 import (
 "fmt"
 "github.com/bmuschko/letsgopher/cmd"
 "os"
 )
 
 var version = "undefined"
 
 func main() {
 cmd.SetVersion(version)
 } main.go
28. Demo
29. Releasing and hosting binaries
30. Release funcFonality Target plaRorm dependent on consumers Versioned binaries trackable to SCM tag Automa5c release notes genera5on
31. Publishing to GitHub/GitLab $ export GITHUB_TOKEN=`YOUR_GH_TOKEN`
 $ export GITLAB_TOKEN=`YOUR_GL_TOKEN`
 
 $ git tag -a v0.2.0 -m "First release"
 $ git push origin v0.2.0
 
 $ goreleaser
32. Publishing to ArFfactory release:
 disable: true
 artifactories:
 - name: production
 target: http://localhost:8081/artifactory/↵
 example-repo-local/{{ .ProjectName }}/↵ {{ .Version }}/
 username: publisher .goreleaser.yml
33. CredenFals as env. variables ARTIFACTORY_ PRODUCTION_ USERNAME ARTIFACTORY_ PRODUCTION_ SECRET
34. Demo
35. Full process modeled as CI/CD pipeline
36. Different tooling opFons Travis CI: Hosted free and paid on-prem solu5on Jenkins: Free and commercial solu5on …and many others like Drone, Circle CI
37. Possible release strategies Manual release by pressing a buBon Automa5c release aka ConFnuous Deployment Release on tagging a SCM commit
38. Delivery pipeline steps Compile Test Code Analysis Release optional Commit
39. Different use cases Compile Release Non-tagged Commit Test Code Analysis ✗ ✓ ✓ ✓ Compile Release Commit tagged v1.2.3 Test Code Analysis ✓ ✓ ✓ ✓
40. Pipeline as code .travis.yml Jenkinsfile Source code in SCM check in evaluate
41. language: go
 
 go:
 - 1.13.x
 
 env:
 - GO111MODULE=on
 
 install: true
 
 cache:
 directories:
 - $GOPATH/pkg/mod Basic setup
42. jobs:
 include:
 - stage: "Compile"
 name: "Compile Packages and Dependencies"
 script: go build
 - stage: "Tests"
 name: "Unit Tests"
 script: go test ./... -coverprofile=coverage.txt -covermode=count
 after_success:
 - bash <(curl -s https://codecov.io/bash)
 - stage: "Code Quality"
 name: "Code Quality Analysis"
 before_script:
 - curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0
 script: golangci-lint run Modeling stages
43. - stage: "Release"
 name: "Release Binaries"
 script: skip
 deploy:
 - provider: script
 skip_cleanup: true
 script: curl -sL https://git.io/goreleaser | bash
 on:
 tags: true
 condition: $TRAVIS_OS_NAME = linux Release binary for tagged commit
44. Pipeline visualizaFon jobs:
 include:
 - stage: ...
 - stage: ...
 - stage: ...
45. AutomaFc Go runFme installaFon Taken care of Jenkins Go plugin hBps:/ /wiki.jenkins.io/display/JENKINS/Go+Plugin
46. pipeline {
 agent any
 tools {
 go 'go-1.13'
 }
 environment {
 GO111MODULE = 'on'
 }
 stages {
 ...
 }
 } Basic setup
47. stage('Compile') {
 steps {
 sh 'go build'
 }
 }
 stage('Test') {
 environment {
 CODECOV_TOKEN = credentials('codecov_token')
 }
 steps {
 sh 'go test ./... -coverprofile=coverage.txt'
 sh 'curl -s https://codecov.io/bash | bash -s -'
 }
 }
 stage('Code Analysis') {
 steps {
 sh 'curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0'
 sh 'golangci-lint run'
 }
 } Modeling stages
48. stage('Release') {
 when {
 buildingTag()
 }
 environment {
 GITHUB_TOKEN = credentials('github_token')
 }
 steps {
 sh 'curl -sL https://git.io/goreleaser | bash'
 }
 } Release binary for tagged commit
49. Pipeline visualizaFon Standard Blue Ocean
50. Demo
51. Thanks! Please ask quesFons.