Fully automated release processes for Go projects

Fully automated release processes for Go projects

Go projects need to rely on an automated process that ushers changes through all aspects of a delivery pipeline. A typical pipeline involves resolving packages, compilation, testing, generation of code coverage, static code analysis, binary creation, and the actual release of the binaries.

This session discusses automation aspects and tools helpful for streamlining Go projects. First, you’ll get to know individual tools and why they are so useful. Then you’ll tie it all together to form a simple delivery pipeline for Go projects powered by Travis CI or Jenkins. You will leave the session with a recipe for implementing a sensible, yet simple end-to-end automation approach for your own Go projects to ensure reliable and reproducible builds.

8f2248c6bfcc6df39a2cd8edf4267cb5?s=128

Benjamin Muschko

September 16, 2019
Tweet

Transcript

  1. Fully automated release processes for projects Benjamin Muschko

  2. AUTOMATED ASCENT bmuschko bmuschko bmuschko.com About the speaker automatedascent.com

  3. Who’s using the language Go?

  4. Typical development tasks Package Management Source Code Analysis Testing Building

    Binaries Publishing Binaries CI/CD Pipeline
  5. Do I need a build tool?

  6. Go Modules The de facto packager manager

  7. High-level usage Enabled via env. variable GO111MODULE Check in generated

    go.mod, go.sum Direct integra5on with go get Will try to download from central proxy first
  8. SemanFc versioning 2 . 13 . 5 hBps:/ /semver.org/ Major

    Minor Patch (Breaking) (Feature) (Fix)
  9. Dependency declaraFon module github.com/bmuschko/letsgopher require ( github.com/Flaque/filet v0.0.0-20190209224823-fc4d33cfcf93 github.com/blang/semver v3.5.1+incompatible

    github.com/ghodss/yaml v1.0.0 github.com/gosuri/uitable v0.0.0-20160404203958-36ee7e946282 github.com/inconshreveable/mousetrap v1.0.0 // indirect github.com/kr/text v0.1.0 github.com/mattn/go-runewidth v0.0.4 // indirect github.com/mitchellh/go-homedir v1.0.0 github.com/spf13/afero v1.2.2 // indirect github.com/spf13/cobra v0.0.3 github.com/spf13/pflag v1.0.3 // indirect github.com/stretchr/testify v1.3.0 gopkg.in/AlecAivazis/survey.v1 v1.8.2 gopkg.in/yaml.v2 v2.2.2 // indirect ) go 1.13
  10. Building blocks go.mod go.sum vendor Optional ⚙ Go runtime External

    packages
  11. Testing is essential for fast feedback

  12. Picking a test framework Standard tesFng package is built-in Eliminate

    asser5on logic with TesFfy Ginkgo supports wri5ng BDD-style tests hBps:/ /bmuschko.com/blog/go-tes5ng-frameworks/
  13. Capturing code coverage metrics $ go test ./... -coverprofile=coverage.txt -covermode=count

    ? github.com/bmuschko/letsgopher [no test files] ok github.com/bmuschko/letsgopher/cmd 0.047s↵ coverage: 72.1% of statements ok github.com/bmuschko/letsgopher/template/archive 0.028s↵ coverage: 69.6% of statements ok github.com/bmuschko/letsgopher/template/config 0.022s↵ coverage: 94.1% of statements Vendor directory excluded with Go 1.9+ Write metrics to coverage.txt for processing
  14. Rendering HTML report $ go tool cover -html=coverage.txt

  15. Codecov coverage visualizaFon $ bash <(curl -s https://codecov.io/bash) hBps:/ /codecov.io

  16. Demo

  17. Code quality needs to be checked continuously

  18. Typical concerns Enforcing coding style convenFons Detec5ng common mistakes Finding

    unused code Calcula5ng cyclomaFc complexiFes
  19. Using GolangCI-Lint Aggregator for mul5ple tools Parallel execuFon of quality

    tools Free for Open Source projects hBps:/ /github.com/golangci/golangci-lint
  20. Example output $ golangci-lint run
 cmd/install.go:40:13: Error return value of

    `errors.New` is not↵ checked (errcheck)
 errors.New("Currently templates can only be installed from↵ a Git repository")
 ^
 templ/gen.go:22:15: Error return value of `utils.CopyDir` is not↵ checked (errcheck)
 utils.CopyDir(templatePath, targetPath)
 ^
 cmd/install.go:38:11: composites: `github.com/bmuschko/↵ lets-gopher-exercise/remote.GitRepo` composite literal uses↵ unkeyed fields (govet)
 repo = &remote.GitRepo{repoUrl, templ.TemplateDir}
 ^
  21. Demo

  22. Cross-compile binaries for target platforms

  23. Assembling binaries Executables for plaSorms & architectures Op5mized for performance

    & includes deps go build + GOOS and GOARCH env vars
  24. OpFons for creaFng binaries Write a script to automate CLI

    invoca5ons Gox: Parallelize builds for mul5ple plaSorms GoReleaser: Binary crea5on and releasing
  25. Using GoReleaser Defini5on in .goreleaser.yml Combina5on of OS and architecture

    Configurable binary crea5on and archives hBps:/ /goreleaser.com
  26. Building binaries $ goreleaser init • Generating .goreleaser.yml file •

    config created; please edit accordingly to your needs↵ file=.goreleaser.yml $ goreleaser --snapshot --skip-publish --rm-dist • BUILDING BINARIES • building binary=dist/darwin_386 • building binary=dist/linux_386 • building binary=dist/linux_amd64 • building binary=dist/darwin_amd64 • ARCHIVES • creating archive=dist/_v0.0.0-next_Darwin_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Linux_i386.tar.gz • creating archive=dist/_v0.0.0-next_Linux_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Darwin_i386.tar.gz
  27. InjecFng version into applicaFon builds:
 - ldflags: -s -w -X↵

    main.version={{.Version}} .goreleaser.yml package main
 
 import (
 "fmt"
 "github.com/bmuschko/letsgopher/cmd"
 "os"
 )
 
 var version = "undefined"
 
 func main() {
 cmd.SetVersion(version)
 } main.go
  28. Demo

  29. Releasing and hosting binaries

  30. Release funcFonality Target plaRorm dependent on consumers Versioned binaries trackable

    to SCM tag Automa5c release notes genera5on
  31. Publishing to GitHub/GitLab $ export GITHUB_TOKEN=`YOUR_GH_TOKEN`
 $ export GITLAB_TOKEN=`YOUR_GL_TOKEN`
 


    $ git tag -a v0.2.0 -m "First release"
 $ git push origin v0.2.0
 
 $ goreleaser
  32. Publishing to ArFfactory release:
 disable: true
 artifactories:
 - name: production


    target: http://localhost:8081/artifactory/↵
 example-repo-local/{{ .ProjectName }}/↵ {{ .Version }}/
 username: publisher .goreleaser.yml
  33. CredenFals as env. variables ARTIFACTORY_ PRODUCTION_ USERNAME ARTIFACTORY_ PRODUCTION_ SECRET

  34. Demo

  35. Full process modeled as CI/CD pipeline

  36. Different tooling opFons Travis CI: Hosted free and paid on-prem

    solu5on Jenkins: Free and commercial solu5on …and many others like Drone, Circle CI
  37. Possible release strategies Manual release by pressing a buBon Automa5c

    release aka ConFnuous Deployment Release on tagging a SCM commit
  38. Delivery pipeline steps Compile Test Code Analysis Release optional Commit

  39. Different use cases Compile Release Non-tagged Commit Test Code Analysis

    ✗ ✓ ✓ ✓ Compile Release Commit tagged v1.2.3 Test Code Analysis ✓ ✓ ✓ ✓
  40. Pipeline as code .travis.yml Jenkinsfile Source code in SCM check

    in evaluate
  41. language: go
 
 go:
 - 1.13.x
 
 env:
 - GO111MODULE=on


    
 install: true
 
 cache:
 directories:
 - $GOPATH/pkg/mod Basic setup
  42. jobs:
 include:
 - stage: "Compile"
 name: "Compile Packages and Dependencies"


    script: go build
 - stage: "Tests"
 name: "Unit Tests"
 script: go test ./... -coverprofile=coverage.txt -covermode=count
 after_success:
 - bash <(curl -s https://codecov.io/bash)
 - stage: "Code Quality"
 name: "Code Quality Analysis"
 before_script:
 - curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0
 script: golangci-lint run Modeling stages
  43. - stage: "Release"
 name: "Release Binaries"
 script: skip
 deploy:
 -

    provider: script
 skip_cleanup: true
 script: curl -sL https://git.io/goreleaser | bash
 on:
 tags: true
 condition: $TRAVIS_OS_NAME = linux Release binary for tagged commit
  44. Pipeline visualizaFon jobs:
 include:
 - stage: ...
 - stage: ...


    - stage: ...
  45. AutomaFc Go runFme installaFon Taken care of Jenkins Go plugin

    hBps:/ /wiki.jenkins.io/display/JENKINS/Go+Plugin
  46. pipeline {
 agent any
 tools {
 go 'go-1.13'
 }
 environment

    {
 GO111MODULE = 'on'
 }
 stages {
 ...
 }
 } Basic setup
  47. stage('Compile') {
 steps {
 sh 'go build'
 }
 }
 stage('Test')

    {
 environment {
 CODECOV_TOKEN = credentials('codecov_token')
 }
 steps {
 sh 'go test ./... -coverprofile=coverage.txt'
 sh 'curl -s https://codecov.io/bash | bash -s -'
 }
 }
 stage('Code Analysis') {
 steps {
 sh 'curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0'
 sh 'golangci-lint run'
 }
 } Modeling stages
  48. stage('Release') {
 when {
 buildingTag()
 }
 environment {
 GITHUB_TOKEN =

    credentials('github_token')
 }
 steps {
 sh 'curl -sL https://git.io/goreleaser | bash'
 }
 } Release binary for tagged commit
  49. Pipeline visualizaFon Standard Blue Ocean

  50. Demo

  51. Thanks! Please ask quesFons.