Fully automated release processes for Go projects

Fully automated release processes for Go projects

Go projects need to rely on an automated process that ushers changes through all aspects of a delivery pipeline. A typical pipeline involves resolving packages, compilation, testing, generation of code coverage, static code analysis, binary creation, and the actual release of the binaries.

This session discusses automation aspects and tools helpful for streamlining Go projects. First, you’ll get to know individual tools and why they are so useful. Then you’ll tie it all together to form a simple delivery pipeline for Go projects powered by Travis CI or Jenkins. You will leave the session with a recipe for implementing a sensible, yet simple end-to-end automation approach for your own Go projects to ensure reliable and reproducible builds.

8f2248c6bfcc6df39a2cd8edf4267cb5?s=128

Benjamin Muschko

September 16, 2019
Tweet

Transcript

  1. 7.

    High-level usage Enabled via env. variable GO111MODULE Check in generated

    go.mod, go.sum Direct integra5on with go get Will try to download from central proxy first
  2. 8.

    SemanFc versioning 2 . 13 . 5 hBps:/ /semver.org/ Major

    Minor Patch (Breaking) (Feature) (Fix)
  3. 9.

    Dependency declaraFon module github.com/bmuschko/letsgopher require ( github.com/Flaque/filet v0.0.0-20190209224823-fc4d33cfcf93 github.com/blang/semver v3.5.1+incompatible

    github.com/ghodss/yaml v1.0.0 github.com/gosuri/uitable v0.0.0-20160404203958-36ee7e946282 github.com/inconshreveable/mousetrap v1.0.0 // indirect github.com/kr/text v0.1.0 github.com/mattn/go-runewidth v0.0.4 // indirect github.com/mitchellh/go-homedir v1.0.0 github.com/spf13/afero v1.2.2 // indirect github.com/spf13/cobra v0.0.3 github.com/spf13/pflag v1.0.3 // indirect github.com/stretchr/testify v1.3.0 gopkg.in/AlecAivazis/survey.v1 v1.8.2 gopkg.in/yaml.v2 v2.2.2 // indirect ) go 1.13
  4. 12.

    Picking a test framework Standard tesFng package is built-in Eliminate

    asser5on logic with TesFfy Ginkgo supports wri5ng BDD-style tests hBps:/ /bmuschko.com/blog/go-tes5ng-frameworks/
  5. 13.

    Capturing code coverage metrics $ go test ./... -coverprofile=coverage.txt -covermode=count

    ? github.com/bmuschko/letsgopher [no test files] ok github.com/bmuschko/letsgopher/cmd 0.047s↵ coverage: 72.1% of statements ok github.com/bmuschko/letsgopher/template/archive 0.028s↵ coverage: 69.6% of statements ok github.com/bmuschko/letsgopher/template/config 0.022s↵ coverage: 94.1% of statements Vendor directory excluded with Go 1.9+ Write metrics to coverage.txt for processing
  6. 16.
  7. 19.

    Using GolangCI-Lint Aggregator for mul5ple tools Parallel execuFon of quality

    tools Free for Open Source projects hBps:/ /github.com/golangci/golangci-lint
  8. 20.

    Example output $ golangci-lint run
 cmd/install.go:40:13: Error return value of

    `errors.New` is not↵ checked (errcheck)
 errors.New("Currently templates can only be installed from↵ a Git repository")
 ^
 templ/gen.go:22:15: Error return value of `utils.CopyDir` is not↵ checked (errcheck)
 utils.CopyDir(templatePath, targetPath)
 ^
 cmd/install.go:38:11: composites: `github.com/bmuschko/↵ lets-gopher-exercise/remote.GitRepo` composite literal uses↵ unkeyed fields (govet)
 repo = &remote.GitRepo{repoUrl, templ.TemplateDir}
 ^
  9. 21.
  10. 23.
  11. 24.

    OpFons for creaFng binaries Write a script to automate CLI

    invoca5ons Gox: Parallelize builds for mul5ple plaSorms GoReleaser: Binary crea5on and releasing
  12. 25.

    Using GoReleaser Defini5on in .goreleaser.yml Combina5on of OS and architecture

    Configurable binary crea5on and archives hBps:/ /goreleaser.com
  13. 26.

    Building binaries $ goreleaser init • Generating .goreleaser.yml file •

    config created; please edit accordingly to your needs↵ file=.goreleaser.yml $ goreleaser --snapshot --skip-publish --rm-dist • BUILDING BINARIES • building binary=dist/darwin_386 • building binary=dist/linux_386 • building binary=dist/linux_amd64 • building binary=dist/darwin_amd64 • ARCHIVES • creating archive=dist/_v0.0.0-next_Darwin_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Linux_i386.tar.gz • creating archive=dist/_v0.0.0-next_Linux_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Darwin_i386.tar.gz
  14. 27.

    InjecFng version into applicaFon builds:
 - ldflags: -s -w -X↵

    main.version={{.Version}} .goreleaser.yml package main
 
 import (
 "fmt"
 "github.com/bmuschko/letsgopher/cmd"
 "os"
 )
 
 var version = "undefined"
 
 func main() {
 cmd.SetVersion(version)
 } main.go
  15. 28.
  16. 31.

    Publishing to GitHub/GitLab $ export GITHUB_TOKEN=`YOUR_GH_TOKEN`
 $ export GITLAB_TOKEN=`YOUR_GL_TOKEN`
 


    $ git tag -a v0.2.0 -m "First release"
 $ git push origin v0.2.0
 
 $ goreleaser
  17. 32.

    Publishing to ArFfactory release:
 disable: true
 artifactories:
 - name: production


    target: http://localhost:8081/artifactory/↵
 example-repo-local/{{ .ProjectName }}/↵ {{ .Version }}/
 username: publisher .goreleaser.yml
  18. 34.
  19. 36.

    Different tooling opFons Travis CI: Hosted free and paid on-prem

    solu5on Jenkins: Free and commercial solu5on …and many others like Drone, Circle CI
  20. 37.

    Possible release strategies Manual release by pressing a buBon Automa5c

    release aka ConFnuous Deployment Release on tagging a SCM commit
  21. 39.

    Different use cases Compile Release Non-tagged Commit Test Code Analysis

    ✗ ✓ ✓ ✓ Compile Release Commit tagged v1.2.3 Test Code Analysis ✓ ✓ ✓ ✓
  22. 41.

    language: go
 
 go:
 - 1.13.x
 
 env:
 - GO111MODULE=on


    
 install: true
 
 cache:
 directories:
 - $GOPATH/pkg/mod Basic setup
  23. 42.

    jobs:
 include:
 - stage: "Compile"
 name: "Compile Packages and Dependencies"


    script: go build
 - stage: "Tests"
 name: "Unit Tests"
 script: go test ./... -coverprofile=coverage.txt -covermode=count
 after_success:
 - bash <(curl -s https://codecov.io/bash)
 - stage: "Code Quality"
 name: "Code Quality Analysis"
 before_script:
 - curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0
 script: golangci-lint run Modeling stages
  24. 43.

    - stage: "Release"
 name: "Release Binaries"
 script: skip
 deploy:
 -

    provider: script
 skip_cleanup: true
 script: curl -sL https://git.io/goreleaser | bash
 on:
 tags: true
 condition: $TRAVIS_OS_NAME = linux Release binary for tagged commit
  25. 45.

    AutomaFc Go runFme installaFon Taken care of Jenkins Go plugin

    hBps:/ /wiki.jenkins.io/display/JENKINS/Go+Plugin
  26. 46.

    pipeline {
 agent any
 tools {
 go 'go-1.13'
 }
 environment

    {
 GO111MODULE = 'on'
 }
 stages {
 ...
 }
 } Basic setup
  27. 47.

    stage('Compile') {
 steps {
 sh 'go build'
 }
 }
 stage('Test')

    {
 environment {
 CODECOV_TOKEN = credentials('codecov_token')
 }
 steps {
 sh 'go test ./... -coverprofile=coverage.txt'
 sh 'curl -s https://codecov.io/bash | bash -s -'
 }
 }
 stage('Code Analysis') {
 steps {
 sh 'curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0'
 sh 'golangci-lint run'
 }
 } Modeling stages
  28. 48.

    stage('Release') {
 when {
 buildingTag()
 }
 environment {
 GITHUB_TOKEN =

    credentials('github_token')
 }
 steps {
 sh 'curl -sL https://git.io/goreleaser | bash'
 }
 } Release binary for tagged commit
  29. 50.