Slide 1

Slide 1 text

Fully automated release processes for projects Benjamin Muschko

Slide 2

Slide 2 text

AUTOMATED ASCENT bmuschko bmuschko bmuschko.com About the speaker automatedascent.com

Slide 3

Slide 3 text

Who’s using the language Go?

Slide 4

Slide 4 text

Typical development tasks Package Management Source Code Analysis Testing Building Binaries Publishing Binaries CI/CD Pipeline

Slide 5

Slide 5 text

Do I need a build tool?

Slide 6

Slide 6 text

Go Modules The de facto packager manager

Slide 7

Slide 7 text

High-level usage Enabled via env. variable GO111MODULE Check in generated go.mod, go.sum Direct integra5on with go get Will try to download from central proxy first

Slide 8

Slide 8 text

SemanFc versioning 2 . 13 . 5 hBps:/ /semver.org/ Major Minor Patch (Breaking) (Feature) (Fix)

Slide 9

Slide 9 text

Dependency declaraFon module github.com/bmuschko/letsgopher require ( github.com/Flaque/filet v0.0.0-20190209224823-fc4d33cfcf93 github.com/blang/semver v3.5.1+incompatible github.com/ghodss/yaml v1.0.0 github.com/gosuri/uitable v0.0.0-20160404203958-36ee7e946282 github.com/inconshreveable/mousetrap v1.0.0 // indirect github.com/kr/text v0.1.0 github.com/mattn/go-runewidth v0.0.4 // indirect github.com/mitchellh/go-homedir v1.0.0 github.com/spf13/afero v1.2.2 // indirect github.com/spf13/cobra v0.0.3 github.com/spf13/pflag v1.0.3 // indirect github.com/stretchr/testify v1.3.0 gopkg.in/AlecAivazis/survey.v1 v1.8.2 gopkg.in/yaml.v2 v2.2.2 // indirect ) go 1.13

Slide 10

Slide 10 text

Building blocks go.mod go.sum vendor Optional ⚙ Go runtime External packages

Slide 11

Slide 11 text

Testing is essential for fast feedback

Slide 12

Slide 12 text

Picking a test framework Standard tesFng package is built-in Eliminate asser5on logic with TesFfy Ginkgo supports wri5ng BDD-style tests hBps:/ /bmuschko.com/blog/go-tes5ng-frameworks/

Slide 13

Slide 13 text

Capturing code coverage metrics $ go test ./... -coverprofile=coverage.txt -covermode=count ? github.com/bmuschko/letsgopher [no test files] ok github.com/bmuschko/letsgopher/cmd 0.047s↵ coverage: 72.1% of statements ok github.com/bmuschko/letsgopher/template/archive 0.028s↵ coverage: 69.6% of statements ok github.com/bmuschko/letsgopher/template/config 0.022s↵ coverage: 94.1% of statements Vendor directory excluded with Go 1.9+ Write metrics to coverage.txt for processing

Slide 14

Slide 14 text

Rendering HTML report $ go tool cover -html=coverage.txt

Slide 15

Slide 15 text

Codecov coverage visualizaFon $ bash <(curl -s https://codecov.io/bash) hBps:/ /codecov.io

Slide 16

Slide 16 text

Demo

Slide 17

Slide 17 text

Code quality needs to be checked continuously

Slide 18

Slide 18 text

Typical concerns Enforcing coding style convenFons Detec5ng common mistakes Finding unused code Calcula5ng cyclomaFc complexiFes

Slide 19

Slide 19 text

Using GolangCI-Lint Aggregator for mul5ple tools Parallel execuFon of quality tools Free for Open Source projects hBps:/ /github.com/golangci/golangci-lint

Slide 20

Slide 20 text

Example output $ golangci-lint run
 cmd/install.go:40:13: Error return value of `errors.New` is not↵ checked (errcheck)
 errors.New("Currently templates can only be installed from↵ a Git repository")
 ^
 templ/gen.go:22:15: Error return value of `utils.CopyDir` is not↵ checked (errcheck)
 utils.CopyDir(templatePath, targetPath)
 ^
 cmd/install.go:38:11: composites: `github.com/bmuschko/↵ lets-gopher-exercise/remote.GitRepo` composite literal uses↵ unkeyed fields (govet)
 repo = &remote.GitRepo{repoUrl, templ.TemplateDir}
 ^

Slide 21

Slide 21 text

Demo

Slide 22

Slide 22 text

Cross-compile binaries for target platforms

Slide 23

Slide 23 text

Assembling binaries Executables for plaSorms & architectures Op5mized for performance & includes deps go build + GOOS and GOARCH env vars

Slide 24

Slide 24 text

OpFons for creaFng binaries Write a script to automate CLI invoca5ons Gox: Parallelize builds for mul5ple plaSorms GoReleaser: Binary crea5on and releasing

Slide 25

Slide 25 text

Using GoReleaser Defini5on in .goreleaser.yml Combina5on of OS and architecture Configurable binary crea5on and archives hBps:/ /goreleaser.com

Slide 26

Slide 26 text

Building binaries $ goreleaser init • Generating .goreleaser.yml file • config created; please edit accordingly to your needs↵ file=.goreleaser.yml $ goreleaser --snapshot --skip-publish --rm-dist • BUILDING BINARIES • building binary=dist/darwin_386 • building binary=dist/linux_386 • building binary=dist/linux_amd64 • building binary=dist/darwin_amd64 • ARCHIVES • creating archive=dist/_v0.0.0-next_Darwin_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Linux_i386.tar.gz • creating archive=dist/_v0.0.0-next_Linux_x86_64.tar.gz • creating archive=dist/_v0.0.0-next_Darwin_i386.tar.gz

Slide 27

Slide 27 text

InjecFng version into applicaFon builds:
 - ldflags: -s -w -X↵ main.version={{.Version}} .goreleaser.yml package main
 
 import (
 "fmt"
 "github.com/bmuschko/letsgopher/cmd"
 "os"
 )
 
 var version = "undefined"
 
 func main() {
 cmd.SetVersion(version)
 } main.go

Slide 28

Slide 28 text

Demo

Slide 29

Slide 29 text

Releasing and hosting binaries

Slide 30

Slide 30 text

Release funcFonality Target plaRorm dependent on consumers Versioned binaries trackable to SCM tag Automa5c release notes genera5on

Slide 31

Slide 31 text

Publishing to GitHub/GitLab $ export GITHUB_TOKEN=`YOUR_GH_TOKEN`
 $ export GITLAB_TOKEN=`YOUR_GL_TOKEN`
 
 $ git tag -a v0.2.0 -m "First release"
 $ git push origin v0.2.0
 
 $ goreleaser

Slide 32

Slide 32 text

Publishing to ArFfactory release:
 disable: true
 artifactories:
 - name: production
 target: http://localhost:8081/artifactory/↵
 example-repo-local/{{ .ProjectName }}/↵ {{ .Version }}/
 username: publisher .goreleaser.yml

Slide 33

Slide 33 text

CredenFals as env. variables ARTIFACTORY_ PRODUCTION_ USERNAME ARTIFACTORY_ PRODUCTION_ SECRET

Slide 34

Slide 34 text

Demo

Slide 35

Slide 35 text

Full process modeled as CI/CD pipeline

Slide 36

Slide 36 text

Different tooling opFons Travis CI: Hosted free and paid on-prem solu5on Jenkins: Free and commercial solu5on …and many others like Drone, Circle CI

Slide 37

Slide 37 text

Possible release strategies Manual release by pressing a buBon Automa5c release aka ConFnuous Deployment Release on tagging a SCM commit

Slide 38

Slide 38 text

Delivery pipeline steps Compile Test Code Analysis Release optional Commit

Slide 39

Slide 39 text

Different use cases Compile Release Non-tagged Commit Test Code Analysis ✗ ✓ ✓ ✓ Compile Release Commit tagged v1.2.3 Test Code Analysis ✓ ✓ ✓ ✓

Slide 40

Slide 40 text

Pipeline as code .travis.yml Jenkinsfile Source code in SCM check in evaluate

Slide 41

Slide 41 text

language: go
 
 go:
 - 1.13.x
 
 env:
 - GO111MODULE=on
 
 install: true
 
 cache:
 directories:
 - $GOPATH/pkg/mod Basic setup

Slide 42

Slide 42 text

jobs:
 include:
 - stage: "Compile"
 name: "Compile Packages and Dependencies"
 script: go build
 - stage: "Tests"
 name: "Unit Tests"
 script: go test ./... -coverprofile=coverage.txt -covermode=count
 after_success:
 - bash <(curl -s https://codecov.io/bash)
 - stage: "Code Quality"
 name: "Code Quality Analysis"
 before_script:
 - curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0
 script: golangci-lint run Modeling stages

Slide 43

Slide 43 text

- stage: "Release"
 name: "Release Binaries"
 script: skip
 deploy:
 - provider: script
 skip_cleanup: true
 script: curl -sL https://git.io/goreleaser | bash
 on:
 tags: true
 condition: $TRAVIS_OS_NAME = linux Release binary for tagged commit

Slide 44

Slide 44 text

Pipeline visualizaFon jobs:
 include:
 - stage: ...
 - stage: ...
 - stage: ...

Slide 45

Slide 45 text

AutomaFc Go runFme installaFon Taken care of Jenkins Go plugin hBps:/ /wiki.jenkins.io/display/JENKINS/Go+Plugin

Slide 46

Slide 46 text

pipeline {
 agent any
 tools {
 go 'go-1.13'
 }
 environment {
 GO111MODULE = 'on'
 }
 stages {
 ...
 }
 } Basic setup

Slide 47

Slide 47 text

stage('Compile') {
 steps {
 sh 'go build'
 }
 }
 stage('Test') {
 environment {
 CODECOV_TOKEN = credentials('codecov_token')
 }
 steps {
 sh 'go test ./... -coverprofile=coverage.txt'
 sh 'curl -s https://codecov.io/bash | bash -s -'
 }
 }
 stage('Code Analysis') {
 steps {
 sh 'curl -sfL https://install.goreleaser.com/github.com/golangci/↵ golangci-lint.sh | bash -s -- -b $GOPATH/bin v1.18.0'
 sh 'golangci-lint run'
 }
 } Modeling stages

Slide 48

Slide 48 text

stage('Release') {
 when {
 buildingTag()
 }
 environment {
 GITHUB_TOKEN = credentials('github_token')
 }
 steps {
 sh 'curl -sL https://git.io/goreleaser | bash'
 }
 } Release binary for tagged commit

Slide 49

Slide 49 text

Pipeline visualizaFon Standard Blue Ocean

Slide 50

Slide 50 text

Demo

Slide 51

Slide 51 text

Thanks! Please ask quesFons.