Concourse CI in the Wild

Slide 1

Slide 1 text

Concourse CI  in the Wild Examples from real projects

Slide 2

Slide 2 text

What is Concourse CI? Pipeline based CI server written in Go Everything runs in containers

Slide 3

Slide 3 text

Core Concepts Task - execution of scripts on dependent resources Resources - entities that can be checked for new versions, speciﬁc versions pulled and created Jobs - Describes actions to be taken when dependent resources changes. An instance of the execution of a job’s plan is called a Build.

Slide 4

Slide 4 text

Pipeline A pipeline is the result of conﬁguring jobs and resources. YAML ﬁle

Slide 5

Slide 5 text

Concourse Architecture ATC - Web UI and build scheduler TSA - Worker registration & forwarding. Custom-built SSH server. Workers - Container runtime & cache management

Slide 6

Slide 6 text

Workers Running Garden and Baggageclaim servers Garden - Golang client / server for container creation and management Baggageclaim - Volume manager for Garden containers Each worker register themselves via TSA via SSH Each worker continuously heartbeats its presence to ATC

Slide 7

Slide 7 text

Access Control Basic Auth - single account only Auth via GitHub - more conﬁgurable, can restrict by organization and team Builds can be made publicly viewable and can be started by unauthenticated users if you prefer. All authentication can be switched off in development mode

Slide 8

Slide 8 text

The Fly CLI Conﬁguration mainly done via a CLI Can be downloaded from the ATC web interface Login to ATC & assign a tag to the server fly -t lite login -c http:// 192.168.100.4:8080 Conﬁgure a build pipeline fly -t lite set-pipeline -p hello- world -c hello.yml

Slide 9

Slide 9 text

The Fly CLI Conﬁgure a build pipeline fly -t lite set-pipeline -p hello-world -c hello.yml

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Tasks Platform - Deﬁnes which pool of workers gets used Image resource - Docker image to run this task with Inputs - What artefacts propagate into the task Outputs - Artefacts of the build Docker speak: Input / outputs are mapped volumes

Slide 15

Slide 15 text

Demo https:/ /github.com/pivotal-sg/concourse- demo

Slide 16

Slide 16 text

Making Concourse Fast Use public Docker Images as they are cached Use Rsync / Archive resource types to download / cache dependencies Use external services if you can (eg. PostgreSQL on host instead of inside the container.) Use inputs / outputs to move artefacts around.