Concourse CI in the Wild

Transcript

1. Concourse CI

   in the Wild
   Examples from real projects
   

   View Slide

2. What is Concourse CI?
   Pipeline based CI server written in Go
   Everything runs in containers
   

   View Slide

3. Core Concepts
   Task - execution of scripts on dependent
   resources
   Resources - entities that can be checked for
   new versions, specific versions pulled and
   created
   Jobs - Describes actions to be taken when
   dependent resources changes. An instance of
   the execution of a job’s plan is called a Build.
   

   View Slide

4. Pipeline
   A pipeline is the result of configuring jobs
   and resources.
   YAML file
   

   View Slide

5. Concourse Architecture
   ATC - Web UI and build scheduler
   TSA - Worker registration &
   forwarding. Custom-built SSH server.
   Workers - Container runtime & cache
   management
   

   View Slide

6. Workers
   Running Garden and Baggageclaim servers
   Garden - Golang client / server for container creation
   and management
   Baggageclaim - Volume manager for Garden
   containers
   Each worker register themselves via TSA via SSH
   Each worker continuously heartbeats its presence to
   ATC
   

   View Slide

7. Access Control
   Basic Auth - single account only
   Auth via GitHub - more configurable, can
   restrict by organization and team
   Builds can be made publicly viewable and can be
   started by unauthenticated users if you prefer.
   All authentication can be switched off in
   development mode
   

   View Slide

8. The Fly CLI
   Configuration mainly done via a CLI
   Can be downloaded from the ATC web interface
   Login to ATC & assign a tag to the server
   fly -t lite login -c http://
   192.168.100.4:8080
   Configure a build pipeline
   fly -t lite set-pipeline -p hello-
   world -c hello.yml
   

   View Slide

9. The Fly CLI
   Configure a build pipeline
   fly -t lite set-pipeline -p hello-world -c
   hello.yml
   

   View Slide

10. View Slide

11. View Slide

12. View Slide

13. View Slide

14. Tasks
    Platform - Defines which pool of workers gets used
    Image resource - Docker image to run this task
    with
    Inputs - What artefacts propagate into the task
    Outputs - Artefacts of the build
    Docker speak: Input / outputs are mapped
    volumes
    

    View Slide

15. Demo
    https:/
    /github.com/pivotal-sg/concourse-
    demo
    

    View Slide

16. Making Concourse Fast
    Use public Docker Images as they are cached
    Use Rsync / Archive resource types to
    download / cache dependencies
    Use external services if you can (eg. PostgreSQL
    on host instead of inside the container.)
    Use inputs / outputs to move artefacts around.
    

    View Slide