Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Concourse CI in the Wild

Concourse CI in the Wild

Michael Cheng

March 20, 2017
Tweet

More Decks by Michael Cheng

Other Decks in Programming

Transcript

  1. Concourse CI

    in the Wild
    Examples from real projects

    View full-size slide

  2. What is Concourse CI?
    Pipeline based CI server written in Go
    Everything runs in containers

    View full-size slide

  3. Core Concepts
    Task - execution of scripts on dependent
    resources
    Resources - entities that can be checked for
    new versions, specific versions pulled and
    created
    Jobs - Describes actions to be taken when
    dependent resources changes. An instance of
    the execution of a job’s plan is called a Build.

    View full-size slide

  4. Pipeline
    A pipeline is the result of configuring jobs
    and resources.
    YAML file

    View full-size slide

  5. Concourse Architecture
    ATC - Web UI and build scheduler
    TSA - Worker registration &
    forwarding. Custom-built SSH server.
    Workers - Container runtime & cache
    management

    View full-size slide

  6. Workers
    Running Garden and Baggageclaim servers
    Garden - Golang client / server for container creation
    and management
    Baggageclaim - Volume manager for Garden
    containers
    Each worker register themselves via TSA via SSH
    Each worker continuously heartbeats its presence to
    ATC

    View full-size slide

  7. Access Control
    Basic Auth - single account only
    Auth via GitHub - more configurable, can
    restrict by organization and team
    Builds can be made publicly viewable and can be
    started by unauthenticated users if you prefer.
    All authentication can be switched off in
    development mode

    View full-size slide

  8. The Fly CLI
    Configuration mainly done via a CLI
    Can be downloaded from the ATC web interface
    Login to ATC & assign a tag to the server
    fly -t lite login -c http://
    192.168.100.4:8080
    Configure a build pipeline
    fly -t lite set-pipeline -p hello-
    world -c hello.yml

    View full-size slide

  9. The Fly CLI
    Configure a build pipeline
    fly -t lite set-pipeline -p hello-world -c
    hello.yml

    View full-size slide

  10. Tasks
    Platform - Defines which pool of workers gets used
    Image resource - Docker image to run this task
    with
    Inputs - What artefacts propagate into the task
    Outputs - Artefacts of the build
    Docker speak: Input / outputs are mapped
    volumes

    View full-size slide

  11. Demo
    https:/
    /github.com/pivotal-sg/concourse-
    demo

    View full-size slide

  12. Making Concourse Fast
    Use public Docker Images as they are cached
    Use Rsync / Archive resource types to
    download / cache dependencies
    Use external services if you can (eg. PostgreSQL
    on host instead of inside the container.)
    Use inputs / outputs to move artefacts around.

    View full-size slide