Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deploying things to BOSH - from spiff to bosh-workspace to concourse ci

Deploying things to BOSH - from spiff to bosh-workspace to concourse ci

At Stark & Wayne we are constantly thinking, planning, exploring and working towards the ultimate continuous deployment pipelines for our production systems & our customer's production systems.

At CF Summit 2015, Dr Nic, overviewed the state of art for deploying safely through to production with BOSH using combinations of spiff, bosh-workspace and the all new Concourse CI

Dr Nic Williams

May 11, 2015
Tweet

More Decks by Dr Nic Williams

Other Decks in Technology

Transcript

  1. How we deploy
    all things
    BOSH

    View full-size slide

  2. @starkandwayne

    View full-size slide

  3. bosh-gen
    bosh-bootstrap
    terraform-*-cf-install
    bosh-workspace

    View full-size slide

  4. blog.starkandwayne.com

    View full-size slide

  5. Or just come
    ask questions
    tomorrow!
    10:40am

    View full-size slide

  6. The Problem
    The next 59 months

    View full-size slide

  7. 12 cf-release
    upgrades in 4
    months
    538 commits to
    cf-release
    99 changes to
    job templates

    View full-size slide

  8. What changed in job template inputs?
    git log v195..v207 -- jobs/*/spec
    commit 295db91d037f8d151621db067c5c7659cfa566ac
    Author: Chris Piraino and Zach Robinson
    Date: Wed Apr 15 11:29:06 2015 -0700
    change the default rootfs to cflinuxfs2
    [#87752612]

    View full-size slide

  9. git diff v195..v207 -- \
    jobs/cloud_controller_ng/spec
    + uaa.clients.cloud_controller_username_lookup.client:
    + description: "Used for fetching usernames from UAA."
    + uaa.clients.cloud_controller_username_lookup.secret:
    + description: "Used for fetching usernames from UAA."
    cc.billing_event_writing_enabled:
    description: "If true, billing events are written to the CCDB."
    - default: true
    + default: false

    View full-size slide

  10. spiff merge
    "OMG please template
    this"

    View full-size slide

  11. spiff merge \
    cf-deployment.yml \
    cf-jobs.yml \
    cf-properties.yml \
    cf-lamb.yml \
    cf-infra-${infra}.yml \
    stub.yml

    View full-size slide

  12. Which release
    version?
    Which
    templates?
    Merge order?
    Changes?

    View full-size slide

  13. Combine
    specific
    releases
    Custom
    templates
    Specific merge
    order

    View full-size slide

  14. Deploy CF
    —bosh-workspace deployment
    file
    —cf/ templates from
    cf-release/templates
    —others are custom
    —specific merge order
    cloudfoundry-community/
    cf-boshworkspace

    View full-size slide

  15. Deploy CF
    —meta.x.y - inputs into spiff
    templates
    cloudfoundry-community/
    cf-boshworkspace

    View full-size slide

  16. Tiny CF
    —6 VMs instead of 20
    —High availability with 12
    instead of 40
    cloudfoundry-community/
    cf-boshworkspace

    View full-size slide

  17. Custom shell script

    View full-size slide

  18. Simple shell scripts easily used by automation
    —make_manifest.sh passed through automation
    with templates

    View full-size slide

  19. Pipelined shell script
    —spiff stub is only config from pipeline inputs

    View full-size slide

  20. Spiff templates & stub:
    * try-anything to pre-prod;
    * pre-prod to production

    View full-size slide

  21. How to make spiff templates?
    $ gem install makemespiffy
    $ cp working_manifest.yml base_template.yml
    $ makemespiffy base_template.yml \
    name environment/name.yml meta.name
    $ makemespiffy base_template.yml \
    networks.cf1.subnets env/networking.yml meta.subnets.cf1

    View full-size slide

  22. Pipeline ProTips - "Docker Docker Docker"
    —Every task runs in a Docker image
    —Own your Docker images
    —Dockerfile in same pipeline repo
    —Pipeline to create Docker image

    View full-size slide

  23. makemespiffy
    redis-bosh-pipeline
    logstash-docker-
    pipeline
    cf-pipeline

    View full-size slide

  24. @starkandwayne

    View full-size slide

  25. 10:40am Tues
    bosh & releases
    concourse
    cloud foundry

    View full-size slide

  26. 3:30pm Tues
    Wear: S&W t-shirt
    Quiz: Read our blog
    $500 / TV / 2hrs with
    @drnic

    View full-size slide