Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Deploying things to BOSH - from spiff to bosh-workspace to concourse ci

Deploying things to BOSH - from spiff to bosh-workspace to concourse ci

At Stark & Wayne we are constantly thinking, planning, exploring and working towards the ultimate continuous deployment pipelines for our production systems & our customer's production systems.

At CF Summit 2015, Dr Nic, overviewed the state of art for deploying safely through to production with BOSH using combinations of spiff, bosh-workspace and the all new Concourse CI

Dr Nic Williams

May 11, 2015

More Decks by Dr Nic Williams

Other Decks in Technology


  1. How we deploy all things BOSH

  2. None
  3. @starkandwayne

  4. None
  5. None
  6. bosh-gen bosh-bootstrap terraform-*-cf-install bosh-workspace

  7. None
  8. blog.starkandwayne.com

  9. Or just come ask questions tomorrow! 10:40am

  10. The Problem The next 59 months

  11. 12 cf-release upgrades in 4 months 538 commits to cf-release

    99 changes to job templates
  12. None
  13. What changed in job template inputs? git log v195..v207 --

    jobs/*/spec commit 295db91d037f8d151621db067c5c7659cfa566ac Author: Chris Piraino and Zach Robinson <[email protected]> Date: Wed Apr 15 11:29:06 2015 -0700 change the default rootfs to cflinuxfs2 [#87752612]
  14. git diff v195..v207 -- \ jobs/cloud_controller_ng/spec + uaa.clients.cloud_controller_username_lookup.client: + description:

    "Used for fetching usernames from UAA." + uaa.clients.cloud_controller_username_lookup.secret: + description: "Used for fetching usernames from UAA." cc.billing_event_writing_enabled: description: "If true, billing events are written to the CCDB." - default: true + default: false
  15. 0 users care

  16. None
  17. None
  18. None
  19. spiff merge "OMG please template this"

  20. None
  21. None
  22. spiff merge \ cf-deployment.yml \ cf-jobs.yml \ cf-properties.yml \ cf-lamb.yml

    \ cf-infra-${infra}.yml \ stub.yml
  23. Which release version? Which templates? Merge order? Changes?

  24. Combine specific releases Custom templates Specific merge order

  25. Deploy CF —bosh-workspace deployment file —cf/ templates from cf-release/templates —others

    are custom —specific merge order cloudfoundry-community/ cf-boshworkspace
  26. None
  27. Deploy CF —meta.x.y - inputs into spiff templates cloudfoundry-community/ cf-boshworkspace

  28. Tiny CF —6 VMs instead of 20 —High availability with

    12 instead of 40 cloudfoundry-community/ cf-boshworkspace
  29. Custom shell script

  30. Simple shell scripts easily used by automation —make_manifest.sh passed through

    automation with templates
  31. Pipelined shell script —spiff stub is only config from pipeline

  32. None
  33. None
  34. None
  35. Spiff templates & stub: * try-anything to pre-prod; * pre-prod

    to production
  36. None
  37. None
  38. None
  39. How to make spiff templates? $ gem install makemespiffy $

    cp working_manifest.yml base_template.yml $ makemespiffy base_template.yml \ name environment/name.yml meta.name $ makemespiffy base_template.yml \ networks.cf1.subnets env/networking.yml meta.subnets.cf1
  40. Pipeline ProTips - "Docker Docker Docker" —Every task runs in

    a Docker image —Own your Docker images —Dockerfile in same pipeline repo —Pipeline to create Docker image
  41. makemespiffy redis-bosh-pipeline logstash-docker- pipeline cf-pipeline

  42. None
  43. @starkandwayne

  44. 10:40am Tues bosh & releases concourse cloud foundry

  45. 3:30pm Tues Wear: S&W t-shirt Quiz: Read our blog $500

    / TV / 2hrs with @drnic