Deploying things to BOSH - from spiff to bosh-workspace to concourse ci

Deploying things to BOSH - from spiff to bosh-workspace to concourse ci

At Stark & Wayne we are constantly thinking, planning, exploring and working towards the ultimate continuous deployment pipelines for our production systems & our customer's production systems.

At CF Summit 2015, Dr Nic, overviewed the state of art for deploying safely through to production with BOSH using combinations of spiff, bosh-workspace and the all new Concourse CI

Cb2b768a5e546b24052ea03334e43676?s=128

Dr Nic Williams

May 11, 2015
Tweet

Transcript

  1. How we deploy all things BOSH

  2. None
  3. @starkandwayne

  4. None
  5. None
  6. bosh-gen bosh-bootstrap terraform-*-cf-install bosh-workspace

  7. None
  8. blog.starkandwayne.com

  9. Or just come ask questions tomorrow! 10:40am

  10. The Problem The next 59 months

  11. 12 cf-release upgrades in 4 months 538 commits to cf-release

    99 changes to job templates
  12. None
  13. What changed in job template inputs? git log v195..v207 --

    jobs/*/spec commit 295db91d037f8d151621db067c5c7659cfa566ac Author: Chris Piraino and Zach Robinson <cpiraino@pivotal.io> Date: Wed Apr 15 11:29:06 2015 -0700 change the default rootfs to cflinuxfs2 [#87752612]
  14. git diff v195..v207 -- \ jobs/cloud_controller_ng/spec + uaa.clients.cloud_controller_username_lookup.client: + description:

    "Used for fetching usernames from UAA." + uaa.clients.cloud_controller_username_lookup.secret: + description: "Used for fetching usernames from UAA." cc.billing_event_writing_enabled: description: "If true, billing events are written to the CCDB." - default: true + default: false
  15. 0 users care

  16. None
  17. None
  18. None
  19. spiff merge "OMG please template this"

  20. None
  21. None
  22. spiff merge \ cf-deployment.yml \ cf-jobs.yml \ cf-properties.yml \ cf-lamb.yml

    \ cf-infra-${infra}.yml \ stub.yml
  23. Which release version? Which templates? Merge order? Changes?

  24. Combine specific releases Custom templates Specific merge order

  25. Deploy CF —bosh-workspace deployment file —cf/ templates from cf-release/templates —others

    are custom —specific merge order cloudfoundry-community/ cf-boshworkspace
  26. None
  27. Deploy CF —meta.x.y - inputs into spiff templates cloudfoundry-community/ cf-boshworkspace

  28. Tiny CF —6 VMs instead of 20 —High availability with

    12 instead of 40 cloudfoundry-community/ cf-boshworkspace
  29. Custom shell script

  30. Simple shell scripts easily used by automation —make_manifest.sh passed through

    automation with templates
  31. Pipelined shell script —spiff stub is only config from pipeline

    inputs
  32. None
  33. None
  34. None
  35. Spiff templates & stub: * try-anything to pre-prod; * pre-prod

    to production
  36. None
  37. None
  38. None
  39. How to make spiff templates? $ gem install makemespiffy $

    cp working_manifest.yml base_template.yml $ makemespiffy base_template.yml \ name environment/name.yml meta.name $ makemespiffy base_template.yml \ networks.cf1.subnets env/networking.yml meta.subnets.cf1
  40. Pipeline ProTips - "Docker Docker Docker" —Every task runs in

    a Docker image —Own your Docker images —Dockerfile in same pipeline repo —Pipeline to create Docker image
  41. makemespiffy redis-bosh-pipeline logstash-docker- pipeline cf-pipeline

  42. None
  43. @starkandwayne

  44. 10:40am Tues bosh & releases concourse cloud foundry

  45. 3:30pm Tues Wear: S&W t-shirt Quiz: Read our blog $500

    / TV / 2hrs with @drnic