Deploying things to BOSH - from spiff to bosh-workspace to concourse ci

At Stark & Wayne we are constantly thinking, planning, exploring and working towards the ultimate continuous deployment pipelines for our production systems & our customer's production systems.

At CF Summit 2015, Dr Nic, overviewed the state of art for deploying safely through to production with BOSH using combinations of spiff, bosh-workspace and the all new Concourse CI

Transcript

1. How we deploy all things BOSH
3. @starkandwayne
6. bosh-gen bosh-bootstrap terraform-*-cf-install bosh-workspace
8. blog.starkandwayne.com
9. Or just come ask questions tomorrow! 10:40am
10. The Problem The next 59 months
11. 12 cf-release upgrades in 4 months 538 commits to cf-release 99 changes to job templates
13. What changed in job template inputs? git log v195..v207 -- jobs/*/spec commit 295db91d037f8d151621db067c5c7659cfa566ac Author: Chris Piraino and Zach Robinson <cpiraino@pivotal.io> Date: Wed Apr 15 11:29:06 2015 -0700 change the default rootfs to cflinuxfs2 [#87752612]
14. git diff v195..v207 -- \ jobs/cloud_controller_ng/spec + uaa.clients.cloud_controller_username_lookup.client: + description: "Used for fetching usernames from UAA." + uaa.clients.cloud_controller_username_lookup.secret: + description: "Used for fetching usernames from UAA." cc.billing_event_writing_enabled: description: "If true, billing events are written to the CCDB." - default: true + default: false
15. 0 users care
19. spiff merge "OMG please template this"
22. spiff merge \ cf-deployment.yml \ cf-jobs.yml \ cf-properties.yml \ cf-lamb.yml \ cf-infra-${infra}.yml \ stub.yml
23. Which release version? Which templates? Merge order? Changes?
24. Combine specific releases Custom templates Specific merge order
25. Deploy CF —bosh-workspace deployment file —cf/ templates from cf-release/templates —others are custom —specific merge order cloudfoundry-community/ cf-boshworkspace
27. Deploy CF —meta.x.y - inputs into spiff templates cloudfoundry-community/ cf-boshworkspace
28. Tiny CF —6 VMs instead of 20 —High availability with 12 instead of 40 cloudfoundry-community/ cf-boshworkspace
29. Custom shell script
30. Simple shell scripts easily used by automation —make_manifest.sh passed through automation with templates
31. Pipelined shell script —spiff stub is only config from pipeline inputs
35. Spiff templates & stub: * try-anything to pre-prod; * pre-prod to production
39. How to make spiff templates? $ gem install makemespiffy $ cp working_manifest.yml base_template.yml $ makemespiffy base_template.yml \ name environment/name.yml meta.name $ makemespiffy base_template.yml \ networks.cf1.subnets env/networking.yml meta.subnets.cf1
40. Pipeline ProTips - "Docker Docker Docker" —Every task runs in a Docker image —Own your Docker images —Dockerfile in same pipeline repo —Pipeline to create Docker image
41. makemespiffy redis-bosh-pipeline logstash-docker- pipeline cf-pipeline
43. @starkandwayne
44. 10:40am Tues bosh & releases concourse cloud foundry
45. 3:30pm Tues Wear: S&W t-shirt Quiz: Read our blog $500 / TV / 2hrs with @drnic