10 ways to create BOSH releases faster

In 2018 there are now 10 great ways you can build, test and ship BOSH releases very quickly. We'll tackle some great ideas around distributing compiled releases, 5-minute CI pipelines, and using Debian packages and Docker images. Plus more!

My goal is to encourage you to package and deploy all your in-house bespoke systems with BOSH and to love it!

Transcript

1. Title Text
   Body Level One
   Body Level Two
   Body Level Three
   Body Level Four
   Body Level Five
   10 ways to create
   BOSH release faster
   CF Summit Basel October 2018
   Dr Nic Williams @drnic
   

   View Slide

2. ultimateguidetobosh.com @drnic
   

   View Slide

3. Docker containers Ǻ
   10 ways
   Packages first 
   @drnic
   Ɏ
   Empty release
   bosh-gen new ɑ
   bosh deploy -o create.yml
   

   View Slide

4. Write jobs with bpm
   Faster BOSH deployments
   Language packs
   10 ways
   Share pre-compiled releases
   Ɲ
   @drnic
   Debian packages
   

   View Slide

5. 1. No Release
   Just Docker
   $ bosh deploy manifests/redis.yml -d redis
   $ bosh logs -f
   $ bosh instances
   Instance State AZ IPs
   docker/9e3bb running z1 10.244.0.3
   $ redis-cli -h 10.244.0.3
   10.244.0.3:6379> set foo 123
   OK
   10.244.0.3:6379> get foo
   "123"
   Ɏ
   github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
   

   View Slide

6. 1. No Release
   Just Docker
   name: redis
   instance_groups:
   - name: docker
   persistent_disk: 65536
   jobs:
   - {name: docker, release: docker}
   - name: containers
   release: docker
   properties:
   containers:
   - name: redis
   image: "redis"
   bind_ports: ["6379:6379"]
   bind_volumes: ["/var/lib/redis"]
   Ɏ
   github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
   

   View Slide

7. 2. Empty
   release
   $ cat config/final.yml
   name: mything
   $ cat config/blobs.yml
   ---{}
   ɑ
   github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
   $ bosh create-release
   Job Digest Packages
   0 jobs
   Package Digest Dependencies
   0 packages
   

   View Slide

8. 3. bosh-gen
   $ bosh-gen new cfsummit
   Confirming: Using AWS/us-east-1
   create README.md
   create jobs/cfsummit/monit
   create jobs/cfsummit/spec
   create manifests/cfsummit.yml
   create manifests/operators/create.yml
   create config/blobs.yml
   create config/private.yml
   create config/final.yml
   $ cd cfsummit-boshrelease
   $
   github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
   

   View Slide

9. 3. bosh-gen
   $ bosh create-release
   Job Digest Packages
   cfsummit/d3115 0930eb -
   1 jobs
   Package Digest Dependencies
   0 packages
   $
   github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
   

   View Slide

10. 4. Create,
    upload,
    deploy
    $ export BOSH_DEPLOYMENT=cfsummit
    $ bosh deploy manifests/cfsummit.yml
    …create dev release
    …upload dev release
    …upload bpm release
    …deploy dev release
    $ bosh instances
    Instance State AZ IPs
    cfsummit/760f running z1 10.244.0.2
    Ɏ
    github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
    

    View Slide

11. $ cat manifests/cfsummit.yml
    …
    releases:
    - name: bpm
    version: 0.12.3
    url: git+https://github.com/…/bpm-release
    - name: cfsummit
    version: create
    url: .
    Ɏ
    github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
    4. Create,
    upload,
    deploy
    

    View Slide

12. $ cat manifests/operators/create.yml
    ---
    - type: replace
    path: /releases/name=cfsummit
    value:
    name: cfsummit
    version: create
    url: file://.
    $ bosh deploy manifests/cfsummit.yml \
    -o manifests/operators/create.yml
    Ɏ
    github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
    4. Create,
    upload,
    deploy
    

    View Slide

13. 5. Packages
    first
    $ bosh-gen package redis
    create packages/redis/packaging
    create packages/redis/spec
    $ cat jobs/cfsummit/spec
    name: cfsummit
    packages: [redis]
    $ bosh deploy manifests/cfsummit.yml
    Compiling packages: redis/ad5bd
    $ bosh ssh
    # cd /var/vcap/packages/redis/
    
    github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
    

    View Slide

14. 6. Language
    Packs
    $ git clone \
    https://github.com/bosh-packages/ruby-release \
    ~/workspace/ruby-release
    $ bosh vendor-package \
    ruby-2.4-r4 \
    ~/workspace/ruby-release
    $ cat packages/myapp/packaging
    source /var/vcap/packages/ruby-2.4-r4/bosh/compile.env
    bosh_bundle
    bosh_generate_runtime_env
    starkandwayne.com/blog/build-bosh-releases-faster-with-language-packs @drnic
    

    View Slide

15. starkandwayne.com/blog/build-bosh-releases-faster-with-language-packs @drnic
    6. Language
    Packs
    

    View Slide

16. 7. Debian
    Packages
    Idea #1
    $ cat manifests/redis-debian.yml
    …
    - name: pre-start-script
    release: os-conf
    properties:
    script: |-
    #!/bin/bash
    apt-get update
    apt-get install redis-server -y
    - name: redis-debian
    release: redis-debian
    properties:
    password: ((redis-password))
    Ɲ
    github.com/cloudfoundry-community/redis-debian-boshrelease @drnic
    

    View Slide

17. 7. Debian
    Packages
    Idea #1
    $ cat manifests/redis-debian.yml
    …
    - name: pre-start-script
    release: os-conf
    properties:
    script: |-
    #!/bin/bash
    apt-get update
    apt-get install redis-server -y
    - name: redis-debian
    release: redis-debian
    properties:
    password: ((redis-password))
    Ɲ
    github.com/cloudfoundry-community/redis-debian-boshrelease @drnic
    

    View Slide

18. 7. Debian
    Packages
    Idea #2
    $ bosh-gen package --apt redis-server
    exist packages
    create packages/redis-server/packaging
    create packages/redis-server/spec
    exist jobs
    create jobs/redis-server-pkg-install/monit
    create jobs/redis-server-pkg-install/spec
    create jobs/redis-server-pkg-install/templates/bin/pre-start.sh
    Next steps:
    1. Edit packages/redis-server/packaging
    to specify list of debian packages to install
    2. Add "redis-server-pkg-install" job into
    deployment manifest
    Ɲ
    github.com/cloudfoundry-community/redis-debian-boshrelease @drnic
    

    View Slide

19. 8. Write
    jobs with
    BPM
    $ bosh-gen job redis-server -d redis
    create jobs/redis-server
    create jobs/redis-server/monit
    create jobs/redis-server/spec
    create jobs/redis-server/templates/config/bpm.yml
    $ cat jobs/redis-server/…/bpm.yml
    processes:
    - name: redis-server
    executable: /var/vcap/packages/redis/bin/redis-server
    args:
    - /var/vcap/jobs/redis-server/config/redis.conf
    ephemeral_disk: true
    persistent_disk: true
    
    github.com/starkandwayne/cfsummit-basel-2018-boshrelease @drnic
    

    View Slide

20. 9. Faster
    BOSH
    bucc up
    bucc up
    bucc up --cpi vsphere --lite
    github.com/starkandwayne/bucc
    BOSH
    UAA
    CredHub
    Concourse
    @drnic
    bucc bosh
    bucc credhub
    

    View Slide

21. 10. Share
    compiled
    releases
    @drnic
    
    bosh export-release
    bosh.io/docs/compiled-releases/ @drnic
    $ bosh create-release \
    --version 1.0.0 —final
    …
    $ bosh export-release \
    cfsummit/1.0.0 ubuntu-xenial/97.19
    $ ls cfsummit*
    cfsummit-1.0.0-ubuntu-xenial-97.19-20181008.…tgz
    

    View Slide

22. @drnic
    github.com/starkandwayne/pipeline-templates @drnic
    $ path/to/pipeline-templates/setup boshrelease
    $ ci/repipe
    

    View Slide