Learn Bosh - Understand the Bosh Agent - Part 3

Video on YouTube https://www.youtube.com/watch?v=0zYU5TY2H-M

If you want to learn about bosh, then you'll want to learn about the bosh agent - it is how every server/instance is controlled by the bosh director. This part 3 of a 3 part short series into the inner workings of bosh from the perspective of the bosh agent itself.

In this screencast, we go deep. CIA deep. Let's steal the raw ingredients for deploying redis in a bosh VM and deploy redis in a vagrant VM without a bosh deployment. Bosh espionage.

Stay tuned for the bonus part 4 - when we explore how microbosh and micro-cloudfoundry work!

Transcript

1. LEARN THE BOSH AGENT Dr Nic Williams http://starkandwayne.com @drnic PART 3
2. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT WORKSHOP Watch a bosh agent inside Vagrant Watch now: redis-­‐server monit bosh_agent
3. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT redis-­‐server :6379
4. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director monit bosh_agent director: the API for a bosh
5. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director nats-­‐server monit bosh_agent “apply spec” nats: the message bus used within bosh and its managed servers
6. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director blobstore monit bosh_agent “get blobs” nats-­‐server blobstore: blobs/packages accessed by HTTP (nginx & filesystem)
7. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director blobstore redis-­‐server monit bosh_agent “start job” nats-­‐server job: a component or thing that runs
8. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT The director assigns a job to the agent
9. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT WE will assign a job to the agent
10. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director blobstore redis-­‐server monit bosh_agent nats-­‐server instead of bosh director creating & controlling...
11. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT blobstore redis-­‐server monit bosh_agent we will be the director...
12. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT VAGRANT Completely independent Local developer VMs
13. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT international espionage 1. normal deploy in a bosh 2. steal the “apply spec” 3. install bosh agent into vagrant vm 4. config agent for that blobstore 5. “apply” with example apply spec
14. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director blobstore redis-­‐server monit bosh_agent nats-­‐server steps 1 & 2: normal deploy, then extract example the apply spec
15. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director blobstore redis-­‐server monit bosh_agent nats-­‐server monit bosh_agent step 3: build vagrant VM with bosh agent & monit
16. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT director blobstore redis-­‐server monit bosh_agent nats-­‐server redis-­‐server monit bosh_agent step 4 & 5: config agent to blobstore & apply spec
17. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT 1. normal deploy • Clone redis-boshrelease • Read the readme • Set uuid & instance type in examples/ openstack-vip-solo.yml • bosh deployment examples/ openstack-vip-solo.yml • bosh deploy
18. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT 2. Steal apply spec • bosh ssh redis/0 • sudo grep apply /var/vcap/bosh/log/ current • extract the Ruby hash (the spec) from the last line above • convert into YAML • place in secret dropbox for spy pickup
19. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT 3. build bosh agent VM • clone chef-bosh_agent repo • kitchen converge • kitchen verify • kitchen login default
20. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT 4. config bosh agent • Change /var/vcap/bosh/settings.json $ sudo vim /var/vcap/bosh/settings.json {..., "blobstore":{"provider":"dav","options": {"endpoint":"http://BLOBSTOREHOST:25250", "user":"agent","password":"agent"}}, "mbus":"https://vcap:vcap@0.0.0.0:6969",...} • Restart agent service $ sudo sv restart agent
21. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT 5. apply stolen spec • store stolen YAML spec at /var/vcap/bosh/apply_spec.yml • use agent client to apply the spec $ pry require "agent_client" require "yaml" api = Bosh::Agent::Client.create("https://localhost:6969", "user" => "vcap", "password" => "vcap") api.apply(YAML.load_file("/var/vcap/bosh/apply_spec.yml")) api.start
22. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT espionage successfully • tail the agent logs $ sudo tail -f /var/vcap/bosh/log/current ... Message: {"method"=>"apply", "arguments"=>[...]... ... Retrieving blob: 59f2df4e-d074-4361-a423-e20827566955 creating /var/vcap/data/jobs/redis/1 ... Retrieving blob: 13db6a1a-c262-4cc1-b765-b401b47cd366 creating /var/vcap/data/packages/redis/1.1 ... Monit: reload • success $ ps ax | grep redis 10379 ... redis-server /var/vcap/jobs/redis/config/redis.conf
23. DR NIC | STARK & WAYNE | THE DEEP MAGIC OF BOSH AGENT blobstore redis-­‐server monit bosh_agent we controlled an agent outside of bosh