dockbot - A Story Of Robots, Whales and Salt Mines

A look at an opinionated CI solution combing Buildbot with Salt Stack provisioning, running tests in isolated Docker containers.

Transcript

1. dockbot A Story Of Robots, Whales and Salt Mines MelbDjango 0.6 Sebastian Vetter @elbaschid github.com/elbaschid
2. Sounds exciting, right?
3. Robots, ...
4. ... whales, ...
5. ... and salt!
6. But What Is It All About?
7. Let me explain! I ❤ Testing
8. I ❤ Continuous Integration
9. Travis CI and others CI-aaS • Limited control over setup • Private repos can be expensive. • Do one job well...but nothing more.
10. An Alternative Idea • Use Buildbot as CI server • Isolate test in Docker container • Reproducible setup with Salt Stack
11. What is Buildbot? • Test & Build Server Framework • Written in Python on top of Twisted • Extremely customisable • Website: http://buildbot.net
12. High-level View
13. What is Docker? • Abstraction on top of LXC • Inlcudes VCS-like versioning • Developed out of dotcloud (PaaS provider) • http://docker.io
14. Looks simple? It is, too! docker run -i -t ubuntu /bin/bash
15. What is Saltstack? • Remote execution & provisioning • Similar to Puppet & Chef • Written in Python • http://saltstack.com
16. The Concept
17. How It Works
18. Saltstack at Tangent • Provision Vagrant locally • Create production like set up • Using salty-vagrant plugin • Lightweight dev boxes to share
19. Project Salt Configuration + fancy_project + salt | + base | + pillar | + salt | minion + docs + www Vagrantfile
20. Set up a project projects: - name: fancy-project repo: git@github.com:fancyman/fancy-project.git
21. Define Dockerslave class DockerLatentBuildSlave(AbstractLatentBuildSlave): default_image = u'dockbot/buildslave' ... def _start_instance(self): instance = docker_client.create_container( image=self.default_image, command="buildslave --verbose start --nodaemon /root/slave" environment=[ 'CI=true', 'BUILDBOT_USER={0}'.format(self.slavename), 'BUILDBOT_PASSWORD={0}'.format(self.password), 'BUILDBOT_HOST={0}'.format(self.master_host), 'BUILDBOT_PORT={0}'.format(self.master_port), ], volumes=['/home/woody/.ssh:/root/.ssh:ro', '/home/woody/wheelhouse:/root/wheelhouse:rw'], ) ...
22. The Builder steps = [Git(repourl=project.get('repo'), mode='incremental')] factory = BuildFactory() steps += get_before_install_steps(project) steps += get_provision_steps(project) steps += get_script_steps(project) factory.addSteps(steps) BuilderConfig(slavenames=['docker-{0}'.format(name)], factory=factory)
23. A Build Overview
24. And A Single Build
25. Still Time?
26. Awesome Goody: $ buildbot try
27. Test Before Commit • Creates patch file from pushed HEAD • Submits it to buildbot master • Applies the patch on the slave • Runs tests
28. All you have to do is... $ buildbot try --wait
29. dockbot • Keep an eye on it at: https://github.com/elbaschid/dockbot
30. I Am Really Excited! Interested? Come and talk to me!