Presentation to internal Engineering team on exactly how our builds and CI work.
On ourCI and BUILDSHOW THEY WORK
View Slide
Or: How shit gets built anddeployed.
Builds involve:
GitHub ∙ CircleCIRuby ∙ ThorDocker
And copious amounts of:
Swearing ∙ FearPunching ∙ HateQuiet Weeping
On thePROCESSOF BUILDING A COMMIT
The steps to build:
1. Make a commit.
2. Push to GitHub.
3. CircleCI is notified.
4. The build begins.
5. The Docker build begins.
6. The sanity build begins.(if master)
7. Deployment can begin.
On thePUSHINGOF A COMMIT
GitHub receives your commitand dispatches a notificationto CircleCI.
Cheers bro.Welcome blud.all of teh infos
CircleCI receives a payload ofinformation telling it whatcommit this was.
CircleCI starts a container upfor the build.
Boom bitches.
It then looks for ourcircle.yml file.
Each step of this file isexecuted.
These steps execute a bunchof shell scripts in /circle.
These steps...∙ Configure versions of software to use (Ruby,Java, Kafka, HBase, etc).∙ Restore the cache.∙ Checkout the repository.∙ Build all the projects.∙ Test all the projects.∙ Save all the artifacts to CircleCI.
We have three differentkinds of builds:
Standard ∙ DockerSanity
We tell it what scripts to useby passing a MODE env var.
Demo: The CircleCI buildscripts.
On theDOCKER BUILDSOUR CODE IN CONTAINERS
Quick Docker refresher:
Docker is built on the idea ofcontainers.
Many container!
You build images.You deploy containers.
How it happens:
A second parameterised buildis started for Docker.
These builds have whateverinfo they need passed asparameters.
It builds, tags and pushes theDocker images.
The building happens on aseparate machine.
The pushed images arestored in the Docker registry.
When we deploy, we pulldown these images and runthem (making containers!)
OnBUILD SCRIPTSUSING AND ABUSING THOR
Thor is a task framework.
I’m actually reallysensitive and kind.
All of the things we do onCircleCI are Thor tasks.
The Thor task sources are in/build-scripts/thor.
Each task is namespaced.thor foo:bar
You can get help for a task byrunning thor help task.
All Thor tasks are just Rubyfiles. Thor is just a nice DSL.
Demo: The Thor buildscripts.
OnCOMMON PROBLEMSSOLVING COMMON ISSUES
Q: Gradle tasks hanginglocally.
A: Start the Nexus server.
Q: Deployments fail as unableto pull from repository.
A: Start the Docker registry.
Q: Docker build fails with “nospace left on device”.
A: Restart the Docker server,if it still fails call for help.
Q: “Expected ID but got nil”.
A: (Sadly) Just retry thebuild. This is aDocker/library bug thathappens randomly.
Q: The tests are failing.
A: Your tests are shit. Fixyour tests.
OnBEING A BUILD EXPERTBECOME A ZOMG HACKER
Demo: How to find the buildto deploy.
Demo: How to see what SHAis being built for Docker.
Fin.I’m gonna eat your hairy legs.