Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
On our CI and Builds
Search
Nathan Kleyn
March 03, 2015
Programming
0
2.5k
On our CI and Builds
Presentation to internal Engineering team on exactly how our builds and CI work.
Nathan Kleyn
March 03, 2015
Tweet
Share
More Decks by Nathan Kleyn
See All by Nathan Kleyn
Distributed ID Generation
nathankleyn
1
2.3k
Semantic Versioning With Git
nathankleyn
2
360
Exception Handling
nathankleyn
0
120
Cucumber Automation Testing
nathankleyn
3
210
Git: Everything You Need To Know (And A Few More Things)
nathankleyn
3
160
How The VM Works
nathankleyn
0
130
Other Decks in Programming
See All in Programming
関数型まつりレポート for JuliaTokai #22
antimon2
0
160
Railsアプリケーションと パフォーマンスチューニング ー 秒間5万リクエストの モバイルオーダーシステムを支える事例 ー Rubyセミナー 大阪
falcon8823
5
1.1k
GitHub Copilot and GitHub Codespaces Hands-on
ymd65536
2
150
Team operations that are not burdened by SRE
kazatohiei
1
310
Discover Metal 4
rei315
2
130
ruby.wasmで多人数リアルタイム通信ゲームを作ろう
lnit
3
450
初学者でも今すぐできる、Claude Codeの生産性を10倍上げるTips
s4yuba
16
11k
設計やレビューに悩んでいるPHPerに贈る、クリーンなオブジェクト設計の指針たち
panda_program
6
2k
Rubyでやりたい駆動開発 / Ruby driven development
chobishiba
1
660
Result型で“失敗”を型にするPHPコードの書き方
kajitack
5
610
Claude Code + Container Use と Cursor で作る ローカル並列開発環境のススメ / ccc local dev
kaelaela
5
2.4k
Kotlin エンジニアへ送る:Swift 案件に参加させられる日に備えて~似てるけど色々違う Swift の仕様 / from Kotlin to Swift
lovee
1
260
Featured
See All Featured
Statistics for Hackers
jakevdp
799
220k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
26
2.9k
Agile that works and the tools we love
rasmusluckow
329
21k
Why You Should Never Use an ORM
jnunemaker
PRO
58
9.4k
Being A Developer After 40
akosma
90
590k
Thoughts on Productivity
jonyablonski
69
4.7k
Making Projects Easy
brettharned
116
6.3k
Speed Design
sergeychernyshev
32
1k
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
20
1.3k
The Art of Programming - Codeland 2020
erikaheidi
54
13k
Fireside Chat
paigeccino
37
3.5k
Transcript
On our CI and BUILDS HOW THEY WORK
Or: How shit gets built and deployed.
Builds involve:
GitHub ∙ CircleCI Ruby ∙ Thor Docker
And copious amounts of:
Swearing ∙ Fear Punching ∙ Hate Quiet Weeping
None
On the PROCESS OF BUILDING A COMMIT
The steps to build:
1. Make a commit.
2. Push to GitHub.
3. CircleCI is notified.
4. The build begins.
5. The Docker build begins.
6. The sanity build begins. (if master)
7. Deployment can begin.
On the PUSHING OF A COMMIT
GitHub receives your commit and dispatches a notification to CircleCI.
Cheers bro. Welcome blud. all of teh infos
CircleCI receives a payload of information telling it what commit
this was.
CircleCI starts a container up for the build.
Boom bitches.
It then looks for our circle.yml file.
Each step of this file is executed.
These steps execute a bunch of shell scripts in /circle.
These steps... ∙ Configure versions of software to use (Ruby,
Java, Kafka, HBase, etc). ∙ Restore the cache. ∙ Checkout the repository. ∙ Build all the projects. ∙ Test all the projects. ∙ Save all the artifacts to CircleCI.
We have three different kinds of builds:
Standard ∙ Docker Sanity
We tell it what scripts to use by passing a
MODE env var.
Demo: The CircleCI build scripts.
On the DOCKER BUILDS OUR CODE IN CONTAINERS
Quick Docker refresher:
Docker is built on the idea of containers.
Many container!
You build images. You deploy containers.
How it happens:
A second parameterised build is started for Docker.
These builds have whatever info they need passed as parameters.
It builds, tags and pushes the Docker images.
The building happens on a separate machine.
The pushed images are stored in the Docker registry.
When we deploy, we pull down these images and run
them (making containers!)
On BUILD SCRIPTS USING AND ABUSING THOR
Thor is a task framework.
I’m actually really sensitive and kind.
All of the things we do on CircleCI are Thor
tasks.
The Thor task sources are in /build-scripts/thor.
Each task is namespaced. thor foo:bar
You can get help for a task by running thor
help task.
All Thor tasks are just Ruby files. Thor is just
a nice DSL.
Demo: The Thor build scripts.
On COMMON PROBLEMS SOLVING COMMON ISSUES
Q: Gradle tasks hanging locally.
A: Start the Nexus server.
Q: Deployments fail as unable to pull from repository.
A: Start the Docker registry.
Q: Docker build fails with “no space left on device”.
A: Restart the Docker server, if it still fails call
for help.
Q: “Expected ID but got nil”.
A: (Sadly) Just retry the build. This is a Docker/library
bug that happens randomly.
Q: The tests are failing.
A: Your tests are shit. Fix your tests.
On BEING A BUILD EXPERT BECOME A ZOMG HACKER
Demo: How to find the build to deploy.
Demo: How to see what SHA is being built for
Docker.
Fin. I’m gonna eat your hairy legs.