Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up for free
On our CI and Builds
Nathan Kleyn
March 03, 2015
Programming
0
2.1k
On our CI and Builds
Presentation to internal Engineering team on exactly how our builds and CI work.
Nathan Kleyn
March 03, 2015
Tweet
Share
More Decks by Nathan Kleyn
See All by Nathan Kleyn
Distributed ID Generation
nathankleyn
1
2.2k
Semantic Versioning With Git
nathankleyn
2
260
Exception Handling
nathankleyn
0
85
Cucumber Automation Testing
nathankleyn
3
140
Git: Everything You Need To Know (And A Few More Things)
nathankleyn
3
95
How The VM Works
nathankleyn
0
55
Other Decks in Programming
See All in Programming
あなたの会社の古いシステム、なんとかしませんか?~システム刷新から考えるDX化への道筋とバリエーション~/webinar20220420-grapecity
grapecity_dev
0
130
Jakarta EE 10 is Coming Your Way
ivargrimstad
0
2.8k
テスト設計技法をなぜ&どのように使うのか体験しよう!
imtnd
0
450
Android入門
hn410
0
310
iOSアプリの技術選択2022
tattn
6
2.5k
Git Rebase
bkuhlmann
7
1k
あなたの会社の古いシステム、なんとかしませんか?~システム刷新から考えるDX化への道筋とバリエーション~/webinar20220420-systems
grapecity_dev
0
130
Android Architecture Design With Koin
agiuliani
0
240
Quartoを使ってみませんか / quarto_get_started
s_uryu
1
290
クリエイティブ系のウェブサイト制作で役立つCSS技法 / CSS for develop creative website
clockmaker
2
1.6k
質とスピード(2022春版、質疑応答用資料付き) / Quality and Speed 2022 Spring Edition
twada
PRO
29
18k
LOWYAの信頼性向上とNew Relic
kazumax55
4
360
Featured
See All Featured
CoffeeScript is Beautiful & I Never Want to Write Plain JavaScript Again
sstephenson
151
12k
Designing with Data
zakiwarfel
91
3.9k
GraphQLの誤解/rethinking-graphql
sonatard
24
6.2k
How New CSS Is Changing Everything About Graphic Design on the Web
jensimmons
212
11k
Bootstrapping a Software Product
garrettdimon
295
110k
Clear Off the Table
cherdarchuk
79
280k
Scaling GitHub
holman
451
140k
Code Review Best Practice
trishagee
41
6.7k
Documentation Writing (for coders)
carmenhchung
48
2.5k
The MySQL Ecosystem @ GitHub 2015
samlambert
238
11k
We Have a Design System, Now What?
morganepeng
35
2.9k
Build The Right Thing And Hit Your Dates
maggiecrowley
19
1.1k
Transcript
On our CI and BUILDS HOW THEY WORK
Or: How shit gets built and deployed.
Builds involve:
GitHub ∙ CircleCI Ruby ∙ Thor Docker
And copious amounts of:
Swearing ∙ Fear Punching ∙ Hate Quiet Weeping
None
On the PROCESS OF BUILDING A COMMIT
The steps to build:
1. Make a commit.
2. Push to GitHub.
3. CircleCI is notified.
4. The build begins.
5. The Docker build begins.
6. The sanity build begins. (if master)
7. Deployment can begin.
On the PUSHING OF A COMMIT
GitHub receives your commit and dispatches a notification to CircleCI.
Cheers bro. Welcome blud. all of teh infos
CircleCI receives a payload of information telling it what commit
this was.
CircleCI starts a container up for the build.
Boom bitches.
It then looks for our circle.yml file.
Each step of this file is executed.
These steps execute a bunch of shell scripts in /circle.
These steps... ∙ Configure versions of software to use (Ruby,
Java, Kafka, HBase, etc). ∙ Restore the cache. ∙ Checkout the repository. ∙ Build all the projects. ∙ Test all the projects. ∙ Save all the artifacts to CircleCI.
We have three different kinds of builds:
Standard ∙ Docker Sanity
We tell it what scripts to use by passing a
MODE env var.
Demo: The CircleCI build scripts.
On the DOCKER BUILDS OUR CODE IN CONTAINERS
Quick Docker refresher:
Docker is built on the idea of containers.
Many container!
You build images. You deploy containers.
How it happens:
A second parameterised build is started for Docker.
These builds have whatever info they need passed as parameters.
It builds, tags and pushes the Docker images.
The building happens on a separate machine.
The pushed images are stored in the Docker registry.
When we deploy, we pull down these images and run
them (making containers!)
On BUILD SCRIPTS USING AND ABUSING THOR
Thor is a task framework.
I’m actually really sensitive and kind.
All of the things we do on CircleCI are Thor
tasks.
The Thor task sources are in /build-scripts/thor.
Each task is namespaced. thor foo:bar
You can get help for a task by running thor
help task.
All Thor tasks are just Ruby files. Thor is just
a nice DSL.
Demo: The Thor build scripts.
On COMMON PROBLEMS SOLVING COMMON ISSUES
Q: Gradle tasks hanging locally.
A: Start the Nexus server.
Q: Deployments fail as unable to pull from repository.
A: Start the Docker registry.
Q: Docker build fails with “no space left on device”.
A: Restart the Docker server, if it still fails call
for help.
Q: “Expected ID but got nil”.
A: (Sadly) Just retry the build. This is a Docker/library
bug that happens randomly.
Q: The tests are failing.
A: Your tests are shit. Fix your tests.
On BEING A BUILD EXPERT BECOME A ZOMG HACKER
Demo: How to find the build to deploy.
Demo: How to see what SHA is being built for
Docker.
Fin. I’m gonna eat your hairy legs.