Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scripting creating AWS based servers
Search
coldclimate
January 27, 2014
Technology
0
270
Scripting creating AWS based servers
in the most Taco Bell way I know
coldclimate
January 27, 2014
Tweet
Share
More Decks by coldclimate
See All by coldclimate
You don't scale like servers scale
coldclimate
0
120
Quick and dirty load testing with Vegeta
coldclimate
3
4.1k
Continuously deploying https served statically compiled sites to S3 with canonical domains
coldclimate
0
70
Scaling CANDDi
coldclimate
0
250
refactoring
coldclimate
1
130
Other Decks in Technology
See All in Technology
CNAPPから考えるAWSガバナンスの実践と最適化
nrinetcom
PRO
1
410
Kubernetesでメールの大量配信をしている話/k8sjp-20250205
hfukamachi
0
330
talk_about_wasmwasi
junkishigaki
0
110
[JAWS-UG栃木]地方だからできたクラウドネイティブ事例大公開! / jawsug_tochigi_tachibana
biatunky
0
210
Oracle Base Database Service 技術詳細
oracle4engineer
PRO
6
56k
SCSAから学ぶセキュリティ管理
masakamayama
0
130
ろう・難聴者のコミュニケーションを円滑化する取り組み
chiemi627
0
120
Larkご案内資料
customercloud
PRO
0
310
ソフトウェアアーキテクトのための意思決定術: Software Architecture and Decision-Making
snoozer05
PRO
20
4.4k
依存関係があるコンポーネントは Barrel ファイルでまとめよう
azukiazusa1
3
470
バクラクの組織とアーキテクチャ(要約)2025/01版
shkomine
13
3.3k
Power BI は、レポート テーマにこだわろう!テーマのティア表付き
ohata_ds
0
140
Featured
See All Featured
Distributed Sagas: A Protocol for Coordinating Microservices
caitiem20
330
21k
Making the Leap to Tech Lead
cromwellryan
133
9.1k
Agile that works and the tools we love
rasmusluckow
328
21k
Bash Introduction
62gerente
610
210k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
365
25k
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
7
240
A Modern Web Designer's Workflow
chriscoyier
693
190k
Adopting Sorbet at Scale
ufuk
74
9.2k
Docker and Python
trallard
44
3.2k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
31
2.1k
The Straight Up "How To Draw Better" Workshop
denniskardys
232
140k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
Transcript
Scripting your servers @coldclimate CANDDi.com
Put your pens down @coldclimate CANDDi.com
There once was a man... @coldclimate CANDDi.com
And then there was a document... @coldclimate CANDDi.com
Which became a wiki... @coldclimate CANDDi.com
Which finally because a script @coldclimate CANDDi.com
Which mostly worked... @coldclimate CANDDi.com
And then the madness had to stop @coldclimate CANDDi.com
You wouldn’t treat code like this (hopefully) @coldclimate CANDDi.com
Man => Document => Wiki => Script => Images @coldclimate
CANDDi.com
Encoding Version controlling (automating) @coldclimate CANDDi.com
Man => Document Removing the bus factor “The man we
can’t fire” @coldclimate CANDDi.com
Document => Wiki The fucker *might* stay up to date
@coldclimate CANDDi.com
Wiki => Script Because script in documentation is stupid @coldclimate
CANDDi.com
@coldclimate CANDDi.com
Why are you not talking about chef, puppet, ansible etc?
@coldclimate CANDDi.com
http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html @coldclimate CANDDi.com
Build once or build twice? @coldclimate CANDDi.com
Build once Raw OS Base Instance Script Working Instance @coldclimate
CANDDi.com
Kick-off instance Install packages Configure them Install your software Hope
it all works @coldclimate CANDDi.com
Build Twice Raw OS Base Instance Script “Base Box” Make
AMI “Base Box” “Instance Box” Script Base AMI @coldclimate CANDDi.com
Build once or build twice? @coldclimate CANDDi.com
Kick-off instance Install packages -------------------------------------------- Configure them Install your software
@coldclimate CANDDi.com
Kick-off instance Install packages -------------------------------------------- Configure them Install your software
Changes rarely Changes frequently
Kick-off instance Install packages -------------------------------------------- Configure them Install your software
AMI Instance
Stability @coldclimate CANDDi.com
Speed @coldclimate CANDDi.com
The game: always runs, no manual fixes @coldclimate CANDDi.com
A script to built an AMI https://gist.github.com/coldclimate/5ff56c96dac116f3234e @coldclimate CANDDi.com
ec2-run-instances --region 'eu-west-1' -g TEMP -k TEMP -f install.sh -t
t1.micro ami-ce7b6fba @coldclimate CANDDi.com
install.sh limited size (16kb) runs as root cock to debug
@coldclimate CANDDi.com
/var/log/cloud-init.log @coldclimate CANDDi.com
set -ex @coldclimate CANDDi.com
Add any sources Update the APT repository Install core packages
@coldclimate CANDDi.com
Add any sources @coldclimate CANDDi.com
echo "deb http://archive.canonical.com/ precise partner" | sudo tee -a /etc/apt/sources.list
@coldclimate CANDDi.com
python-setuptools add-apt-repository -y ppa:ondrej/php5 @coldclimate CANDDi.com
apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10 echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist
10gen' | sudo tee /etc/apt/sources.list.d/10gen.list @coldclimate CANDDi.com
Update the APT repository @coldclimate CANDDi.com
apt-get update @coldclimate CANDDi.com
Install core packages @coldclimate CANDDi.com
apt-get -y install zip unzip xmlstarlet sysstat iftop ntp sysvbanner
build- essential python-setuptools nginx nodejs @coldclimate CANDDi.com
easy_install -U boto @coldclimate CANDDi.com
wget https://github.com/nicolasff/phpredis/archive/2.2.4.zip -P /tmp/ mkdir -p /opt/code unzip /tmp/2.2.4.zip -d
/opt/code/ cd /opt/code/phpredis-2.2.4 phpize ./configure make make install @coldclimate CANDDi.com
banner “*** DONE BABY ***” @coldclimate CANDDi.com
ec2-create-image -n<ami-name> instance_id @coldclimate CANDDi.com
Making running instance @coldclimate CANDDi.com
ec2-run-instances --region 'eu-west-1' -g sg-bb18b66c -k LIVE-PEM-KEY -p Instance_Profile -f
install.sh -t m1.small --availability-zone eu-west-1a --disable-api-termination ami-XXXXXX @coldclimate CANDDi.com
Roles http://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html Instance Profiles http://docs.aws.amazon.com/cli/latest/reference/iam/create-instance-profile.html Adding a Role to an
Instance Profile http://docs.aws.amazon.com/cli/latest/reference/iam/add-role-to-instance-profile.html @coldclimate CANDDi.com
Download from S3 Setup directories Setup config Install code Start
it up Register it @coldclimate CANDDi.com
Download from S3 python <<PY import boto import sys from
boto.s3.key import Key def get_wonka(bucket_name, file_name, destination_file): conn = boto.connect_s3() bucket = conn.get_bucket(bucket_name) k = Key(bucket) k.name = file_name destination = "/tmp/%s" % (destination_file) k.get_contents_to_filename(destination) get_wonka("BUCKET","SOURCEFILE","DESTINATION") PY @coldclimate CANDDi.com
Setup directories Your script is running as root so remember
to chown @coldclimate CANDDi.com
Setup config Copy it from the download into place Restart
the service @coldclimate CANDDi.com
Install code https://github.com/jordansissel/fpm @coldclimate CANDDi.com
Start it up Supervisord Ubuntu service Avoid own scripts @coldclimate
CANDDi.com
Register it @coldclimate CANDDi.com
Monitoring (zabbix) @coldclimate CANDDi.com
Load Balancer (ec2 call) @coldclimate CANDDi.com
Logging (papertrail) @coldclimate CANDDi.com
DNS (R53 call) @coldclimate CANDDi.com
And it works! @coldclimate CANDDi.com
@coldclimate CANDDi.com
Need to change the code or config? @coldclimate CANDDi.com
JFDI and roll a fresh one @coldclimate CANDDi.com
You need to change the base AMI @coldclimate CANDDi.com
It’s simple, it just doesn’t feel it @coldclimate CANDDi.com
Update the scripts until you can “launch clean” @coldclimate CANDDi.com
Lastly @coldclimate CANDDi.com
@coldclimate CANDDi.com
speakerdeck. com/coldclimate/scripting-creating- aws-based-servers @coldclimate canddi.com/blog @coldclimate CANDDi.com