Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Scripting creating AWS based servers
Search
coldclimate
January 27, 2014
Technology
0
300
Scripting creating AWS based servers
in the most Taco Bell way I know
coldclimate
January 27, 2014
Tweet
Share
More Decks by coldclimate
See All by coldclimate
You don't scale like servers scale
coldclimate
0
140
Quick and dirty load testing with Vegeta
coldclimate
3
4.2k
Continuously deploying https served statically compiled sites to S3 with canonical domains
coldclimate
0
72
Scaling CANDDi
coldclimate
0
280
refactoring
coldclimate
1
130
Other Decks in Technology
See All in Technology
パフォーマンスチューニングのために普段からできること/Performance Tuning: Daily Practices
fujiwara3
2
140
serverless team topology
_kensh
3
240
頭部ふわふわ浄酔器
uyupun
0
230
Amazon Athena で JSON・Parquet・Iceberg のデータを検索し、性能を比較してみた
shigeruoda
1
170
DSPy入門
tomehirata
3
480
AI時代の発信活動 ~技術者として認知してもらうための発信法~ / 20251028 Masaki Okuda
shift_evolve
PRO
1
120
re:Inventに行くまでにやっておきたいこと
nagisa53
0
700
AI機能プロジェクト炎上の 3つのしくじりと学び
nakawai
0
130
OSSで50の競合と戦うためにやったこと
yamadashy
3
1k
abema-trace-sampling-observability-cost-optimization
tetsuya28
0
350
webpack依存からの脱却!快適フロントエンド開発をViteで実現する #vuefes
bengo4com
4
3.6k
プロファイルとAIエージェントによる効率的なデバッグ / Effective debugging with profiler and AI assistant
ymotongpoo
1
510
Featured
See All Featured
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
16
1.7k
The Pragmatic Product Professional
lauravandoore
36
7k
Stop Working from a Prison Cell
hatefulcrawdad
272
21k
Rails Girls Zürich Keynote
gr2m
95
14k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
249
1.3M
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
55
3k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
PRO
190
55k
A designer walks into a library…
pauljervisheath
209
24k
Practical Orchestrator
shlominoach
190
11k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
The Straight Up "How To Draw Better" Workshop
denniskardys
239
140k
YesSQL, Process and Tooling at Scale
rocio
173
15k
Transcript
Scripting your servers @coldclimate CANDDi.com
Put your pens down @coldclimate CANDDi.com
There once was a man... @coldclimate CANDDi.com
And then there was a document... @coldclimate CANDDi.com
Which became a wiki... @coldclimate CANDDi.com
Which finally because a script @coldclimate CANDDi.com
Which mostly worked... @coldclimate CANDDi.com
And then the madness had to stop @coldclimate CANDDi.com
You wouldn’t treat code like this (hopefully) @coldclimate CANDDi.com
Man => Document => Wiki => Script => Images @coldclimate
CANDDi.com
Encoding Version controlling (automating) @coldclimate CANDDi.com
Man => Document Removing the bus factor “The man we
can’t fire” @coldclimate CANDDi.com
Document => Wiki The fucker *might* stay up to date
@coldclimate CANDDi.com
Wiki => Script Because script in documentation is stupid @coldclimate
CANDDi.com
@coldclimate CANDDi.com
Why are you not talking about chef, puppet, ansible etc?
@coldclimate CANDDi.com
http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html @coldclimate CANDDi.com
Build once or build twice? @coldclimate CANDDi.com
Build once Raw OS Base Instance Script Working Instance @coldclimate
CANDDi.com
Kick-off instance Install packages Configure them Install your software Hope
it all works @coldclimate CANDDi.com
Build Twice Raw OS Base Instance Script “Base Box” Make
AMI “Base Box” “Instance Box” Script Base AMI @coldclimate CANDDi.com
Build once or build twice? @coldclimate CANDDi.com
Kick-off instance Install packages -------------------------------------------- Configure them Install your software
@coldclimate CANDDi.com
Kick-off instance Install packages -------------------------------------------- Configure them Install your software
Changes rarely Changes frequently
Kick-off instance Install packages -------------------------------------------- Configure them Install your software
AMI Instance
Stability @coldclimate CANDDi.com
Speed @coldclimate CANDDi.com
The game: always runs, no manual fixes @coldclimate CANDDi.com
A script to built an AMI https://gist.github.com/coldclimate/5ff56c96dac116f3234e @coldclimate CANDDi.com
ec2-run-instances --region 'eu-west-1' -g TEMP -k TEMP -f install.sh -t
t1.micro ami-ce7b6fba @coldclimate CANDDi.com
install.sh limited size (16kb) runs as root cock to debug
@coldclimate CANDDi.com
/var/log/cloud-init.log @coldclimate CANDDi.com
set -ex @coldclimate CANDDi.com
Add any sources Update the APT repository Install core packages
@coldclimate CANDDi.com
Add any sources @coldclimate CANDDi.com
echo "deb http://archive.canonical.com/ precise partner" | sudo tee -a /etc/apt/sources.list
@coldclimate CANDDi.com
python-setuptools add-apt-repository -y ppa:ondrej/php5 @coldclimate CANDDi.com
apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10 echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist
10gen' | sudo tee /etc/apt/sources.list.d/10gen.list @coldclimate CANDDi.com
Update the APT repository @coldclimate CANDDi.com
apt-get update @coldclimate CANDDi.com
Install core packages @coldclimate CANDDi.com
apt-get -y install zip unzip xmlstarlet sysstat iftop ntp sysvbanner
build- essential python-setuptools nginx nodejs @coldclimate CANDDi.com
easy_install -U boto @coldclimate CANDDi.com
wget https://github.com/nicolasff/phpredis/archive/2.2.4.zip -P /tmp/ mkdir -p /opt/code unzip /tmp/2.2.4.zip -d
/opt/code/ cd /opt/code/phpredis-2.2.4 phpize ./configure make make install @coldclimate CANDDi.com
banner “*** DONE BABY ***” @coldclimate CANDDi.com
ec2-create-image -n<ami-name> instance_id @coldclimate CANDDi.com
Making running instance @coldclimate CANDDi.com
ec2-run-instances --region 'eu-west-1' -g sg-bb18b66c -k LIVE-PEM-KEY -p Instance_Profile -f
install.sh -t m1.small --availability-zone eu-west-1a --disable-api-termination ami-XXXXXX @coldclimate CANDDi.com
Roles http://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html Instance Profiles http://docs.aws.amazon.com/cli/latest/reference/iam/create-instance-profile.html Adding a Role to an
Instance Profile http://docs.aws.amazon.com/cli/latest/reference/iam/add-role-to-instance-profile.html @coldclimate CANDDi.com
Download from S3 Setup directories Setup config Install code Start
it up Register it @coldclimate CANDDi.com
Download from S3 python <<PY import boto import sys from
boto.s3.key import Key def get_wonka(bucket_name, file_name, destination_file): conn = boto.connect_s3() bucket = conn.get_bucket(bucket_name) k = Key(bucket) k.name = file_name destination = "/tmp/%s" % (destination_file) k.get_contents_to_filename(destination) get_wonka("BUCKET","SOURCEFILE","DESTINATION") PY @coldclimate CANDDi.com
Setup directories Your script is running as root so remember
to chown @coldclimate CANDDi.com
Setup config Copy it from the download into place Restart
the service @coldclimate CANDDi.com
Install code https://github.com/jordansissel/fpm @coldclimate CANDDi.com
Start it up Supervisord Ubuntu service Avoid own scripts @coldclimate
CANDDi.com
Register it @coldclimate CANDDi.com
Monitoring (zabbix) @coldclimate CANDDi.com
Load Balancer (ec2 call) @coldclimate CANDDi.com
Logging (papertrail) @coldclimate CANDDi.com
DNS (R53 call) @coldclimate CANDDi.com
And it works! @coldclimate CANDDi.com
@coldclimate CANDDi.com
Need to change the code or config? @coldclimate CANDDi.com
JFDI and roll a fresh one @coldclimate CANDDi.com
You need to change the base AMI @coldclimate CANDDi.com
It’s simple, it just doesn’t feel it @coldclimate CANDDi.com
Update the scripts until you can “launch clean” @coldclimate CANDDi.com
Lastly @coldclimate CANDDi.com
@coldclimate CANDDi.com
speakerdeck. com/coldclimate/scripting-creating- aws-based-servers @coldclimate canddi.com/blog @coldclimate CANDDi.com