Scripting creating AWS based servers

in the most Taco Bell way I know

Transcript

1. Scripting your servers @coldclimate CANDDi.com
2. Put your pens down @coldclimate CANDDi.com
3. There once was a man... @coldclimate CANDDi.com
4. And then there was a document... @coldclimate CANDDi.com
5. Which became a wiki... @coldclimate CANDDi.com
6. Which finally because a script @coldclimate CANDDi.com
7. Which mostly worked... @coldclimate CANDDi.com
8. And then the madness had to stop @coldclimate CANDDi.com
9. You wouldn’t treat code like this (hopefully) @coldclimate CANDDi.com
10. Man => Document => Wiki => Script => Images @coldclimate CANDDi.com
11. Encoding Version controlling (automating) @coldclimate CANDDi.com
12. Man => Document Removing the bus factor “The man we can’t fire” @coldclimate CANDDi.com
13. Document => Wiki The fucker *might* stay up to date @coldclimate CANDDi.com
14. Wiki => Script Because script in documentation is stupid @coldclimate CANDDi.com
15. @coldclimate CANDDi.com
16. Why are you not talking about chef, puppet, ansible etc? @coldclimate CANDDi.com
17. http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html @coldclimate CANDDi.com
18. Build once or build twice? @coldclimate CANDDi.com
19. Build once Raw OS Base Instance Script Working Instance @coldclimate CANDDi.com
20. Kick-off instance Install packages Configure them Install your software Hope it all works @coldclimate CANDDi.com
21. Build Twice Raw OS Base Instance Script “Base Box” Make AMI “Base Box” “Instance Box” Script Base AMI @coldclimate CANDDi.com
22. Build once or build twice? @coldclimate CANDDi.com
23. Kick-off instance Install packages -------------------------------------------- Configure them Install your software @coldclimate CANDDi.com
24. Kick-off instance Install packages -------------------------------------------- Configure them Install your software Changes rarely Changes frequently
25. Kick-off instance Install packages -------------------------------------------- Configure them Install your software AMI Instance
26. Stability @coldclimate CANDDi.com
27. Speed @coldclimate CANDDi.com
28. The game: always runs, no manual fixes @coldclimate CANDDi.com
29. A script to built an AMI https://gist.github.com/coldclimate/5ff56c96dac116f3234e @coldclimate CANDDi.com
30. ec2-run-instances --region 'eu-west-1' -g TEMP -k TEMP -f install.sh -t t1.micro ami-ce7b6fba @coldclimate CANDDi.com
31. install.sh limited size (16kb) runs as root cock to debug @coldclimate CANDDi.com
32. /var/log/cloud-init.log @coldclimate CANDDi.com
33. set -ex @coldclimate CANDDi.com
34. Add any sources Update the APT repository Install core packages @coldclimate CANDDi.com
35. Add any sources @coldclimate CANDDi.com
36. echo "deb http://archive.canonical.com/ precise partner" | sudo tee -a /etc/apt/sources.list @coldclimate CANDDi.com
37. python-setuptools add-apt-repository -y ppa:ondrej/php5 @coldclimate CANDDi.com
38. apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10 echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | sudo tee /etc/apt/sources.list.d/10gen.list @coldclimate CANDDi.com
39. Update the APT repository @coldclimate CANDDi.com
40. apt-get update @coldclimate CANDDi.com
41. Install core packages @coldclimate CANDDi.com
42. apt-get -y install zip unzip xmlstarlet sysstat iftop ntp sysvbanner build- essential python-setuptools nginx nodejs @coldclimate CANDDi.com
43. easy_install -U boto @coldclimate CANDDi.com
44. wget https://github.com/nicolasff/phpredis/archive/2.2.4.zip -P /tmp/ mkdir -p /opt/code unzip /tmp/2.2.4.zip -d /opt/code/ cd /opt/code/phpredis-2.2.4 phpize ./configure make make install @coldclimate CANDDi.com
45. banner “*** DONE BABY ***” @coldclimate CANDDi.com
46. ec2-create-image -n<ami-name> instance_id @coldclimate CANDDi.com
47. Making running instance @coldclimate CANDDi.com
48. ec2-run-instances --region 'eu-west-1' -g sg-bb18b66c -k LIVE-PEM-KEY -p Instance_Profile -f install.sh -t m1.small --availability-zone eu-west-1a --disable-api-termination ami-XXXXXX @coldclimate CANDDi.com
49. Roles http://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html Instance Profiles http://docs.aws.amazon.com/cli/latest/reference/iam/create-instance-profile.html Adding a Role to an Instance Profile http://docs.aws.amazon.com/cli/latest/reference/iam/add-role-to-instance-profile.html @coldclimate CANDDi.com
50. Download from S3 Setup directories Setup config Install code Start it up Register it @coldclimate CANDDi.com
51. Download from S3 python <<PY import boto import sys from boto.s3.key import Key def get_wonka(bucket_name, file_name, destination_file): conn = boto.connect_s3() bucket = conn.get_bucket(bucket_name) k = Key(bucket) k.name = file_name destination = "/tmp/%s" % (destination_file) k.get_contents_to_filename(destination) get_wonka("BUCKET","SOURCEFILE","DESTINATION") PY @coldclimate CANDDi.com
52. Setup directories Your script is running as root so remember to chown @coldclimate CANDDi.com
53. Setup config Copy it from the download into place Restart the service @coldclimate CANDDi.com
54. Install code https://github.com/jordansissel/fpm @coldclimate CANDDi.com
55. Start it up Supervisord Ubuntu service Avoid own scripts @coldclimate CANDDi.com
56. Register it @coldclimate CANDDi.com
57. Monitoring (zabbix) @coldclimate CANDDi.com
58. Load Balancer (ec2 call) @coldclimate CANDDi.com
59. Logging (papertrail) @coldclimate CANDDi.com
60. DNS (R53 call) @coldclimate CANDDi.com
61. And it works! @coldclimate CANDDi.com
62. @coldclimate CANDDi.com
63. Need to change the code or config? @coldclimate CANDDi.com
64. JFDI and roll a fresh one @coldclimate CANDDi.com
65. You need to change the base AMI @coldclimate CANDDi.com
66. It’s simple, it just doesn’t feel it @coldclimate CANDDi.com
67. Update the scripts until you can “launch clean” @coldclimate CANDDi.com
68. Lastly @coldclimate CANDDi.com
69. @coldclimate CANDDi.com
70. speakerdeck. com/coldclimate/scripting-creating- aws-based-servers @coldclimate canddi.com/blog @coldclimate CANDDi.com