Scripting creating AWS based servers

Transcript

1. Scripting your servers @coldclimate CANDDi.com

2. Put your pens down @coldclimate CANDDi.com

3. There once was a man... @coldclimate CANDDi.com

4. And then there was a document... @coldclimate CANDDi.com

5. Which became a wiki... @coldclimate CANDDi.com

6. Which finally because a script @coldclimate CANDDi.com

7. Which mostly worked... @coldclimate CANDDi.com

8. And then the madness had to stop @coldclimate CANDDi.com

9. You wouldn’t treat code like this (hopefully) @coldclimate CANDDi.com

10. Man => Document => Wiki => Script => Images @coldclimate

    CANDDi.com

11. Encoding Version controlling (automating) @coldclimate CANDDi.com

12. Man => Document Removing the bus factor “The man we

    can’t fire” @coldclimate CANDDi.com

13. Document => Wiki The fucker *might* stay up to date

    @coldclimate CANDDi.com

14. Wiki => Script Because script in documentation is stupid @coldclimate

    CANDDi.com

15. @coldclimate CANDDi.com

16. Why are you not talking about chef, puppet, ansible etc?

    @coldclimate CANDDi.com

17. http://widgetsandshit.com/teddziuba/2010/10/taco-bell-programming.html @coldclimate CANDDi.com

18. Build once or build twice? @coldclimate CANDDi.com

19. Build once Raw OS Base Instance Script Working Instance @coldclimate

    CANDDi.com

20. Kick-off instance Install packages Configure them Install your software Hope

    it all works @coldclimate CANDDi.com

21. Build Twice Raw OS Base Instance Script “Base Box” Make

    AMI “Base Box” “Instance Box” Script Base AMI @coldclimate CANDDi.com

22. Build once or build twice? @coldclimate CANDDi.com

23. Kick-off instance Install packages -------------------------------------------- Configure them Install your software

    @coldclimate CANDDi.com

24. Kick-off instance Install packages -------------------------------------------- Configure them Install your software

    Changes rarely Changes frequently

25. Kick-off instance Install packages -------------------------------------------- Configure them Install your software

    AMI Instance

26. Stability @coldclimate CANDDi.com

27. Speed @coldclimate CANDDi.com

28. The game: always runs, no manual fixes @coldclimate CANDDi.com

29. A script to built an AMI https://gist.github.com/coldclimate/5ff56c96dac116f3234e @coldclimate CANDDi.com

30. ec2-run-instances --region 'eu-west-1' -g TEMP -k TEMP -f install.sh -t

    t1.micro ami-ce7b6fba @coldclimate CANDDi.com

31. install.sh limited size (16kb) runs as root cock to debug

    @coldclimate CANDDi.com

32. /var/log/cloud-init.log @coldclimate CANDDi.com

33. set -ex @coldclimate CANDDi.com

34. Add any sources Update the APT repository Install core packages

    @coldclimate CANDDi.com

35. Add any sources @coldclimate CANDDi.com

36. echo "deb http://archive.canonical.com/ precise partner" | sudo tee -a /etc/apt/sources.list

    @coldclimate CANDDi.com

37. python-setuptools add-apt-repository -y ppa:ondrej/php5 @coldclimate CANDDi.com

38. apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10 echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist

    10gen' | sudo tee /etc/apt/sources.list.d/10gen.list @coldclimate CANDDi.com

39. Update the APT repository @coldclimate CANDDi.com

40. apt-get update @coldclimate CANDDi.com

41. Install core packages @coldclimate CANDDi.com

42. apt-get -y install zip unzip xmlstarlet sysstat iftop ntp sysvbanner

    build- essential python-setuptools nginx nodejs @coldclimate CANDDi.com

43. easy_install -U boto @coldclimate CANDDi.com

44. wget https://github.com/nicolasff/phpredis/archive/2.2.4.zip -P /tmp/ mkdir -p /opt/code unzip /tmp/2.2.4.zip -d

    /opt/code/ cd /opt/code/phpredis-2.2.4 phpize ./configure make make install @coldclimate CANDDi.com

45. banner “*** DONE BABY ***” @coldclimate CANDDi.com

46. ec2-create-image -n<ami-name> instance_id @coldclimate CANDDi.com

47. Making running instance @coldclimate CANDDi.com

48. ec2-run-instances --region 'eu-west-1' -g sg-bb18b66c -k LIVE-PEM-KEY -p Instance_Profile -f

    install.sh -t m1.small --availability-zone eu-west-1a --disable-api-termination ami-XXXXXX @coldclimate CANDDi.com

49. Roles http://docs.aws.amazon.com/cli/latest/reference/iam/create-role.html Instance Profiles http://docs.aws.amazon.com/cli/latest/reference/iam/create-instance-profile.html Adding a Role to an

    Instance Profile http://docs.aws.amazon.com/cli/latest/reference/iam/add-role-to-instance-profile.html @coldclimate CANDDi.com

50. Download from S3 Setup directories Setup config Install code Start

    it up Register it @coldclimate CANDDi.com

51. Download from S3 python <<PY import boto import sys from

    boto.s3.key import Key def get_wonka(bucket_name, file_name, destination_file): conn = boto.connect_s3() bucket = conn.get_bucket(bucket_name) k = Key(bucket) k.name = file_name destination = "/tmp/%s" % (destination_file) k.get_contents_to_filename(destination) get_wonka("BUCKET","SOURCEFILE","DESTINATION") PY @coldclimate CANDDi.com

52. Setup directories Your script is running as root so remember

    to chown @coldclimate CANDDi.com

53. Setup config Copy it from the download into place Restart

    the service @coldclimate CANDDi.com

54. Install code https://github.com/jordansissel/fpm @coldclimate CANDDi.com

55. Start it up Supervisord Ubuntu service Avoid own scripts @coldclimate

    CANDDi.com

56. Register it @coldclimate CANDDi.com

57. Monitoring (zabbix) @coldclimate CANDDi.com

58. Load Balancer (ec2 call) @coldclimate CANDDi.com

59. Logging (papertrail) @coldclimate CANDDi.com

60. DNS (R53 call) @coldclimate CANDDi.com

61. And it works! @coldclimate CANDDi.com

62. @coldclimate CANDDi.com

63. Need to change the code or config? @coldclimate CANDDi.com

64. JFDI and roll a fresh one @coldclimate CANDDi.com

65. You need to change the base AMI @coldclimate CANDDi.com

66. It’s simple, it just doesn’t feel it @coldclimate CANDDi.com

67. Update the scripts until you can “launch clean” @coldclimate CANDDi.com

68. Lastly @coldclimate CANDDi.com

69. @coldclimate CANDDi.com

70. speakerdeck. com/coldclimate/scripting-creating- aws-based-servers @coldclimate canddi.com/blog @coldclimate CANDDi.com