AWS Provisioning with Chef - AMI Factory

Creating an AMI Factory for use in auto-scaling groups. Presented at the AWS Pop-Up Loft in San Francisco.

Transcript

1. AWS Provisioning with Chef Creating an AMI Factory George Miranda Partner Integration Engineer AWS Pop-up Loft June 23, 2014
2. Introductions • George Miranda ! • Partner Integration Engineer • Consultant • Infrastructure guy ! • gmiranda@getchef.com • gmiranda23 Twitter | GitHub | IRC | *:*
3. Deploying to AWS EC2 • A pretty BIG topic • FAQ: where does Chef fit? • Here’s what we’ll cover • Brief intro to Chef • Basic intro to provisioning options • AWS specific challenge: auto-scaling • Focus on AMI management • Live Demo • Next Steps
4. Chef Basics In 90 seconds or less • Configuration Management & Automation Framework • Scalable, Extensible, Testable • Infrastructure as Code • Data Driven Cookbooks • Run the same code: Dev/Stg/Prod, On-Prem or AWS • “Recipes” are a collection of “resources” • “Cookbooks” are a collection of “recipes” • 1,400+ Community Cookbooks Available • “Roles” are an ordered list of recipes to apply • “Nodes” can have one or more “roles” • “Knife” is a Chef’s go-to tool
5. Provisioning a node with knife-ec2 ! knife  ec2  server  create  -­‐r  "role[webserver]"  -­‐I  ami-­‐2d4aa444          -­‐-­‐flavor  m1.small  -­‐S  aws_ssh_key  -­‐Z  us-­‐east-­‐1b  -­‐N  my_webserver1 • Provisions a new m1.small in us-east-1b • Waits for instance-id & network settings from AWS • Waits for ssh or winrm • Issues remote commands • Install Chef • Configure Chef • Run chef-client
6. Provisioning with chef-metal require  'chef_metal_fog'   ! with_machine_options  :bootstrap_options  =>  {      :key_name  =>  'aws_ssh_key',      :image_id  =>  'ami-­‐2d4aa444',      :flavor_id  =>  'm1.small',      :region  =>  'us-­‐east-­‐1b'   }   ! machine  'my_database1'  do      role  'database'      tag  'mysql_master'   end   ! num_webservers  =  5   ! 1.upto(num_webservers)  do  |i|      machine  "my_webserver#{i}"  do          role  'webserver'    end   end • Declare infrastructure topology in a recipe • Configure multiple VM "drivers" • Ensure my_database1 is present • Then ensure 5 web servers are present • Self-healing (convergent) • Version your infrastructure • Almost 1.0.0
7. {        ~500  lines  of  JSON   !      "UserData"  :  {  "Fn::Base64"  :  {  "Fn::Join"  :  ["",  [                "#!/bin/bash  -­‐v\n",                "function  error_exit\n",                "{\n",                "  cfn-­‐signal  -­‐e  1  -­‐r  \"$1\"  '",  {  "Ref"  :   "ChefClientWaitHandle"  },  "'\n",                "exit  1\n",                "}\n",     !            #{Bunch  of  stuff  to  get  Chef  installed/configured}   !            "#  If  all  went  well,  signal  success\n",                "cfn-­‐signal  -­‐e  $?  -­‐r  'Chef  Server  configuration'   '",  {  "Ref"  :  "ChefClientWaitHandle"  },  "'\n"            ]]}},   !      ~500  more  lines  of  JSON   } Provisioning with CloudFormation
8. Why CloudFormation? JSON All The Things! • Create new Security Groups • Provision EC2 stack topology • Elastic Load Balancers • Elastic IPs • Provision AWS PaaS connections • RDS • SQS • S3 Buckets • etc • Create EC2 Auto-Scaling Groups
9. The Auto-Scaling Challenge
 Meeting Demand Fast • Typically triggered by utilization alerts • You are already falling behind by the time your infrastructure responds • You need capacity… FAST! • You already lose response time during instantiation • 3-5 minutes is an eternity when your application is choking
10. • Flexibility • Managing Configuration Drift • Rapid incremental change • Avoid golden image sprawl • Systems are not always disposable • Avoid asynchronous VM configuration • Infrastructure as Code Deploying fully-baked AMIs
 Why Would You Still Use Configuration Management?
11. Pre-Loaded AMIs
 Flexibility and Performance • Bake in the most expensive components • Minimal Configuration occurs after initialization (e.g. systems integration tasks) • Manage your pre-loaded AMI creation in a Chef Recipe • Chef Recipes are idempotent • Include those same recipes in your node’s run_list
12. Today’s Demo
 A Live Demo? SRSLY? • Creating an AMI Factory with Chef • Chef Recipe • Packer • Berkshelf ! • For today’s Demo, we’ll pre-bake a simple Nginx Web Server* ! * (you would probably never bake in something that inexpensive in “the real world”)
13. Packer • Packer is a tool for creating identical machine images for multiple platforms from a single source configuration • A HashiCorp Project • http://www.packer.io
14. Berkshelf • Berkshelf is a bundler-like dependency manager for Chef cookbooks • Encourages treating your cookbooks as Libraries or Applications • Makes it very easy to automatically download and use Chef Community Cookbooks • Started at Riot Games • http://berkshelf.com
15. Live Demo Creating a Pre-Loaded AMI Instance-Store AMI & EBS-Backed AMI: building in parallel
16. A Tour of Today’s Demo
 The Recipe include_recipe  'apt'   include_recipe  'nginx'   ! directory  '/var/www/nginx-­‐default'  do      owner  'www-­‐data'      group  'www-­‐data'      mode  '0755'      recursive  true      action  :create   end   ! file  '/var/www/nginx-­‐default/index.html'  do      owner  'www-­‐data'      group  'www-­‐data'      mode  '0755'      content  'Hello  World  from  the  AWS  Pop-­‐up  Loft!'      action  :create   end
17. A Tour of Today’s Demo
 The Berksfile site  :opscode   ! cookbook  'apt'   cookbook  'nginx'   ! metadata
18. A Tour of Today’s Demo
 The packer.json file — building Instance-Store AMIs    "builders":  [          {  "type":  "amazon-­‐instance",              "access_key":  "{{user  `aws_access_key_id`}}",              "secret_key":  "{{user  `aws_secret_key`}}",              "region":  "us-­‐east-­‐1",              "source_ami":  "ami-­‐6f969506",              "instance_type":  "m1.xlarge",              "ssh_username":  "ubuntu",              "ami_name":  "packer-­‐instance_store_nginx_{{timestamp}}",   !            "bundle_vol_command":  "sudo  -­‐n  #{very  long  ami  tool  e2-­‐bundle-­‐vol  command}”,              "bundle_upload_command":  "sudo  -­‐n  #{very  long  ami  tool  e2-­‐upload-­‐bundle  command}”,   !            "account_id":  "{{user  `account_id`}}",              "s3_bucket":  "{{user  `s3_bucket`}}",              "x509_cert_path":  "{{user  `x509_cert_path`}}",              "x509_key_path":  "{{user  `x509_key_path`}}"          },
19. A Tour of Today’s Demo
 The packer.json file — building EBS-backed AMIs builders  (continued)   !        {  "type":  "amazon-­‐ebs",              "access_key":  "{{user  `aws_access_key_id`}}",              "secret_key":  "{{user  `aws_secret_key`}}",              "region":  "us-­‐east-­‐1",              "source_ami":  "ami-­‐2b0b1442",              "instance_type":  "m1.xlarge",              "ssh_username":  "ubuntu",              "ami_name":  "packer-­‐ebs_nginx_{{timestamp}}"          }      ],
20. A Tour of Today’s Demo
 The packer.json file — provisioners    "provisioners"  :  [          {  "type":  "chef-­‐solo",              "cookbook_paths":  [  "../vendor/cookbooks"  ],              "run_list":  [  "packer::default"  ]          },   !        {  "type":  "shell",              "inline":  [                  "sudo  apt-­‐get  install  -­‐y  ruby  unzip  kpartx",                  "wget  http://s3.amazonaws.com/ec2-­‐downloads/ec2-­‐ami-­‐tools.zip",                  "sudo  mkdir  -­‐p  /usr/local/ec2",                  "sudo  unzip  ec2-­‐ami-­‐tools.zip  -­‐d  /usr/local/ec2",                  "sudo  mv  /usr/local/ec2/ec2-­‐ami-­‐tools-­‐*  /usr/local/ec2/ec2-­‐ami-­‐tools/"              ],              "only":  [  "amazon-­‐instance"  ]          }    ]
21. A Tour of Today’s Demo
 The packer command (script) #!/bin/bash   rm  -­‐r  ../vendor/cookbooks   berks  install  -­‐-­‐path  ../vendor/cookbooks   packer  build  \      -­‐var  "account_id=$AWS_ACCOUNT_ID"  \      -­‐var  "aws_access_key_id=$AWS_ACCESS_KEY_ID"  \      -­‐var  "aws_secret_key=$AWS_SECRET_ACCESS_KEY"  \      -­‐var  "x509_cert_path=$AWS_X509_CERT_PATH"  \      -­‐var  "x509_key_path=$AWS_X509_KEY_PATH"  \      -­‐var  "s3_bucket=chef-­‐packer-­‐bucket"  \      packer.json
22. Live Demo Checkpoint Creating a Pre-Loaded AMI Instance-Store AMI & EBS-Backed AMI: building in parallel
24. Ready to Use Our AMIs
 Run it the best way for your environment • Use these AMIs in your CloudFormation Templates • Define UserData to run "chef-client -j run_list.json" • Example run_list.json { "run_list": [ "role[webserver]" ] } ! • Leave integration tasks to Chef • Recommended: run chef-client periodically
25. Pitfalls
 Or How I Learned to Stop Worrying and Not Rage Against Pre-Baked AMIs • Beware Image Sprawl • Pre-Load AMIs only for critical Autoscaling Instances • Just Enough OS everywhere else • Generate AMIs with a CI Pipeline • Purge previous AMIs methodically • Cycle running instances methodically ! • Note: I am NOT suggesting everyone start making pre-loaded AMIs
26. Further Reading Next Steps From here • https://github.com/gmiranda23/chef-ami-factory • Unit testing your AMIs (e.g. serverspec) • AWS Chef Cookbook • ebs_volume • ebs_raid • elastic_ip • elastic_lb • resource_tag • http://packer.io • http://community.opscode.com/cookbooks • http://learnchef.com/
27. Thank You! George Miranda ! gmiranda@getchef.com @gmiranda23 ! Demo Repo — https://github.com/gmiranda23/chef-ami-factory ! Packer Demo primer found at — http://engineering.cotap.com