Chef and the Cloud

Over the past year, we've been using Chef to both configure instances in EC2 and build AMIs. Chef is an integral part of our workflow, but it doesn't always play perfectly in a world of auto scaling groups and ephemeral instances. In this talk, I cover what's worked and what's failed and explain how to best utilize Chef in the world of Amazon Web Services. It's better with audio: http://www.youtube.com/watch?v=KSScdObVdCM&feature=youtu.be

Transcript

1. Chef and the Cloud Trotter Cashion (@cashion) - Mashion
2. A Play in Three Acts
3. EC2-Classic + Chef Solo EC2-VPC + Chef Server EC2-Classic + Chef Solo + RPMs + AMIs + Netflix
4. Low Maintenance Boxes Starting Quickly No Wakeup Calls Easy to Add New Services
5. Chef Solo! March & April 2012
6. One Java Application
7. JAR
8. Make all buckets private Use IAM roles to give access
9. Librarian
10. JAR Cookbooks
11. JAR Cookbooks node.json
12. Ran on single instances ELB in front of instances AMI must support userdata.sh
13. Use Auto Scaling Groups Always Especially for Singletons!
14. > ec2-start-instances ...
15. > # install s3cmd > # install chef > # download cookbooks > chef-solo ...
16. No SPF Everything automated
17. Environment Support Config Changes Multiple Applications
18. Low Maintenance Boxes Starting Quickly No Wakeup Calls Easy to Add New Services
19. Chef Server! May 2012 - January 2013
20. Many Java Applications
21. Environment Support Chef Search
22. Moved to VPC VGW with data center
23. ELB inside VPC NAT box in public VPC
24. Use big NAT boxes Routing tables are hard
25. 10.0.0.0/8 # 10.x.x.x 10.75.0.0/16 # 10.75.x.x 10.75.43.0/24 # 10.75.43.x 10.75.43.17/32 # 10.75.43.17
26. > mvn clean install > # upload to assets.internal.host
27. Berkshelf Jenkins builds cookbooks
28. Nothing in S3
29. Protip: Put your assets in S3
30. Data Bags for AWS Keys Encrypted
31. > require ‘chef’ > s = Chef::EDBI.load_secret(key) > Chef::EDBI.encrypt_value(input, s) # Chef::EDBI = Chef::EncryptedDataBagItem
32. Cloned Asgard in Ruby Command line only Not open source :/
33. > ud = UserData.generate(“config.yml”) > lc = LaunchConfig.generate(ud) > asg = ASG.generate(lc) > CloudWatch.monitor(asg)
34. ELBs + ASGs make rolling deploys easy
35. > # write node.json > # write client.rb > # install chef > chef-client ...
36. Environments Configuration Updates Multiple Applications
37. Convergence Failures Cookbook Maintenance Start Time Environment Properties
38. > knife node delete > knife client delete
39. Node Garbage Collector
40. Chef Search
41. Low Maintenance Boxes Starting Quickly No Wakeup Calls Easy to Add New Services
42. RPMs & AMIs! February 2013 - Present
43. Leave VPC Time to Boot Cookbook Maintenance Failures to Converge
44. AMI per app-version Promote AMIs through environments
45. Back to Chef Solo One cookbook RPM per app-version
46. Archaius Eureka Asgard
47. ./conf for configs Config per env Web based config service
48. > main_class: net.mashion.App
49. > mvn clean install > # create upstart > fpm -s dir -t rpm > # upload to repo.internal.host
50. FPM is one of the best things since sliced bread. I love, love, love, love, love, love, love, love, love, love, love it.
51. Run Chef inside VPC Save AMI Use AMI outside VPC
52. > cat “PROD” > /etc/our_env > start our-service
53. Changing userdata in Asgard is... fun :/
54. Boot Time Rolling Deploys No restarts on service discovery
55. Little cookbook work New instances always boot
56. Applications we didn’t write
57. zk_watcher may help with that
58. Low Maintenance Boxes Starting Quickly No Wakeup Calls Easy to Add New Services
59. netflix.github.com github.com/jordansissel/fpm aws.amazon.com mashion.net