Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure

This is the presentation deck I used for the talk I gave at rootconf - A conference at Bangalore for sysadmins.
Gist of the talk:-
Puppet is a great configuration management tool and git is great at version controlling.AWS lets you create instances in few clicks. But when it comes to large deployments only automation(where tools come together) can make you productive and happy. I will take you through following.. Fog - The Ruby cloud services library and how it helps you to create vendor neutral cloud deployments, Puppet- Multi region puppet masters, Ruby- How Ruby pulls the strings together in EC2/ELB/RDS creation, Security group creation, IP authorization, Route53 DNS etc, Git- how we use git to version control deployment configs/configurations.

Transcript

1. Cloud meets Fog & Puppet A Story of Version Controlled Infrastructure Habeeb Rahman | apigee | @habi_pk
2. About:apigee We love APIs Developers => API <= Enterprises Usergrid
3. Take 1 Third-party web tool and ...
4. ….Following Manual Tasks ShareImages CreateInstancesOnebyOne InstallPackages InstallLicense CreateEBSVolumes OpenPorts AttachEIP GenerateServerConfig Apply Patches GenerateNagiosConfig + a Few Misc
5. Take 2 Cloud meets puppet and ... (aws console, git, puppet)
6. ….Following Manual Tasks ShareImages CreateInstancesOnebyOne InstallPackages InstallLicense CreateEBSVolumes OpenPorts AttachEIP GenerateServerConfig ApplyPatches GenerateNagiosConfig + a Few Misc
7. Take 3 Cloud meets Fog and … (puppet cloud provisioner, fog, ruby)
8. ….Following Manual Tasks ShareImages CreateInstancesOnebyOne InstallPackages InstallLicense CreateEBSVolumes OpenPorts AttachEIP GenerateServerConfig ApplyPatches GenerateNagiosConfig + a Few Misc
9. Customer in a 'yaml' file...
11. Git version controls Puppet Configurations & Cloud Deployments (Yes! Version Controlled Infrastructure)
13. Git hooks Pre-update => puppet syntax checks (puppet parser, puppet lint) Post-update => sync with regional puppet masters
14. Puppet Git branches => Puppet environments Single puppet master => Test Dev Production manifests and files/data => Separate git repo Puppet masters per AWS regions Hiera: a pluggable hierarchical data store
15. Why Fog?
16. Cross service compatibility (multitude of services=> Storage, Compute, DNS…) Avoid vendor lock-in (Vendor independent=>EC2, Rackspace, Slicehost, Linode…) Power of Ruby
17. gem install fog
18. require ‘fog’
19. ~/.fog
20. Fog quick starter kit Look at the test suites (thanks to TDD) https://github.com/fog/fog/blob/master/tests/aws/requests/compute/instance_tests.rb
21. Fog command line [root@zebra ~]$ less ~/.fog [root@zebra ~]$ fog testaws Welcome to fog interactive! :testaws provides AWS >>
22. ec2 = Fog::Compute.new( provider: "AWS", aws_access_key_id: "KEY", aws_secret_access_key: "SECRET" ) rsc = Fog::Compute.new( provider: "Rackspace", rackspace_api_key: "KEY", rackspace_username: "SECRET" ) ec2.servers.create( flavor_id: "c1.medium", image_id: "ami-6936fb00", # Ubuntu 10.04 LTS key_name: "mykeyname", groups: ["default"], ) rsc.servers.create( flavor_id: 5, # 4GB Memory image_id: 49, # Ubuntu 10.04 LTS name: 'my_server_name' )
23. Open Source Fog alternatives libcloud (Python) jclouds (Java) deltacloud (Ruby)
24. Why Ruby?
25. Ruby is designed for programmer productivity and fun Matsumoto(creator of the ruby language)
26. 'everything-ruby' by Ajey Gore http://confreaks.com/videos/921-rubyconfindia2012-everything-ruby
27. irb is your friend (ruby interactive shell) root@zebra:~/$ irb 1.9.3-p125 :001 > irb alternative = > pry (gem install pry)
28. The Layers of Abstraction...
29. Fog => Cloud abstraction
30. Puppet => OS abstraction
31. Notes to self >>
32. Start small...
33. ...and iterate reiterate & re reiterate
34. Select right tools (Plenty of tools => Puppet/Chef/Cfengine/Ansible but what is good for you/team/culture?)
35. Design your SaaS but make sure it's 'ops' ready (from the day you start the design)
36. http://puppetlabs.com/ http://fog.io http://git-scm.com/ http://ansible.github.com/
37. We are hiring Join apigee ops team