Automating the Cloud - Integrating Puppet with Cloud Infrastructures 2016

So, you've applied your manifests to a number of servers, and puppet is working great! Now you've grown more ambitious and want to puppetize an infrastructure on a public cloud like AWS. But after weeks of clicking around in the AWS console and manually hooking up EC2 instances in Puppet you find yourself thinking: "There must be an easier and more structured way to do this, RIGHT?" Well, there is! This talk combines all the secrets I learned at PuppetConf 2015 and will help you leverage tools like puppetlabs-aws, terraform, hiera, cloud-init, CodeDeploy and foreman (autosigning and default_hostgroup) to deploy your "Infrastructure as Code" and overcome the difficulties of bootstrapping and managing a variable number of distributed systems using Puppet. We focus primarily on AWS, but the principles and difficulties apply to most public cloud platforms and all distributed systems in general.

Transcript

1. Remco Overdijk LEAD OPERATIONS ENGINEER Automating the Cloud Integrating Puppet with Cloud Infrastructures @MaxServ @RemzJay
2. Automating the Cloud 1
3. 3 AUTOMATING THE CLOUD Traditional Website Hosting MySQL
4. AUTOMATING THE CLOUD Single Server Infrastructure Issues PROBLEM Limited headroom Service issues affect other services One outage means downtime Maintenance during deployment 4
5. Image credit: https://commons.wikimedia.org/wiki/File:Grumpy-Cat.jpg - Rjommabolli (CC 4.0)
6. AUTOMATING THE CLOUD Single Server Infrastructure Issues 6 Scalability Service Isolation Redundancy Zero-Downtime Deployments SOLUTION PROBLEM Limited headroom Affected services One outage means downtime Maintenance during deployment
7. • AWS specific, but applies to most (if not any) Cloud platforms. AUTOMATING THE CLOUD Scope 7 • LAMP stack, but works for most stacks. • Mix & match for best results. • The method that works best depends on your own setup. • Based on Puppet Open Source. • Things may be different in Puppet Enterprise (Orchestrator). • Ready-to-go AMI’s may be faster to launch, but harder to maintain.
8. AUTOMATING THE CLOUD AWS Infrastructure Principles Read more https://media.amazonwebservices.com/AWS_Cloud_Best_Practices.pdf 8 • Infrastructure is failure-prone; Service robustness is achieved through redundancy. • EC2 instances should be considered volatile. • Scaling should be both vertical and horizontal. • Legacy applications aren’t magically cloud-ready. • Loose coupling improves scalability. • Isolation increases security and decreases dependencies.
9. AUTOMATING THE CLOUD AWS Infra: A lot of moving parts 9
10. Provisioning Infrastructure 2
11. • IAM Server Certificates • IAM Instance Profiles • IAM Roles • IAM Policies • IAM Role Policies • CodeDeploy Apps • CodeDeploy Deployment Groups • EC2 Instances • EC2 Elastic IP’s • CloudWatch Metrics • CloudWatch Alarms • EIP Attachments • ElastiCache Subnet Groups Provisioning Infrastructure AWS Resources • ElastiCache Clusters • ElastiCache Parameter Groups • EC2 Elastic Load Balancers • ELB Health Checks • ELB App Cookie Stickiness Policies • Key Pairs • RDS Subnet Groups • RDS Parameter Groups • RDS DB Instances • Route53 Zones • Route53 Records • S3 Buckets • S3 Policies • S3 CORS Configuration • Security Groups • SNS Topics • SNS Topic Subscriptions • VPC’s • VPC Subnets • VPC Internet Gateways • VPC (S3) Endpoints • VPC Route Tables • VPC Customer Gateways • VPC VPN Gateways • VPC DHCP Option Sets • VPC VPN Routes 11
13. Provisioning Infrastructure That’s a lot of clicking 13 • Manual changes are extremely error-prone. • Manual changes result in an inconsistent platform. • Collaboration is difficult; People are scared to break things. • Changes are scattered throughout the AWS console.
14. Provisioning Infrastructure Provisioning 14
15. • Uses the same DSL as your ”regular” Puppet code. Provisioning Infrastructure Puppetlabs-AWS 15 • Most benefits from ”regular” Puppet; like relationships, defined types and the graph. • At the moment this module only supports a few of the resources in the AWS API. • Does NOT run using agents / puppet masters, but manually from your workstation using `puppet apply`. • Not as idempotent as you’d like at times. Read more https://github.com/puppetlabs/puppetlabs-aws https://forge.puppetlabs.com/puppetlabs/aws
16. `AWS_PROFILE=my-aws-account AWS_REGION=eu-west-1 puppet apply aws-stack.pp --templatedir ./templates/`
17. Provisioning Infrastructure Hashicorp Terraform • Uses HCL, which looks a lot like Puppet DSL. • Maintains a dependency graph, just like Puppet. • Runs from your workstation. • Is not AWS API feature complete, but covers most services. • Allows flexible scaling and destruction. 17 Read more https://github.com/hashicorp/terraform https://www.terraform.io/
18. Provisioning Infrastructure Hashicorp Terraform 18
19. View & try the full source https://github.com/MaxServ/Terraform-Puppet-Demo/tree/master/Terraform
20. Provisioning Infrastructure Puppet? 20 • Automated Infrastructure is awesome. • We like Puppet too. • Empty EC2 instances don’t serve websites. • How do we connect Puppet to the infrastructure we’ve just provisioned?
21. Bootstrapping Puppet 3
22. Bootstrapping Puppet Machine Configuration States 22
23. SLIDE CREDIT: Tim Bell, CERN – PuppetConf 2012 https://speakerdeck.com/puppetlabs/cern-accelerating-science-with-puppet https://youtu.be/-Ykb2j2ojYU?t=19m33s
24. Image credit: Geert Orye, https://flic.kr/p/7zDqXY (CC BY 2.0)
25. Bootstrapping Puppet So, Cattle. • Configuration should be at type level. Not node level. • Nodes should be replaceable. • Data on the nodes should be considered volatile. • Only versioned and cached content should be present. 25 • Provisioning should be automated; No manual intervention should be required.
26. Bootstrapping Puppet Puppet: Autosigning • Automates indoctrination for new nodes. • Multiple mechanisms available: Naïve, Basic and Policy-based Autosigning. 26 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Puppet/puppet.conf Read more https://docs.puppetlabs.com/puppet/latest/reference/ssl_autosign.html Be very careful with naïve autosigning. Don’t do it in production without strict firewalls.
27. Bootstrapping Puppet Puppet: Autosigning 27
28. Bootstrapping Puppet Node Manifests using Regex 28 Read more https://docs.puppetlabs.com/puppet/latest/reference/lang_node_definitions.html#regular-expression-names
29. Bootstrapping Puppet Hiera • Configuration at `type` level instead of `clientcert`. • Use node-level overrides when required. • Combine with host `%H` mount points for master/slave. 29 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/tree/master/Puppet/hieradata Read more https://docs.puppetlabs.com/hiera/latest/ https://docs.puppetlabs.com/guides/file_serving.html#file-server-configuration
30. Bootstrapping Puppet Hiera 30
31. Bootstrapping Puppet ENC’s in Foreman 31 Read more https://docs.puppetlabs.com/guides/external_nodes.html http://theforeman.org/manuals/1.10/index.html#2.Quickstart
32. Bootstrapping Puppet Foreman: Default Host Groups • The same idea as Autosigning in Puppet. • Uses a Foreman Plugin. • Use facts for assigning host groups. 32 Read more https://github.com/theforeman/foreman_default_hostgroup
33. Bootstrapping Puppet Foreman: Default Host Groups 33
34. Bootstrapping Puppet Assigning Configuration: Facter 34 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Terraform/templates/userdata.tpl https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Puppet/Vagrantfile Read more https://docs.puppetlabs.com/facter/2.4/custom_facts.html#structured-data-facts
35. Bootstrapping Puppet AWS: User data • User data allows you to pass configuration data to an instance during launch. • User data can be used to provide cloud-init configuration. 35
36. Bootstrapping Puppet AWS: User data 36
37. Bootstrapping Puppet Cloud-init 37 • Handles early initialization of cloud instances. • Supports EC2, CloudStack, OpenStack, OnApp, OpenNebula, RHEVm, vSphere & more. • Can install packages, do basic config and more. • Is able to bootstrap Puppet. Read more http://cloudinit.readthedocs.org/en/latest/index.html http://cloudinit.readthedocs.org/en/latest/topics/examples.html#setup-and-run-puppet
38. Bootstrapping Puppet Cloud-init with Puppetlabs-Aws 38 Read more https://docs.puppetlabs.com/puppet/latest/reference/lang_template_erb.html
39. Bootstrapping Puppet Cloud-init with Terraform 39 View & try https://github.com/MaxServ/Terraform-Puppet-Demo/blob/master/Terraform/ec2.tf Read more https://docs.puppetlabs.com/puppet/latest/reference/lang_template_erb.html
40. Bootstrapping Puppet Don’t forget to clean up! • Revoke Puppet-CA certificates for decommissioned nodes. • Clean up Salt keys. • Remove old reports, exported resources and catalogs from PuppetDB. • Clean connected resources like Load Balancers. 40 Sloppiness will catch up with you when it hurts the most.
41. Demo 4
42. Demo Terraform & Puppet THIS DEMONSTRATION INCLUDES: VIEW & TRY THE FULL SOURCE OF THIS DEMO https://github.com/MaxServ/Terraform-Puppet-Demo A B C Docker container running a Puppetmaster. AWS Stack with 4 webservers using Terraform. Webserver provisioning using Puppet based on type. 42
43. Demo Virtual Private Cloud 43
45. 0416 - 30 10 00 Remco Overdijk LEAD OPERATIONS ENGINEER info@maxserv.com Questions? @MaxServ @RemzJay