Modern Infrastructure From Scratch

Provision infrastructure from code on an IaaS platform and establish a continuous workflow to make changes in deployment and promote them into production via testing.

This talk/workshop showcases Puppet concepts like Roles & Profiles, node-less classification and r10k deployment. Using the above methods we will install and configure some of the modern FOSS operations tools available to help us manage our infrastructure. Some of the areas that will be covered are monitoring and metrics, log aggregation and of course infrastructure automation.

The talk will highlight infrastructure as code having an impact on the future of computing as we see the industry move from hardware to software driven tooling which brings quite a number of benefits.

Transcript

1. Getting it right from the start with Puppet Modern Infrastructure From Scratch
2. Senior DevOps Engineer at Ecetera simon.kp@ecetera.com.au eskp almost everywhere else Who am I
3. Overview • Define, model and provision new infrastructure • Establish an efficient development workflow • Get familiar with some of the Puppet concepts
4. Why automate infrastructure? • Consistently Repeatable and Portable • Efficient • Captured Effort • Executable/Live Self-Documentation
5. Define Modern Infrastructure • Infrastructure Automation (VirtualBox+Vagrant) • Version Control (Git+Gitlab) • Configuration Management (Puppet) • Test + Deployment Automation (Jenkins+r10k) • Centralised Log Aggregation (ELK stack) • Monitoring (Sensu)
6. Infrastructure Model
7. The code Two repositories: • Vagrant repo https://github.com/Ecetera/puppet-vagrant • Control repo https://github.com/Ecetera/puppet-control
8. Puppet Roles & Profiles Resources > Classes > Modules > Profiles > Role > Node Allow the business to manage how the infrastructure looks without defining what it is Data is abstracted by Hiera
9. Roles definition # site/role/manifests/init.pp class role::git { include profile::jenkins include profile::rabbit include profile::mco::client } ...
10. Profiles definition # site/profile/manifests/init.pp class profile::jenkins { class { '::jenkins': } jenkins::job { 'puppet-control': } create_resources(jenkins::credentials, hiera('jenkins_credentials')) } ...
11. Nodeless Classification # site.pp node default { hiera_include(‘role’) }
12. Hiera overview --- # /etc/puppet/hiera.yaml :yaml: :datadir: “/etc/puppet/environments/%{::environment}/data” :hierarchy: - “%{::hostname}”
13. Hiera overview --- # data/puppet.yaml role: - role::puppetserver apache::default_vhost: false ...
14. Don’t write modules unless you absolutely have to. Modules
15. Look for modules • Puppet Labs approved and supported • Lots of downloads • No weird dependencies • Documentation • Source is on Github
16. Module Deployment r10k - manages modules defined in Puppetfile Deploys Puppet’s dynamic directory environments based on puppet-control repo’s git branches Does not resolve modules dependencies
17. Puppetfile mod ‘puppetlabs/ntp’, ‘3.2.1’ mod ‘rabbitmq’, :git => https://github.com/puppetlabs/puppetlabs-rabbitmq.git, :commit => '15e2421048a3495e60482837910441a1248bf6f0' ...
18. Demonstration http://puppet.eskp.net:5000 Puppetboard view of Puppet infrastructure http://control.eskp.net:9090 Jenkins http://log.eskp.net Kibana dashboard for centralised logs access http://mon.eskp.net:3000 Uchiwa dashboard overview of Sensu
19. Todo • Write infrastructure tests • Address scale • Provision with puppetlabs-aws module • Upgrade to Puppet 4
20. Questions? simon.kp@ecetera.com.au (eskp) “Modern Infrastructure From Scratch”