A metadata ocean in Chef and Puppet

Transcript

1. A metadata ocean in Puppet and Chef FOSDEM’14 Marc Cluet DevOps & Automation Advisory Rackspace http://www.flickr.com/photos/elias_daniel/7470265444/
2. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Who am I?! Marc Cluet (@lynxman) More than 16 years of experience as a SysAdmin Founding member of Juju and MAAS while at Canonical Leading a DevOps Engineering Team at Rackspace Likes DevOps, programming and long walks on the beach
3. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK What is Rackspace?! Founded in 1998 in San Antonio, TX by three guys that wanted to create a hosting company Home of Fanatical Support /o/ Second biggest Public Cloud in the world OpenStack Project co-founder
4. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK What is Metadata?! /http://www.flickr.com/photos/boojee/3743753784/
5. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK What is Metadata?! http://www.flickr.com/photos/ggvaidya/8256283184/ Metadata is “data about data”! Structural Metadata! Descriptive Metadata!
6. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Metadata in Chef! Metadata is compiled on the client at execution time! Metadata is always stored in its entirety in the node[] dictionary! $node[var] = val
7. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Metadata in Chef! You can add metadata in your! Attribute File! Node/Recipe! Environment! Role!
8. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Organisation in Chef! http://docs.opscode.com/essentials_environments.html
9. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Metadata in Puppet! Metadata is always compiled on execution time Metadata vars can’t be reassigned! $::var = val
10. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Metadata in Puppet! You can add metadata in your! !Class vars! !Nodes! !Top Scope!
11. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Scoping in Puppet! http://docs.puppetlabs.com/puppet/latest/reference/lang_scope.html
12. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Hiera in Puppet! Solves metadata fragmentation issues! Defines hierarchy for easy organisation! All in clean and properly styled in either YAML or JSON!
13. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Hiera in Puppet! :backends: - yaml - json :yaml: :datadir: /etc/puppet/hieradata :json: :datadir: /etc/puppet/hieradata :hierarchy: - "%{::clientcert}" - "%{::custom_location}" - common
14. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/musebrarian/3289649684/
15. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Bad metadata practices! In Chef! Not keeping consistency between cookbook scope and environment scope! Not removing metadata from nodes when deprecated! Directly assigning metadata in a cookbook without a default attribute!
16. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Bad metadata practices! In Puppet! Directly assigning metadata in a class without a default value! Assigning vars in nodes.pp! Not assigning global vars in site.pp!
17. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/deepstereo/9616389940/
18. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Good behaviours! Separate vars from execution! Do not store vars on the execution area (classes/cookbooks)! High abstraction, good hierarchy! Metadata is all about context! !
19. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Good behaviours! Global Environment Node
20. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/monica_andre/3119991125/
21. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Metadata Across Providers! Provisioner cloud-init Config Mgmt Puppet $dog = cute $::dog = big WHO IS RIGHT?!?!
22. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Metadata Across Providers! http://www.flickr.com/photos/pss/233841567/
23. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Hard Problem to Solve! Metadata brain split problem! Gets worse by adding more players! System view varies! Not same info from bios than from a user level program as facter or ohai! !
24. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Possible Solutions! Central DB using something fast! !Redis, mongoDB, mySQL! Metadata translators might fix it?! !Write for everything! Creates new SPoF! !Hard to solve without throwing $$!
25. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Hard Problem to Solve! http://xkcd.com/927/
26. RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK Possible Solutions! Scope vars from top of chain! Pass them over as ext facts! $dog in cloud-init becomes $::dog in puppet!
27. 27 RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.CO.UK RACKSPACE® HOSTING | 5 MILLINGTON ROAD | HAYES, UNITED KINGDOM UB3 4AZ UK SALES: +44 (0)20 8712 6507 | UK SUPPORT: 0800 988 0300 | WWW.RACKSPACE.CO.UK @lynxman! http://slideshare.net/lynxmanuk/! http://www.rackspace.co.uk/technicalservices/ ! !