Hiera

Slides for a hiera talk I gave at Infracoders Melbourne.

Transcript

1. HIERA
2. STOP PUTTING VARIABLES IN MANIFESTS (no really, stop it)
3. WHAT HAPPENED ! ! case $environment {! dev: ! ! { $sudoers = ['dev','ops','test']}! test: !! { $sudoers = [ 'ops','test']}! prod: !! { $sudoers = 'ops'}! default: { $sudoers = 'ops'}! }
4. WHAT YOU THINK YOU DID (It's all Jim's fault, honest!) ! if $hostname != 'snowflake' {! case $environment {! dev: ! ! { $sudoers = ['dev','ops','test']}! test: ! ! { $sudoers = [ 'ops','test']}! prod: ! ! { $sudoers = 'ops'}! default: ! { $sudoers = 'ops'}! }! } else {! # clever hack to grant tmp access to Jim! $sudoers = ['dev' ,'ops','jim']! } !
5. https://www.flickr.com/photos/sicolan/9713040984 What you’ve actually done. (good luck when you need to update it in 2 months)
6. https://www.flickr.com/photos/luckywhitegirl/2636803289/ HIERA TO THE RESCUE Let’s untangle that mess!
7. DEFINE YOUR HIERARCHY --- :backends: - yaml - json :hierarchy: - common - 'nodes/%{hostname}' - '%{environment}'
8. POPULATE YOUR FILES --- dev.yaml sudoers: - dev - ops - test ! --- test.yaml sudoers: - ops - test ! --- prod.yaml sudoers: - ops ! --- nodes/snowflake.yaml sudoers: - dev - ops - jim
9. HIERA MAGIC! ! $sudoers = hiera(‘sudoers’) ! if $hostname != 'snowflake' {! case $environment {! dev: ! ! { $sudoers = ['dev','ops','test']}! test: ! ! { $sudoers = [ 'ops','test']}! prod: ! ! { $sudoers = 'ops'}! default: ! { $sudoers = 'ops'}! }! } else {! # clever hack to grant tmp access to Jim! $sudoers = ['dev' ,'ops','jim']! } ! = replace that ugly code with a one-liner!
10. SO…WHAT IS IT GOOD FOR? Things that are often repeated across hosts: - User accounts - config options - passwords?!
11. SURE, I’LL COMMIT MY PASSWORDS TO GIT. What’s the worse that can happen?
13. HIERA-GPG
15. BUT… How to let other people edit encrypted files?
16. https://www.flickr.com/photos/kalexanderson/ Start from scratch every time? (and have people keep a collection of clear- text files out of version control)
17. SHARE MY PRIVATE KEY WITH EVERYONE? https://www.flickr.com/photos/manu_le_manu/ Share my private key with everyone? (assuming no one ever leaves on bad terms, get their laptop stolen, chmod 777 their key, sell their computer without cleaning the HDD, etc)
18. HIERA-EYAML ! - Address the shortcomings of hiera-gpg - Allow fine grained encryption for the fields that require it - let people edit your files and add new values with only your public key
19. ---! plain-property: You can see me! ! encrypted-property: >! ENC[PKCS7,Y22exl+OvjDe+drmik2XEeD3VQtl1uZJXFFF2NnrMXDWx0csyqLB/2NOWefv! NBTZfOlPvMlAesyr4bUY4I5XeVbVk38XKxeriH69EFAD4CahIZlC8lkE/uDh! jJGQfh052eonkungHIcuGKY/5sEbbZl/qufjAtp/ufor15VBJtsXt17tXP4y! l5ZP119Fwq8xiREGOL0lVvFYJz2hZc1ppPCNG5lwuLnTekXN/OazNYpf4CMd! /HjZFXwcXRtTlzewJLc+/gox2IfByQRhsI/AgogRfYQKocZgFb/DOZoXR7wm! IZGeunzwhqfmEtGiqpvJJQ5wVRdzJVpTnANBA5qxeA==]! ! ---! plain-property: You can see me! ! encrypted-property: >! DEC(1)::PKCS7[You can't see me]!! ! Only encrypts the variables you need to hide And decrypt them when you actually need them eyaml encrypt -s ‘string’ to encrypt a string, -f for a file, etc.
 More on their github page
20. OK, SO NOW IT'S SECURE If you remember one of the first slides, there is still a lot of information that gets typed several times. My example has only 3 groups, but what if you have 25 across 600 hosts? Do you type them all every time? Or do you merge them?
21. BACK TO OUR EXAMPLE --- common.yaml sudoers: - ops ! --- dev.yaml sudoers: - dev - test ! --- test.yaml sudoers: - test ! --- nodes/snowflake.yaml sudoers: - dev - jim We didn’t put prod because it should be matched by common.
22. ! $sudoers = hiera(‘sudoers’) A bit of hiera magic and we should be having our groups all set up on every host.
23. EVERYTHING IS NOW OPS That was my plan all along. ! Totally.
24. SURELY IT HANDLES HASHES BETTER? Ruby, yaml, and hashes. ! A match made in heaven.
25. --- common.yaml! config:! sudoers: ! - ops! ! --- dev.yaml! config:! sudoers:! - dev! - test! gateway: 192.168.0.12! ! --- test.yaml! config:! sudoers:! - test! gateway: 192.168.1.12
26. ! config = hiera(config)! sudoers = config['sudoers']!
27. EVERYTHING IS NOW OPS (again) Ok, so what are we doing wrong? For starters, it’s called a hierarchy for a reason: First match (the file on top) takes all! ! The other bad thing is it won’t merge by default :(
28. WHAT’S THE POINT OF COMMON VARIABLES THEN? Common variables act as a catch-all when using the default implementation. You put them last so they apply if nothing else does! ! How do I apply variables across different files though?
29. –Not Yoda “Use the merge, Luke.”
30. hiera_array to merge arrays: ! sudoers = hiera_array('sudoers')! Dev: sudoers == ['dev', 'test','ops'] hiera_array will parse all files that match your host following the hierarchy. A dev host will match both dev.yaml and common.yaml, and flatten the text and array values in a single array
31. hiera_hash to merge…. hashes: --- dev.yaml! config:! sudoers:! - dev! - test! gateway: 192.168.0.12! --- common.yaml! config:! sudoers: ! - ops! Similarly hiera_hash will parse all the files that match and merge the multiple hashes into a single one.
32. config = hiera_hash('config') Dev: config == { gateway => "192.168.1.12" , sudoers => ['dev', 'test'] } With a twist…if there are common keys, it will only take the highest key/value pair and ignore the rest.
34. ---! :backends:! - yaml! - json! :hierarchy:! - common ! - 'nodes/%{hostname}'! - '%{environment}'! :merge_behavior:! - deeper You need to change the merge behaviour in your hiera configuration. ! This require the “deep_merge” gem to be present on the system.
35. --- dev.yaml! config:! sudoers:! - dev! - test! gateway: 192.168.0.12! --- common.yaml! config:! sudoers: ! ! - ops
36. ! config = hiera_hash('config')! Dev: ! config == {! ! gateway => '192.168.1.12', ! sudoers => ['dev','test','ops'] ! } Looking much better already! Deep merge works as follow:
37. --- dev.yaml! config:! ! gateway: 192.168.0.12! ! --- common.yaml ! config:! sudoers: ops ! gateway => '192.168.1.12', ! sudoers => 'ops'! } For each key, if only is present, it goes in the hash
38. --- dev.yaml! config:! sudoers: dev --- common.yaml ! config:! sudoers: ops sudoers == 'dev' If there’s more than one string/number/boolean, the first one goes in the hash.
39. --- dev.yaml! config:! sudoers:! - dev! - test --- common.yaml ! config:! sudoers: ! ! - ops sudoers == [dev, test, ops] if an array exist in more than a hash, they’re flattened and the result goes in the final hash. Additionally, hashes are merged recursively but I was too lazy to write up an exemple.
40. THANK YOU FOR YOUR TIME! @frconil blog.conil.id.au