in "Once in a Lifetime" -- a deceptively simple song about the loss of identity -- You may ask yourself, HOW DID I GET HERE? So I want to start off by establishing how WE got here, to Puppet 4.
3.75 4.375 5 Sep 2009 Jul 2010 June 2011 Sep 2012 Apr 2015 Jan 2016 0.25 2.60 2.70 3.00 4.00 4.32 This is my "Give Edward Tufte an Aneurism" slide. It's kind of deliberately terrible data visualization. The X axis is The Linear Flow of Time. The Y axis is the amount of Puppet. Time, for us, starts in 2009, when Puppet sort of emerged from the pre-alpha stage and began to get used in large-scale production. This was about version 0.25 - any Oh-Dot-Twenty-Five users besides me out here? Ugh fistbump. Between 2009 and 2010, Luke decided that the Pre-1.0 number was actually pretty inaccurate, and the next "major" release of puppet became, not 0.26. But 2.6. We suddenly got TEN TIMES MORE PUPPET. As you can see from the graph here. Then in the 2.6 to 2.7 time frame, features were added, but we weren't really "semantically versioned", in the sense that breaking changes would never creep in. There actually was a ton of change in here, despite the shallow slope in the line, and by the time 3.0 rolled around in Sep 2012, we HAD to hew closely to the semantic versioning spec: X dot Y dot Z, with bug fixes only in Z, features in Y, and breaking backwards incompatible changes in X. That started with 3, and we released 3.0 through 3.8 as feature releases with pretty tight adherence and a ton of awesome innovation: directory environments, the future parser hidden behind an opt-in configuration setting, performance boosts, and more. Last year we released 4.0 having changes across a broad swath of areas and-- because it's a semver major -- a few backwards-incompatible changes you need to watch out for. It shipped in the commercial product PE2015.2 in July and we've been tweaking and refining since then,
of changes - LANGUAGE is a big one - biggest change to the language since... well since, ever, probably. PACKAGING -- Batteries-included All-in-One NETWORKING -- agent/master communication has changed for the better. MISC -- catch-all for a bunch of changes that don't fit.
photos. In addition to the "frustrated person using computer messing their hair up", I love the "multicultural handshake" photo. But let's talk about the agent-server communication in puppet, which is a sort of cross-cultural handshake.
that describes the agent-master interaction. I won't go through the whole thing right now. The important point here is that each of the requests has the environment prefixed onto it, even requests that make no sense to have an environment, like the certificate activity. This caused some problems as we started to move the network stack out of the Ruby master running under Rack (usually apache+passenger) into the new Puppet Server stack that I talked about this morning. Not to get too philisophical, but some things are just unknowable, like a brand new agent knowing what environment to request, so it's nonsense to hard code that into the request. This was interfering with the ability to do some seriously cool stuff -- like point requests for a given environment at a different host, transparently -- so we wanted to change it. Now, the environment is a query parameter, the requests are their own thing, and they're all versioned with a '/v3' element in the path and namespaced under '/puppet' so you can have a puppetmaster and say a puppetdb on the same port and the puppetserver will know which requests to route to each. But that's a breaking change. So it's in Puppet 4 and Puppet Server 2. We released a puppet server version that transparently rewrites the URLS from this form to their Puppet 4 form, so you can seamlessly upgrade your masters, then upgrade the agents.
package that would put files and config down in the same place for all unix like operating systems (windows did this from day 1) * normalize the locations and purpose of the directories we used to bring the 2007-era paths in line with how puppet is actually used today. so we have a 1 package download, call it Omnibus if you must, that puts some things under /opt/puppetlabs and others under /etc/ puppetlabs - notable changes Code directory aggregates everything that was under "environments", "manifests", "modules" and "hieradata" into one tree - to bring it in line with how r10k works and how revision control works more generally won't be in your PATH - happy to talk this over when i have a beer in hand but we wanted to keep things out of the system path. extensible for things like puppetdb and puppet server without having to go through contortions. SAME AGENT between open source and PE, so if you are using OSP today and want to start paying us money -- which I highly advise you to do -- you don't have to reinstall the agents. There's an awesome upgrade module coming out next week which will seamlessly upgrade your agents.
the transition is awkward. Previous parser had been the same since 2006 with just some incremental additions. New one is expression-based and has a model and a specification, which we lacked before- this is important because it allows multiple implementations that can all behave "correctly" instead of having to try things in the implementation to see what they were supposed to do. like there can be multiple rubies, there can be multiple puppets. But we're in an awkward transition phase right now - so the power is flowing but it looks a little weird. There's some tooling to help out with this: if you run a 3.8 series puppet and use the puppetlabs-catalog-preview module, it'll show you the differences between catalogs for the same node compiled under the 3.x code and 4.x. So what specifically is better?
We know it's bad. It's not your fault. It's puppet's fault. It's better now, thanks to the puppet 4 language. In general, the language has more strong typing - so things that were sorta kinda working, like comparison against undef, you can eliminate a lot of this kind of hackery - and things that were slippery are a lot more solid - the biggest example is file modes, where something that looks octal needs to be octal.
- Justin Lambert at EvenUp - solving the same problem in Puppet 4 - you get a real "type" system which can tell whether it's getting passed a value with the correct data type *inherently*, without having to do a bunch of archaeology. So your class parameters become much easier to validate and look like the actual API for your module. It's from Justin Lambert at Evenup, check out this repo for some awesome real-world examples of puppet 4 ready code. The voxpupuli - community maintained module team - are working through the same transition with the modules they curate.
or method call that doesn't have a name. it's inline. the content of a 'foreach' loop -- the stuff *inside* the control flow that tells it what data to take and when to stop -- is a lambda. puppet now has the ability to use lambdas in a pretty wide variety of situations, but the most useful is to iterate over a list of things. like a foreach loop. take the list, on line 1. call 'each' on it in line 4, and use that TYPE system we just talked about, to assign a loop local variable... then make a file resource FOR EACH of the things in the list. pretty awesome, eh? I'm not going to show a before/after comparison here because it's too awful. but this is cool and it can be used in all kinds of situations that had us scratching our heads or calling out to Ruby functions before. this example by the way is from the docs.puppetlabs.com section on Iteration. check it out, there's a ton more you can do.
catch-all way to do "something" at catalog compilation time. You can do pretty much whatever you want in a function - call out to an external web service to pull data in ( check erik dalén's pupetdbquery for an example ) but their internals were a bit wonky because they were loaded directly into the puppet master's runtime. So now you can use a cleaner API that doesn't have that side-effect - which means you can have different versions of the function in different environments and they won't clobber each other. So that's nice. Some details, there's an optional "dispatch" method that allows you to do type checking and assignment - this says to send the arguments as strings a and b to a method 'versioncmp' inside the function definition. this function just lets you get at the same version comparison routine that puppet's package system uses internally.
for a minute - William James is credited with this story about cosmology - "your theories about the solar system are nonsense, the earth rests on the back of a giant turtle." "what does the turtle rest on?" "it's turtles all the way down" hiera is puppet's system for separating your data from code - generally considered a Good Thing regardless of what you're doing because it lets you modify values without changing (and therefore having to re-test) behaviour. hiera lets you describe a sequence of lookups in terms of overrides - a hierarchy that tries to bind a value to a variable by looking in really specific places then more and more general places, falling through to global defaults if nothing's found. cfengine hackers may remember an article i wrote in 2003 about singlecopy nirvana - same concept. in chef, this is attribute precedence.
classic hiera /etc/puppetlabs/puppet/hiera.yaml # environment_data_provider = hiera /code/${environment}/hiera.yaml # {"data_provider" = "hiera" } modules/${modulename}/hiera.yaml But there were some problems with hiera so in puppet 4 we have an improved system called Puppet Lookup that layers on top of hiera. * Classic Hiera is for global overrides, when you need to fix something before a change can roll through your environments, or until you're all-in on environment data. It's consulted first and if the keys are found there (in non- merging lookups anyway) the rest of the hieras won't be consulted. * Environment data is the core of Puppet lookup. It's where most of your data should live. In fact one of the main requirements for this project was to make hiera environment-aware so you could have different hierarchies in your environments - this way you can test and promote changes to the hierarchy itself through environments, along with the code they affect. * Module data can only provide default values for a module's own parameters. Puppet lookup enforces that by only using it for keys in a given module's namespace. This was also a primary use-case for the lookup work - RI's data in modules code and the big discussions about it showed there was a lot of demand. The main goal here is to let you replace params.pp with hiera as a more natural extension of global hiera data.
the new "puppet lookup" subcommand which lets you navigate some of these complexities and see how they work. Do I have time for a live demo? Do i have guts? !294 - ls over the code directory !242 - env lookup !247 - module data
want to encourage you to try Puppet 4. Don't be afraid but be cautious - test your code using the future parser flag and catalog preview PE module, gain confidence, trial upgrade with the puppet_agent module and gain MORE confidence, then roll it out across your fleet using Puppet-Server 2.1 , or the upcoming PE release based on it, and move on into the next level of Puppet.
ahpook
tkikuc
yussugi
hotekagi
nokonoko1203
iotcomjpadmin
siva_official
pfn
akiroom
leveragestech
minorun365
livesense
gappy50
shpigford
samanthasiow
ddemaree
chriscoyier
tanoku
garrettdimon
sferik
speakerdeck
gr2m
danielanewman
jonyablonski
panda_program
