Better Together: Managing Windows with Puppet, PowerShell and DSC

PowerShell is a critical skillset if you want to manage Windows systems no matter what solutions or tools that you use. This session will explain the platform approach we are taking with PowerShell and show you how to use it to manage systems. There will be a particular focus on writing Desired State Configuration resource providers so that you can light up value-add Configuration Management solutions like Puppet.

Transcript

2. Better Together: Managing Windows with Puppet, PowerShell and DSC
3. Introductions Bruce Payette Principal Software Engineer Microsoft @BrucePayette Ethan Brown Senior Software Engineer Puppet Labs Iristyle @ethanjbrown
4. v • Overview of PowerShell and DSC • Puppet + DSC • All The Way Live! • Feedback? AGENDA
5. PowerShell and DSC
6. v Prelude: A Touch of PowerShell • Modern Interactive object-oriented shell – Inspired by ksh, syntax from C#, blocks from Ruby, cmd syntax from DCL – Being a shell makes the syntax a bit weird • All the usual stuff: regex, first-class functions, array and hashtable literals, modules, classes (as of PowerShell V5) • Underlying type system is .NET – Can access any .NET API directly – Can inline code written C#, VB, F#, IronPython, IronRuby etc. • Built-in “remoting” • Type-preserving remote execution over HTTP using MS-PSRP • Hostable and embeddable (like TCL) – Can be loaded into multiple host (including Visual Studio thru PoshTools)
7. v 1. #requires -version 5 2. using namespace System.Net 3. using namespace System.Text 4. function Start-HTTPListener ( [int] $Port = 8888, $Url = "“ ) # supports named arguments e.g. Start-HttpListener –port 8080 5. { 6. [HttpListener] $listener = [HttpListener]::new() 7. $prefix = "http://*:$Port/$Url" # variable expansion in strings 8. $listener.Prefixes.Add($prefix) 9. $listener.AuthenticationSchemes = [AuthenticationSchemes]::IntegratedWindowsAuthentication # static member reference 10. $listener.Start() 11. while ($true) 12. { 13. $statusCode = 200 14. $context = $listener.GetContext() # block waiting for a connection 15. $request, $identity = $context.Request, $context.User.Identity # supports multiple assignment 16. $request | Format-List * | Out-String | Write-Verbose # display request object as a list 17. $command = $request.QueryString.Item("command"); $Format = $request.QueryString.Item("format") # ; separates statements 18. $sb = [ScriptBlock]::Create($command) 19. $commandOutput = switch (& $sb) {# expression oriented; statements are lvalues 20. TEXT { $commandOutput | Out-String ; break } 21. JSON { $commandOutput | ConvertTo-JSON; break } 22. XML { $commandOutput | ConvertTo-XML -As String; break } 23. default { "Invalid output format selected, valid choices are TEXT, JSON and XML"; $statusCode = 501; break } 24. } 25. $response, $response.StatusCode = $context.Response, $statusCode 26. $buffer = [Encoding]::UTF8.GetBytes($commandOutput) 27. $response.ContentLength64 = $buffer.Length 28. $response.OutputStream.Write($buffer, 0, $buffer.Length) 29. $response.OutputStream.Close() 30. } 31. } PowerShell Example: HttpListener
8. v • We take it as given that Configuration Management is a Good Thing. • BUT Unix/Linux management idioms don't work well on Windows – This is why we built PowerShell in the first place For Configuration Management SOLUTIONS to work well on Windows, we needed to change the Windows PLATFORM to work well with Configuration Management • Thus DSC exposes a set of language-independent API’s that management solutions can use to configure Windows • Three Main Pieces to DSC 1. Local Configuration Manager Service 2. Source Language 3. Resource Providers Why create DSC?
9. • The LCM is a built-in component on Windows supporting a declarative configuration management model – Runs on every node • Open Standards based (WSMAN, REST/ODATA, DMTF MOF for schema and instance data) • The LCM receives "configuration catalogs" expressed in MOF describing the desired final state of the system – Push (RPC-like invocation) – Pull (REST call to a pull server/config master) • The LCM consistency engine then processes the configuration catalog delegating the enactment of system changes to "providers" What is DSC Part 1 The Local Configuration Manager Service
10. v Example DSC MOF for a Node 1. instance of MSFT_FileDirectoryConfiguration as $MSFT_FileDirectoryConfiguration1ref 2. { 3. ResourceID = "[File]f1"; 4. Contents = "A file from brucepaypro3"; 5. DestinationPath = "c:\\temp\\zork.txt"; 6. ModuleName = "PSDesiredStateConfiguration"; 7. SourceInfo = "C:\\Users\\brucepay\\documents\\dsc\\server.ps1::6::9::File"; 8. ModuleVersion = "1.0"; 9. ConfigurationName = "Main"; 10.}; 11. 12.instance of OMI_ConfigurationDocument 13.{ 14. Version="2.0.0"; 15. MinimumCompatibleVersion = "1.0.0"; 16. CompatibleVersionAdditionalProperties= {"Omi_BaseResource:ConfigurationName"}; 17. Author="brucepay"; 18. GenerationDate="09/10/2015 17:06:23"; 19. GenerationHost="BRUCEPAYPRO3"; 20. Name="Main"; 21.};
11. • We added a set of extensions to the PowerShell language to express configuration in a (Windows) natural way – Statically typed, data declaration language – Deeply integrating into PowerShell allows for intellisense + parse- time validation of configurations • Configurations are reusable, composable, parameterized – Essentially equivalent to functions in PowerShell • We use a convention for separating configuration “templates” from bulk parameter data – called ConfigurationData expressed as JSON (or PowerShell hashtables) What is DSC Part 2 The Source Language
12. v Example PSDSC Configuration Script 1. configuration Main 2. { 3. param ($NodeList, $targetDir) 4. 5. node ($NodeList) 6. { 7. File f1 8. { 9. Contents = "A file from $(hostname) generated at $(Get-Date)" 10. DestinationPath = "$targetDir\file1.txt" 11. } 12. File f2 13. { 14. Contents = "My node name is $($node.Name)" 15. DestinationPath = Join-Path $targetDir file2.txt 16. } 17. } 18.} 19. 20.# generate the configuration MOF for each machine 21.Main -TargetDir "c:\temp" -NodeList '169.254.48.23', '169.254.48.24', '169.254.48.25' 22. 23.# configure all 3 machines concurrently, display status, wait until done 24.Start-DscConfiguration -Verbose -Wait
13. • Interface for a resource is defined by schema – DMTF MOF class in V1 – PowerShell Classes in V2 • All resources are Open Source on GitHub – Providers on Windows are written in C/C++ or PowerShell – Providers on Linux are (currently) written in Python • Nuget-based PowerShell Gallery to simplify installation of providers – Install manually or auto-installed via the “pull protocol” What is DSC Part 3 Implementing Resource Providers
14. v Example Resource Implementation 1. #requires -version 5 2. enum Ensure 3. { 4. Absent 5. Present 6. } 7. 8. [DscResource()] 9. class MyResource 10.{ 11. [DscProperty(Key)] [string]$P1 # must be set to a unique value for each instance 12. [DscProperty(Mandatory)] [Ensure] $P2 13. [DscProperty(NotConfigurable)] [Nullable[datetime]] $P3 14. [DscProperty()] [ValidateSet("val1", "val2")] [string] $P4 15. 16. [void] Set() { <# Sets the desired state of the resource. #> } 17. [bool] Test() { <# Tests if the resource is in the desired state. #> return $true } 18. [MyResource] Get() { <# Gets the resource's current state. #> return $this } 19.}
15. DSC End-to-End Configuration Staging Area (Contains catalog data) 3rd party languages and tools Authoring Phase (May include imperative as well as declarative code) Staging Phase - Fully declarative configuration representation using DMTF standard MOF instances - Configuration is calculated for all nodes PowerShell Parser and Dispatcher Resources When authoring in PowerShell: • Declarative syntax extensions • Schema-driven Intellisense • Schema validation (early- binding) Local Configuration Store “Make it So” Phase (Declarative configuration is reified through imperative providers.) Resources implement changes: • Monotonic • Imperative • Idempotent
16. v Using DSC in Other Solutions • Possible Approaches: – Generating MOF from other source languages (e.g. Puppet) then physically placing it on a machine – Implementing the PowerShell Pull Protocol • Very simple REST-based protocol • Documented under the Microsoft’s Open Protocols Initiative as MS-PSDSC – Using the PowerShell API to invoke configuration scripts • Generate ConfigurationData from a CMDB – ***** Direct invocation of providers: Invoke-DscResource -Name File -Method Get -Property @{ DestinationPath = "c:\temp\foo.txt“ } • The Big Challenge: – Honoring the semantics of the higher layer
17. v • Built in to Windows since Windows Server 2012 R2 – No install needed • Installable on older versions of Windows through the Windows Management Framework – WMF 4.0 – WMF 5.0 (Production Preview) • DSC on Linux • DSC is available as a resource extension in Azure • Resource development on GitHub • Resource sharing on Nuget-based PowerShell Gallery Where is DSC Available?
18. Puppet + DSC
19. v • Marc Sutter, community originated • Released 0.1.0 in Sept 2014 • Puppet adopted April 2015 • Puppet unsupported release 0.8.0 in Sept 2015 – Requires WMF5 – Speedier Invoke-DscResource vs XXX-DscConfiguration – Windows on DSC only • Fully supported Q4 Changelog
21. Batteries Included 180+ Resources
22. MOF -> PUPPET TYPE
23. v • dsc_ prefixes – i.e. dsc_ensure • Include validation from MOF schema primitive types – Can abort prior to calling DSC / PS • Special handling for MSFT_KeyValuePair, MSFT_Credentials • Understand EmbeddedInstance schemas • Propagate meaningful DSC errors to Puppet • Support on-demand reboots • Not supporting WaitForOne, WaitForSome, WaitForAll • Don’t need Internet access to PowerShell Gallery Puppet Types
24. v • WMF5 in-box resources resolved by PS ModuleName • Symlink vendored modules • Vendored matched on ModuleName / ModuleVersion – Ensures compatibility with codegen’d Puppet types Finding DSC Resources
25. Toward Supported – UNIT TestS
26. Toward Supported - CI
27. Toward Supported - Docs
28. Technet Docs
29. v • Support integration with DSC • LCM does heavy-lifting of resource application • xPrefix style resources included from “community” Good fences…
30. ALL THE WAY LIVE!
31. In The Year 3000 • Property Reporting / --noop * • MI APIs to improve perf • Generic resource support – Codegen tooling – Dsc_ wrapper for custom / class-based resources • Puppet resource support • DSC OS compat metadata ** * Needs Get-TargetResource ** Needs MS implementation
32. v • WMF 5.0 Download - http://bit.ly/1idWxvp • Puppet DSC module source - http://bit.ly/1Qm5ddR • Vendored PowerShell DSC_resources - http://bit.ly/1jbauua • Technet DSC Docs - http://bit.ly/1HoS4yp • James Pogran “Learning PowerShell DSC” - http://bit.ly/learndsc STILL HUNGRY?
33. Feedback?
34. Thanks!