Better Together: Managing Windows with Puppet, PowerShell and DSC

PowerShell is a critical skillset if you want to manage Windows systems no matter what solutions or tools that you use. This session will explain the platform approach we are taking with PowerShell and show you how to use it to manage systems. There will be a particular focus on writing Desired State Configuration resource providers so that you can light up value-add Configuration Management solutions like Puppet.

Transcript

1. View Slide

2. Better Together:
   Managing Windows with
   Puppet, PowerShell and DSC
   

   View Slide

3. Introductions
   Bruce Payette
   Principal Software Engineer
   Microsoft
   @BrucePayette
   Ethan Brown
   Senior Software Engineer
   Puppet Labs
   Iristyle
   @ethanjbrown
   

   View Slide

4. v
   • Overview of PowerShell and DSC
   • Puppet + DSC
   • All The Way Live!
   • Feedback?
   AGENDA
   

   View Slide

5. PowerShell and DSC
   

   View Slide

6. v
   Prelude: A Touch of PowerShell
   • Modern Interactive object-oriented shell
   – Inspired by ksh, syntax from C#, blocks from Ruby, cmd syntax from
   DCL
   – Being a shell makes the syntax a bit weird
   • All the usual stuff: regex, first-class functions, array and hashtable literals,
   modules, classes (as of PowerShell V5)
   • Underlying type system is .NET
   – Can access any .NET API directly
   – Can inline code written C#, VB, F#, IronPython, IronRuby etc.
   • Built-in “remoting”
   • Type-preserving remote execution over HTTP using MS-PSRP
   • Hostable and embeddable (like TCL)
   – Can be loaded into multiple host (including Visual Studio thru PoshTools)
   

   View Slide

7. v
   1. #requires -version 5
   2. using namespace System.Net
   3. using namespace System.Text
   4. function Start-HTTPListener ( [int] $Port = 8888, $Url = "“ ) # supports named arguments e.g. Start-HttpListener –port 8080
   5. {
   6. [HttpListener] $listener = [HttpListener]::new()
   7. $prefix = "http://*:$Port/$Url" # variable expansion in strings
   8. $listener.Prefixes.Add($prefix)
   9. $listener.AuthenticationSchemes = [AuthenticationSchemes]::IntegratedWindowsAuthentication # static member reference
   10. $listener.Start()
   11. while ($true)
   12. {
   13. $statusCode = 200
   14. $context = $listener.GetContext() # block waiting for a connection
   15. $request, $identity = $context.Request, $context.User.Identity # supports multiple assignment
   16. $request | Format-List * | Out-String | Write-Verbose # display request object as a list
   17. $command = $request.QueryString.Item("command"); $Format = $request.QueryString.Item("format") # ; separates statements
   18. $sb = [ScriptBlock]::Create($command)
   19. $commandOutput = switch (& $sb) {# expression oriented; statements are lvalues
   20. TEXT { $commandOutput | Out-String ; break }
   21. JSON { $commandOutput | ConvertTo-JSON; break }
   22. XML { $commandOutput | ConvertTo-XML -As String; break }
   23. default { "Invalid output format selected, valid choices are TEXT, JSON and XML"; $statusCode = 501; break }
   24. }
   25. $response, $response.StatusCode = $context.Response, $statusCode
   26. $buffer = [Encoding]::UTF8.GetBytes($commandOutput)
   27. $response.ContentLength64 = $buffer.Length
   28. $response.OutputStream.Write($buffer, 0, $buffer.Length)
   29. $response.OutputStream.Close()
   30. }
   31. }
   PowerShell Example: HttpListener
   

   View Slide

8. v
   • We take it as given that Configuration Management is a Good
   Thing.
   • BUT Unix/Linux management idioms don't work well on Windows
   – This is why we built PowerShell in the first place
   For Configuration Management SOLUTIONS to work well on
   Windows, we needed to change the Windows PLATFORM to work
   well with Configuration Management
   • Thus DSC exposes a set of language-independent API’s that
   management solutions can use to configure Windows
   • Three Main Pieces to DSC
   1. Local Configuration Manager Service
   2. Source Language
   3. Resource Providers
   Why create DSC?
   

   View Slide

9. • The LCM is a built-in component on Windows supporting a declarative
   configuration management model
   – Runs on every node
   • Open Standards based (WSMAN, REST/ODATA, DMTF MOF for schema
   and instance data)
   • The LCM receives "configuration catalogs" expressed in MOF
   describing the desired final state of the system
   – Push (RPC-like invocation)
   – Pull (REST call to a pull server/config master)
   • The LCM consistency engine then processes the configuration catalog
   delegating the enactment of system changes to "providers"
   What is DSC Part 1
   The Local Configuration Manager Service
   

   View Slide

10. v
    Example DSC MOF for a Node
    1. instance of MSFT_FileDirectoryConfiguration as $MSFT_FileDirectoryConfiguration1ref
    2. {
    3. ResourceID = "[File]f1";
    4. Contents = "A file from brucepaypro3";
    5. DestinationPath = "c:\\temp\\zork.txt";
    6. ModuleName = "PSDesiredStateConfiguration";
    7. SourceInfo = "C:\\Users\\brucepay\\documents\\dsc\\server.ps1::6::9::File";
    8. ModuleVersion = "1.0";
    9. ConfigurationName = "Main";
    10.};
    11.
    12.instance of OMI_ConfigurationDocument
    13.{
    14. Version="2.0.0";
    15. MinimumCompatibleVersion = "1.0.0";
    16. CompatibleVersionAdditionalProperties= {"Omi_BaseResource:ConfigurationName"};
    17. Author="brucepay";
    18. GenerationDate="09/10/2015 17:06:23";
    19. GenerationHost="BRUCEPAYPRO3";
    20. Name="Main";
    21.};
    

    View Slide

11. • We added a set of extensions to the PowerShell language to
    express configuration in a (Windows) natural way
    – Statically typed, data declaration language
    – Deeply integrating into PowerShell allows for intellisense + parse-
    time validation of configurations
    • Configurations are reusable, composable, parameterized
    – Essentially equivalent to functions in PowerShell
    • We use a convention for separating configuration “templates” from
    bulk parameter data
    – called ConfigurationData expressed as JSON (or PowerShell
    hashtables)
    What is DSC Part 2
    The Source Language
    

    View Slide

12. v
    Example PSDSC Configuration Script
    1. configuration Main
    2. {
    3. param ($NodeList, $targetDir)
    4.
    5. node ($NodeList)
    6. {
    7. File f1
    8. {
    9. Contents = "A file from $(hostname) generated at $(Get-Date)"
    10. DestinationPath = "$targetDir\file1.txt"
    11. }
    12. File f2
    13. {
    14. Contents = "My node name is $($node.Name)"
    15. DestinationPath = Join-Path $targetDir file2.txt
    16. }
    17. }
    18.}
    19.
    20.# generate the configuration MOF for each machine
    21.Main -TargetDir "c:\temp" -NodeList '169.254.48.23', '169.254.48.24', '169.254.48.25'
    22.
    23.# configure all 3 machines concurrently, display status, wait until done
    24.Start-DscConfiguration -Verbose -Wait
    

    View Slide

13. • Interface for a resource is defined by schema
    – DMTF MOF class in V1
    – PowerShell Classes in V2
    • All resources are Open Source on GitHub
    – Providers on Windows are written in C/C++ or PowerShell
    – Providers on Linux are (currently) written in Python
    • Nuget-based PowerShell Gallery to simplify installation of
    providers
    – Install manually or auto-installed via the “pull protocol”
    What is DSC Part 3
    Implementing Resource Providers
    

    View Slide

14. v
    Example Resource Implementation
    1. #requires -version 5
    2. enum Ensure
    3. {
    4. Absent
    5. Present
    6. }
    7.
    8. [DscResource()]
    9. class MyResource
    10.{
    11. [DscProperty(Key)] [string]$P1 # must be set to a unique value for each instance
    12. [DscProperty(Mandatory)] [Ensure] $P2
    13. [DscProperty(NotConfigurable)] [Nullable[datetime]] $P3
    14. [DscProperty()] [ValidateSet("val1", "val2")] [string] $P4
    15.
    16. [void] Set() { }
    17. [bool] Test() { return $true }
    18. [MyResource] Get() { return $this }
    19.}
    

    View Slide

15. DSC End-to-End
    Configuration
    Staging Area
    (Contains catalog
    data)
    3rd party
    languages and
    tools
    Authoring Phase
    (May include imperative as
    well as declarative code)
    Staging Phase
    - Fully declarative
    configuration representation
    using DMTF standard MOF
    instances
    - Configuration is calculated for
    all nodes
    PowerShell
    Parser and
    Dispatcher
    Resources
    When authoring in PowerShell:
    • Declarative syntax extensions
    • Schema-driven Intellisense
    • Schema validation (early-
    binding)
    Local
    Configuration
    Store
    “Make it So” Phase
    (Declarative configuration is reified
    through imperative providers.)
    Resources implement
    changes:
    • Monotonic
    • Imperative
    • Idempotent
    

    View Slide

16. v
    Using DSC in Other Solutions
    • Possible Approaches:
    – Generating MOF from other source languages (e.g. Puppet) then
    physically placing it on a machine
    – Implementing the PowerShell Pull Protocol
    • Very simple REST-based protocol
    • Documented under the Microsoft’s Open Protocols Initiative as
    MS-PSDSC
    – Using the PowerShell API to invoke configuration scripts
    • Generate ConfigurationData from a CMDB
    – ***** Direct invocation of providers:
    Invoke-DscResource -Name File -Method Get -Property @{
    DestinationPath = "c:\temp\foo.txt“ }
    • The Big Challenge:
    – Honoring the semantics of the higher layer
    

    View Slide

17. v
    • Built in to Windows since Windows Server 2012 R2
    – No install needed
    • Installable on older versions of Windows through the
    Windows Management Framework
    – WMF 4.0
    – WMF 5.0 (Production Preview)
    • DSC on Linux
    • DSC is available as a resource extension in Azure
    • Resource development on GitHub
    • Resource sharing on Nuget-based PowerShell Gallery
    Where is DSC Available?
    

    View Slide

18. Puppet + DSC
    

    View Slide

19. v
    • Marc Sutter, community originated
    • Released 0.1.0 in Sept 2014
    • Puppet adopted April 2015
    • Puppet unsupported release 0.8.0 in Sept 2015
    – Requires WMF5
    – Speedier Invoke-DscResource vs XXX-DscConfiguration
    – Windows on DSC only
    • Fully supported Q4
    Changelog
    

    View Slide

20. View Slide

21. Batteries Included
    180+
    Resources
    

    View Slide

22. MOF -> PUPPET TYPE
    

    View Slide

23. v
    • dsc_ prefixes – i.e. dsc_ensure
    • Include validation from MOF schema primitive types
    – Can abort prior to calling DSC / PS
    • Special handling for MSFT_KeyValuePair, MSFT_Credentials
    • Understand EmbeddedInstance schemas
    • Propagate meaningful DSC errors to Puppet
    • Support on-demand reboots
    • Not supporting WaitForOne, WaitForSome, WaitForAll
    • Don’t need Internet access to PowerShell Gallery
    Puppet Types
    

    View Slide

24. v
    • WMF5 in-box resources resolved by PS ModuleName
    • Symlink vendored modules
    • Vendored matched on ModuleName / ModuleVersion
    – Ensures compatibility with codegen’d Puppet types
    Finding DSC Resources
    

    View Slide

25. Toward Supported – UNIT TestS
    

    View Slide

26. Toward Supported - CI
    

    View Slide

27. Toward Supported - Docs
    

    View Slide

28. Technet Docs
    

    View Slide

29. v
    • Support integration with DSC
    • LCM does heavy-lifting of resource application
    • xPrefix style resources included from “community”
    Good fences…
    

    View Slide

30. ALL THE WAY LIVE!
    

    View Slide

31. In The Year 3000
    • Property Reporting / --noop *
    • MI APIs to improve perf
    • Generic resource support
    – Codegen tooling
    – Dsc_ wrapper for custom /
    class-based resources
    • Puppet resource support
    • DSC OS compat metadata **
    * Needs Get-TargetResource
    ** Needs MS implementation
    

    View Slide

32. v
    • WMF 5.0 Download - http://bit.ly/1idWxvp
    • Puppet DSC module source - http://bit.ly/1Qm5ddR
    • Vendored PowerShell DSC_resources - http://bit.ly/1jbauua
    • Technet DSC Docs - http://bit.ly/1HoS4yp
    • James Pogran “Learning PowerShell DSC” - http://bit.ly/learndsc
    STILL HUNGRY?
    

    View Slide

33. Feedback?
    

    View Slide

34. Thanks!
    

    View Slide