$30 off During Our Annual Pro Sale. View Details »

Better Together: Managing Windows with Puppet, PowerShell and DSC

Better Together: Managing Windows with Puppet, PowerShell and DSC

PowerShell is a critical skillset if you want to manage Windows systems no matter what solutions or tools that you use. This session will explain the platform approach we are taking with PowerShell and show you how to use it to manage systems. There will be a particular focus on writing Desired State Configuration resource providers so that you can light up value-add Configuration Management solutions like Puppet.

Ethan J. Brown

October 09, 2015
Tweet

Other Decks in Technology

Transcript

  1. View Slide

  2. Better Together:
    Managing Windows with
    Puppet, PowerShell and DSC

    View Slide

  3. Introductions
    Bruce Payette
    Principal Software Engineer
    Microsoft
    @BrucePayette
    Ethan Brown
    Senior Software Engineer
    Puppet Labs
    Iristyle
    @ethanjbrown

    View Slide

  4. v
    • Overview of PowerShell and DSC
    • Puppet + DSC
    • All The Way Live!
    • Feedback?
    AGENDA

    View Slide

  5. PowerShell and DSC

    View Slide

  6. v
    Prelude: A Touch of PowerShell
    • Modern Interactive object-oriented shell
    – Inspired by ksh, syntax from C#, blocks from Ruby, cmd syntax from
    DCL
    – Being a shell makes the syntax a bit weird
    • All the usual stuff: regex, first-class functions, array and hashtable literals,
    modules, classes (as of PowerShell V5)
    • Underlying type system is .NET
    – Can access any .NET API directly
    – Can inline code written C#, VB, F#, IronPython, IronRuby etc.
    • Built-in “remoting”
    • Type-preserving remote execution over HTTP using MS-PSRP
    • Hostable and embeddable (like TCL)
    – Can be loaded into multiple host (including Visual Studio thru PoshTools)

    View Slide

  7. v
    1. #requires -version 5
    2. using namespace System.Net
    3. using namespace System.Text
    4. function Start-HTTPListener ( [int] $Port = 8888, $Url = "“ ) # supports named arguments e.g. Start-HttpListener –port 8080
    5. {
    6. [HttpListener] $listener = [HttpListener]::new()
    7. $prefix = "http://*:$Port/$Url" # variable expansion in strings
    8. $listener.Prefixes.Add($prefix)
    9. $listener.AuthenticationSchemes = [AuthenticationSchemes]::IntegratedWindowsAuthentication # static member reference
    10. $listener.Start()
    11. while ($true)
    12. {
    13. $statusCode = 200
    14. $context = $listener.GetContext() # block waiting for a connection
    15. $request, $identity = $context.Request, $context.User.Identity # supports multiple assignment
    16. $request | Format-List * | Out-String | Write-Verbose # display request object as a list
    17. $command = $request.QueryString.Item("command"); $Format = $request.QueryString.Item("format") # ; separates statements
    18. $sb = [ScriptBlock]::Create($command)
    19. $commandOutput = switch (& $sb) {# expression oriented; statements are lvalues
    20. TEXT { $commandOutput | Out-String ; break }
    21. JSON { $commandOutput | ConvertTo-JSON; break }
    22. XML { $commandOutput | ConvertTo-XML -As String; break }
    23. default { "Invalid output format selected, valid choices are TEXT, JSON and XML"; $statusCode = 501; break }
    24. }
    25. $response, $response.StatusCode = $context.Response, $statusCode
    26. $buffer = [Encoding]::UTF8.GetBytes($commandOutput)
    27. $response.ContentLength64 = $buffer.Length
    28. $response.OutputStream.Write($buffer, 0, $buffer.Length)
    29. $response.OutputStream.Close()
    30. }
    31. }
    PowerShell Example: HttpListener

    View Slide

  8. v
    • We take it as given that Configuration Management is a Good
    Thing.
    • BUT Unix/Linux management idioms don't work well on Windows
    – This is why we built PowerShell in the first place
    For Configuration Management SOLUTIONS to work well on
    Windows, we needed to change the Windows PLATFORM to work
    well with Configuration Management
    • Thus DSC exposes a set of language-independent API’s that
    management solutions can use to configure Windows
    • Three Main Pieces to DSC
    1. Local Configuration Manager Service
    2. Source Language
    3. Resource Providers
    Why create DSC?

    View Slide

  9. • The LCM is a built-in component on Windows supporting a declarative
    configuration management model
    – Runs on every node
    • Open Standards based (WSMAN, REST/ODATA, DMTF MOF for schema
    and instance data)
    • The LCM receives "configuration catalogs" expressed in MOF
    describing the desired final state of the system
    – Push (RPC-like invocation)
    – Pull (REST call to a pull server/config master)
    • The LCM consistency engine then processes the configuration catalog
    delegating the enactment of system changes to "providers"
    What is DSC Part 1
    The Local Configuration Manager Service

    View Slide

  10. v
    Example DSC MOF for a Node
    1. instance of MSFT_FileDirectoryConfiguration as $MSFT_FileDirectoryConfiguration1ref
    2. {
    3. ResourceID = "[File]f1";
    4. Contents = "A file from brucepaypro3";
    5. DestinationPath = "c:\\temp\\zork.txt";
    6. ModuleName = "PSDesiredStateConfiguration";
    7. SourceInfo = "C:\\Users\\brucepay\\documents\\dsc\\server.ps1::6::9::File";
    8. ModuleVersion = "1.0";
    9. ConfigurationName = "Main";
    10.};
    11.
    12.instance of OMI_ConfigurationDocument
    13.{
    14. Version="2.0.0";
    15. MinimumCompatibleVersion = "1.0.0";
    16. CompatibleVersionAdditionalProperties= {"Omi_BaseResource:ConfigurationName"};
    17. Author="brucepay";
    18. GenerationDate="09/10/2015 17:06:23";
    19. GenerationHost="BRUCEPAYPRO3";
    20. Name="Main";
    21.};

    View Slide

  11. • We added a set of extensions to the PowerShell language to
    express configuration in a (Windows) natural way
    – Statically typed, data declaration language
    – Deeply integrating into PowerShell allows for intellisense + parse-
    time validation of configurations
    • Configurations are reusable, composable, parameterized
    – Essentially equivalent to functions in PowerShell
    • We use a convention for separating configuration “templates” from
    bulk parameter data
    – called ConfigurationData expressed as JSON (or PowerShell
    hashtables)
    What is DSC Part 2
    The Source Language

    View Slide

  12. v
    Example PSDSC Configuration Script
    1. configuration Main
    2. {
    3. param ($NodeList, $targetDir)
    4.
    5. node ($NodeList)
    6. {
    7. File f1
    8. {
    9. Contents = "A file from $(hostname) generated at $(Get-Date)"
    10. DestinationPath = "$targetDir\file1.txt"
    11. }
    12. File f2
    13. {
    14. Contents = "My node name is $($node.Name)"
    15. DestinationPath = Join-Path $targetDir file2.txt
    16. }
    17. }
    18.}
    19.
    20.# generate the configuration MOF for each machine
    21.Main -TargetDir "c:\temp" -NodeList '169.254.48.23', '169.254.48.24', '169.254.48.25'
    22.
    23.# configure all 3 machines concurrently, display status, wait until done
    24.Start-DscConfiguration -Verbose -Wait

    View Slide

  13. • Interface for a resource is defined by schema
    – DMTF MOF class in V1
    – PowerShell Classes in V2
    • All resources are Open Source on GitHub
    – Providers on Windows are written in C/C++ or PowerShell
    – Providers on Linux are (currently) written in Python
    • Nuget-based PowerShell Gallery to simplify installation of
    providers
    – Install manually or auto-installed via the “pull protocol”
    What is DSC Part 3
    Implementing Resource Providers

    View Slide

  14. v
    Example Resource Implementation
    1. #requires -version 5
    2. enum Ensure
    3. {
    4. Absent
    5. Present
    6. }
    7.
    8. [DscResource()]
    9. class MyResource
    10.{
    11. [DscProperty(Key)] [string]$P1 # must be set to a unique value for each instance
    12. [DscProperty(Mandatory)] [Ensure] $P2
    13. [DscProperty(NotConfigurable)] [Nullable[datetime]] $P3
    14. [DscProperty()] [ValidateSet("val1", "val2")] [string] $P4
    15.
    16. [void] Set() { <# Sets the desired state of the resource. #> }
    17. [bool] Test() { <# Tests if the resource is in the desired state. #> return $true }
    18. [MyResource] Get() { <# Gets the resource's current state. #> return $this }
    19.}

    View Slide

  15. DSC End-to-End
    Configuration
    Staging Area
    (Contains catalog
    data)
    3rd party
    languages and
    tools
    Authoring Phase
    (May include imperative as
    well as declarative code)
    Staging Phase
    - Fully declarative
    configuration representation
    using DMTF standard MOF
    instances
    - Configuration is calculated for
    all nodes
    PowerShell
    Parser and
    Dispatcher
    Resources
    When authoring in PowerShell:
    • Declarative syntax extensions
    • Schema-driven Intellisense
    • Schema validation (early-
    binding)
    Local
    Configuration
    Store
    “Make it So” Phase
    (Declarative configuration is reified
    through imperative providers.)
    Resources implement
    changes:
    • Monotonic
    • Imperative
    • Idempotent

    View Slide

  16. v
    Using DSC in Other Solutions
    • Possible Approaches:
    – Generating MOF from other source languages (e.g. Puppet) then
    physically placing it on a machine
    – Implementing the PowerShell Pull Protocol
    • Very simple REST-based protocol
    • Documented under the Microsoft’s Open Protocols Initiative as
    MS-PSDSC
    – Using the PowerShell API to invoke configuration scripts
    • Generate ConfigurationData from a CMDB
    – ***** Direct invocation of providers:
    Invoke-DscResource -Name File -Method Get -Property @{
    DestinationPath = "c:\temp\foo.txt“ }
    • The Big Challenge:
    – Honoring the semantics of the higher layer

    View Slide

  17. v
    • Built in to Windows since Windows Server 2012 R2
    – No install needed
    • Installable on older versions of Windows through the
    Windows Management Framework
    – WMF 4.0
    – WMF 5.0 (Production Preview)
    • DSC on Linux
    • DSC is available as a resource extension in Azure
    • Resource development on GitHub
    • Resource sharing on Nuget-based PowerShell Gallery
    Where is DSC Available?

    View Slide

  18. Puppet + DSC

    View Slide

  19. v
    • Marc Sutter, community originated
    • Released 0.1.0 in Sept 2014
    • Puppet adopted April 2015
    • Puppet unsupported release 0.8.0 in Sept 2015
    – Requires WMF5
    – Speedier Invoke-DscResource vs XXX-DscConfiguration
    – Windows on DSC only
    • Fully supported Q4
    Changelog

    View Slide

  20. View Slide

  21. Batteries Included
    180+
    Resources

    View Slide

  22. MOF -> PUPPET TYPE

    View Slide

  23. v
    • dsc_ prefixes – i.e. dsc_ensure
    • Include validation from MOF schema primitive types
    – Can abort prior to calling DSC / PS
    • Special handling for MSFT_KeyValuePair, MSFT_Credentials
    • Understand EmbeddedInstance schemas
    • Propagate meaningful DSC errors to Puppet
    • Support on-demand reboots
    • Not supporting WaitForOne, WaitForSome, WaitForAll
    • Don’t need Internet access to PowerShell Gallery
    Puppet Types

    View Slide

  24. v
    • WMF5 in-box resources resolved by PS ModuleName
    • Symlink vendored modules
    • Vendored matched on ModuleName / ModuleVersion
    – Ensures compatibility with codegen’d Puppet types
    Finding DSC Resources

    View Slide

  25. Toward Supported – UNIT TestS

    View Slide

  26. Toward Supported - CI

    View Slide

  27. Toward Supported - Docs

    View Slide

  28. Technet Docs

    View Slide

  29. v
    • Support integration with DSC
    • LCM does heavy-lifting of resource application
    • xPrefix style resources included from “community”
    Good fences…

    View Slide

  30. ALL THE WAY LIVE!

    View Slide

  31. In The Year 3000
    • Property Reporting / --noop *
    • MI APIs to improve perf
    • Generic resource support
    – Codegen tooling
    – Dsc_ wrapper for custom /
    class-based resources
    • Puppet resource support
    • DSC OS compat metadata **
    * Needs Get-TargetResource
    ** Needs MS implementation

    View Slide

  32. v
    • WMF 5.0 Download - http://bit.ly/1idWxvp
    • Puppet DSC module source - http://bit.ly/1Qm5ddR
    • Vendored PowerShell DSC_resources - http://bit.ly/1jbauua
    • Technet DSC Docs - http://bit.ly/1HoS4yp
    • James Pogran “Learning PowerShell DSC” - http://bit.ly/learndsc
    STILL HUNGRY?

    View Slide

  33. Feedback?

    View Slide

  34. Thanks!

    View Slide