VMworld 2015 INF4529 - VMware vSphere Certificate Management for Mere Mortals

Presented at VMworld 2015
Recording at https://youtu.be/SDaL9Rj32BQ
Microsite at http://vmware.com/go/inf4529

SSL certificates have become essential in securing access to our data centers and infrastructure. As VMware vSphere has evolved, so too have the processes, applications, and management of SSL certificates. Gone are the days of simply relying on self-signed certificates and never having to validate, renew, or really manage those certificates. In this session, we will discuss and demonstrate the new VMware Certificate Authority to show how to deploy and manage SSL certificates in vSphere 6. We'll also discuss upgrade considerations and how to evaluate the different deployment models for the VMware Certificate Authority in order to decide which is best for the user's organization.

Transcript

1. VMware vSphere Certificate Management for Mere Mortals
   Ryan Johnson, VMware, Inc
   @tenthirtyam
   Adam Eckerle, VMware, Inc
   @eck79
   vmware.com/go/podcast
   INF4529
   #INF4529
   

   View Slide

2. • This presentation may contain product features that are currently under development.
   • This overview of new technology represents no commitment from VMware to deliver these
   features in any generally available product.
   • Features are subject to change, and must not be included in contracts, purchase orders, or
   sales agreements of any kind.
   • Technical feasibility and market demand will affect final delivery.
   • Pricing and packaging for any new technologies or features discussed or presented have not
   been determined.
   Disclaimer
   2
   

   View Slide

3. View Slide

4. Certificate Lifecycle Management
   VMware vSphere 6.0 Solutions for Complete Certificate Lifecycle Management
   VMware
   Certificate
   Authority
   VMCA
   VMware
   Endpoint
   Certificate Store
   VECS
   Located on:
   Embedded Deployment, and
   Platform Services Controller
   Located on:
   Embedded Deployment, and
   vCenter Management Node
   4
   

   View Slide

5. VMware Certificate Authority (VMCA)
   5
   Dual Operational Modes
   Root CA
   • During installation, VMCA automatically
   creates a root CA certificate.
   • This certificate is capable of issuing
   other certificates.
   • All solutions and endpoint
   certificates are created and trusted
   through to this certificate.
   Issuer CA
   • Can replace the default root CA
   certificate created during installation.
   • Requires a CSR issued from VMCA to
   be used by an enterprise or 3rd party
   CA to generate a new issuing
   certificate.
   • Requires replacement of all issued
   default certificates after
   implementation.
   

   View Slide

6. VMware Endpoint Certificate Store (VECS)
   § Repository for Certificates and Private Keys
   § Mandatory Component
   (Used even if you don’t sign your certificates with the VMCA… )
   § Key Stores:
   – Machine SSL Certificates
   – Trusted Roots
   – Certificate Revocation Lists (CRLs)
   – Solution Users Certificates
   – Others (e.g. Virtual Volumes)
   § Managing VECS is done via vecs-cli
   (Or better yet, use the vSphere 6.0 Certificate Manager… coming up in a bit… )
   § Does Not Manage Single Sign-On Certificates
   6
   VMware vSphere 6.0
   

   View Slide

7. VECS
   VMCA
   VMware Endpoint Certificate Store (VECS)
   7
   VMware vSphere 6.0
   Signed
   VMCA
   Certificate
   Machine SSL
   Certificate
   

   View Slide

8. VMware vSphere 6.0 Certificate Types
   § ESXi Certificates
   § Machine SSL Certificate
   § Solution User Certificates
   § Single Sign-On Certificates
   8
   

   View Slide

9. ESXi Certificates
   9
   VMware vSphere 6.0
   § Post-install, ESXi always has an auto-generated
   certificate
   § VMCA will provision a signed certificate when host is
   joined to vCenter (default mode)
   § Custom certificates can be use if desired (custom mode)
   § ESXi certificates are stored locally on each host in
   the /etc/vmware/ssl
   § VMCA issued certificates can be renewed via the
   vSphere Web Client or PowerCLI
   

   View Slide

10. ESXi Certificates
    10
    VMware vSphere 6.0
    Example:
    function refreshcerts {
    process {
    $hostid = Get-VMHost $vmhost | Get-View
    $hostParam = New-Object VMware.Vim.ManagedObjectReference[] (1)
    $hostParam[0] = New-Object VMware.Vim.ManagedObjectReference
    $hostParam[0].value = $hostid.moref.value
    $hostParam[0].type = 'HostSystem'
    $_this = Get-View -Id 'CertificateManager-certificateManager'
    $_this.CertMgrRefreshCertificates_Task($hostParam)
    }
    }
    

    View Slide

11. Machine SSL Certificates
    § Creates a server-side SSL socket
    § Server verification and secure communication
    e.g. HTTPS or LDAPS
    § Each node has its own Machine SSL Certificate.
    i.e. Embedded Deployment; Management Node; or Platform Services Controller
    § All services use a Machine SSL Certificate for endpoint encryption.
    § All services communicate through the reverse proxy
    § Traffic does not go to the services themselves
    e.g. The vpxd service uses the MACHINE_SSL_CERT to expose its endpoint.
    11
    VMware vSphere 6.0
    

    View Slide

12. Solution User Certificate
    Certificate stores are located in VECS on each management node and
    embedded deployment:
    § machine – Used by component manager, license server, and the
    logging service
    § vpxd – vCenter service daemon (vpxd) store on management nodes
    and embedded deployments. vpxd uses the solution user certificate
    to authenticate to vCenter Single Sign-On
    § vpxd-extensions – Includes the Auto Deploy service, inventory
    service, and other services that are not part of other solution users
    § vsphere-webclient – Includes the vSphere Web Client and some
    additional services such as the performance chart service
    12
    VMware vSphere 6.0 – More Services but Consolidated Behind Solution Users that Hold the Certificate
    

    View Slide

13. Solution User Certificates
    § Encapsulates one or more vCenter Server services
    § Certificate authenticated by vCenter Single Sign-On
    and issued a SAML token to authenticate to other
    solution user and services
    § Each solution user must be authenticated to vCenter
    Single Sign-On
    § Re-authentication occurs after a reboot and after a
    timeout
    § The timeout configurable in the vSphere Web Client
    and defaults to 2592000 seconds (30 days)
    Maximum Holder-of-Key Token Lifetime
    13
    VMware vSphere 6.0
    30 DAYS
    

    View Slide

14. Single Sign-On Certificates
    VMware Directory Service SSL Certificate –
    § With custom certificates you may need to replace this SSL
    certificate explicitly.
    VMware vCenter Single Sign-On Signing Certificate –
    § Security Token Service (STS) – an identity provider that
    issues, validates, and renews SAML tokens that are used for
    authentication throughout vSphere
    § By default, the STS signing certificate is generated by VMCA
    § Manually refresh STS certificate via vSphere Web Client when
    the certificate expires or changes
    14
    VMware vSphere 6.0
    

    View Slide

15. Single Sign-On Certificates
    § Not stored in VECS.
    § Not managed with certificate management tools.
    § Changes are not necessary, but in special situations,
    you can replace these certificates.
    15
    Remember…
    

    View Slide

16. VMware vSphere 6.0 Certificates
    16
    Summary
    Certificate Type Provisioning Storage
    ESXi Certificates VMCA (Default) Locally on ESXi Hosts
    Machine SSL Certificates VMCA (Default) VECS
    Solution User Certificates VMCA (Default) VECS
    Single Sign-On Certificates Provisioned During Installation Manage in vSphere Web Client.
    Directory Service Certificates Provisioned During Installation In certain custom certificate
    corner cases, you may need to
    replace this certificate.
    

    View Slide

17. Certificate Replacement Options
    17
    VMware vCenter Server 6.0
    VMCA
    as Root CA
    VMCA
    as Enterprise CA
    Subordinate
    Custom CA Hybrid
    

    View Slide

18. VMware vSphere 6.0 Certificate Manager
    18
    Let’s Make Certificate Replacement Simple
    Appliance Deployment
    /usr/lib/vmware-vmca/
    bin/certificate-manager
    Windows Deployment
    :\Program Files\VMware\
    vCenter Server\vmcad\
    certificate-manager
    

    View Slide

19. Common Certificate Manager Use Cases
    19
    VMCA
    as Root CA
    (Default or
    Option 4)
    VMCA
    as Enterprise
    CA
    Subordinate
    (Option 2)
    Custom CA
    (Option 1 & 5)
    Hybrid
    (Combination)
    

    View Slide

20. VMCA as Root CA
    20
    VMware KB 2108294
    

    View Slide

21. VMCA as Enterprise CA Subordinate
    Private Key Algorithm: RSA with 2048 bits.
    Standard: X.509 v3
    Format: PEM (PKCS8 and PKCS1) with a header of ---BEGIN CERTIFICATE---
    Recommended Signature Algorithms: SHA256, SHA384, or SHA512
    § Does NOT support wildcard cards or SubjectAltName
    § You CANNOT create subsidiary CAs of VMCA.
    § No explicit limit to the length of the certificate chain.
    § Synchronize time for all nodes in environment.
    21
    Requirements
    

    View Slide

22. VMCA as Enterprise CA Subordinate
    § Create and publish custom Subordinate Certificate Authority template per KB 2112009
    § Generate Certificate Signing Request and Key in Certificate Manager with Option 2
    § On VCSA run chsh –s /bin/bash root to enable WinSCP file transfers.
    § Submit Certificate Signing Request – root_signing_cert.csr – to Enterprise Certificate Authority
    § Create the Full Certificate Chain – root_signing_chain.pem
    § Import the Full Certificate Chain and Key to Replace VMCA Root Signing Certificate in Certificate Manager with Option 2
    § Configure certool.cfg with proper values.
    § Restart vCenter Services on Connected vCenter to Reflect the Change
    § service-control –stop | --start –all
    § Replace Machine SSL Certificate with VMCA Certificate on Connected vCenter(s) with Option 3
    § Provide the FQDN or IP of Platform Service Controller
    § Configure certool.cfg with proper values.
    § Replace Solution User Certificates with VMCA Certificates on Connected vCenter(s) with Option 6
    § Provide the FQDN or IP of Platform Service Controller
    22
    Workflow
    

    View Slide

23. Demo Time
    VMCA as Enterprise CA Subordinate:
    Certificate Replacement
    

    View Slide

24. VECS
    VMCA
    Demo Scenario
    VMCA
    Signing Certificate
    Machine SSL
    Certificate
    Root CA
    Certificate
    Enterprise CA
    Certificate
    Microsoft Enterprise
    Certificate Authority
    mgmt01dc01.sddc.local vSphere 6 Platform Services Controller
    mgmt01psc01.sddc.local
    Signed Signed Signed
    VECS
    Machine SSL
    Solution Users
    Certificates
    vCenter 6 Server
    mgmt01vc01.sddc.local
    24
    

    View Slide

25. ESXi Certificate Management Modes
    25
    VMware ESXi 6.0
    VMCA
    Authority
    Mode
    Custom
    Mode
    Thumbprint
    Mode
    

    View Slide

26. 26
    Default Value = vmca
    Possible Values = vmca | custom | thumbprint
    Serach for certmgmt
    

    View Slide

27. VMCA Authority Mode
    § The default mode
    § Post-install ESXi always has an auto-generated certificate
    § ESXi certificates are stored locally on each host in the /etc/vmware/ssl
    § VMCA provisions the host a signed certificate when added to vCenter Server
    § Host certificates include the full chain to VMCA
    § ESXi certificates can be renewed via the vSphere Web Client or PowerCLI
    vpxd.certmgmt.mode = vmca
    § 24 Hour Rule – VMCA as Enterprise CA Subordinate
    § Signing certificate must have a valid date of 24 hours prior before renewing host certificates or
    adding new hosts to vCenter
    § Plan for this aging period when configuring an environment
    § Replace certificates prior to putting an environment into production
    27
    

    View Slide

28. Custom Mode
    § Replacement is the same as vSphere 5.5
    – ESXi Shell
    – HTTPS GET/PUT
    § vifs will wrap these operations.
    § Custom / 3rd Party certificates
    – Must change vpxd.certmgmt.mode to custom or risk replacement by VMCA
    – Must update TRUSTED_ROOTS store in VECS on vCenter with the custom root certificates to
    ensure trust relationship – use the vecs-cli entry create command
    vpxd.certmgmt.mode = custom
    28
    

    View Slide

29. Thumbprint Mode
    § Legacy mode
    § Fallback option for vSphere 6.0
    § May be used to retains vSphere 5.5 certificates during an upgrade
    § DO NOT use this mode unless encountering issues with vmca or custom mode
    § vCenter 6.0 and later services may not work correctly in thumbprint mode
    § Switching from thumbprint to vmca mode requires extensive planning
    29
    vpxd.certmgmt.mode = thumbprint
    

    View Slide

30. Demo Time
    VMCA as Enterprise CA Subordinate:
    ESXi Certificate Replacement
    

    View Slide

31. VECS
    VMCA
    Demo Scenario
    31
    VMCA
    Signing Certificate
    Machine SSL
    Certificate
    Root CA
    Certificate
    Enterprise CA
    Certificate
    Microsoft Enterprise
    Certificate Authority
    mgmt01dc01.sddc.local vSphere 6 Platform Services Controller
    mgmt01psc01.sddc.local
    Signed Signed Signed
    VECS
    Machine SSL
    Solution Users
    Certificates
    vCenter 6 Server
    mgmt01vc01.sddc.local
    /etc/vmware/ssl/
    ESXi Certificate
    ESXi 6.0 Host
    mgmt01esx01.sddc.local
    Signed
    

    View Slide

32. Upgrades and Operational
    Considerations
    VMware vSphere 6.0 Certificate Management
    

    View Slide

33. Deployment Considerations
    § VMCA as Enterprise CA Subordinate
    – Perform the signing certificate replacement on all Platform Services Controllers to
    ensure trusted certificates for all vCenter Server 6.0 installations
    • Remember the ‘24 Hour Rule’
    – Signing certificate must have a valid date of 24 hours prior before renewing host
    certificates or adding new hosts to vCenter
    – Plan for this aging period when configuring an existing environment
    – Replace certificates prior to putting a new environment into production
    33
    VMware vSphere 6.0
    

    View Slide

34. Managing Certificates
    • Supports replacing certificates
    • No CRL enforcement against PKI for vCenter Server and ESXi hosts
    • If you suspect that one of your certificates has been compromised, revoke and
    replace all existing certificates, including the VMCA root certificate
    • If you do not remove revoked certificates, a man-in-the-middle attack might
    enable compromise through impersonation with the account's credentials.
    34
    VMware vSphere 6.0
    

    View Slide

35. Upgrades & Auto Deploy
    § Host Upgrades and VMCA Signed Certificates
    – Upgrade process replaces self-signed certificates with VMCA-signed certificates
    – vCenter then monitors certificates and displays details vSphere Web Client
    § Host Upgrades and Custom Certificates
    – Custom certificates are retained – even if expired or invalid
    – Change vxd.certmgmt.mode to custom to ensure certificates are not replaced accidentally
    § Update Manager
    – Not compatible with the Machine SSL certificate template in vSphere 6.0.
    § Use the vSphere 5.5 certificate template for Update Manager 6.0
    35
    

    View Slide

36. A Call to Action
    Determine the Best Approach for Your Organization.
    VMCA
    as Root CA
    (Default or
    Option 4)
    VMCA
    as Enterprise
    CA
    Subordinate
    (Option 2)
    Custom CA
    (Option 1 & 5)
    Hybrid
    (Combination)
    36
    

    View Slide

37. CONFIDENTIAL 37
    vmware.com/go/inf4529
    

    View Slide

38. Ryan Johnson
    Senior Technical Marketing Manager
    @tenthirtyam
    Adam Eckerle
    Technical Account Manager
    @eck79
    vmware.com/go/podcast
    

    View Slide

39. View Slide

40. VMware vSphere Certificate Management for Mere Mortals
    Ryan Johnson, VMware, Inc
    @tenthirtyam
    Adam Eckerle, VMware, Inc
    @eck79
    vmware.com/go/podcast
    INF4529
    #INF4529
    

    View Slide