Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PKI at Scale Using Short-Lived Certificates

Bryan Payne
January 25, 2016

PKI at Scale Using Short-Lived Certificates

This talk was originally given at Engima 2016.
Video available at https://www.youtube.com/watch?v=7YPIsbz8Pig

While TLS is considered a “best practice” for security, deploying the underlying PKI at scale for cloud applications presents many challenges. This starts with the need to securely bootstrap secrets into each instance. The challenges continue at runtime with the need for insight into the continued trustworthiness of each instance. Unfortunately, in practice, it can be difficult to deploy and maintain such a PKI. In an effort to solve both scale and management challenges, some advocate for the use of short-­lived certificates in lieu of revocation lists. The idea is that a compromised private key is less valuable because it will only work for a limited timespan. But what is really required to deploy such a system?

This talk will take a deep dive into the world of PKI deployments at scale. We will start with a brief overview of PKIs in general before drilling into the specific use case of protecting internally facing microservices using TLS with mutual authentication. From here we will explore the pros and cons of using short-­lived certificates. Then we will look at the operational challenges around such deployments, including scaling certificate authority services, handling reloading of certificates into services at run­time, and determining if an instance is trustworthy enough to receive renewed credentials. We will close with some parting thoughts about the remaining challenges in this space.

Bryan Payne

January 25, 2016
Tweet

More Decks by Bryan Payne

Other Decks in Technology

Transcript

  1. PKI at Scale Using Short-Lived Certificates
    Bryan D. Payne Engineering Manager, Platform Security

    View Slide

  2. weeks <6 months 6-12 months >1 year
    *Notified via extortion attempt
    2 weeks
    3 weeks
    1 month
    3 months
    4 months
    8 months
    17 months
    13 months

    View Slide

  3. Elastic
    Load
    Balancers
    Web Service
    Web Service
    Web Service
    Web Service
    . . .
    Internet Cloud / Data Center / Etc

    View Slide

  4. Securely Deploy Certificate / Key
    Communicate Securely
    API & UI
    for Certificate
    Creation
    Lemur
    Get Certificate & Key
    Public CA
    Private CA
    CloudCA
    Seal Secrets
    Metatron
    Deployment
    Management
    Spinnaker
    Version Control
    Git
    AMI
    Server with TLS
    Karyon
    Tomcat
    Apache
    Metatron
    Client with TLS
    Ribbon
    Metatron

    View Slide

  5. Securely Deploy Certificate / Key
    Communicate Securely
    API & UI
    for Certificate
    Creation
    Lemur
    Get Certificate & Key
    Public CA
    Private CA
    CloudCA
    Seal Secrets
    Metatron
    Deployment
    Management
    Spinnaker
    Version Control
    Git
    AMI
    Server with TLS
    Karyon
    Tomcat
    Apache
    Metatron
    Client with TLS
    Ribbon
    Metatron

    View Slide

  6. Securely Deploy Certificate / Key
    Communicate Securely
    API & UI
    for Certificate
    Creation
    Lemur
    Get Certificate & Key
    Public CA
    Private CA
    CloudCA
    Seal Secrets
    Metatron
    Deployment
    Management
    Spinnaker
    Version Control
    Git
    AMI
    Server with TLS
    Karyon
    Tomcat
    Apache
    Metatron
    Client with TLS
    Ribbon
    Metatron

    View Slide

  7. Revocation Is Hard

    View Slide

  8. CRL (rfc2459)
    OCSP (rfc2560)
    OCSP stapling (rfc6066)
    OCSP must staple (draft-hallambaker-muststaple-00)

    View Slide

  9. CRL: Certificate Revocation List
    Browser Web
    Server
    (Content)
    Web
    Server
    (CRL)
    Certificate
    Authority
    Update
    Internet
    CRL
    Cache
    1: TLS Handshake
    2: Check CRL

    View Slide

  10. CRL (rfc2459)
    OCSP (rfc2560)
    OCSP stapling (rfc6066)
    OCSP must staple (draft-hallambaker-muststaple-00)

    View Slide

  11. OCSP: Online Certificate Status Protocol
    Browser
    Web
    Server
    (Content)
    OCSP
    Responder
    Certificate
    Authority
    Update
    Internet
    1: TLS Handshake
    2: Get Certificate Status

    View Slide

  12. CRL (rfc2459)
    OCSP (rfc2560)
    OCSP stapling (rfc6066)
    OCSP must staple (draft-hallambaker-muststaple-00)

    View Slide

  13. OCSP Stapling
    Browser
    Web
    Server
    (Content)
    Certificate
    Authority
    Update
    Internet
    1: TLS Handshake
    2: Return Certificate Status
    OCSP
    Responder

    View Slide

  14. CRL (rfc2459)
    OCSP (rfc2560)
    OCSP stapling (rfc6066)
    OCSP must staple (draft-hallambaker-muststaple-00)

    View Slide

  15. OCSP
    must-staple
    OCSP
    staple
    OCSP CRL
    Java
    C
    Python
    JavaScript
    M Georgiev et al., “The most dangerous code in the world:
    validating SSL certificates in non-browser software”, In
    Proceedings of ACM CCS, 2012.

    View Slide

  16. View Slide

  17. Photo Credit: Kayamon (CC BY-SA 3.0)
    https://en.wikipedia.org/wiki/File:Penny_Harvest_Field_2007.jpg

    View Slide

  18. Short-Lived Certificates
    • R Rivest, “Can We Eliminate Certificate Revocation Lists?”, In
    Proceedings of Financial Cryptography, 1998.
    • E Topalovic et al., “Towards Short-Lived Certificates”, In Proceedings
    of IEEE Oakland Web 2.0 Security and Privacy (W2SP), 2012.
    6 months
    3 months
    1 month
    1 week
    4 days
    4 Hours

    View Slide

  19. Photo Credit: Bhernandez (CC BY 2.0)
    https://www.flickr.com/photos/kennyuhh/2917293212

    View Slide

  20. View Slide

  21. AWS HMAC
    Generation
    Not real secret keys, sorry.
    Lifecycle of AccessKeyID and SecretKey
    is of utmost interest here.
    AKIAIOSFODNN7EXAMPLE:iX
    KQe8qXbhnN0jUe7JGVqFNXM
    mTxP5pI6example
    DELETE\n
    \n
    \n
    Tue, 27 Mar 2007 21:20:26 +0000\n
    /johnsmith/photos/puppy.jpg
    AccessKeyID and SecretKey
    HMAC-
    SHA-1
    Customer Request
    lx3byBScXR6KzyMaifNkardMwNk
    Digest Verified by AWS

    View Slide

  22. Circa 2012: AWS SDKs Introduce the Provider Paradigm
    // provider paradigm dynamically asks for keys every time
    AWSCredentialsProvider prov = new AWSCredentialsProvider(){
    public AWSCredentials getCredentials(){
    RESTfulObj AWSKey = RESTService.get(“server/getAWSKey”);
    return new BasicAWSCredentials(
    AWSKey.getAccessID(), AWSKey.getSecretKey());
    }
    };
    AmazonSimpleDBClient client = new AmazonSimpleDBClient(prov);
    client.listDomains();
    The client object in the above code example no longer caches keys.

    View Slide

  23. On Instance Credentials
    $curl http://169.254.169.254/latest/meta-data/iam/security-credentials/role
    {
    "Code" : "Success",
    "LastUpdated" : "2015-09-17T01:29:49Z",
    "Type" : "AWS-HMAC",
    "AccessKeyId" : "ASIAIL6IJJCXLEXAMPLE",
    "SecretAccessKey" : "iXKQe8qXbhnN0jUe7JGVqFNXMmTxP5pI6example",
    "Token" : "...",
    "Expiration" : "2015-09-17T07:47:45Z"
    }

    View Slide

  24. Alice
    Alice
    Bob
    Bob
    ID Proof + Credential Request
    New Short-Lived Credential
    Validate ID
    Generate
    Credential

    View Slide

  25. Don’t use
    short-lived
    cred to get
    updated cred!
    Alice
    Alice
    Bob
    Bob
    ID Proof + Credential Request
    New Short-Lived Credential
    Validate ID
    Generate
    Credential

    View Slide

  26. Linux Kernel (with AppArmor or SELinux)
    Credential
    Management
    Process
    Service with TLS
    Short-Lived
    Certificate and
    Key Files
    (write)
    (read)
    TLS Session
    System Identity
    Credentials
    (TPM or SGX)
    Credential Renewal
    Protocol

    View Slide

  27. nel (with AppArmor or SELinux)
    Service with TLS
    Short-Lived
    Certificate and
    Key Files
    (write)
    (read)
    TLS Session
    Loading new certificates
    into service…
    • Send signal to service
    • Restart service
    • Design service to reload
    certificates periodically

    View Slide

  28. How to load a new certificate and
    private key?
    Zero downtime?
    Apache graceful restart Maybe
    Nginx reload Yes
    Tomcat restart No
    HAProxy reload No
    Stunnel HUP No
    Ghostunnel SIGUSR1 Yes

    View Slide

  29. Develop & Deploy Code
    Communicate Securely
    Provision Credentials at Startup
    API & UI
    for Certificate
    Creation
    Lemur
    Public CA
    Private CA
    CloudCA
    Initialize
    Secrets
    Metatron
    Deployment
    Management
    Spinnaker
    Version Control
    Git
    AMI
    Server with TLS
    Karyon
    Tomcat
    Nginx
    Metatron
    Client with TLS
    Ribbon
    Metatron

    View Slide

  30. Develop & Deploy Code
    Communicate Securely
    Provision Credentials at Startup
    API & UI
    for Certificate
    Creation
    Lemur
    Public CA
    Private CA
    CloudCA
    Initialize
    Secrets
    Metatron
    Deployment
    Management
    Spinnaker
    Version Control
    Git
    AMI
    Server with TLS
    Karyon
    Tomcat
    Nginx
    Metatron
    Client with TLS
    Ribbon
    Metatron

    View Slide

  31. Develop & Deploy Code
    Communicate Securely
    Provision Credentials at Startup
    API & UI
    for Certificate
    Creation
    Lemur
    Public CA
    Private CA
    CloudCA
    Initialize
    Secrets
    Metatron
    Deployment
    Management
    Spinnaker
    Version Control
    Git
    AMI
    Server with TLS
    Karyon
    Tomcat
    Nginx
    Metatron
    Client with TLS
    Ribbon
    Metatron

    View Slide

  32. Develop & Deploy Code
    Communicate Securely
    Provision Credentials at Startup
    API & UI
    for Certificate
    Creation
    Lemur
    Public CA
    Private CA
    CloudCA
    Initialize
    Secrets
    Metatron
    Deployment
    Management
    Spinnaker
    Version Control
    Git
    AMI
    Server with TLS
    Karyon
    Tomcat
    Nginx
    Metatron
    Client with TLS
    Ribbon
    Metatron

    View Slide

  33. Long-Lived
    Certificates
    Short-Lived Certificates
    • Improve attack detection, in
    practice
    • Retrofit your applications to
    support revocation
    • Refresh certificates
    • Update server / client to support
    graceful reloading of certificates

    View Slide

  34. From Vision to Reality…

    View Slide

  35. Questions?
    [email protected]flix.com
    http://bryanpayne.org
    [PS… I’m hiring!]

    View Slide