Trust Unearned? Evaluating CA Trustworthiness Across 5 Billion Certificates

Security relies on trust, especially when it comes to Certificate Authorities. Browsers ship with many root CAs built in, but are they all equally trustworthy? I examined over 5 billion recent TLS certificates and now I know! In this session I’ll reveal the most and least trustworthy CAs, factors influencing their trust ratings, and how to use this data to protect your organization.

[As presented at RSAC 2023, 27 April 2023]

Transcript

1. SESSION ID:
   #RSAC
   David J. Bianco
   Trust Unearned? Evaluating CA
   Trustworthiness Across 5 Billion
   Certificates
   Staff Security Strategist
   SURGe by Splunk
   @DavidJBianco / @DavidJBianco@infosec.exchange
   HT-R03
   

   View Slide

2. #RSAC
   Disclaimer
   Presentations are intended for educational purposes only and do not replace independent professional
   judgment. Statements of fact and opinions expressed are those of the presenters individually and,
   unless expressly stated to the contrary, are not the opinion or position of RSA Conference™ or any other
   co-sponsors. RSA Conference does not endorse or approve, and assumes no responsibility for, the
   content, accuracy or completeness of the information presented.
   Attendees should note that sessions may be audio- or video-recorded and may be published in various
   media, including print, audio and video formats without further notice. The presentation template and
   any media capture are subject to copyright protection.
   © 2023 RSA Conference LLC or its affiliates. The RSA Conference logo and other trademarks are proprietary. All rights reserved.
   2
   

   View Slide

3. #RSAC
   The Secret Origins of the CA Trust Project
   As a new SURGeon, your first project is an
   important choice!
   I wanted something research-y, but
   practical. Ideally, something that could
   benefit from our unlimited Splunk license.
   I’d worked with Certificate Transparency
   before and thought it was neat.
   What could I do with ALL THE TLS CERTS
   ON THE INTERNET?
   3
   Source: DALL-E
   

   View Slide

4. #RSAC
   CA Trust: Is it Truly Earned?
   Internet security is built on TLS, which
   anchors its trust with Root CAs.
   OS and browser manufacturers decide for
   us which roots we trust.
   • Chrome: 138 trusted roots
   • Firefox: 54 trusted roots
   • Safari / MacOS: 154 trusted roots
   How do we know they are all worthy of
   our trust? Let’s measure!
   4
   

   View Slide

5. #RSAC
   Splunkvengers, Assemble!
   5
   Mikael
   Bjerkeland
   Apps/Sys
   Engineer
   Kelcie
   Bourne
   Security Strategist,
   SURGe
   Philipp
   Drieger
   Principal
   Architect
   

   View Slide

6. #RSAC
   Methodology
   6
   In theory, this is simple:
   1. Download all TLS certificates on
   the Internet and load them into
   Splunk
   2. Mark all certificates observed
   doing Bad Things
   3. Calculate the ”worst” and “best”
   CAs
   Source: DALL-E
   

   View Slide

7. #RSAC
   Certificate Review
   7
   Subject
   (“Whose cert
   is this?”)
   Issuer
   (“Who created
   this cert?”)
   

   View Slide

8. #RSAC
   Certificate Review
   8
   SANs
   (“What other
   names is this
   good for?”)
   Chain of
   Trust
   (“Can I trust
   this cert?”)
   Root CA
   (“Who says I
   can trust it?”)
   

   View Slide

9. #RSAC
   Superpower: Certificate Validation
   9
   Browsers must validate a certificate’s chain
   of trust, but that’s not all!
   What if the site owner loses control of their
   secret key?
   • Certificate Revocation Lists (CRLs) or
   Online Certificate Status Protocol (OCSP)
   But what if the CA loses control of its
   signing key?
   Source: DALL-E
   

   View Slide

10. #RSAC
    Certificate Transparency Explained Simply
    10
    Certificate Transparency (CT) provides public, verifiable
    records of WebPKI certificate issuance.
    Chrome has required CT since April 2018.
    CTLs are open to all via a standard API. I used this to
    download certificates from (mostly) 2021 and 2022.
    Source: https://certificate.transparency.dev/howctworks/
    

    View Slide

11. #RSAC
    Data: Certificates
    11
    5 Billion
    Unique Certificates
    78k
    Issuing CAs
    497
    Root CAs
    15
    Certificate Transparency Logs
    

    View Slide

12. #RSAC
    Root CAs and Issuers
    12
    

    View Slide

13. #RSAC
    Typical Root/Issuer CA Relationships
    13
    

    View Slide

14. #RSAC
    Highly-Connected Relationships
    14
    

    View Slide

15. #RSAC
    Data: Intel
    15
    185 Million
    Malicious Observations
    7
    Providers
    

    View Slide

16. #RSAC
    Data: Intel - Big THANK YOUs to the Providers!
    16
    

    View Slide

17. #RSAC
    Analysis: Challenges and Limitations
    • This was some big “tres commas” data!
    – Needed custom download & parsing tool
    – Ingest was tricky
    • Matching intel to certificate subjects isn’t straightforward
    • Extreme variation in CA population sizes makes fair comparisons
    difficult
    17
    

    View Slide

18. #RSAC
    Riskiest Root CAs (Single Pool)
    18
    

    View Slide

19. #RSAC
    Riskiest Issuing CAs (Single Pool)
    19
    

    View Slide

20. #RSAC
    Total Certificates by Root CA
    20
    

    View Slide

21. #RSAC
    Total Certificates by Issuing CA
    21
    

    View Slide

22. #RSAC
    Creating Root CA Tiers is Much Fairer!
    22
    

    View Slide

23. #RSAC
    All Roots – Tier 1
    23
    

    View Slide

24. #RSAC
    All Roots – Tier 2
    24
    

    View Slide

25. #RSAC
    All Roots – Tier 3
    25
    

    View Slide

26. #RSAC
    Riskiest Roots – Tier 4
    26
    

    View Slide

27. #RSAC
    Trustiest Roots – Tier 4
    27
    

    View Slide

28. #RSAC
    Issuing CA Tiers
    28
    

    View Slide

29. #RSAC
    Riskiest Issuers – Tier 1
    29
    

    View Slide

30. #RSAC
    Trustiest Issuers – Tier 1
    30
    

    View Slide

31. #RSAC
    Riskiest Issuers – Tier 2
    31
    

    View Slide

32. #RSAC
    Trustiest Issuers – Tier 2
    32
    

    View Slide

33. #RSAC
    Riskiest Issuers – Tier 3
    33
    

    View Slide

34. #RSAC
    Trustiest Issuers – Tier 3
    34
    

    View Slide

35. #RSAC
    Riskiest Issuers – Tier 4
    35
    

    View Slide

36. #RSAC
    Trustiest Issuers – Tier 4
    36
    

    View Slide

37. #RSAC
    Conclusions
    • We identified 4 risky outlier root CAs and 10 issuers
    – Tier 4 roots were drowned in noise, so we ignored those
    • Blocking specific CAs is probably not warranted for most orgs
    • Logging certificates enables CAs to be used as observations with Risk-
    Based Alerting (RBA) or as data enrichment for threat hunting
    37
    

    View Slide

38. #RSAC
    Room for Future Work
    • Longer certificate history + longer intel history would allow us to
    track trustworthiness changes over time, which could be interesting
    • More precise subject matching
    • Why don’t more intel providers track malicious certificates?
    • Someone (else) should compute & publish CA risk rankings on a
    regular basis
    38
    

    View Slide

39. #RSAC
    Apply
    Days
    • Download the CA Trust risk rankings
    from https://splk.it/CATrust
    • Start logging TLS certificates w/Root
    and Issuing CA details
    Weeks
    • Do some HTTPS-focused threat
    hunting, using certificates and our
    risk data for enrichment
    39
    Months
    • Beg your CTI vendors to make CA
    risk rankings a regular part of their
    service offering!
    

    View Slide

40. SESSION ID:
    #RSAC
    David J. Bianco
    Trust Unearned? Evaluating CA
    Trustworthiness Across 5 Billion
    Certificates
    Staff Security Strategist
    SURGe by Splunk
    @DavidJBianco / @DavidJBianco@infosec.exchange
    HT-R03
    

    View Slide