Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[THC19] Certificate Transparency & threats detection, 24 months later

[THC19] Certificate Transparency & threats detection, 24 months later

Talk written by /me & Thomas Damonneville. Given by /me at Toulouse Hacking Convention (@ToulouseHacking), March, 8 2019. It is the last iteration of our talk (with @o0tAd0o ) about Certificate Transparency & Threats detection. It gives a specific focus on the 24 months journey of CT, coming from an niche technology to become a global TLS users protection and a blueteam tool.

Christophe Brocas

March 08, 2019
Tweet

More Decks by Christophe Brocas

Other Decks in Technology

Transcript

  1. Certificate Transparency
    & threats detection
    24 months later
    Christophe Brocas
    Thomas Damonneville
    Caisse Nationale d’Assurance Maladie – Security Department
    Toulouse Hacking Convention
    Toulouse | 08/03/2019

    View full-size slide

  2. 1) Certificate Transparency
    - Risk / Answer
    - How Certificate Transparency works
    2) Benefits for threats monitoring
    - Usages for blue teams
    - CertStreamMonitor
    3) CT & threats monitoring: a 24 months story
    Agenda

    View full-size slide

  3. Risk & Answer

    View full-size slide

  4. Attacker
    Certificate
    authority
    www.mydomain.com

    View full-size slide

  5. Attaquant
    Attacker
    www.mydomain.com
    Attacker
    Certificate
    authority
    www.mydomain.com

    View full-size slide

  6. Attacker
    Abused users
    www.mydomain.com
    Attaquant
    Attacker
    www.mydomain.com
    Attacker
    Certificate
    authority
    www.mydomain.com

    View full-size slide

  7. And « www.mydomain.com » owner?

    View full-size slide

  8. And « www.mydomain.com » owner?

    View full-size slide

  9. And « www.mydomain.com » owner?

    View full-size slide

  10. Public CA have to submit all certificates they signed to
    publicly auditable and accessible, append-only,
    cryptographically signed logs.
    Certificate Transparency

    View full-size slide

  11. Public CA have to submit all certificates they signed to
    publicly auditable and accessible, append-only,
    cryptographically signed logs.
    Timeline :
    2013 : Google (RFC 6962) then IETF (RFC 6962bis)

    → 2015 : CT mandatory for EV certificates
    → 30/04/2018 : CT for all certificates
    → 24/07/2018 : interstitial blocking page Chrome 68
    → 15/10/2018 : CT mandatory for Apple products
    Certificate Transparency

    View full-size slide

  12. How CT works

    View full-size slide

  13. Site web
    CA Logs
    Monitors
    Browser
    Web site

    View full-size slide

  14. 1
    Ask for a
    certificate
    Site web
    CA Logs
    Monitors
    Browser
    Web site

    View full-size slide

  15. 2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Browser
    Web site
    Logs
    Monitors

    View full-size slide

  16. 3
    Receive SCT (*)
    (*) Signed Certificate Timestamp
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Browser
    Web site
    Logs
    Monitors

    View full-size slide

  17. 4
    sends certificate+SCT
    (*) Signed Certificate Timestamp
    3
    Receive SCT (*)
    (*) Signed Certificate Timestamp
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Browser
    Web site
    Logs
    Monitors

    View full-size slide

  18. 5
    (*) Signed Certificate Timestamp
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site
    Logs
    Monitors
    Browser
    TLS request

    View full-size slide

  19. (*) Signed Certificate Timestamp
    6 TLS answer with cert + SCT
    5
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site
    Logs
    Monitors
    Browser
    TLS request

    View full-size slide

  20. (*) Signed Certificate Timestamp
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    Chrome 68 requires CT for all certificates signed after 30 April 2018.
    Safari does it since October 2018.
    6 TLS answer with cert + SCT
    5
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site
    Logs
    Monitors
    Browser
    TLS request

    View full-size slide

  21. Searching for certificates
    Collecting certificates
    (*) Signed Certificate Timestamp
    (*) Signed Certificate Timestamp
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    Chrome 68 requires CT for all certificates signed after 30 April 2018.
    Safari does it since October 2018.
    6 TLS answer with cert + SCT
    5
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site Monitors
    Browser
    TLS request
    Logs

    View full-size slide

  22. … for Blue Teams

    View full-size slide

  23. CT : benefits for Blue Teams
    FQDN (!= DNS)

    View full-size slide

  24. FQDN (!= DNS)
    Internet wide logging
    +
    Open access to the data
    FQDN (!= DNS)
    FQDN (!= DNS)
    CT : benefits for Blue Teams

    View full-size slide

  25. #1 Find certificates for our domains
    hacked / malicious CA

    → hacked DNS server (*)
    → legit web site but not using corporate security best
    practices (hosting, certificate, DNS etc)
    * : https://blog.talosintelligence.com/2018/11/dnspionage-campaign-targets-middle-east.html
    CT: 2 useful usages (for us)

    View full-size slide

  26. #2 Find certificates for « near » domains
    → phishing campaigns
    → image damage
    CT: 2 useful usages (for us)

    View full-size slide

  27. Current choice:
    → hosted service
    daily notification

    managed by our team

    dealing with certificates
    (efficiency)
    Usage #1: our domains monitoring

    View full-size slide

  28. Code: CertStreamMonitor

    View full-size slide

  29. Usage #2: « near » domains monitoring
    CertStreamMonitor :
    use CT to monitor threats
    in « real time »
    AssuranceMaladieSec

    View full-size slide

  30. CertStreamMonitor.py
    . works on multi CT logs flow
    . keywords detection with
    threshold
    . real time
    . runs in daemon mode
    CertStreamMonitor

    View full-size slide

  31. Tailor your configuration file (conf/filename.conf)
    → Choose your keywords : ex: apple|account|login
    → Set your threshold: ex: 2 (defaut value)
    CertStreamMonitor.py: how it works

    View full-size slide

  32. Tailor your configuration file (conf/filename.conf)
    → Choose your keywords : ex: apple|account|login
    → Set your threshold: ex: 2 (defaut value)
    hostnames with a number of keywords ≥ threshold
    insert in DB (ex :
    → login.apple-connect.com)
    CertStreamMonitor.py: how it works

    View full-size slide

  33. Tailor your configuration file (conf/filename.conf)
    → Choose your keywords : ex: apple|account|login
    → Set your threshold: ex: 2 (defaut value)
    hostnames with a number of keywords ≥ threshold
    insert in DB (ex :
    → login.apple-connect.com)
    hostnames with a number of keywords < threshold but >0
    write to log file (ex : webmail.
    → apple-mail.com)
    CertStreamMonitor.py: how it works

    View full-size slide

  34. → run on demand (ex. : 1/day)
    → test all hostnames not already
    logged as up
    if hostname is up:

    * update DB
    * JSON report file
    (ip, AS, abuse email...)
    scanhost.py: how it works

    View full-size slide

  35. JSON report file
    scanhost.py: how it works

    View full-size slide

  36. Screenshots are not a demo

    View full-size slide

  37. Example #1 :
    customers abuse
    cpam-{78,75,13,...}.fr
    service potentially

    abusing our customers
    (over priced phone
    number, personal data
    theft)
    Results

    View full-size slide

  38. Example #1 :
    customers abuse
    cpam-{78,75,13,...}.fr
    service potentially

    abusing our customers
    (over priced phone
    number, personal data
    theft)
    → service inactivation
    Results

    View full-size slide

  39. Example #2 : IT management
    social-ameli.fr
    . Legit website
    . Best practices not applied :
    (domainname, hosting etc)
    Results

    View full-size slide

  40. TLS, not HTTP – only detect hostnames accessed through TLS
    RegExp – relying on regexp to find hostnames can lead to
    miss some of them. Wildcard certificates also beat us.
    Trust- we use tier service to get CT certificates (Calidog
    Security in our case). Can we trust it?
    Limits

    View full-size slide

  41. TLS, not HTTP – only detect hostnames accessed through TLS
    RegExp – relying on regexp to find hostnames can lead to
    miss some of them. Wildcard certificates also beat us.
    Trust- we use tier service to get CT certificates (Calidog
    Security in our case). Can we trust it?
    Limits

    View full-size slide

  42. TLS, not HTTP – only detect hostnames accessed through TLS
    RegExp – relying on regexp to find hostnames can lead to
    miss some of them. Wildcard certificates also beat us.
    Trust- we use tier service to get CT certificates (Calidog
    Security in our case). Can we trust it?
    But we rely on their code, a potential single point of failure.
    it is a
    → call for action to the Infosec community
    Limits

    View full-size slide

  43. CertStreamMonitor evolution in 9 months

    View full-size slide

  44. You can now choose your
    CT logs aggregator
    service :
    * Calidog Security one
    * your own using Calidog
    code

    View full-size slide

  45. Setting the threshold for
    keywords detection is
    available in config file

    View full-size slide

  46. (optional) check Google
    SafeBrowsing status of
    the hostname

    View full-size slide

  47. Name of the alerts
    directory can be hashed
    with date + hostname
    (PR of @xme)

    View full-size slide

  48. (optional) notification
    by mail or instant
    messaging like Slack or
    Rocket.

    View full-size slide

  49. CT & Threats Monitoring:
    a 24 months story

    View full-size slide

  50. April 2017
    The
    announcement

    View full-size slide

  51. June 2017
    Why CT
    becomes
    interesting?

    View full-size slide

  52. 01/2015 : 31 %
    June 2017

    View full-size slide

  53. 01/2015 : 31 %
    06/2017 : 57 %
    → 83% of growth
    in 2,5 years
    June 2017

    View full-size slide

  54. Nov. 2017
    First tools
    show up

    View full-size slide

  55. May 2018
    More complex
    tools
    + CT required for
    all certificates

    View full-size slide

  56. July 2018
    Chrome
    implements
    CT as a strict
    requirement

    View full-size slide

  57. Nov. 2018
    When CT
    becomes a
    DNS hacks
    detection
    tool

    View full-size slide

  58. Nov. 2018
    When CT
    becomes a
    DNS hacks
    detection
    tool

    View full-size slide

  59. Jan. 2019
    CT appears in Blue Teams
    best practices

    View full-size slide

  60. Feb. 2019
    CT is point out as one of
    the tools able to control
    TLS grey/dark activities

    View full-size slide

  61. low cost
    tools and services
    are there, just use
    them
    efficiency
    notified before
    or soon after the
    the attacks comes
    online
    blind
    vision at Internet
    scale
    + bonus track: compliance
    CT monitoring is now part of best practices requirements

    View full-size slide

  62. Thanks!
    Some questions?
    https://github.com/AssuranceMaladieSec
    [email protected]
    [email protected]
    @cbrocas | @o0tAd0o

    View full-size slide

  63. Photos credits :
    Images under Creative Commons licence:
    Clair de lune : https://www.flickr.com/photos/cbrocas/4200102493/
    danger : https://www.flickr.com/photos/adulau/26003405317/
    complexity : https://www.flickr.com/photos/70023venus2009/6032939635
    gain : https://www.flickr.com/photos/143106192@N03/29307455407/
    book : https://www.flickr.com/photos/thesoulofhope/14545003924/
    evolution : https://www.flickr.com/photos/elle_florio/26750479006/
    Flaticons : Freepik from https://www.flaticon.com/

    View full-size slide