[THC19] Certificate Transparency & threats detection, 24 months later

Talk written by /me & Thomas Damonneville. Given by /me at Toulouse Hacking Convention (@ToulouseHacking), March, 8 2019. It is the last iteration of our talk (with @o0tAd0o ) about Certificate Transparency & Threats detection. It gives a specific focus on the 24 months journey of CT, coming from an niche technology to become a global TLS users protection and a blueteam tool.

Transcript

1. Certificate Transparency
   & threats detection
   24 months later
   Christophe Brocas
   Thomas Damonneville
   Caisse Nationale d’Assurance Maladie – Security Department
   Toulouse Hacking Convention
   Toulouse | 08/03/2019
   

   View Slide

2. 1) Certificate Transparency
   - Risk / Answer
   - How Certificate Transparency works
   2) Benefits for threats monitoring
   - Usages for blue teams
   - CertStreamMonitor
   3) CT & threats monitoring: a 24 months story
   Agenda
   

   View Slide

3. Risk & Answer
   

   View Slide

4. Attacker
   Certificate
   authority
   www.mydomain.com
   

   View Slide

5. Attaquant
   Attacker
   www.mydomain.com
   Attacker
   Certificate
   authority
   www.mydomain.com
   

   View Slide

6. Attacker
   Abused users
   www.mydomain.com
   Attaquant
   Attacker
   www.mydomain.com
   Attacker
   Certificate
   authority
   www.mydomain.com
   

   View Slide

7. And « www.mydomain.com » owner?
   

   View Slide

8. And « www.mydomain.com » owner?
   

   View Slide

9. And « www.mydomain.com » owner?
   

   View Slide

10. Example
    

    View Slide

11. Public CA have to submit all certificates they signed to
    publicly auditable and accessible, append-only,
    cryptographically signed logs.
    Certificate Transparency
    

    View Slide

12. Public CA have to submit all certificates they signed to
    publicly auditable and accessible, append-only,
    cryptographically signed logs.
    Timeline :
    2013 : Google (RFC 6962) then IETF (RFC 6962bis)
    →
    → 2015 : CT mandatory for EV certificates
    → 30/04/2018 : CT for all certificates
    → 24/07/2018 : interstitial blocking page Chrome 68
    → 15/10/2018 : CT mandatory for Apple products
    Certificate Transparency
    

    View Slide

13. View Slide

14. How CT works
    

    View Slide

15. Site web
    CA Logs
    Monitors
    Browser
    Web site
    

    View Slide

16. 1
    Ask for a
    certificate
    Site web
    CA Logs
    Monitors
    Browser
    Web site
    

    View Slide

17. 2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Browser
    Web site
    Logs
    Monitors
    

    View Slide

18. 3
    Receive SCT (*)
    (*) Signed Certificate Timestamp
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Browser
    Web site
    Logs
    Monitors
    

    View Slide

19. 4
    sends certificate+SCT
    (*) Signed Certificate Timestamp
    3
    Receive SCT (*)
    (*) Signed Certificate Timestamp
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Browser
    Web site
    Logs
    Monitors
    

    View Slide

20. 5
    (*) Signed Certificate Timestamp
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site
    Logs
    Monitors
    Browser
    TLS request
    

    View Slide

21. (*) Signed Certificate Timestamp
    6 TLS answer with cert + SCT
    5
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site
    Logs
    Monitors
    Browser
    TLS request
    

    View Slide

22. (*) Signed Certificate Timestamp
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    Chrome 68 requires CT for all certificates signed after 30 April 2018.
    Safari does it since October 2018.
    6 TLS answer with cert + SCT
    5
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site
    Logs
    Monitors
    Browser
    TLS request
    

    View Slide

23. Searching for certificates
    Collecting certificates
    (*) Signed Certificate Timestamp
    (*) Signed Certificate Timestamp
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    TLS answer with cert + SCT
    Chrome 68 requires CT for all certificates signed after 30 April 2018.
    Safari does it since October 2018.
    6 TLS answer with cert + SCT
    5
    5
    4
    sends certificate+SCT
    3
    Receive SCT (*)
    2
    Log pre-certificate
    1
    Ask for a
    certificate
    Site web
    CA
    Web site Monitors
    Browser
    TLS request
    Logs
    

    View Slide

24. … for Blue Teams
    

    View Slide

25. CT : benefits for Blue Teams
    FQDN (!= DNS)
    

    View Slide

26. FQDN (!= DNS)
    Internet wide logging
    +
    Open access to the data
    FQDN (!= DNS)
    FQDN (!= DNS)
    CT : benefits for Blue Teams
    

    View Slide

27. #1 Find certificates for our domains
    hacked / malicious CA
    →
    → hacked DNS server (*)
    → legit web site but not using corporate security best
    practices (hosting, certificate, DNS etc)
    * : https://blog.talosintelligence.com/2018/11/dnspionage-campaign-targets-middle-east.html
    CT: 2 useful usages (for us)
    

    View Slide

28. #2 Find certificates for « near » domains
    → phishing campaigns
    → image damage
    CT: 2 useful usages (for us)
    

    View Slide

29. Current choice:
    → hosted service
    daily notification
    →
    managed by our team
    →
    dealing with certificates
    (efficiency)
    Usage #1: our domains monitoring
    

    View Slide

30. Code: CertStreamMonitor
    

    View Slide

31. Usage #2: « near » domains monitoring
    CertStreamMonitor :
    use CT to monitor threats
    in « real time »
    AssuranceMaladieSec
    

    View Slide

32. CertStreamMonitor.py
    . works on multi CT logs flow
    . keywords detection with
    threshold
    . real time
    . runs in daemon mode
    CertStreamMonitor
    

    View Slide

33. Tailor your configuration file (conf/filename.conf)
    → Choose your keywords : ex: apple|account|login
    → Set your threshold: ex: 2 (defaut value)
    CertStreamMonitor.py: how it works
    

    View Slide

34. Tailor your configuration file (conf/filename.conf)
    → Choose your keywords : ex: apple|account|login
    → Set your threshold: ex: 2 (defaut value)
    hostnames with a number of keywords ≥ threshold
    insert in DB (ex :
    → login.apple-connect.com)
    CertStreamMonitor.py: how it works
    

    View Slide

35. Tailor your configuration file (conf/filename.conf)
    → Choose your keywords : ex: apple|account|login
    → Set your threshold: ex: 2 (defaut value)
    hostnames with a number of keywords ≥ threshold
    insert in DB (ex :
    → login.apple-connect.com)
    hostnames with a number of keywords < threshold but >0
    write to log file (ex : webmail.
    → apple-mail.com)
    CertStreamMonitor.py: how it works
    

    View Slide

36. → run on demand (ex. : 1/day)
    → test all hostnames not already
    logged as up
    if hostname is up:
    →
    * update DB
    * JSON report file
    (ip, AS, abuse email...)
    scanhost.py: how it works
    

    View Slide

37. JSON report file
    scanhost.py: how it works
    

    View Slide

38. Screenshots are not a demo
    

    View Slide

39. View Slide

40. View Slide

41. View Slide

42. View Slide

43. View Slide

44. Example #1 :
    customers abuse
    cpam-{78,75,13,...}.fr
    service potentially
    →
    abusing our customers
    (over priced phone
    number, personal data
    theft)
    Results
    

    View Slide

45. Example #1 :
    customers abuse
    cpam-{78,75,13,...}.fr
    service potentially
    →
    abusing our customers
    (over priced phone
    number, personal data
    theft)
    → service inactivation
    Results
    

    View Slide

46. Example #2 : IT management
    social-ameli.fr
    . Legit website
    . Best practices not applied :
    (domainname, hosting etc)
    Results
    

    View Slide

47. TLS, not HTTP – only detect hostnames accessed through TLS
    RegExp – relying on regexp to find hostnames can lead to
    miss some of them. Wildcard certificates also beat us.
    Trust- we use tier service to get CT certificates (Calidog
    Security in our case). Can we trust it?
    Limits
    

    View Slide

48. TLS, not HTTP – only detect hostnames accessed through TLS
    RegExp – relying on regexp to find hostnames can lead to
    miss some of them. Wildcard certificates also beat us.
    Trust- we use tier service to get CT certificates (Calidog
    Security in our case). Can we trust it?
    Limits
    

    View Slide

49. TLS, not HTTP – only detect hostnames accessed through TLS
    RegExp – relying on regexp to find hostnames can lead to
    miss some of them. Wildcard certificates also beat us.
    Trust- we use tier service to get CT certificates (Calidog
    Security in our case). Can we trust it?
    But we rely on their code, a potential single point of failure.
    it is a
    → call for action to the Infosec community
    Limits
    

    View Slide

50. CertStreamMonitor evolution in 9 months
    

    View Slide

51. View Slide

52. You can now choose your
    CT logs aggregator
    service :
    * Calidog Security one
    * your own using Calidog
    code
    

    View Slide

53. Setting the threshold for
    keywords detection is
    available in config file
    

    View Slide

54. (optional) check Google
    SafeBrowsing status of
    the hostname
    

    View Slide

55. Name of the alerts
    directory can be hashed
    with date + hostname
    (PR of @xme)
    

    View Slide

56. (optional) notification
    by mail or instant
    messaging like Slack or
    Rocket.
    

    View Slide

57. CT & Threats Monitoring:
    a 24 months story
    

    View Slide

58. April 2017
    The
    announcement
    

    View Slide

59. June 2017
    Why CT
    becomes
    interesting?
    

    View Slide

60. 01/2015 : 31 %
    June 2017
    

    View Slide

61. 01/2015 : 31 %
    06/2017 : 57 %
    → 83% of growth
    in 2,5 years
    June 2017
    

    View Slide

62. Nov. 2017
    First tools
    show up
    

    View Slide

63. May 2018
    More complex
    tools
    + CT required for
    all certificates
    

    View Slide

64. July 2018
    Chrome
    implements
    CT as a strict
    requirement
    

    View Slide

65. Nov. 2018
    When CT
    becomes a
    DNS hacks
    detection
    tool
    

    View Slide

66. Nov. 2018
    When CT
    becomes a
    DNS hacks
    detection
    tool
    

    View Slide

67. Jan. 2019
    CT appears in Blue Teams
    best practices
    

    View Slide

68. Feb. 2019
    CT is point out as one of
    the tools able to control
    TLS grey/dark activities
    

    View Slide

69. low cost
    tools and services
    are there, just use
    them
    efficiency
    notified before
    or soon after the
    the attacks comes
    online
    blind
    vision at Internet
    scale
    + bonus track: compliance
    CT monitoring is now part of best practices requirements
    

    View Slide

70. Thanks!
    Some questions?
    https://github.com/AssuranceMaladieSec
    christophe.brocas@assurance-maladie.fr
    thomas.damonneville@assurance-maladie.fr
    @cbrocas | @o0tAd0o
    

    View Slide

71. Photos credits :
    Images under Creative Commons licence:
    Clair de lune : https://www.flickr.com/photos/cbrocas/4200102493/
    danger : https://www.flickr.com/photos/adulau/26003405317/
    complexity : https://www.flickr.com/photos/70023venus2009/6032939635
    gain : https://www.flickr.com/photos/143106192@N03/29307455407/
    book : https://www.flickr.com/photos/thesoulofhope/14545003924/
    evolution : https://www.flickr.com/photos/elle_florio/26750479006/
    Flaticons : Freepik from https://www.flaticon.com/
    

    View Slide