[2018bis Hack-it-n] CertStreamMonitor: use Certificate Transparency to improve your threats detection

Talk written by myself and Thomas Damonneville. Given by myself during the 2018bis edition of the Hack-it-n conference.

Transcript

1. CertStreamMonitor use Certificate Transparency to improve your threats detection Christophe Brocas Thomas Damonneville Caisse Nationale d’Assurance Maladie – Security team hack-it-n 2018 bis Bordeaux, 12/11/2018
2. 1) Risk / Answer 2) How Certificate Transparency works 3) Benefits for threats monitoring 4) CertStreamMonitor : usage, results, limits → Agenda
3. #1 Risk & Answer
4. Attacker Certificate authority www.mydomain.com
5. Attaquant Attacker www.mydomain.com Attacker Certificate authority www.mydomain.com
6. Attacker Abused users www.mydomain.com Attaquant Attacker www.mydomain.com Attacker Certificate authority www.mydomain.com
7. And « www.mydomain.com » owner?
8. And « www.mydomain.com » owner?
9. And « www.mydomain.com » owner?
10. Example
11. Public CA have to submit all certificates they signed to publicly auditable and accessible, append-only, cryptographically signed logs. Certificate Transparency
12. Public CA have to submit all certificates they signed to publicly auditable and accessible, append-only, cryptographically signed logs. Timeline : 2013 : Google (RFC 6962) then IETF (RFC 6962bis) → → 2015 : CT mandatory for EV certificates → 30/04/2018 : CT for all certificates → 24/07/2018 : interstitial blocking page Chrome 68 → 15/10/2018 : CT mandatory for Apple products Certificate Transparency
14. #2 How CT works
15. Site web CA Logs Monitors Browser Web site
16. 1 Ask for a certificate Site web CA Logs Monitors Browser Web site
17. 2 Log pre-certificate 1 Ask for a certificate Site web CA Browser Web site Logs Monitors
18. 3 Receive SCT (*) (*) Signed Certificate Timestamp 2 Log pre-certificate 1 Ask for a certificate Site web CA Browser Web site Logs Monitors
19. 4 sends certificate+SCT (*) Signed Certificate Timestamp 3 Receive SCT (*) (*) Signed Certificate Timestamp 2 Log pre-certificate 1 Ask for a certificate Site web CA Browser Web site Logs Monitors
20. 5 (*) Signed Certificate Timestamp 5 4 sends certificate+SCT 3 Receive SCT (*) 2 Log pre-certificate 1 Ask for a certificate Site web CA Web site Logs Monitors Browser TLS request
21. (*) Signed Certificate Timestamp 6 TLS answer with cert + SCT 5 (*) Signed Certificate Timestamp 5 4 sends certificate+SCT 3 Receive SCT (*) 2 Log pre-certificate 1 Ask for a certificate Site web CA Web site Logs Monitors Browser TLS request
22. (*) Signed Certificate Timestamp TLS answer with cert + SCT TLS answer with cert + SCT TLS answer with cert + SCT Chrome 68 requires CT for all certificates signed after 30 April 2018. Safari does it since October 2018. 6 TLS answer with cert + SCT 5 5 4 sends certificate+SCT 3 Receive SCT (*) 2 Log pre-certificate 1 Ask for a certificate Site web CA Web site Logs Monitors Browser TLS request
23. Rechercher des certificats Collecte des certificats (*) Signed Certificate Timestamp (*) Signed Certificate Timestamp TLS answer with cert + SCT TLS answer with cert + SCT TLS answer with cert + SCT Chrome 68 requires CT for all certificates signed after 30 April 2018. Safari does it since October 2018. 6 TLS answer with cert + SCT 5 5 4 sends certificate+SCT 3 Receive SCT (*) 2 Log pre-certificate 1 Ask for a certificate Site web CA Web site Logs Monitors Browser TLS request
24. #3 … for Blue Teams
25. CT : benefits for Blue Teams FQDN (!= DNS)
26. FQDN (!= DNS) Internet wide logging + Opened to all « database » FQDN (!= DNS) FQDN (!= DNS) CT : benefits for Blue Teams
27. #1 Find certificates for our domains hacked / malicious CA → → hacked DNS server (*) → legit web site but not using corporate security best practices (hosting, certificate, DNS etc) * : https://blog.talosintelligence.com/2018/11/dnspionage-campaign-targets-middle-east.html CT : 2 useful (for us) usages
28. #2 Find certificates for « near » domains → phishing campaigns → image damage CT : 2 useful (for us) usages
29. Current choice: → hosted service daily notification → managed by our team → dealing with certificates (efficiency) Our domains monitoring
30. #4 code : CertStreamMonitor
31. Usage #2 : « near » domains monitoring CertStreamMonitor : use CT to monitor threats in « real time » AssuranceMaladieSec
32. CertStreamMonitor.py . works on multi CT logs flow . keywords detection with threshold . real time . runs in daemon mode CertStreamMonitor
33. Tailor your configuration file (conf/filename.conf) → Choose your keywords : ex: apple|account|login → Set your threshold: ex: 2 (defaut value) CertStreamMonitor.py : how it works
34. Tailor your configuration file (conf/filename.conf) → Choose your keywords : ex: apple|account|login → Set your threshold: ex: 2 (defaut value) hostnames with a number of keywords ≥ threshold insert in DB (ex : → login.apple-connect.com) CertStreamMonitor.py : how it works
35. Tailor your configuration file (conf/filename.conf) → Choose your keywords : ex: apple|account|login → Set your threshold: ex: 2 (defaut value) hostnames with a number of keywords ≥ threshold insert in DB (ex : → login.apple-connect.com) hostnames with a number of keywords < threshold but >0 write to log file (ex : webmail. → apple-mail.com) CertStreamMonitor.py : how it works
36. → run on demand (ex. : 1/day) → test all hostnames not already logged as up if hostname is up: → * update DB * JSON report file (ip, AS, abuse email...) scanhost.py : how it works
37. JSON report file scanhost.py : how it works
38. DEMO TIME !
44. Stats : « near » domains monitoring
45. Example #1 : customers abuse cpam-{78,75,13,...}.fr service potentially → abusing our customers (over priced phone number, personal data theft) Results
46. Example #1 : customers abuse cpam-{78,75,13,...}.fr service potentially → abusing our customers (over priced phone number, personal data theft) → service inactivation Results
47. Example #2 : IT management social-ameli.fr . Legit website . Best practices not applied : (domainname, hosting etc) Results
48. TLS, not HTTP – only detect hostnames accessed through TLS RegExp – relying on regexp to find hostnames can lead to miss some of them. Wildcard certificates also beat us. Trust- we use tier service to get CT certificates (Calidog Security in our case). Can we trust it? Limits
49. TLS, not HTTP – only detect hostnames accessed through TLS RegExp – relying on regexp to find hostnames can lead to miss some of them. Wildcard certificates also beat us. Trust- we use tier service to get CT certificates (Calidog Security in our case). Can we trust it? Limits
50. TLS, pas HTTP - détection uniquement des hostnames protégés par TLS RegExp - si le hostname n’a pas de chaînes de caractères contenues dans vos mots clefs pas de détection. → Les certificats wildcards nous mettent aussi en échec. Confiance - le volume de données engendré oblige à passer par des intermédiaires (moniteurs). A qui peut-on faire confiance ? Limites de l'approche
51. low cost tools and services are there, just use them efficiency notified before or soon after the the attacks comes online blind vision at Internet scale Benefits
52. Project: evolution (06/2018 ) →
53. Can choose your CT logs aggregator service end of the dependency → to Calidog Security infra using open source code libre from … Calidog Security <3 Project: evolution (06/2018 ) →
54. Can use a HTTP proxy to connect to the websocket of CT logs aggregator server Project: evolution (06/2018 ) →
55. Setting the threshold for keywords detection is now available in config file Project: evolution (06/2018 ) →
56. Répertoire d’alertes pouvant être hashés date + hostname (PR X. Mertens aka @xme) Project: evolution (06/2018 ) →
57. Thanks! Some questions? https://github.com/AssuranceMaladieSec christophe.brocas@assurance-maladie.fr thomas.damonneville@assurance-maladie.fr @cbrocas | @o0tAd0o
58. Photos credits : Images under Creative Commons licence: danger : https://www.flickr.com/photos/adulau/26003405317/ complexity : https://www.flickr.com/photos/70023venus2009/6032939635 gain : https://www.flickr.com/photos/143106192@N03/29307455407/