CertStreamMonitor
use Certificate Transparency to improve your threats detection
Christophe Brocas
Thomas Damonneville
Caisse Nationale d’Assurance Maladie – Security team
hack-it-n 2018 bis
Bordeaux, 12/11/2018
Slide 2
Slide 2 text
1) Risk / Answer
2) How Certificate Transparency works
3) Benefits for threats monitoring
4) CertStreamMonitor :
usage, results, limits
→
Agenda
Public CA have to submit all certificates they signed to
publicly auditable and accessible, append-only,
cryptographically signed logs.
Certificate Transparency
Slide 12
Slide 12 text
Public CA have to submit all certificates they signed to
publicly auditable and accessible, append-only,
cryptographically signed logs.
Timeline :
2013 : Google (RFC 6962) then IETF (RFC 6962bis)
→
→ 2015 : CT mandatory for EV certificates
→ 30/04/2018 : CT for all certificates
→ 24/07/2018 : interstitial blocking page Chrome 68
→ 15/10/2018 : CT mandatory for Apple products
Certificate Transparency
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
#2 How CT works
Slide 15
Slide 15 text
Site web
CA Logs
Monitors
Browser
Web site
Slide 16
Slide 16 text
1
Ask for a
certificate
Site web
CA Logs
Monitors
Browser
Web site
Slide 17
Slide 17 text
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Browser
Web site
Logs
Monitors
Slide 18
Slide 18 text
3
Receive SCT (*)
(*) Signed Certificate Timestamp
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Browser
Web site
Logs
Monitors
Slide 19
Slide 19 text
4
sends certificate+SCT
(*) Signed Certificate Timestamp
3
Receive SCT (*)
(*) Signed Certificate Timestamp
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Browser
Web site
Logs
Monitors
Slide 20
Slide 20 text
5
(*) Signed Certificate Timestamp
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site
Logs
Monitors
Browser
TLS request
Slide 21
Slide 21 text
(*) Signed Certificate Timestamp
6 TLS answer with cert + SCT
5
(*) Signed Certificate Timestamp
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site
Logs
Monitors
Browser
TLS request
Slide 22
Slide 22 text
(*) Signed Certificate Timestamp
TLS answer with cert + SCT
TLS answer with cert + SCT
TLS answer with cert + SCT
Chrome 68 requires CT for all certificates signed after 30 April 2018.
Safari does it since October 2018.
6 TLS answer with cert + SCT
5
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site
Logs
Monitors
Browser
TLS request
Slide 23
Slide 23 text
Rechercher des certificats
Collecte des certificats
(*) Signed Certificate Timestamp
(*) Signed Certificate Timestamp
TLS answer with cert + SCT
TLS answer with cert + SCT
TLS answer with cert + SCT
Chrome 68 requires CT for all certificates signed after 30 April 2018.
Safari does it since October 2018.
6 TLS answer with cert + SCT
5
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site
Logs
Monitors
Browser
TLS request
Slide 24
Slide 24 text
#3
… for Blue Teams
Slide 25
Slide 25 text
CT : benefits for Blue Teams
FQDN (!= DNS)
Slide 26
Slide 26 text
FQDN (!= DNS)
Internet wide logging
+
Opened to all « database »
FQDN (!= DNS)
FQDN (!= DNS)
CT : benefits for Blue Teams
Slide 27
Slide 27 text
#1 Find certificates for our domains
hacked / malicious CA
→
→ hacked DNS server (*)
→ legit web site but not using corporate security best
practices (hosting, certificate, DNS etc)
* : https://blog.talosintelligence.com/2018/11/dnspionage-campaign-targets-middle-east.html
CT : 2 useful (for us) usages
Current choice:
→ hosted service
daily notification
→
managed by our team
→
dealing with certificates
(efficiency)
Our domains monitoring
Slide 30
Slide 30 text
#4 code : CertStreamMonitor
Slide 31
Slide 31 text
Usage #2 : « near » domains monitoring
CertStreamMonitor :
use CT to monitor threats
in « real time »
AssuranceMaladieSec
Slide 32
Slide 32 text
CertStreamMonitor.py
. works on multi CT logs flow
. keywords detection with
threshold
. real time
. runs in daemon mode
CertStreamMonitor
Slide 33
Slide 33 text
Tailor your configuration file (conf/filename.conf)
→ Choose your keywords : ex: apple|account|login
→ Set your threshold: ex: 2 (defaut value)
CertStreamMonitor.py : how it works
Slide 34
Slide 34 text
Tailor your configuration file (conf/filename.conf)
→ Choose your keywords : ex: apple|account|login
→ Set your threshold: ex: 2 (defaut value)
hostnames with a number of keywords ≥ threshold
insert in DB (ex :
→ login.apple-connect.com)
CertStreamMonitor.py : how it works
Slide 35
Slide 35 text
Tailor your configuration file (conf/filename.conf)
→ Choose your keywords : ex: apple|account|login
→ Set your threshold: ex: 2 (defaut value)
hostnames with a number of keywords ≥ threshold
insert in DB (ex :
→ login.apple-connect.com)
hostnames with a number of keywords < threshold but >0
write to log file (ex : webmail.
→ apple-mail.com)
CertStreamMonitor.py : how it works
Slide 36
Slide 36 text
→ run on demand (ex. : 1/day)
→ test all hostnames not already
logged as up
if hostname is up:
→
* update DB
* JSON report file
(ip, AS, abuse email...)
scanhost.py : how it works
Slide 37
Slide 37 text
JSON report file
scanhost.py : how it works
Slide 38
Slide 38 text
DEMO TIME !
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
Stats : « near » domains monitoring
Slide 45
Slide 45 text
Example #1 :
customers abuse
cpam-{78,75,13,...}.fr
service potentially
→
abusing our customers
(over priced phone
number, personal data
theft)
Results
Slide 46
Slide 46 text
Example #1 :
customers abuse
cpam-{78,75,13,...}.fr
service potentially
→
abusing our customers
(over priced phone
number, personal data
theft)
→ service inactivation
Results
Slide 47
Slide 47 text
Example #2 : IT management
social-ameli.fr
. Legit website
. Best practices not applied :
(domainname, hosting etc)
Results
Slide 48
Slide 48 text
TLS, not HTTP – only detect hostnames accessed through TLS
RegExp – relying on regexp to find hostnames can lead to
miss some of them. Wildcard certificates also beat us.
Trust- we use tier service to get CT certificates (Calidog
Security in our case). Can we trust it?
Limits
Slide 49
Slide 49 text
TLS, not HTTP – only detect hostnames accessed through TLS
RegExp – relying on regexp to find hostnames can lead to
miss some of them. Wildcard certificates also beat us.
Trust- we use tier service to get CT certificates (Calidog
Security in our case). Can we trust it?
Limits
Slide 50
Slide 50 text
TLS, pas HTTP - détection uniquement des hostnames
protégés par TLS
RegExp - si le hostname n’a pas de chaînes de caractères
contenues dans vos mots clefs pas de détection.
→
Les certificats wildcards nous mettent aussi en échec.
Confiance - le volume de données engendré oblige à passer
par des intermédiaires (moniteurs). A qui peut-on faire
confiance ?
Limites de l'approche
Slide 51
Slide 51 text
low cost
tools and services
are there, just use
them
efficiency
notified before
or soon after the
the attacks comes
online
blind
vision at Internet
scale
Benefits
Slide 52
Slide 52 text
Project: evolution (06/2018 )
→
Slide 53
Slide 53 text
Can choose your CT logs
aggregator service
end of the dependency
→
to Calidog Security infra
using open source code
libre from … Calidog
Security <3
Project: evolution (06/2018 )
→
Slide 54
Slide 54 text
Can use a HTTP proxy to
connect to the websocket
of CT logs aggregator
server
Project: evolution (06/2018 )
→
Slide 55
Slide 55 text
Setting the threshold for
keywords detection is now
available in config file
Project: evolution (06/2018 )
→
Slide 56
Slide 56 text
Répertoire d’alertes
pouvant être
hashés date + hostname
(PR X. Mertens aka @xme)
Project: evolution (06/2018 )
→