Certificate Transparency
& threats detection
24 months later
Christophe Brocas
Thomas Damonneville
Caisse Nationale d’Assurance Maladie – Security Department
Toulouse Hacking Convention
Toulouse | 08/03/2019
Slide 2
Slide 2 text
1) Certificate Transparency
- Risk / Answer
- How Certificate Transparency works
2) Benefits for threats monitoring
- Usages for blue teams
- CertStreamMonitor
3) CT & threats monitoring: a 24 months story
Agenda
Public CA have to submit all certificates they signed to
publicly auditable and accessible, append-only,
cryptographically signed logs.
Certificate Transparency
Slide 12
Slide 12 text
Public CA have to submit all certificates they signed to
publicly auditable and accessible, append-only,
cryptographically signed logs.
Timeline :
2013 : Google (RFC 6962) then IETF (RFC 6962bis)
→
→ 2015 : CT mandatory for EV certificates
→ 30/04/2018 : CT for all certificates
→ 24/07/2018 : interstitial blocking page Chrome 68
→ 15/10/2018 : CT mandatory for Apple products
Certificate Transparency
Slide 13
Slide 13 text
No content
Slide 14
Slide 14 text
How CT works
Slide 15
Slide 15 text
Site web
CA Logs
Monitors
Browser
Web site
Slide 16
Slide 16 text
1
Ask for a
certificate
Site web
CA Logs
Monitors
Browser
Web site
Slide 17
Slide 17 text
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Browser
Web site
Logs
Monitors
Slide 18
Slide 18 text
3
Receive SCT (*)
(*) Signed Certificate Timestamp
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Browser
Web site
Logs
Monitors
Slide 19
Slide 19 text
4
sends certificate+SCT
(*) Signed Certificate Timestamp
3
Receive SCT (*)
(*) Signed Certificate Timestamp
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Browser
Web site
Logs
Monitors
Slide 20
Slide 20 text
5
(*) Signed Certificate Timestamp
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site
Logs
Monitors
Browser
TLS request
Slide 21
Slide 21 text
(*) Signed Certificate Timestamp
6 TLS answer with cert + SCT
5
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site
Logs
Monitors
Browser
TLS request
Slide 22
Slide 22 text
(*) Signed Certificate Timestamp
TLS answer with cert + SCT
TLS answer with cert + SCT
TLS answer with cert + SCT
Chrome 68 requires CT for all certificates signed after 30 April 2018.
Safari does it since October 2018.
6 TLS answer with cert + SCT
5
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site
Logs
Monitors
Browser
TLS request
Slide 23
Slide 23 text
Searching for certificates
Collecting certificates
(*) Signed Certificate Timestamp
(*) Signed Certificate Timestamp
TLS answer with cert + SCT
TLS answer with cert + SCT
TLS answer with cert + SCT
Chrome 68 requires CT for all certificates signed after 30 April 2018.
Safari does it since October 2018.
6 TLS answer with cert + SCT
5
5
4
sends certificate+SCT
3
Receive SCT (*)
2
Log pre-certificate
1
Ask for a
certificate
Site web
CA
Web site Monitors
Browser
TLS request
Logs
Slide 24
Slide 24 text
… for Blue Teams
Slide 25
Slide 25 text
CT : benefits for Blue Teams
FQDN (!= DNS)
Slide 26
Slide 26 text
FQDN (!= DNS)
Internet wide logging
+
Open access to the data
FQDN (!= DNS)
FQDN (!= DNS)
CT : benefits for Blue Teams
Slide 27
Slide 27 text
#1 Find certificates for our domains
hacked / malicious CA
→
→ hacked DNS server (*)
→ legit web site but not using corporate security best
practices (hosting, certificate, DNS etc)
* : https://blog.talosintelligence.com/2018/11/dnspionage-campaign-targets-middle-east.html
CT: 2 useful usages (for us)
Slide 28
Slide 28 text
#2 Find certificates for « near » domains
→ phishing campaigns
→ image damage
CT: 2 useful usages (for us)
Slide 29
Slide 29 text
Current choice:
→ hosted service
daily notification
→
managed by our team
→
dealing with certificates
(efficiency)
Usage #1: our domains monitoring
Slide 30
Slide 30 text
Code: CertStreamMonitor
Slide 31
Slide 31 text
Usage #2: « near » domains monitoring
CertStreamMonitor :
use CT to monitor threats
in « real time »
AssuranceMaladieSec
Slide 32
Slide 32 text
CertStreamMonitor.py
. works on multi CT logs flow
. keywords detection with
threshold
. real time
. runs in daemon mode
CertStreamMonitor
Slide 33
Slide 33 text
Tailor your configuration file (conf/filename.conf)
→ Choose your keywords : ex: apple|account|login
→ Set your threshold: ex: 2 (defaut value)
CertStreamMonitor.py: how it works
Slide 34
Slide 34 text
Tailor your configuration file (conf/filename.conf)
→ Choose your keywords : ex: apple|account|login
→ Set your threshold: ex: 2 (defaut value)
hostnames with a number of keywords ≥ threshold
insert in DB (ex :
→ login.apple-connect.com)
CertStreamMonitor.py: how it works
Slide 35
Slide 35 text
Tailor your configuration file (conf/filename.conf)
→ Choose your keywords : ex: apple|account|login
→ Set your threshold: ex: 2 (defaut value)
hostnames with a number of keywords ≥ threshold
insert in DB (ex :
→ login.apple-connect.com)
hostnames with a number of keywords < threshold but >0
write to log file (ex : webmail.
→ apple-mail.com)
CertStreamMonitor.py: how it works
Slide 36
Slide 36 text
→ run on demand (ex. : 1/day)
→ test all hostnames not already
logged as up
if hostname is up:
→
* update DB
* JSON report file
(ip, AS, abuse email...)
scanhost.py: how it works
Slide 37
Slide 37 text
JSON report file
scanhost.py: how it works
Slide 38
Slide 38 text
Screenshots are not a demo
Slide 39
Slide 39 text
No content
Slide 40
Slide 40 text
No content
Slide 41
Slide 41 text
No content
Slide 42
Slide 42 text
No content
Slide 43
Slide 43 text
No content
Slide 44
Slide 44 text
Example #1 :
customers abuse
cpam-{78,75,13,...}.fr
service potentially
→
abusing our customers
(over priced phone
number, personal data
theft)
Results
Slide 45
Slide 45 text
Example #1 :
customers abuse
cpam-{78,75,13,...}.fr
service potentially
→
abusing our customers
(over priced phone
number, personal data
theft)
→ service inactivation
Results
Slide 46
Slide 46 text
Example #2 : IT management
social-ameli.fr
. Legit website
. Best practices not applied :
(domainname, hosting etc)
Results
Slide 47
Slide 47 text
TLS, not HTTP – only detect hostnames accessed through TLS
RegExp – relying on regexp to find hostnames can lead to
miss some of them. Wildcard certificates also beat us.
Trust- we use tier service to get CT certificates (Calidog
Security in our case). Can we trust it?
Limits
Slide 48
Slide 48 text
TLS, not HTTP – only detect hostnames accessed through TLS
RegExp – relying on regexp to find hostnames can lead to
miss some of them. Wildcard certificates also beat us.
Trust- we use tier service to get CT certificates (Calidog
Security in our case). Can we trust it?
Limits
Slide 49
Slide 49 text
TLS, not HTTP – only detect hostnames accessed through TLS
RegExp – relying on regexp to find hostnames can lead to
miss some of them. Wildcard certificates also beat us.
Trust- we use tier service to get CT certificates (Calidog
Security in our case). Can we trust it?
But we rely on their code, a potential single point of failure.
it is a
→ call for action to the Infosec community
Limits
Slide 50
Slide 50 text
CertStreamMonitor evolution in 9 months
Slide 51
Slide 51 text
No content
Slide 52
Slide 52 text
You can now choose your
CT logs aggregator
service :
* Calidog Security one
* your own using Calidog
code
Slide 53
Slide 53 text
Setting the threshold for
keywords detection is
available in config file
Slide 54
Slide 54 text
(optional) check Google
SafeBrowsing status of
the hostname
Slide 55
Slide 55 text
Name of the alerts
directory can be hashed
with date + hostname
(PR of @xme)
Slide 56
Slide 56 text
(optional) notification
by mail or instant
messaging like Slack or
Rocket.
Slide 57
Slide 57 text
CT & Threats Monitoring:
a 24 months story
Slide 58
Slide 58 text
April 2017
The
announcement
Slide 59
Slide 59 text
June 2017
Why CT
becomes
interesting?
Slide 60
Slide 60 text
01/2015 : 31 %
June 2017
Slide 61
Slide 61 text
01/2015 : 31 %
06/2017 : 57 %
→ 83% of growth
in 2,5 years
June 2017
Slide 62
Slide 62 text
Nov. 2017
First tools
show up
Slide 63
Slide 63 text
May 2018
More complex
tools
+ CT required for
all certificates
Slide 64
Slide 64 text
July 2018
Chrome
implements
CT as a strict
requirement
Slide 65
Slide 65 text
Nov. 2018
When CT
becomes a
DNS hacks
detection
tool
Slide 66
Slide 66 text
Nov. 2018
When CT
becomes a
DNS hacks
detection
tool
Slide 67
Slide 67 text
Jan. 2019
CT appears in Blue Teams
best practices
Slide 68
Slide 68 text
Feb. 2019
CT is point out as one of
the tools able to control
TLS grey/dark activities
Slide 69
Slide 69 text
low cost
tools and services
are there, just use
them
efficiency
notified before
or soon after the
the attacks comes
online
blind
vision at Internet
scale
+ bonus track: compliance
CT monitoring is now part of best practices requirements