Botconf 2019 OSINT 101

OSINT 101 section from Botconf 2019 workshop

Transcript

1. OSINT hunting
2. Basic Knowledge & Concepts
3. Good Friends With Bad Habits • Attackers are good friends with bad habits. ◦ Reusing infrastructures ▪ Same IP, same domain ◦ Reusing components ▪ Same software, same HTML, same tracker ◦ Reusing SSL certificates ◦ Reusing SSH keys • Reusing something increases a possibility of tracking. ◦ Let’s say it's a fingerprint of an attacker. ◦ You can track him down based on his fingerprint.
4. Data Sources / Fingerprints
5. Search Engines • IPv4(v6) search engines: ◦ Censys ◦ Shodan ◦ Onyphey ◦ BinaryEdge • Search engines for IP, domain, URL, hash, etc. ◦ RiskIQ Community / PassiveTotal ◦ DomainTools ◦ VirusTotal ◦ SecurityTrails ◦ urlscan.io ◦ crt.sh
6. OSINT? (souce: https://twitter.com/infosystir/status/1086394141234421760)
7. Search Engines 101
8. Shodan 101 • Shodan crawls the entire Internet at least once a month. • Search query syntax: ◦ filtername:value ◦ Logical operators: ▪ +, - ◦ Query examples: ▪ country:FR ▪ country:FR +port:80 • You can scan a host manually. (If you are a paid user)
9. Censys 101 • Censys scans IPv4, popular websites and certificates. ◦ Censys performs all IPv4 scan at least once a week and scan popular websites daily. ◦ Censys monitors certificates in near real time with leveraging Certificate Transparency. • Search query syntax: ◦ filtername:value ◦ Logical operators: ▪ AND, OR, NOT ◦ Query examples: ▪ location.country_code:FR ▪ location.country_code:FR AND ports:80
10. Onyphe 101 • Onyphe crawls the Internet at least once a month. • Other unique features: ◦ Paste sites lookup, dark web crawling, historical records to search, etc. • Search query syntax: ◦ filtername:value ◦ Query examples: ▪ country:FR ▪ country:FR port:80 ▪ Functions: • -wildcard, -hourago, -dayago, -weekago, -monthago ◦ country:FR port:80 -wildcard:hostname,*ovh* -monthago:6
11. BinaryEdge 101 • BinaryEdge crawls the entire Internet at least once a month. • Other unique features: ◦ DHT(Distributed Hash Table) activity, data leaks, risk score, honeypots data, etc. • Search query syntax: ◦ filtername:value ◦ Logical operators: ▪ AND, OR, NOT ◦ Query examples: ▪ country:FR ▪ country:FR AND port:80
12. RiskIQ 101 • RiskIQ provides search functions for Passive DNS, components, trackers, WHOIS, certificates and cookies. ◦ Components mean server-side / client side technologies (e.g. Nginx, jQuery, etc.) ◦ Trackers mean analytics trackers (e.g. Google Analytics)
13. VirusTotal 101 • VirusTotal provides search functions for files, URLs, domains and IPs.
14. VirusTotal 101 • VirusTotal data (which can be used outside of the paid wall) ◦ Passive DNS ◦ Detection data ▪ IP, domain, URL and hash ◦ Sandbox data
15. VirusTotal 101 • Relations and Behavior sections contain sandbox data.
16. urlscan.io 101 • urlscan.io provides scan and search functions for websites. ◦ urlscan.io data include manual submissions and automatic-submissions. ▪ Automatic-submissions: • urlscan.io scans URLs from OpenPhish, PhishTank, URLhaus, etc. with auto. ◦ You can use ElasticSearch syntax to search. ▪ domain:kuronekoyamao.com ▪ domain:jppost-*.top AND page.country:FR ◦ Be careful with the limitations. ▪ urlscan.io scans a website via rotating European VPN exit IPs. ▪ Default UA equals to the latest Google Chrome Stable on Mac OS X.