Slide 1

Slide 1 text

Ripping Apart Surface and Dark Web via OSINT

Slide 2

Slide 2 text

#whoami πŸ‘‰ Ashwani kumar a.k.a CyberK@Lki πŸ‘‰ Hobbies – Gaming & finding new places to go solo πŸ‘‰ Interest area – OSINT, Network Pentest, Malware analysis, Social engineering , DevSecOps, Geopolitics & Quantum physics πŸ‘‰ Professional CS GO & Fortnite Player : Nick : GodWA πŸ‘‰ Certified ICS Security skills from US Homeland Security & CISA πŸ‘‰ Reported Scada bugs for BSNL, Railtel & other National critical infrastructures πŸ‘‰ Provide OSINT training and consultancy to govt agencies & Private entities πŸ‘‰ Work closely with CyberPeace NGO helping Indian masses to stay safe in cyberspace and build next-gen cyber warriors 2 Website – hack.cyberkalki.com

Slide 3

Slide 3 text

Key takeaways World Wide Web is composed of the three layers: Surface Web, Deep Web and Darknet. ● Surface Web covers only 5 % of the World Wide Web. ● Deep Web includes any kind of web services and constitutes a very interesting data source for OSINT investigations. ● Darknet can be accessed only by using dedicated technology, such as TOR or I2P.

Slide 4

Slide 4 text

OSINT integration with DML model

Slide 5

Slide 5 text

Surface Web Efficient OSINT analysis requires Federated Search including the following data sources: ● Search engines, such as Google, Bing, Yahoo, and others. ● Deep web search engines, such as DuckDuckGo. ● Social media, for example Twitter, Weibo, Instagram, Facebook, or LinkedIn. ● RSS feeds from websites of interest. ● New Data-as-a-Service providers, e.g. for financial figures or other information.

Slide 6

Slide 6 text

Surface Web Federated search retrieves information from a variety of sources via a search application built on top of one or more search engines whereas metasearch engine is an online Information retrieval tool that take input from a user and immediately query search engines for results. Sufficient data is gathered, ranked, and presented to the users. Ref: OSINT Open Source Intelligence tools resources methods techniques

Slide 7

Slide 7 text

Deep Web Contents on the Deep Web cannot be indexed because: ● it is either protected with a password, such as your cloud storage, webmail solution, digital libraries, online magazines, or newspapers. ● or it is stored behind web services or APIs preventing direct access to the raw data. ● There are many deep web data sources available: ● Google patent database ● Google academic database ● EU Sanction Lists ● HaveIBeenPwned The extracted deep-web links helps OSINT analyst in use cases like crypto, fraud, criminal intelligence investigation scenarios to articulate and correlate hidden data points.

Slide 8

Slide 8 text

Dark Web - The Cult of OSINT The dark web is a subset of the internet that is accessed via special means, such as a TOR browser, and not immediately available from the clear net. Another common darknet is Zeronet. Each has different access requirements or methods. Different darknet details: ● TOR: https://www.torproject.org ● Zeronet: https://zeronet.io/ ● I2P: https://geti2p.net/en/ ● Freenet: https://freenetproject.org/index.html

Slide 9

Slide 9 text

Dark Web - Illegal Services

Slide 10

Slide 10 text

Dark Web - Legal Services

Slide 11

Slide 11 text

Is TOR Really Safe ?

Slide 12

Slide 12 text

How is traffic on TOR network ? Torflow is an uncharted app which places each relay on a world map and illustrate traffic exchanged between relays as animated dots. Similarly, tor-metrics shows list of exit nodes as per geolocation tagged. Some investigators will have a requirement to identify & monitor new .onion sites as they arise. This could be to observe patterns, identify new vectors, or simply to create additional pipelines of new .onion URLs to feed into custom crawling engines for advanced users.

Slide 13

Slide 13 text

DarkWeb TTP Tools , techniques and tactics for OSINT Investigation

Slide 14

Slide 14 text

Dark Web - Marketplace to watch ● General Markets ● PII & PHI ● Credit Cards ● Digital identities ● Information Trading ● Remote Access ● Personal Documents ● Electronic Wallets ● Insider Threats

Slide 15

Slide 15 text

Dark Web - Sites examples

Slide 16

Slide 16 text

Dark Web - Product examples

Slide 17

Slide 17 text

Dark Web - Automating Threat hunting

Slide 18

Slide 18 text

Dark Web - Automating Crawler

Slide 19

Slide 19 text

Dark Web - Tools to try out ● Scrapy ● Tor ● OnionScan ● Privoxy ● Elastic ● Redis ● Torbot ● OnionSearch ● Darkdump / Darksearch ● Maltego / Lampyre ● Hunchly ● Searchlight / Spectrum ● OnionIngestor / Poopak : Hidden service crawler

Slide 20

Slide 20 text

Dark Web - Scrapy

Slide 21

Slide 21 text

Dark Web : Katana Scanner Katana-ds (ds for dork_scanner) is a simple python tool that automates Google Hacking/Dorking and supports Tor. It becomes a more powerful in combination with GHDB. IT supports google dorking for finding exposure points & enumeration, help finding exposed PLC and SCADA devices with tor and proxy support.

Slide 22

Slide 22 text

Dark Web : Crawler & bots TorCrawl.py is a python script to crawl and extract (regular or onion) webpages through TOR network. Similarly, TorBot is an open source intelligence tool developed in python. The main objective of this project is to collect open data from the deep web (aka dark web) and with the help of data mining algorithms, collect as much information as possible and produce an interactive tree graph. The interactive tree graph module will be able to display the relations of the collected intelligence data. On same pattern, onionscan and other tools are made and widely used with variant features and scope of data intel.

Slide 23

Slide 23 text

Dark Web OnionScan OnionScan is a free and open source tool for investigating the Dark Web. It helps to detect :- ● Build a better fingerprint of your server, including php and other software versions. ● Determine client IP addresses if you are co-hosting a clearnet site. ● Determine your IP address if your setup allows. ● Determine other sites you are co-hosting. ● Determine how active your site is. ● Find secret or hidden areas of your site ● Open Directories ● Server Fingerprint ● Analytics IDs ● Protocol Detection

Slide 24

Slide 24 text

Dark Web : Dumping darkweb links Tools like Onionsearch, darkdump, darksearch performs scanning of top darknet forums, websites and marketplaces against provided keywords and collate it to user for further analysis.

Slide 25

Slide 25 text

Dark Web :Monitoring hidden services

Slide 26

Slide 26 text

Q/A Session

Slide 27

Slide 27 text

Next Session Agenda ● Understanding Darkweb and TOr ● Planning and readiness for conducting darkweb investigation ● Strategies and approaches based on use cases ● Investigative workflow ● Toolkit and tactics ● Reporting and concerns ● Challenges and workarounds

Slide 28

Slide 28 text

Follow my social media handles & website i.am.psy.ber_kalki