Upgrade to Pro — share decks privately, control downloads, hide ads and more …

OSINT tools for security auditing

7c4b1ae16723b56facc7a8a8f95aa6ce?s=47 jmortegac
October 09, 2016

OSINT tools for security auditing

OSINT tools for security auditing

7c4b1ae16723b56facc7a8a8f95aa6ce?s=128

jmortegac

October 09, 2016
Tweet

Transcript

  1. José Manuel Ortega @jmortegac PYCONES 7-9 OCT 2016

  2. None
  3. https://github.com/jmortega/osint_tools_security_auditing

  4. ▪ OSINT introduction ▪ Server information(Censys,Shodan) ▪ OSINT tools developed

    with python ▪ Geolocation,Metadata ▪ Twitter,Footprinting,FullContact
  5. ▪ Define a specific target and data you wish to

    obtain ▪ Technical-Accounts,servers,services,software ▪ Social-Social Media,Email,Photos ▪ Physical-Address,Home IP address,Footprinting ▪ Logical-Network,Operational intelligence
  6. ▪ GeoLocation ▪ IP address ▪ Email address ▪ Telephone

    Number ▪ Usernames in social network profiles ▪ Metadata information from images ▪ Server information & vulnerabilities
  7. None
  8. ▪ https://www.censys.io/api/v1/view/ipv4/<ip_ address> ▪ https://www.censys.io/api/v1/view/websites/<domain>

  9. None
  10. None
  11. None
  12. None
  13. ▪ Checking data with ip address ▪ https://www.shodan.io/host/144.76.246.116

  14. None
  15. https://developer.shodan.io/api

  16. ▪ https://bitbucket.org/LaNMaSteR53/recon-ng ▪ Open Source OSINT toolkit written in python

    ▪ Actively maintained ▪ Uses modules and saves all recollected information in databases
  17. ▪ dnspython - http://www.dnspython.org/ ▪ dicttoxml - https://github.com/quandyfactory/dicttoxml/ ▪ jsonrpclib

    - https://github.com/joshmarshall/jsonrpclib/ ▪ lxml - http://lxml.de/ ▪ slowaes - https://code.google.com/p/slowaes/ ▪ XlsxWriter - https://github.com/jmcnamara/XlsxWriter/ ▪ Mechanize ▪ PyPDF2 ▪ sqlite3
  18. None
  19. None
  20. None
  21. None
  22. None
  23. None
  24. None
  25. ▪ https://github.com/laramies/theHarvester

  26. None
  27. ▪ httplib ▪ socket ▪ requests ▪ shodan

  28. None
  29. ▪ pip install osrframework ▪ Developed in python 2.7 ▪

    Integrates with maltego transforms ▪ https://pypi.python.org/pypi/osrframework/0.13.2 ▪ https://github.com/i3visio/osrframework
  30. ▪ BeautifulSoup ▪ Requests ▪ Mechanize ▪ pyDNSresolving name servers

    ▪ python-whoisto recover the whois info from a domain ▪ tweepyfor connecting with Twitter API ▪ Skype4Py for connecting with Skype API ▪ Python-emailahoyfor checking email address ▪ Multiprocessingimport Process, Queue, Pool
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. None
  39. Source Location Notes abuse.ch http://www.abuse.ch Various malware trackers. AdBlock https://easylist-

    downloads.adblockplus.org/easylist.txt AdBlock pattern matches AlienVault https://reputation.alienvault.com AlienVault’s IP reputation database. Autoshun.org http://www.autoshun.org Blacklists. AVG Site Safety Report http://www.avgthreatlabas.com Site safety checker. Bing http://www.bing.com Scraping but future version to also use API. Blocklist.de http://lists.blocklist.de Blacklists. Checkusernames.com http://www.checkusernames.com Look up username availability on popular sites. DNS Your configured DNS server. Defaults to your local DNS but can be configured to whatever IP address you supply SpiderFoot. DomainTools http://www.domaintools.com DroneBL http://www.dronebl.org Facebook http://www.facebook.com Scraping but future version to also use API. FreeGeoIP http://freegeoip.net Google http://www.google.com Scraping but future version to also use API. Google+ http://plus.google.com Scraping but future version to also use API. Google Safe Browsing http://www.google.com/safebrowsing Site safety checker. LinkedIn http://www.linkedin.com Scraping but future version to also use API. malc0de.com http://malc0de.com Blacklists. malwaredomainlist.com http://www.malwaredomainlist.com Blacklists.
  40. Source Location Notes malwaredomains.com http://www.malwaredomains.com Blacklists. McAfee SiteAdvisor http://www.siteadvisor.com Site

    safety checker. NameDroppers http://www.namedroppers.org Nothink.org http://www.nothink.org Blacklists. OpenBL http://www.openbl.org Blacklists. PasteBin http://www.pastebin.com Achieved through Google scraping. PGP Servers http://pgp.mit.edu/pks/ PGP public keys. PhishTank http://www.phishtank.org Identified phishing sites. Project Honeypot http://www.projecthoneypot.org Blacklists. API key needed. RIPE/ARIN http://stat.ripe.net/ Robtex http://www.robtex.com SANS ISC http://isc.sans.edu Internet Storm Center IP reputation database. SHODAN http://www.shodanhq.com API key needed. SORBS http://www.sorbs.net Blacklists. SpamHaus http://www.spamhaus.org Blacklists. ThreatExpert http://www.threatexpert.com Blacklists. TOR Node List http://torstatus.blutmagie.de TotalHash.com http://www.totalhash.com Domains/IPs used by malware. UCEPROTECT http://www.uceprotect.net Blacklists. VirusTotal http://www.virustotal.com Domains/IPs used by malware. API key needed. Whois Various Whois servers for different TLDs. Yahoo http://www.yahoo.com Scraping but future version to also use API. Zone-H http://www.zone-h.org Easy to get black-listed. Log onto the site in a browser from the IP you’re scanning from first and enter the CAPTCHA, then it should be fine.
  41. ▪ Python 2.7 ▪ BeautifulSoup ▪ DNSPython ▪ Socks ▪

    Socket ▪ SSL ▪ CherryPy ▪ M2MCrypto ▪ Netaddr ▪ pyPDF
  42. from bs4 import BeautifulSoup, SoupStrainer

  43. None
  44. None
  45. None
  46. ▪ PDFPyPDF2,PDFMiner ▪ ImagesPillow,pyexiv2(python 2.7),gexiv2(python 3)

  47. import geoip2 import geoip2.database http://dev.maxmind.com/geoip/geoip2/geolite2/

  48. ▪ Orb(Python 2.x) • https://github.com/epsylon/orb • python-whois - Python module

    for retrieving WHOIS information • python-dnspython - DNS toolkit for Python • python-nmap - Python interface to the Nmap port scanner • InstaRecon(Python 2.x) • https://github.com/vergl4s/instarecon • dnspython,ipaddress • ipwhois,python-whois • requests,shodan
  49. None
  50. None
  51. ▪ BeautifulSoup for parsing web information ▪ Requests,urllib3 for synchronous

    requests ▪ Asyncio,aiohttp for asynchronous requests ▪ Robobrowser,Scrapy for web crawling ▪ PyGeoIP,geoip2,geojson for GeoLocation ▪ python-twitter,tweepy for connecting with twitter ▪ Shodan for obtain information for servers ▪ DNSPython,netaddr for resolving ip address
  52. None
  53. python tinfoleak.py pycones -i -s --sdate 2016/01/01 --hashtags --mentions --meta

    --media [d] --geo GEOFILE --top 10 -o report.html
  54. None
  55. ▪ import tweepyTwitter API library for Python ▪ from PIL

    import Image, ExifTags, ImageCmsmetadata from images ▪ import pyexiv2metadata from images ▪ import urllib2requests ▪ from OpenSSL import SSL ▪ from jinja2 import Template, Environment, FileSystemLoaderreport
  56. None
  57. None
  58. None
  59. None
  60. None
  61. ▪ We know we have a valid email address ▪

    What other profiles are associated with this address? ▪ Go to fullcontact.com for an API key…..
  62. None
  63. None
  64. None
  65. None
  66. None
  67. ▪ https://sourceforge.net/projects/spiderfoot ▪ http://www.edge-security.com/theharvester.php ▪ https://developer.shodan.io/api ▪ http://www.clips.ua.ac.be/pattern ▪ http://www.pentest-

    standard.org/index.php/PTES_Technical_Guidelines#OSINT ▪ http://www.vicenteaguileradiaz.com/tools ▪ https://github.com/automatingosint/osint_public ▪ http://www.automatingosint.com/blog/
  68. None
  69. Thanks! @jmortegac AMSTERDAM 9-12 MAY 2016