Upgrade to Pro — share decks privately, control downloads, hide ads and more …

No todo es DNS: enumeración de software en tiempos modernos

alertot
October 26, 2017

No todo es DNS: enumeración de software en tiempos modernos

alertot

October 26, 2017
Tweet

More Decks by alertot

Other Decks in Programming

Transcript

  1. No todo es DNS:
    enumeración de software
    en tiempos modernos
    Claudio Salazar

    View Slide

  2. Whois
    ● CEO alertot (SUP18)
    ● Founder spect.cl
    ★ Web Security Workshop, UTFSM (2013)
    ★ Pinguinux’s member

    View Slide

  3. Agenda
    1. Enumeration techniques
    2. Software’s enumeration state of the art
    3. detectem, the project
    4. Long-term challenges

    View Slide

  4. Enumeration techniques
    [ ref: https://www.slideshare.net/bugcrowd/ekoparty-2017-the-bug-hunters-methodology/ ]

    View Slide

  5. Enumeration techniques - OSINT
    jhaddix/domain
    aboul3la/Sublist3r

    View Slide

  6. Enumeration techniques - Port scanning
    nmap
    masscan

    View Slide

  7. Enumeration techniques - Port scanning

    View Slide

  8. Enumeration techniques - Web software
    urbanadventurer/WhatWeb
    AliasIO/Wappalyzer
    builtwith.com

    View Slide

  9. Enumeration techniques - Web software

    View Slide

  10. Software’s enumeration state
    of the art

    View Slide

  11. State of the art - Status

    View Slide

  12. State of the art - Little development

    View Slide

  13. State of the art - No Javascript support

    View Slide

  14. State of the art - False positives

    Use: https://code.jquery.com/jquery-3.2.1.js

    View Slide

  15. State of the art - False positives

    View Slide

  16. State of the art - False positives

    View Slide

  17. State of the art - Lack of testing suite

    View Slide

  18. detectem, the project

    View Slide

  19. Detectem - Goals
    ● Reliable version extraction
    ● Avoid false positives as much as possible
    ● Work on sites with intensive use of
    Javascript
    ● Have Test Driven Development as rule

    View Slide

  20. Detectem - Features
    ● Passive detection with a browser (Splash)
    ● Works on the list of requests/responses (HAR)
    ● Javascript support through Splash
    ● Based on a plugin system
    ● Command-line executable & Web service

    View Slide

  21. detectem - Architecture

    View Slide

  22. detectem - demo

    View Slide

  23. Detectem - Plugin system
    ● Idea taken from WhatWeb
    ● Provide enough flexibility for corner cases
    ● Plugins contain metadata
    ● Plugins contain matchers, indicators and
    hints

    View Slide

  24. Detectem - Matchers
    ● Unit in charge of version extraction
    ● Could be simple regular expressions or functions
    ● Adapt to the context to avoid false positives
    ● Main strongness in the framework

    View Slide

  25. How it works - URL matcher
    https://code.jquery.com/jquery-3.1.1.js

    View Slide

  26. How it looks - URL matcher
    class JqueryPlugin(Plugin):
    name = 'jquery'
    homepage = 'https://jquery.com/'
    matchers = [
    {'url': ‘/jquery-(?P[0-9\.]+)\.js'},
    ]

    View Slide

  27. How it works - URL matcher scope
    Every request/response url
    except the first one

    View Slide

  28. How it works - Body matcher
    https://code.jquery.com/jquery.js
    /*! * jQuery JavaScript Library v3.1.1

    View Slide

  29. How it looks - Body matcher
    class JqueryPlugin(Plugin):
    name = 'jquery'
    homepage = 'https://jquery.com/'
    matchers = [
    {'body': '/\*\! jQuery v(?P[0-9\.]+)'},
    ]

    View Slide

  30. How it works - Body matcher scope
    Every response
    except the first one

    View Slide

  31. How it works - Header matcher
    Server: Apache/2.4.18 (Ubuntu)

    View Slide

  32. How it looks - Header matcher
    class Apache(Plugin):
    name = 'apache'
    homepage = 'http://httpd.apache.org/'
    matchers = [{
    'header': (
    'Server',
    'Apache/(?P[0-9\.]+)'
    )
    }]

    View Slide

  33. How it works - Header matcher scope
    Only first response

    View Slide

  34. How it works - XPath matcher
    name="generator"
    content="Ghost 0.11"
    >

    View Slide

  35. How it looks - XPath matcher
    class GhostPlugin(Plugin):
    name = 'ghost'
    homepage = 'https://www.ghost.org/'
    matchers = [{
    ‘xpath': (
    meta_generator('Ghost'),
    '(?P[0-9\.]+)'
    )
    }]

    View Slide

  36. How it works - XPath matcher scope
    Only first response

    View Slide

  37. How it works - DOM matcher
    > d3
    <- {version: "3.5.17", ascending: ƒ, ...}
    > d3.version
    <- "3.5.17"

    View Slide

  38. How it looks - DOM matcher
    class D3JSPlugin(Plugin):
    name = 'd3.js'
    homepage = 'https://d3js.org'
    js_matchers = [{
    'check': 'window.d3',
    'version': 'window.d3.version'
    }]

    View Slide

  39. How it works - DOM matcher scope
    Page DOM

    View Slide

  40. detectem - Additional information
    Indicators
    Hints

    View Slide

  41. detectem - Indicator mission
    Indicates the presence of a software
    but not its version

    View Slide

  42. detectem - Indicator look
    class NewsShowProGk5Plugin(Plugin):
    name = 'news-show-pro-gk5'
    homepage = 'https://gavick.com/news-show-pro'
    indicators = [
    {'body': '\* @package News Show Pro GK5\n'},
    ]

    View Slide

  43. detectem - Indicator return value
    {
    'from_url': ‘http://d.tld/mod_news_pro_gk5/sty.css',
    'homepage': 'https://www.gavick.com/news-show-pro',
    'name': 'news-show-pro-gk5',
    'type': 'indicator'
    }

    View Slide

  44. detectem - Hint mission
    A software is hinted
    in presence of a software
    directly related to it.

    View Slide

  45. detectem - Hint look
    class NewsShowProGk5Plugin(Plugin):
    name = 'news-show-pro-gk5'
    homepage = 'https://gavick.com/news-show-pro'
    hints = ["joomla"]

    View Slide

  46. detectem - Hint return value
    {
    'from_url': 'https://domain.tld',
    'homepage': 'https://www.joomla.org/',
    'name': 'joomla',
    'type': 'hint'
    }

    View Slide

  47. Long-term challenges

    View Slide

  48. Challenges - Web is Wild
    ● Multiple versions in the same page
    ● Version pollution
    ● There’s no standard software

    View Slide

  49. Long-term challenges
    ● Increase number of supported softwares
    ● Improvement from NLP/ML/AI
    ● Add concurrency
    ● Improve performance at scale

    View Slide

  50. Challenges - Some news
    ● Migrate Wappalyzer DB
    ● Add bundle splitting feature
    ● Add DOM hash matcher
    ● Tests with Chrome headless

    View Slide

  51. github.com/alertot/detectem

    View Slide