Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building Offensive Web Security Framework in Python

Building Offensive Web Security Framework in Python

A draft version of slides for a proposed talk at PyCon India 2015

Bharadwaj Machiraju

July 14, 2015
Tweet

More Decks by Bharadwaj Machiraju

Other Decks in Programming

Transcript

  1. Building Offensive Web Security Framework in Python Bharadwaj `tunnelshade` Machiraju

  2. subprocess.check_output(“whoami”).decode(“utf-8”) • Senior @ IIT (BHU) Varanasi • Application Security

    Engineer @Yodlee • Project Leader of OWASP OWTF (along with Abraham Aranguren) • Loves writing security tools • Blogs @ blog.tunnelshade.in
  3. import Agenda; print(Agenda.pycon_in) ▪ Introduction to OWASP and OWTF ▪

    Do we really need another web security tool? ▪ Basic architecture & dependencies ▪ Test categorization and implementation ▪ Some interesting modules & their implementations – HTTP(S) MiTM proxy – Snooping on what all the tools are doing – Ajax web crawler – Crawling dynamic sites for better idea of attack surface – Botnet Mode – Attacking targets with botnet style – Pentester’sTools Parser (PTP) – A python library to parse tool outputs ▪ Extending with own tests and integration ▪ Pre-requisites to know if planning on building a similar one
  4. Open Web Application Security Project • OWASP is an open

    community dedicated to enabling organizations to conceive, develop, acquire, operate, and maintain applications that can be trusted. • All of the OWASP tools, documents, forums, and chapters are free and open to anyone interested in improving application security. • OWASP advocates approaching application security as a people, process, and technology problem because the most effective approaches to application security include improvements in all of these areas. • Popular projects include OWASP Top 10 & OWASP ZAP • www.owasp.org.
  5. Openhub Stats!

  6. Offensive Web Testing Framework • OWTF!! • A project aimed

    to penetration testing efficiency and alignment of security tests to standards like OWASP v4, NIST etc.. • Core framework is written in python2. Openhub stats below!
  7. Is OWTF really needed? • There are many good security

    tools out there, but none of them serve all our purposes. It is practically impossible to create one tool for everything. • So Abraham decided to write a framework which let you run all those tools along with your custom tests. • The main goal was to present all the output in an organised fashion. • Some similar frameworks are Golismero Project (Open Source) & Faraday IDE (Community, Pro & Corp versions)
  8. Basic OWTF Architecture Interface & API Server Proxy Server Worker

    Worker Database Transaction Logger (P) HTTP TRAFFIC API & WEB UI
  9. CORE DB Handler Plugin Helper File Handler Config Handler Plugin

    Handler Developer Perspective HTTP Requester
  10. Python libraries used! • Tornado • pycurl • SQLAlchemy •

    Selenium • BeautifulSoup4 • lxml • markdown • pexpect • psycopg2 • pyOpenSSL • python –owasp-zap-v2 • rdflib • PyVirtualDisplay
  11. Well most developers are careful about XSS, SQLi etc.. But

    does OWTF check for all these??
  12. Extensibility in OWTF • Tests are classified into three main

    categories • WEB • Active • Passive • Semi-Passive • Grep • External • NET • AUX
  13. So what we do? • Provide Core object to the

    plugins, so they have all the features of the framework available. • Import the plugin source using python “imp” module to get the plugin code. • A sample plugin
  14. Modules!!!

  15. HTTP(S) Proxy • Built in Tornado! Why?? Because our pre-implementation

    research said so! • We wanted to build a really fast proxy but at the same time, not implement all the request parsing code. So, decided to use tornado Application instance. • Few of our crazy requirements were • Caching • SSL MiTM • Serving HTTP, HTTPS, WS & WSS on the same port!! • Change the outbound proxy randomly for different requests • Lets dig a bit into internals of this module!
  16. File System as Cache Instance 1 Instance 2 Instance 3

    Instance 4 File System When writing to the file system, a file lock is used and this prevents race conditions. A shared memory for all the instances also exists which is mostly read only.
  17. None
  18. SSL MiTM Web Server 2 Web Server 1 Proxy 8OO8

    Worker X HTTP Traffic Worker Y SSL Session 1 SSL Session 2 Cert. Store (File System) Self-Signed CA
  19. Supporting HTTP & WebSocket • Tornado has “RequestHandler” and “WebSocketHandler”

    classes and instances of these are used to handle http requests and websockets accordingly. • So, when a request arrives at tornado, it will create one of the instances according to the request type and call certain methods of that instance. • Since we are using the application aspect of tornado to build our proxy there was a small problem. • As a proxy we never know on which path you will get a websocket request, so we had the requirement of changing the Handler class once tornado calls for a new object. There is one popular way :P
  20. None
  21. Performance Improvements