Security Regression Tests Victor Pereira Security Engineer @ SUSE 

2 Security Regression Tests Agenda: • ideas behind • existent difficulties • define requirements • proposed solution • conclusions 

3 Security Regression Tests Ideas behind: • Provide a way to self-assess your application without any extra resource • Provide a way to automatically check for new security regression* introduced in your code • Provide a way to reproduce a security regression 

4 Security Regression Tests * Definition of Security Regression Test: • As described by Wikipedia (Software Regression): A software regression is a software bug which makes a feature stop functioning as intended after a certain event (for example, a system upgrade, system patching or a change to daylight saving time) • For us, we are looking for possible vulnerabilities that appear after a new release (or commit) 

5 Security Regression Tests Existent Difficulties: • security tests are expensive • need an expert to execute it and to interpret the results • Existing tools are difficult to integrate with existing test-suite • Not enough literature about it 

6 Security Regression Tests Requirements: To be able to run the security regression tests we will need: • Web application • Existent integration test-suite* using Selenium (or any other web driver with proxy support) • Jenkins and java 

7 Security Regression Tests Existent integration test-suite* But what is Integration tests? • It is the phase in software testing in which individual software modules are combined and tested as a group • It tests all components together 

8 Security Regression Tests Proposed solution (technologies): • Selenium • Ruby* • OWASP Zed Attack Proxy (ZAP) ** • Jenkins *** 

9 Security Regression Tests * Is ruby necessary? • No, its not. Actually ZAP has a HTTP API and originally it supports java and python • But because I know that @ SUSE we use quite often ruby, I developed a client to the API in ruby, to make easier to integrate with existing ruby test infra-structure. • Repository https://github.com/vpereira/ruby-zap 

10 Security Regression Test ** Is ZAP necessary? • No, but the advantages of ZAP are: • Easy to install and run it headless • Powerful security tool with proxy support • Actively being developed • Offer a remote API, making it easier to automate • Extensive security tests • Another option would be the Arachni scanner (any other idea?) 

11 Security Regression Tests Is Jenkins necessary? • No, if you have a different way to run your tests automatically you can use it. • However if you are doing continuous integration, probably you are using it already • You can integrate it to your rake test tasks 

12 Security Regression Tests Start and Stop server: 

13 Security Regression Tests How it works: • configure Selenium to use HTTP Proxy • run the Selenium tests normally and they will be sent through the proxy 

14 Security Regression Tests Selenium with proxy support: 

15 Security Regression Tests Test runs through the proxy. The proxy supports spidering, but works better with Selenium as exploratory test (it stores authentication tokens and etc) How it works (cont.) 

16 Security Regression Tests How it works (cont.) After the selenium tests, we starts via HTTP API the security tests using as baseline the traffic from Selenium 

17 Security Regression Tests Integration with Jenkins: 

18 Security Regression Tests Possible extension • Integrate with threadfix (threadfix.org) • ThreadFix is a software vulnerability aggregation and management system that reduces the time it takes to fix software vulnerabilities. ThreadFix imports the results from dynamic, static and manual testing to provide a centralized view of software security defects across development teams and applications. • It supports reports from different tools, it has a HTTP API to help with the automation and etc. 

19 Security Regression Tests Links: • ZAP Homepage http://www.owasp.org/index.php/ZAP • Selenium Homepage http://docs.seleniumhq.org/ • Jenkins Homepage http://jenkins-ci.org/ • Threadfix Homepage http://threadfix.org 

20 Security Regression Tests Thank You!