AppSec Broken Windows Theory - Why we are winning battles but losing the war

Conventional wisdom says fix the most severe flaws first and triage the rest. While this may impact the security of one application, it won’t improve AppSec over the long term. During this presentation Erik Peterson proposes applying the sociological principles of Broken Window Theory to AppSec to change the development culture within an organization, making long term improvements to AppSec.

Transcript

1. AppSec & Broken Window Theory Why we are wining battles but losing the war Erik Peterson - February 2014 - BSidesSF
2. About Your Speaker • Hi, I’m Erik Peterson • AppSec guy for 16 years now, currently I work at Veracode • If you have used a dynamic scanner, in some way i’ve probably influenced it’s design • I’ve directly contributed to the design or development of AppScan, WebInspect, Qualys WAS and Veracode Discovery, DynamicDS & DynamicMP email: epeterson@veracode.com twitter: @silvexis
3. Software Security is the greatest challenge of all humankind
4. And we are failing
5. Our Journey Today • Application Security as a discipline, has performed poorly • I have really wanted to know why • Sneaking suspicion that it might be my fault
6. Fortune 100
 An AppSec Failure • Automated dynamic analysis tools have been available for over 10 years now • At least 82% of the Fortune 100 use a dynamic scanning tool* • Every Fortune 100 company to suffer a web site breach in the last 5 years uses a dynamic scanning tool* • Most however are not scanning all of their
 web applications and this is where they
 are getting hit.** **My personal observations, not a formal study *Based on a review of known AppScan, WebInspect & Veracode customers
7. ‣Too Expensive ‣Headcount ‣Scanning costs ‣Permission ‣Arranging scan windows ‣No security mandate ‣Poor Results ‣Too much noise in reports ‣Don’t know what to do with the data ‣Can’t keep up ‣New sites are being deployed too fast ‣Security not involved early enough ! Bottom line: Today’s security professionals have to pick their battles...or do they? Why are the Fortune 100 not scanning 100%? •In a informal survey of 2 dozen F100 companies No Permission 22% Poor Results 11% Can’t Keep Up 52% Cost 15%
8. Triage - Picking our battles • Triage (noun) - The process of determining the most important people or things from amongst a large number that require attention. • Because we can’t keep up, are unhappy with the results, lack authority and budget, we triage what we will scan • Triage ≠ Application Security Program • What if we could scan everything
 and not have to choose, would that
 solve the problem?
9. Picking all the Battles • In 2012 I created DynamicMP that could Dynamically scan an unlimited* number of web applications simultaneously • For the first time we could scan everything and not have to triage • It had a dramatic effect on Veracode’s Scan volume, but what was the net effect?
10. Winning the Battle Vulns Found 0 2250 4500 6750 9000 Apps Scanned 0 400 800 1200 1600 201304 201305 201306 201307 201308 201309 201310 201311 201312 Apps Scanned Vulns Found 6,112 7,421 6,538 3,419 6,632 4,180 6,756 8,341 183
11. Losing the War Average number of applications vulnerable to SQLi or XSS remains mostly…unchanged? 201304 201305 201306 201307 201308 201309 201310 201311 201312 DynMP DynDS
12. I’M FINDING THOUSANDS OF VULNERABILITIES STILL RIGHT WHERE I STARTED
13. Losing the War New, vulnerable, applications are added at a increasing pace per month, erasing overall gains 201304 201305 201306 201307 201308 201309 201310 201311 201312 DynMP DynDS New Apps Added
14. I Do it in Production • This data might make sense if all these applications were in development • Unfortunately, all of these applications were scanned in production
15. Perhaps the issue isn't technology, but people, and perhaps it's not really people but human behavior?
17. 1980’s • Despite all the great hair and neon • Crime rates were at all time highs • Cities like New York were falling apart • The conventual wisdom of the time was to “Get tough on crime”
18. GET TOUGH ON APPSEC YOU HAVE 60 SECONDS TO COMPLY
19. But are we getting to the root of the problem?
20. Broken Window Theory • The appearance of decay in the surroundings (like litter, broken building windows or graffiti) lead to even more decay creating an environment more vulnerable to serious crimes • Introduced in a 1982 article by social scientists James Q. Wilson and George L. Kelling Original Article: http://www.theatlantic.com/magazine/archive/1982/03/broken-windows/304465/
22. Broken Window Theory • Broken window theory is often credited with saving New York in the 80’s ! ! ! • Which started to put broken window theory to the test in 1985
23. Key Take-Aways from Broken Window Theory in Actual Practice • Focus on community - Improve the environment (fix the broken windows, litter, graffiti) • Make it personal - Police on foot vs. police in squad cars • Rapid, Persistent Response - Address environment issues quickly and stick with it
24. How can we apply this to AppSec?
25. Technology is Simple, People are Complex
26. Get to know your People
27. Make it Clear That Someone Cares about AppSec
28. Improve Their Environment
29. This doesn’t mean send everyone off to training and call it a day
30. Respond Quickly to calls for help
32. Develop a Better Definition of Success Running a bunch of scans Mostly harmless, and pointless Tracking Success by # of flaws found Bad Running an AppSec Program Good Tracking success by # of flaws fixed Great Tracking Success by # of flaws not found in new applications Awesome
33. SCAN EVERYTHING CARRY ON AND
34. Thank You @Silvexis Follow me on Twitter: Email me: epeterson@veracode.com