AppSec Broken Windows Theory - Why we are winning battles but losing the war

Conventional wisdom says fix the most severe flaws first and triage the rest. While this may impact the security of one application, it won’t improve AppSec over the long term. During this presentation Erik Peterson proposes applying the sociological principles of Broken Window Theory to AppSec to change the development culture within an organization, making long term improvements to AppSec.

Transcript

1. AppSec & Broken
   Window Theory
   Why we are wining battles but losing the war
   Erik Peterson - February 2014 - BSidesSF
   

   View Slide

2. About Your Speaker
   • Hi, I’m Erik Peterson
   • AppSec guy for 16 years now, currently I
   work at Veracode
   • If you have used a dynamic scanner, in
   some way i’ve probably influenced it’s
   design
   • I’ve directly contributed to the design
   or development of AppScan,
   WebInspect, Qualys WAS and Veracode
   Discovery, DynamicDS & DynamicMP
   email: epeterson@veracode.com
   twitter: @silvexis
   

   View Slide

3. Software Security is
   the greatest challenge
   of all humankind
   

   View Slide

4. And we are failing
   

   View Slide

5. Our Journey Today
   • Application Security as a discipline, has
   performed poorly
   • I have really wanted to know why
   • Sneaking suspicion that it might be my fault
   

   View Slide

6. Fortune 100

   An AppSec Failure
   • Automated dynamic analysis tools have been available
   for over 10 years now
   • At least 82% of the Fortune 100 use a dynamic
   scanning tool*
   • Every Fortune 100 company to suffer a web site breach
   in the last 5 years uses a dynamic scanning tool*
   • Most however are not scanning all of their

   web applications and this is where they

   are getting hit.**
   **My personal observations, not a formal study
   *Based on a review of known AppScan, WebInspect & Veracode customers
   

   View Slide

7. ‣Too Expensive
   ‣Headcount
   ‣Scanning costs
   ‣Permission
   ‣Arranging scan windows
   ‣No security mandate
   ‣Poor Results
   ‣Too much noise in reports
   ‣Don’t know what to do with the data
   ‣Can’t keep up
   ‣New sites are being deployed too fast
   ‣Security not involved early enough
   !
   Bottom line: Today’s security professionals have to pick their battles...or do
   they?
   Why are the Fortune 100 not scanning
   100%?
   •In a informal survey of 2 dozen F100 companies
   No Permission
   22%
   Poor Results
   11%
   Can’t Keep Up
   52%
   Cost
   15%
   

   View Slide

8. Triage - Picking our battles
   • Triage (noun) - The process of determining the most
   important people or things from amongst a large
   number that require attention.
   • Because we can’t keep up, are unhappy with the
   results, lack authority and budget, we triage what
   we will scan
   • Triage ≠ Application Security Program
   • What if we could scan everything

   and not have to choose, would that

   solve the problem?
   

   View Slide

9. Picking all the Battles
   • In 2012 I created DynamicMP that could
   Dynamically scan an unlimited* number
   of web applications simultaneously
   • For the first time we could scan
   everything and not have to triage
   • It had a dramatic effect on Veracode’s
   Scan volume, but what was the net
   effect?
   

   View Slide

10. Winning the Battle
    Vulns Found
    0
    2250
    4500
    6750
    9000
    Apps Scanned
    0
    400
    800
    1200
    1600
    201304
    201305
    201306
    201307
    201308
    201309
    201310
    201311
    201312
    Apps Scanned Vulns Found
    6,112
    7,421
    6,538
    3,419
    6,632
    4,180
    6,756
    8,341
    183
    

    View Slide

11. Losing the War
    Average number of applications vulnerable to SQLi or XSS
    remains mostly…unchanged?
    201304
    201305
    201306
    201307
    201308
    201309
    201310
    201311
    201312
    DynMP DynDS
    

    View Slide

12. I’M FINDING THOUSANDS OF VULNERABILITIES
    STILL RIGHT WHERE I STARTED
    

    View Slide

13. Losing the War
    New, vulnerable, applications are added at a increasing
    pace per month, erasing overall gains
    201304
    201305
    201306
    201307
    201308
    201309
    201310
    201311
    201312
    DynMP DynDS New Apps Added
    

    View Slide

14. I Do it in Production
    • This data might make
    sense if all these
    applications were in
    development
    • Unfortunately, all of
    these applications
    were scanned in
    production
    

    View Slide

15. Perhaps the issue isn't
    technology, but people, and
    perhaps it's not really people
    but human behavior?
    

    View Slide

16. View Slide

17. 1980’s
    • Despite all the great hair and neon
    • Crime rates were at all time highs
    • Cities like New York were falling apart
    • The conventual wisdom of the time
    was to “Get tough on crime”
    

    View Slide

18. GET TOUGH ON APPSEC
    YOU HAVE 60 SECONDS
    TO COMPLY
    

    View Slide

19. But are we getting to the
    root of the problem?
    

    View Slide

20. Broken Window Theory
    • The appearance of decay in the
    surroundings (like litter, broken building
    windows or graffiti) lead to even more
    decay creating an environment more
    vulnerable to serious crimes
    • Introduced in a 1982 article by social
    scientists James Q. Wilson and George
    L. Kelling
    Original Article: http://www.theatlantic.com/magazine/archive/1982/03/broken-windows/304465/
    

    View Slide

21. View Slide

22. Broken Window Theory
    • Broken window theory is often credited
    with saving New York in the 80’s
    !
    !
    !
    • Which started to put broken window
    theory to the test in 1985
    

    View Slide

23. Key Take-Aways from Broken
    Window Theory in Actual Practice
    • Focus on community - Improve the environment
    (fix the broken windows, litter, graffiti)
    • Make it personal - Police on foot vs. police in
    squad cars
    • Rapid, Persistent Response - Address
    environment issues quickly and stick with it
    

    View Slide

24. How can we apply this
    to AppSec?
    

    View Slide

25. Technology is Simple,
    People are Complex
    

    View Slide

26. Get to know your
    People
    

    View Slide

27. Make it Clear That Someone
    Cares about AppSec
    

    View Slide

28. Improve Their
    Environment
    

    View Slide

29. This doesn’t mean send everyone
    off to training and call it a day
    

    View Slide

30. Respond Quickly to
    calls for help
    

    View Slide

31. View Slide

32. Develop a Better Definition
    of Success
    Running a bunch of scans
    Mostly harmless, and
    pointless
    Tracking Success by # of flaws found Bad
    Running an AppSec Program Good
    Tracking success by # of flaws fixed Great
    Tracking Success by # of flaws not found
    in new applications
    Awesome
    

    View Slide

33. SCAN
    EVERYTHING
    CARRY
    ON
    AND
    

    View Slide

34. Thank You
    @Silvexis
    Follow me on Twitter:
    Email me:
    epeterson@veracode.com
    

    View Slide