Upgrade to Pro — share decks privately, control downloads, hide ads and more …

AppSec Broken Windows Theory - Why we are winning battles but losing the war

Erik Peterson
February 23, 2014

AppSec Broken Windows Theory - Why we are winning battles but losing the war

Conventional wisdom says fix the most severe flaws first and triage the rest. While this may impact the security of one application, it won’t improve AppSec over the long term. During this presentation Erik Peterson proposes applying the sociological principles of Broken Window Theory to AppSec to change the development culture within an organization, making long term improvements to AppSec.

Erik Peterson

February 23, 2014
Tweet

More Decks by Erik Peterson

Other Decks in Technology

Transcript

  1. AppSec & Broken
    Window Theory
    Why we are wining battles but losing the war
    Erik Peterson - February 2014 - BSidesSF

    View Slide

  2. About Your Speaker
    • Hi, I’m Erik Peterson
    • AppSec guy for 16 years now, currently I
    work at Veracode
    • If you have used a dynamic scanner, in
    some way i’ve probably influenced it’s
    design
    • I’ve directly contributed to the design
    or development of AppScan,
    WebInspect, Qualys WAS and Veracode
    Discovery, DynamicDS & DynamicMP
    email: [email protected]
    twitter: @silvexis

    View Slide

  3. Software Security is
    the greatest challenge
    of all humankind

    View Slide

  4. And we are failing

    View Slide

  5. Our Journey Today
    • Application Security as a discipline, has
    performed poorly
    • I have really wanted to know why
    • Sneaking suspicion that it might be my fault

    View Slide

  6. Fortune 100

    An AppSec Failure
    • Automated dynamic analysis tools have been available
    for over 10 years now
    • At least 82% of the Fortune 100 use a dynamic
    scanning tool*
    • Every Fortune 100 company to suffer a web site breach
    in the last 5 years uses a dynamic scanning tool*
    • Most however are not scanning all of their

    web applications and this is where they

    are getting hit.**
    **My personal observations, not a formal study
    *Based on a review of known AppScan, WebInspect & Veracode customers

    View Slide

  7. ‣Too Expensive
    ‣Headcount
    ‣Scanning costs
    ‣Permission
    ‣Arranging scan windows
    ‣No security mandate
    ‣Poor Results
    ‣Too much noise in reports
    ‣Don’t know what to do with the data
    ‣Can’t keep up
    ‣New sites are being deployed too fast
    ‣Security not involved early enough
    !
    Bottom line: Today’s security professionals have to pick their battles...or do
    they?
    Why are the Fortune 100 not scanning
    100%?
    •In a informal survey of 2 dozen F100 companies
    No Permission
    22%
    Poor Results
    11%
    Can’t Keep Up
    52%
    Cost
    15%

    View Slide

  8. Triage - Picking our battles
    • Triage (noun) - The process of determining the most
    important people or things from amongst a large
    number that require attention.
    • Because we can’t keep up, are unhappy with the
    results, lack authority and budget, we triage what
    we will scan
    • Triage ≠ Application Security Program
    • What if we could scan everything

    and not have to choose, would that

    solve the problem?

    View Slide

  9. Picking all the Battles
    • In 2012 I created DynamicMP that could
    Dynamically scan an unlimited* number
    of web applications simultaneously
    • For the first time we could scan
    everything and not have to triage
    • It had a dramatic effect on Veracode’s
    Scan volume, but what was the net
    effect?

    View Slide

  10. Winning the Battle
    Vulns Found
    0
    2250
    4500
    6750
    9000
    Apps Scanned
    0
    400
    800
    1200
    1600
    201304
    201305
    201306
    201307
    201308
    201309
    201310
    201311
    201312
    Apps Scanned Vulns Found
    6,112
    7,421
    6,538
    3,419
    6,632
    4,180
    6,756
    8,341
    183

    View Slide

  11. Losing the War
    Average number of applications vulnerable to SQLi or XSS
    remains mostly…unchanged?
    201304
    201305
    201306
    201307
    201308
    201309
    201310
    201311
    201312
    DynMP DynDS

    View Slide

  12. I’M FINDING THOUSANDS OF VULNERABILITIES
    STILL RIGHT WHERE I STARTED

    View Slide

  13. Losing the War
    New, vulnerable, applications are added at a increasing
    pace per month, erasing overall gains
    201304
    201305
    201306
    201307
    201308
    201309
    201310
    201311
    201312
    DynMP DynDS New Apps Added

    View Slide

  14. I Do it in Production
    • This data might make
    sense if all these
    applications were in
    development
    • Unfortunately, all of
    these applications
    were scanned in
    production

    View Slide

  15. Perhaps the issue isn't
    technology, but people, and
    perhaps it's not really people
    but human behavior?

    View Slide

  16. View Slide

  17. 1980’s
    • Despite all the great hair and neon
    • Crime rates were at all time highs
    • Cities like New York were falling apart
    • The conventual wisdom of the time
    was to “Get tough on crime”

    View Slide

  18. GET TOUGH ON APPSEC
    YOU HAVE 60 SECONDS
    TO COMPLY

    View Slide

  19. But are we getting to the
    root of the problem?

    View Slide

  20. Broken Window Theory
    • The appearance of decay in the
    surroundings (like litter, broken building
    windows or graffiti) lead to even more
    decay creating an environment more
    vulnerable to serious crimes
    • Introduced in a 1982 article by social
    scientists James Q. Wilson and George
    L. Kelling
    Original Article: http://www.theatlantic.com/magazine/archive/1982/03/broken-windows/304465/

    View Slide

  21. View Slide

  22. Broken Window Theory
    • Broken window theory is often credited
    with saving New York in the 80’s
    !
    !
    !
    • Which started to put broken window
    theory to the test in 1985

    View Slide

  23. Key Take-Aways from Broken
    Window Theory in Actual Practice
    • Focus on community - Improve the environment
    (fix the broken windows, litter, graffiti)
    • Make it personal - Police on foot vs. police in
    squad cars
    • Rapid, Persistent Response - Address
    environment issues quickly and stick with it

    View Slide

  24. How can we apply this
    to AppSec?

    View Slide

  25. Technology is Simple,
    People are Complex

    View Slide

  26. Get to know your
    People

    View Slide

  27. Make it Clear That Someone
    Cares about AppSec

    View Slide

  28. Improve Their
    Environment

    View Slide

  29. This doesn’t mean send everyone
    off to training and call it a day

    View Slide

  30. Respond Quickly to
    calls for help

    View Slide

  31. View Slide

  32. Develop a Better Definition
    of Success
    Running a bunch of scans
    Mostly harmless, and
    pointless
    Tracking Success by # of flaws found Bad
    Running an AppSec Program Good
    Tracking success by # of flaws fixed Great
    Tracking Success by # of flaws not found
    in new applications
    Awesome

    View Slide

  33. SCAN
    EVERYTHING
    CARRY
    ON
    AND

    View Slide

  34. Thank You
    @Silvexis
    Follow me on Twitter:
    Email me:
    [email protected]

    View Slide