Upgrade to Pro — share decks privately, control downloads, hide ads and more …

[OWASP Sofia] Dan Cornell - AppSec Fast and Slow: Your DevSecOps CI⁄CD Pipeline Isn’t an SSA Program (July 27th, 2021)

[OWASP Sofia] Dan Cornell - AppSec Fast and Slow: Your DevSecOps CI⁄CD Pipeline Isn’t an SSA Program (July 27th, 2021)

OWASP Sofia is proud to invite you to join us and Dan Cornell, at the "AppSec Fast and Slow: Your DevSecOps CI/CD Pipeline Isn’t an SSA Program" session.

With all the focus on DevSecOps and integrating security into Continuous Integration/Continuous Delivery (CI/CD) pipelines, some teams may be lured into thinking that the entirety of a Software Security Assurance (SSA) program can be baked into these pipelines. While integrating security into CI/CD offers many benefits, it is critical to understand that a full SSA program encompasses a variety of activities – many of which are incompatible with run-time restrictions and other constraints imposed by these pipelines. This webinar looks at the breadth of activities involved in a mature SSA program and steps through the aspects of a program that can be realistically included in a pipeline, as well as those that cannot. It also reviews how these activities and related tooling have evolved over time as the application security discipline has matured and as development teams started to focus on cloud-native development techniques and technologies.

Speaker: Dan Cornell

A globally recognized software security expert, Dan Cornell has over 20 years of experience architecting, developing, and securing software systems. As Vice President of Product Strategy at Coalfire, Dan works with customers and industry partners to help drive the direction of their product portfolio. Prior to its acquisition by Coalfire, Dan was a founder of and the Chief Technology Officer at Denim Group, where he helped Fortune 500 companies and government organizations integrate security throughout the development process.

Cornell is an active member of the development community and a sought-after speaker on topics of application and software security, speaking at international conferences including RSA Security Conference, OWASP AppSec USA and EU, TEDx, and Black Hat Arsenal. He holds three patents in the area of software security.



July 31, 2021

More Decks by OWASP Sofia

Other Decks in Technology


  1. AppSec Fast and Slow Your DevSecOps CI/CD Pipeline Isn’t a

    SSA Program July 27, 2021 Dan Cornell
  2. Agenda • Cool Kids: Moving FAST • SSA Programs •

    Fast and Slow • OWASP SAMM Walkthrough • Conclusions • Questions 2
  3. Cool Kids: Moving FAST

  4. Security in the DevOps Pipeline Organizations like Etsy and Netflix

    are doing amazing things to secure application via their DevOps pipelines
  5. All About the Pipeline • Security checks in the pipeline

    • Application • Infrastructure • Cloud • Automation is king 5
  6. But What Doesn’t Fit Into a Pipeline? • Dangers of

    DevSecOps fundamentalism • The Pipeline Isn’t the Program 6
  7. SSA Programs

  8. What is Your “Why?” • Simon Sinek TED Talk •

    (If you have seen this before, rolling your eyes at this point is acceptable) • Why -> How -> What https://www.youtube.com/watch?v=q p0HIF3SfI4
  9. What is an SSA Program • SSA = Software Security

    Assurance • Set of practices and activities used to reliably create, maintain, and deploy secure software • “We do an annual app pen test for PCI” is not an SSA program • Or at least probably not a very effective one • “Here are the security checks we figured out how to stuff into our CI/CD pipeline” is also not an SSA program • Danger: Don’t let the pipeline become your program • “Shifting left” isn’t bad – it just isn’t everything 9
  10. SSA Program References • OWASP SAMM • BSIMM 10

  11. OWASP SAMM • Originally OpenSAMM from Pravir Chandra • OWASP’s

    evolution/fork • Five Business Functions • Three Security Practices for each • Two Streams for each https://owaspsamm.org/ 11
  12. OWASP SAMM 12

  13. BSIMM • Originally from Cigital (now Synopsys) • Based on

    data collection from participating organizations • Four domains • Three Practices for each • Total of 119 Activities https://www.bsimm.com/ 13
  14. BSIMM 14

  15. OWASP SAMM Walkthrough • We will use OWASP SAMM for

    the purposes of this webinar • More prescriptive • Less vendor-centric • If you are using BSIMM it is pretty trivial to translate 15
  16. If You Are Just Starting Out • Assessing your program

    using either tool is less-than-ideal • Better: • Define your scope/mandate • Do some testing • Run some vulnerabilities through resolution • Proceed from there 16
  17. Fast and Slow

  18. Thinking Fast and Slow 18 • Written by Daniel Kahneman

    • System 1 (Fast): Instinctive, emotional • System 2 (Slow): Deliberative, logical • (For AppSec purposes, use configuration/customization to minimize the “emotional”) https://www.amazon.com/Thinking-Fast-Slow-Daniel- Kahneman/dp/0141033576/ref=asc_df_0141033576/
  19. An Aside: What Horrible Names! • System 1 and System

    2 ??? • Almost as bad as Type I and Type II Errors 19 https://www.simplypsychology.org/type_I_and_type_II_errors.html
  20. Another Aside: The Undoing Project • Michael Lewis book on

    the research of and the collaboration between Daniel Kahneman and Amos Tversky https://www.amazon.com/Undoing-Project-Friendship- Changed-Minds/dp/0393354776/ref=sr_1_2 20
  21. Fast and Slow In a culture like DevSecOps that is

    so focused on FAST, what is still critical, but has to go SLOW? 21
  22. What Do We Mean By FAST? Blog post: Power, Responsibility,

    and Security’s Role in the DevOps Pipeline https://www.denimgroup.com/resources/blog/2019/02/power- responsibility-and-securitys-role-in-the-devops-pipeline/ 22
  23. To Be DevSecOps FAST 1. Available quickly 2. High-value 3.

    Low (NO) false positives (no Type I errors) • Limited time budget • Developers have to care • Don’t waste developers’ time 23
  24. OWASP SAMM Walkthrough

  25. Governance • Strategy and Metrics • Policy and Compliance •

    Education and Guidance 25
  26. Strategy and Metrics • You can’t automate strategy • SLOW

    • You can use CI/CD to feed your metrics • Kinda FAST • Metrics in general: very automatable 26
  27. Policy and Compliance • You can’t automate the creation of

    your policies • SLOW • You can use CI/CD to automate some policy checks • CI/CD pass/fail • Be careful of limitations – this is a helper, not definitive • Kinda FAST 27
  28. CI/CD Policy Configuration • Testing • Synchronous • Asynchronous •

    Decision • Reporting 28 Blog Post: Effective Application Security Testing in DevOps Pipelines http://www.denimgroup.com/blog/2016/12/effective-application-security-testing-in-devops-pipelines/ https://www.denimgroup.com/resources/effective-application-security-for-devops/
  29. Education and Guidance • Instructor-led training: SLOW • eLearning •

    Monolithic: SLOW • Targeted: Not FAST, but increasingly interesting • Security Champions • Common responsibility is to configure security testing in CI/CD environments and tune scanning • They make things FASTer 29
  30. Security Champions Webinar: Security Champions: Pushing Security Expertise to the

    Edges of Your Organization https://www.denimgroup.com/resources/webinar/security-champions- pushing-security-expertise-to-the-edges-of-your-organization/ 30
  31. Design • Threat Assessment • Security Requirements • Security Architecture

  32. Threat Assessment • Determining your general application threat profiles can’t

    be automated • SLOW • Threat Modeling also requires a lot of manual work • Some new interesting automation, but nothing in CI/CD pipelines • Some vendors providing tooling support • Can allow for manual incremental changes – not CI/CD, but fits better into Agile environments • SLOW 32
  33. Security Requirements • Determining your requirements is largely manual •

    Some tooling support is available • SLOW • Validating if they are met is largely manual, but we will look at this later during the Verification/Requirements-Driven Testing activity 33
  34. Secure Architecture • Determining your architectural security requirements is largely

    manual • SLOW • Validating if they are met is largely manual, but we will look at this later during the Verification/Architecture Assessment activity 34
  35. Implementation • Secure Build • Secure Deployment • Defect Management

  36. Secure Build • This is really the crux of what

    we are discussing today • FAST • How can you integrate security into the build process? • SAST/DAST/IAST • SCA • OWASP Dependency Check https://owasp.org/www-project-dependency-check/ • If you are even considering this you have to have a repeatable build process • Otherwise please log off this webinar and pick up a Jenkins for Dummies book. You can pick this back up later. • Software Bill of Materials (SBOM) • OWASP Dependency Track https://dependencytrack.org/ 36
  37. Architectural Bill of Materials Webinar: The As, Bs, and Four

    Cs of Testing Cloud-Native Applications https://www.denimgroup.com/resources/webinar/the-as-bs-and- four-cs-of-testing-cloud-native-applications/ 37
  38. Secure Deployment • An extension of Secure Build • Organizations

    tend to be a little less mature • FAST • Technologies like Puppet, Chef, Terraform 38
  39. Defect Management • Subsets of this can be FAST •

    But you have to tune scanners or you will run into problems • High-value, no false positives • Technically automated defect creation is usually possible • In practice, it takes a while to get to this level • Limited coverage: only works for vulnerabilities you can find with automation in CI/CD pipelines • We will talk more about these testing limitations in the Verification discussions 39
  40. Bundling Strategies • Turning vulnerabilities into defects • 1:1 approach?

    • More time spent administering defects than fixing issues • Bundling • By vulnerability type • By severity (more mature applications) • Other approaches 40
  41. Metrics and Feedback Stream • Scanner / developer provide separation

    of duties • Scanners find vulns, developers say they fixed them, scanners confirm they did • Obviously only applies to vulnerabilities identified by automation • Tracking mean-time-to-remediation (MTTR) • Good metric for Agile/DevOps teams – how fast can you fix? • (Better than defects per KLoC) • Benchmark against data from Veracode/WhiteHat 41
  42. Verification • Architecture Assessment • Requirements-driven Testing • Security Testing

  43. Architecture Assessment • This largely has to be done manually

    • SLOW • Some architectural policies may be checked automatically • Cloud configuration 43
  44. ScoutSuite • Check configuration of cloud environments • Checks for:

    • Open S3 buckets • IAM configuration https://github.com/nccgroup/ScoutSuite 44
  45. Requirements-Driven Testing • Control verification: largely a manual process •

    SLOW • Misuse/abuse testing: • Fuzzing can be automated, but runtimes can extend beyond the time budget for FAST • Abuse case and business logic testing is manual • DoS testing does not fit in most general pipelines • Mostly SLOW • Some automation and integration possible 45
  46. Security Testing • THIS is really what the discussion comes

    down to • How sufficient is the security testing you can stuff into a CI/CD pipeline? • OWASP SAMM has two streams: • Scalable baseline • Deep understanding 46
  47. OWASP and Testing • OWASP has traditionally had a cultural

    focus on the strengths (and weaknesses) of automated testing tools • Consultants vs scanner vendors • Testing Guide • https://owasp.org/www-project-web-security-testing-guide/ • ASVS • https://owasp.org/www-project-application-security-verification- standard/ 47
  48. Scalable Baseline Stream • Three levels of maturity 1. Use

    an automated tool 2. Employ application-specific automation (tuning) 3. Integrate into the build process • This webinar presupposes the top level of maturity • You did remember to tune your scanner before you put it in the build process, right? 48
  49. Deep Understanding Stream • This is all manual • Manual

    test high-risk components • Perform penetration testing • Integrate testing into the development process • Tooling can help • Focus efforts on diffs / new or altered functionality 49
  50. Testing in CI/CD Pipelines 50

  51. SAST in CI/CD • Mostly open source linting tools •

    Need for speed • Commercial-grade tools are less prevalent • Run SAST on diffs? • Cross-method/class data and control flow takes time • Cut down the rules • Shorten run times • Limit false positives 51
  52. DAST in CI/CD • Concerns about run times • Approaches

    for targeted DAST • Focus on changes in the app 52
  53. Targeting DAST Testing Webinar: Monitoring Application Attack Surface and Integrating

    Security into DevOps Pipelines https://threadfix.it/resources/monitori ng-application-attack-surface-and- integrating-security-into-devops- pipelines/ 53
  54. IAST in CI/CD • Great! • Typically relies on generated

    traffic • Use DAST testing to generate traffic • Use integration tests to generate traffic 54
  55. SCA in CI/CD • Great! • Look at run time

    tradeoffs vs. velocity of new components and new vulnerabilities 55
  56. Operations • Incident Management • Environmental Management • Operational Management

  57. Incident Management • Not in a pipeline • Use automation

    for detection where possible • Some automation frameworks available for response 57
  58. Application Logging for Security Video: Top Strategies to Capture Security

    Intelligence for Applications https://www.denimgroup.com/resources/article/top-strategies-to-capture- security-intelligence-for-applications-includes-educational-video/ 58
  59. Environment Management • Servers should be cattle, not pets •

    Configuration Handling stream: • Hopefully you have this sorted given the work you have done for Secure Deployment • Chef, Puppet, Terraform • ScoutSuite • Patching and Updating stream: • Detection: FAST • Actual patching: SLOW 59
  60. Operational Management • Data Protection stream: SLOW • Oh, wait,

    your DLP solution will sort this out for you • Decommissioning: SLOW 60
  61. Conclusions

  62. What Goes in a Pipeline? • Linting SAST • DAST

    if you can target it • IAST if you can generate meaningful traffic • SCA if you want 62
  63. What Likely Has to be Done Outside? • Full, commercial-grade

    SAST • Full DAST • Manual code review • Penetration testing • Threat modeling 63
  64. What Has to be Done Outside? • Most everything else

    • Strategy • Policy • Training • Architecture • Security requirements 64
  65. Shifting Left is Awesome… • But it is only one

    aspect of a far more complicated landscape • For testing: think coverage • Classes of vulnerabilities • Detection approaches • Quality of approaches • For everything else: • Thing programmatically 65
  66. Questions daniel.cornell@coalfire.com Twitter: @danielcornell