[OWASP Sofia] Dan Cornell - AppSec Fast and Slow: Your DevSecOps CI⁄CD Pipeline Isn’t an SSA Program (July 27th, 2021)

OWASP Sofia is proud to invite you to join us and Dan Cornell, at the "AppSec Fast and Slow: Your DevSecOps CI/CD Pipeline Isn’t an SSA Program" session.

With all the focus on DevSecOps and integrating security into Continuous Integration/Continuous Delivery (CI/CD) pipelines, some teams may be lured into thinking that the entirety of a Software Security Assurance (SSA) program can be baked into these pipelines. While integrating security into CI/CD offers many benefits, it is critical to understand that a full SSA program encompasses a variety of activities – many of which are incompatible with run-time restrictions and other constraints imposed by these pipelines. This webinar looks at the breadth of activities involved in a mature SSA program and steps through the aspects of a program that can be realistically included in a pipeline, as well as those that cannot. It also reviews how these activities and related tooling have evolved over time as the application security discipline has matured and as development teams started to focus on cloud-native development techniques and technologies.

Speaker: Dan Cornell

A globally recognized software security expert, Dan Cornell has over 20 years of experience architecting, developing, and securing software systems. As Vice President of Product Strategy at Coalfire, Dan works with customers and industry partners to help drive the direction of their product portfolio. Prior to its acquisition by Coalfire, Dan was a founder of and the Chief Technology Officer at Denim Group, where he helped Fortune 500 companies and government organizations integrate security throughout the development process.

Cornell is an active member of the development community and a sought-after speaker on topics of application and software security, speaking at international conferences including RSA Security Conference, OWASP AppSec USA and EU, TEDx, and Black Hat Arsenal. He holds three patents in the area of software security.

Transcript

1. AppSec Fast and Slow Your DevSecOps CI/CD Pipeline Isn’t a SSA Program July 27, 2021 Dan Cornell
2. Agenda • Cool Kids: Moving FAST • SSA Programs • Fast and Slow • OWASP SAMM Walkthrough • Conclusions • Questions 2
3. Cool Kids: Moving FAST
4. Security in the DevOps Pipeline Organizations like Etsy and Netflix are doing amazing things to secure application via their DevOps pipelines
5. All About the Pipeline • Security checks in the pipeline • Application • Infrastructure • Cloud • Automation is king 5
6. But What Doesn’t Fit Into a Pipeline? • Dangers of DevSecOps fundamentalism • The Pipeline Isn’t the Program 6
7. SSA Programs
8. What is Your “Why?” • Simon Sinek TED Talk • (If you have seen this before, rolling your eyes at this point is acceptable) • Why -> How -> What https://www.youtube.com/watch?v=q p0HIF3SfI4
9. What is an SSA Program • SSA = Software Security Assurance • Set of practices and activities used to reliably create, maintain, and deploy secure software • “We do an annual app pen test for PCI” is not an SSA program • Or at least probably not a very effective one • “Here are the security checks we figured out how to stuff into our CI/CD pipeline” is also not an SSA program • Danger: Don’t let the pipeline become your program • “Shifting left” isn’t bad – it just isn’t everything 9
10. SSA Program References • OWASP SAMM • BSIMM 10
11. OWASP SAMM • Originally OpenSAMM from Pravir Chandra • OWASP’s evolution/fork • Five Business Functions • Three Security Practices for each • Two Streams for each https://owaspsamm.org/ 11
12. OWASP SAMM 12
13. BSIMM • Originally from Cigital (now Synopsys) • Based on data collection from participating organizations • Four domains • Three Practices for each • Total of 119 Activities https://www.bsimm.com/ 13
14. BSIMM 14
15. OWASP SAMM Walkthrough • We will use OWASP SAMM for the purposes of this webinar • More prescriptive • Less vendor-centric • If you are using BSIMM it is pretty trivial to translate 15
16. If You Are Just Starting Out • Assessing your program using either tool is less-than-ideal • Better: • Define your scope/mandate • Do some testing • Run some vulnerabilities through resolution • Proceed from there 16
17. Fast and Slow
18. Thinking Fast and Slow 18 • Written by Daniel Kahneman • System 1 (Fast): Instinctive, emotional • System 2 (Slow): Deliberative, logical • (For AppSec purposes, use configuration/customization to minimize the “emotional”) https://www.amazon.com/Thinking-Fast-Slow-Daniel- Kahneman/dp/0141033576/ref=asc_df_0141033576/
19. An Aside: What Horrible Names! • System 1 and System 2 ??? • Almost as bad as Type I and Type II Errors 19 https://www.simplypsychology.org/type_I_and_type_II_errors.html
20. Another Aside: The Undoing Project • Michael Lewis book on the research of and the collaboration between Daniel Kahneman and Amos Tversky https://www.amazon.com/Undoing-Project-Friendship- Changed-Minds/dp/0393354776/ref=sr_1_2 20
21. Fast and Slow In a culture like DevSecOps that is so focused on FAST, what is still critical, but has to go SLOW? 21
22. What Do We Mean By FAST? Blog post: Power, Responsibility, and Security’s Role in the DevOps Pipeline https://www.denimgroup.com/resources/blog/2019/02/power- responsibility-and-securitys-role-in-the-devops-pipeline/ 22
23. To Be DevSecOps FAST 1. Available quickly 2. High-value 3. Low (NO) false positives (no Type I errors) • Limited time budget • Developers have to care • Don’t waste developers’ time 23
24. OWASP SAMM Walkthrough
25. Governance • Strategy and Metrics • Policy and Compliance • Education and Guidance 25
26. Strategy and Metrics • You can’t automate strategy • SLOW • You can use CI/CD to feed your metrics • Kinda FAST • Metrics in general: very automatable 26
27. Policy and Compliance • You can’t automate the creation of your policies • SLOW • You can use CI/CD to automate some policy checks • CI/CD pass/fail • Be careful of limitations – this is a helper, not definitive • Kinda FAST 27
28. CI/CD Policy Configuration • Testing • Synchronous • Asynchronous • Decision • Reporting 28 Blog Post: Effective Application Security Testing in DevOps Pipelines http://www.denimgroup.com/blog/2016/12/effective-application-security-testing-in-devops-pipelines/ https://www.denimgroup.com/resources/effective-application-security-for-devops/
29. Education and Guidance • Instructor-led training: SLOW • eLearning • Monolithic: SLOW • Targeted: Not FAST, but increasingly interesting • Security Champions • Common responsibility is to configure security testing in CI/CD environments and tune scanning • They make things FASTer 29
30. Security Champions Webinar: Security Champions: Pushing Security Expertise to the Edges of Your Organization https://www.denimgroup.com/resources/webinar/security-champions- pushing-security-expertise-to-the-edges-of-your-organization/ 30
31. Design • Threat Assessment • Security Requirements • Security Architecture 31
32. Threat Assessment • Determining your general application threat profiles can’t be automated • SLOW • Threat Modeling also requires a lot of manual work • Some new interesting automation, but nothing in CI/CD pipelines • Some vendors providing tooling support • Can allow for manual incremental changes – not CI/CD, but fits better into Agile environments • SLOW 32
33. Security Requirements • Determining your requirements is largely manual • Some tooling support is available • SLOW • Validating if they are met is largely manual, but we will look at this later during the Verification/Requirements-Driven Testing activity 33
34. Secure Architecture • Determining your architectural security requirements is largely manual • SLOW • Validating if they are met is largely manual, but we will look at this later during the Verification/Architecture Assessment activity 34
35. Implementation • Secure Build • Secure Deployment • Defect Management 35
36. Secure Build • This is really the crux of what we are discussing today • FAST • How can you integrate security into the build process? • SAST/DAST/IAST • SCA • OWASP Dependency Check https://owasp.org/www-project-dependency-check/ • If you are even considering this you have to have a repeatable build process • Otherwise please log off this webinar and pick up a Jenkins for Dummies book. You can pick this back up later. • Software Bill of Materials (SBOM) • OWASP Dependency Track https://dependencytrack.org/ 36
37. Architectural Bill of Materials Webinar: The As, Bs, and Four Cs of Testing Cloud-Native Applications https://www.denimgroup.com/resources/webinar/the-as-bs-and- four-cs-of-testing-cloud-native-applications/ 37
38. Secure Deployment • An extension of Secure Build • Organizations tend to be a little less mature • FAST • Technologies like Puppet, Chef, Terraform 38
39. Defect Management • Subsets of this can be FAST • But you have to tune scanners or you will run into problems • High-value, no false positives • Technically automated defect creation is usually possible • In practice, it takes a while to get to this level • Limited coverage: only works for vulnerabilities you can find with automation in CI/CD pipelines • We will talk more about these testing limitations in the Verification discussions 39
40. Bundling Strategies • Turning vulnerabilities into defects • 1:1 approach? • More time spent administering defects than fixing issues • Bundling • By vulnerability type • By severity (more mature applications) • Other approaches 40
41. Metrics and Feedback Stream • Scanner / developer provide separation of duties • Scanners find vulns, developers say they fixed them, scanners confirm they did • Obviously only applies to vulnerabilities identified by automation • Tracking mean-time-to-remediation (MTTR) • Good metric for Agile/DevOps teams – how fast can you fix? • (Better than defects per KLoC) • Benchmark against data from Veracode/WhiteHat 41
42. Verification • Architecture Assessment • Requirements-driven Testing • Security Testing 42
43. Architecture Assessment • This largely has to be done manually • SLOW • Some architectural policies may be checked automatically • Cloud configuration 43
44. ScoutSuite • Check configuration of cloud environments • Checks for: • Open S3 buckets • IAM configuration https://github.com/nccgroup/ScoutSuite 44
45. Requirements-Driven Testing • Control verification: largely a manual process • SLOW • Misuse/abuse testing: • Fuzzing can be automated, but runtimes can extend beyond the time budget for FAST • Abuse case and business logic testing is manual • DoS testing does not fit in most general pipelines • Mostly SLOW • Some automation and integration possible 45
46. Security Testing • THIS is really what the discussion comes down to • How sufficient is the security testing you can stuff into a CI/CD pipeline? • OWASP SAMM has two streams: • Scalable baseline • Deep understanding 46
47. OWASP and Testing • OWASP has traditionally had a cultural focus on the strengths (and weaknesses) of automated testing tools • Consultants vs scanner vendors • Testing Guide • https://owasp.org/www-project-web-security-testing-guide/ • ASVS • https://owasp.org/www-project-application-security-verification- standard/ 47
48. Scalable Baseline Stream • Three levels of maturity 1. Use an automated tool 2. Employ application-specific automation (tuning) 3. Integrate into the build process • This webinar presupposes the top level of maturity • You did remember to tune your scanner before you put it in the build process, right? 48
49. Deep Understanding Stream • This is all manual • Manual test high-risk components • Perform penetration testing • Integrate testing into the development process • Tooling can help • Focus efforts on diffs / new or altered functionality 49
50. Testing in CI/CD Pipelines 50
51. SAST in CI/CD • Mostly open source linting tools • Need for speed • Commercial-grade tools are less prevalent • Run SAST on diffs? • Cross-method/class data and control flow takes time • Cut down the rules • Shorten run times • Limit false positives 51
52. DAST in CI/CD • Concerns about run times • Approaches for targeted DAST • Focus on changes in the app 52
53. Targeting DAST Testing Webinar: Monitoring Application Attack Surface and Integrating Security into DevOps Pipelines https://threadfix.it/resources/monitori ng-application-attack-surface-and- integrating-security-into-devops- pipelines/ 53
54. IAST in CI/CD • Great! • Typically relies on generated traffic • Use DAST testing to generate traffic • Use integration tests to generate traffic 54
55. SCA in CI/CD • Great! • Look at run time tradeoffs vs. velocity of new components and new vulnerabilities 55
56. Operations • Incident Management • Environmental Management • Operational Management 56
57. Incident Management • Not in a pipeline • Use automation for detection where possible • Some automation frameworks available for response 57
58. Application Logging for Security Video: Top Strategies to Capture Security Intelligence for Applications https://www.denimgroup.com/resources/article/top-strategies-to-capture- security-intelligence-for-applications-includes-educational-video/ 58
59. Environment Management • Servers should be cattle, not pets • Configuration Handling stream: • Hopefully you have this sorted given the work you have done for Secure Deployment • Chef, Puppet, Terraform • ScoutSuite • Patching and Updating stream: • Detection: FAST • Actual patching: SLOW 59
60. Operational Management • Data Protection stream: SLOW • Oh, wait, your DLP solution will sort this out for you • Decommissioning: SLOW 60
61. Conclusions
62. What Goes in a Pipeline? • Linting SAST • DAST if you can target it • IAST if you can generate meaningful traffic • SCA if you want 62
63. What Likely Has to be Done Outside? • Full, commercial-grade SAST • Full DAST • Manual code review • Penetration testing • Threat modeling 63
64. What Has to be Done Outside? • Most everything else • Strategy • Policy • Training • Architecture • Security requirements 64
65. Shifting Left is Awesome… • But it is only one aspect of a far more complicated landscape • For testing: think coverage • Classes of vulnerabilities • Detection approaches • Quality of approaches • For everything else: • Thing programmatically 65
66. Questions daniel.cornell@coalfire.com Twitter: @danielcornell