Effective Test Automation Scripting

2019 Test Automation & Digital QA Summit - RTP, NC

Transcript

1. Tony Rice DevSecOps Architect Government Trust and Technology Services Test Automation and Digital QA Summit, RTP, 2019-10-01 Moving beyond pass/fail Note: all host names and IP addresses are fictitious
2. QA Marketing Information Security
5. Photo Metro Centric/Flickr:
6. Ten Most Critical Web Application Security Risks - 2017 1 Injection 2 Broken Authentication 3 Sensitive Data Exposure 4 XML External Entities 5 Broken Access Control 6 Security Misconfiguration 7 Cross-Site Scripting 8 Insecure Deserialization 9 Components with Known Vulnerabilities 10 Insufficient Logging & Monitoring
7. 2004 2017 2013 2010 2007 2003 OWASP Top 10 through the years 1 Input Validation Input Validation XSS Injection Injection Injection 2 Access Control Access Control Injection XSS Broken Auth Broken Auth 3 Auth/Session Mgmt Auth/Session Mgmt Malicious File Exec Auth/Session Mgmt XSS Data Exposure 4 XSS XSS Direct Object Ref Direct Object Ref Direct Object Ref XXE 5 Buffer Overflow Buffer Overflow CSRF CSRF Security Misconfig Access Control 6 Injection Injection Info Leakage / Error Handling Security Misconfig Data Exposure Security Misconfig 7 Error Handling Error Handling Auth/Session Mgmt Insecure Crypto Storage RBAC XSS 8 Insecure Storage Insecure Storage Insecure Crypto Storage URL Access CSRF Deserialization 9 Remote Admin DOS Insecure Comms Transport Layer Protection Known Vulns Known Vulns 10 Sec Misconfig Insecure Config Mgmt URL Access Unvalidated Redirects Unvalidated Redirects Logging & Monitoring
8. xkcd.com/327 by Randall Monroe
9. Test output: verbosity ≠ value What the test expects What are the gaps? Why should I care? Results: concise and to the point Make gory details and test documentation available out of band Assets tested Where do I go to reproduce/fix this problem? What wasn’t tested Photo courtesy William Warby
10. Result: PASS If the traffic to your load balancer must be secure, use either the HTTPS or the SSL protocol for the front-end connection Upgrade your load balancer to the latest version of the predefined SSL security policy. Use only the recommended ciphers and protocols. Alert Criteria: A load balancer has no listener that uses a secure protocol (HTTPS or SSL). A load balancer listener uses an outdated predefined SSL security policy. A load balancer listener uses a cipher or protocol that is not recommended. and it goes on and on and on Tested On: ????? Example 1 (before)
11. Title: HTTP Strict Transport Security (HSTS) Findings: 1) HSTS headers are not set to be accepted by load balancer XYZ on AWS 461422971532 2) HTTPS server 250.62.33.33 accepts insecure 40 bit AES cipher on Related Standards • Pass: FedRamp Low • Fail: FedRamp Moderate/High, NIST 800-53 SC-13 Cryptographic Protection • Fail: CIS control 5.2 Ensure only strong ciphers are used • Fail: OWASP Top 10: A3 Sensitive Data Exposure • Fail: PCI ver 3.2.1 controls 4.1 & 6.5.4 • Fail: CWE ID 319 • Fail: RFC 6797 Example 1 (after) EXPECTATIONS RESULTS ASSETS TESTED
12. Result: FAIL AWS CloudTrails log file integrity validation SHA-256 hashing with RSA digital signing failed. Log file may have been modified, deleted or forged. AWS S3 Bucket logging enabled on all buckets Tested On: AWS Account 774521919539 arn:aws:s3:::elasticbeanstalk-us-east-1-212386523837 Example 2 (before)
13. Title: Secure log administration Findings: 1) Logging not enabled on S3 bucket arn:aws:s3:::elasticbeanstalk-us-west-1-202216827631 2) AWS CloudTrail not enabled in AWS us-west-1 461422971532 3) AWS CloudTrail enabled in AWS us-west-1 461422971532 4) Logging enabled on S3 bucket arn:aws:s3:::elasticbeanstalk-us-east-1-212386523837 Related Standards • Pass: FedRamp Low/Medium • Fail: FedRamp High, NIST 800-53 AU-12 AUDIT GENERATION • Fail: CIS 1.4 Ensure only strong ciphers are used • Fail: OWASP Top 10: A10 Insecure Configuration Management • Fail: PCI ver 3.2.1 Control 6.4.6/ Example 2 (after) EXPECTATIONS REULTS ASSETS TESTED
14. Result: PASS The SSL certificates for CloudFront alternate domain names in the IAM certificate store are valid. Qualified Certificate Authority (CA) found No insecure crypto algorithms or key lengths found. OpenSSL TLS library up to date Tested On: www.foobar.com Example 3 (before)
15. Title: X.509 Certificates Assets tested: www.foobar.com, admin.foobar.com, bastion host at 118.127.198.137 Findings: 1) Fully qualified domain names found on all certificates 2) Approved Verisign CA found on all certificates 3) Approved crypto algorithms RSA/AES-256 found on web hosts, ECDHE/SHA-384 found on bastion host 4) OpenSSL 1.1.d (Sept 10, 2019) found Related Standards • Pass: FedRamp High, NIST 800-53 CM-5 Signed Components • Pass: DoD Cloud SRG Medium Example 3 (after) EXPECTATIONS REULTS ASSETS TESTED ECDSA/
16. create evidence not stoplights Photo: USAF/Tech. Sgt. Bob Oldham
17. trice@cisco.com rtphokie rtphokie tonyrice speakerdeck.com/tonyrice