Secure Software Development for Cloud (AWS) Teams Akash Mahajan Co-Founder Appsecco

• Software related weaknesses can compromise the • Cloud Platform • Cloud Infrastructure (because platform is compromised) • Source Code and end user data (Intellectual Property) • Machine Learning Training Data • If one or more of the above is compromised it can lead to • Financial loss • Reputation damage • Business sentiment loss • Investor confidence loss and trust deficit Why apply framework for security for dev teams in AWS?

Your public IaaS cloud provider (AWS) demands this Your Job Not Your Job

Secure Coding Standards Cloud Configuration Standards Production Environment – Patterns & Practices Standard Operating Procedures/KB Account & Financial Governance A Broad Framework Should Cover these 5 areas

Secure Coding Standards

Kind of Artefact Reference Standard 1. Web Applications, Web Sites, HTTP based APIs OWASP ASVS version 4.x https://owasp.org/www-project-application-security-verification-standard/ 2. Mobile Applications OWASP MASVS version 1.2 https://github.com/OWASP/owasp-masvs 3. Infrastructure as Code For Terraform https://blog.gruntwork.io/a-comprehensive-guide-to-managing-secrets-in-your-terraform-code-1d586955ace1 Secure Coding Standards should be in place • Ideally led by the roles who set coding standards such as VP Engineering, Team Leads etc. • Security coding standards should be part of other standards related to performance etc. • Depending on what artefact gets generated (application files/APK/.app/.jar files) additional security standards maybe required

Cloud Configuration Standards

Target Reference Standard 1. AWS Platform AWS Well Architected Framework Security Pillar https://d1.awsstatic.com/whitepapers/architecture/AWS-Security-Pillar.pdf AWS CIS Foundations Benchmark https://d1.awsstatic.com/whitepapers/compliance/AWS_CIS_Foundations_Benchmark.pdf 2. Infrastructure you use Start here https://www.cisecurity.org/cis-benchmarks/ 3. Specific security docs https://aws.amazon.com/security/security-learning/?whitepapers-main.sort- by=item.additionalFields.sortDate&whitepapers-main.sort-order=desc 4. Kubernetes Amazon EKS Best Practices for Security https://aws.github.io/aws-eks-best-practices/ https://aws.amazon.com/blogs/containers/introducing-cis-amazon-eks-benchmark/ Cloud Configuration Standards should be in place • Ideally led by the roles VP Infrastructure, VP Platform, CISO • There should be a clear separation of workload and environments • For e.g. different accounts for Dev, R&D, Production

Example of an Ideal Secure Internal DevOps Process Dev Env. Secured Laptops with developer tools End point security No root/admin to developer Test + Staging Env. Automated Build Automated Deploy Privileged Access Whitelisted Manual Access Security Testing Manual Security Testing Automated Production Env. Automated Build Automated Deploy Centralised Logging Soft Boundary Hard Boundary

Network Access Control Application Access Control Secrets Management Enabling developers with dummy data Automated Build & Deploy Separation of Dev, Test and Prod Updated Secure Knowledge Base Security Operations Dashboard Main Components of Secure DevOps Process A c c o u n t G o v e r n a n c e CTO Add all your security SAST, DAST here

# Security Control AWS Service 1 Network Access Control • VPC Security Groups • Network ACLs 2 Application Access Control • AWS IAM 3 Secrets Management • AWS Parameter Store with KMS • AWS Secret Manager with KMS 4 Dev, Test and Prod • AWS Accounts • Private Subnets (No use of Public Subnet ever) • NAT Gateways 1st choice instead of Internet Gateways • Jenkins/GitHub Actions 5 Dummy Data • Generated using tools 6 Security Knowledge Base • Static Site generated using GitOps 7 Logging, Monitoring • AWS Cloudtrail, AWS CloudWatch + AWS Lambda + AWS ECS, SecurityHub, Detective, GuardDuty • AWS ElasticSearch & Kibana Stack (Exclusive for security) • Self-Hosted Logstash/Fluentd Examples of security controls to use This will get obsolete so most important to rely on latest security guidance as provided by AWS

Production Environment - Patterns & Practices

Production Environment – Patterns & Practices 1. Automation – Ideally no login to SSH, RDP and/or ad-hoc Terraform plans, Ansible playbooks. Everything deployed using automated pipelines 2. Secrets Management – Secrets baked in during pipeline build time from secure store with no human intervention required 3. Troubleshooting and Observability - of production apps should be done with specialist tools provided with limited access and data masking in place

Production Environment – Secrets Management • Use different set of secrets for Dev and Production environment • Maintain secrets in configuration instead of being hard coded in source code that gets checked in to Git repositories • Scan for secrets in source code every time application is deployed • For 3rd party secrets that are shared amongst team members, create • a clear revocation procedure in-case of leakage • a simple to follow rotation procedure as part of regular security hygiene

Production Environment – Secrets Management Lifecycle © Appsecco - All Rights Reserved

Production Environment – Secrets Management Pipeline Source Code CI Server Artefacts (Automated) Monitoring Deploy (Automated) Test & Scan From Code to Operations Automated Build and Deployment Dev/Test Secret Prod Secret ✓ X Done Not Done Status Markers

• Prod data should be considered sacred and never used in the Dev environment • Dummy data should be generated for use of Developers and sharing with 3rd parties • If usage requires using actual prod data (Data Science Team) create a data security policy covering use case Production Environment – Dummy Data Generation

Parity • Staging and production software environments should be similar • Same version of application should be running on prod which is tested on staging Differences • The secrets and configuration should be different on staging and production • Database being accessed should be different on both Production Environment – If you use a staging env

• Ideally, production should have APM/Observability processes • But since the applications can be released quite often and production issues can hamper business there is a need for a secured access to production database for troubleshooting • There needs to be a standard defined way for privileged use to access production by Breaking Glass Production Environment - Troubleshooting in Production

Standard Operating Process and Procedures (SOPs)

• Maintain SOPs as static site of docs of all code, configuration, production patterns • Deploy as a web app for team to refer and updated using GitOps • Tag any SOP document with maturity level (crawl, walk, run) • Crawl – Still evolving • Walk – A team member can complete procedure as walkthrough by following it • Run – SOP has been walked so many times that it can be automated Document SOPs

• Monthly patching and updating of systems • Quarterly rotation of production database secrets • Monthly AWS IAM Audit • Alerts for sensitive operations • Monthly review of exceptions given SOP – Examples of Security Operations Tasks

Example - A possible secure deployment process Code in Github Quality Assurance of staging app Automated build & deploy in staging Artefact read from S3 Artefact created and written to S3 Staging release final Automated build & deploy in production Deployed in production server STAGE PROD STAGE PROD Amazon ECS Amazon ECS Amazon ECS

Account & Financial Governance

• All accounts, communication etc. in a company using public cloud starts from its domain name being secure • If the DNS records are hijacked, password resets can be initiated, and accounts can be hijacked • Users can be spoofed; invoice fraud can be perpetrated • Therefore it is extremely important that the primary domain name is renewed on time, domain records are kept secure from tampering • The root account email of the AWS org should be kept safe and secure as well Accounts and Financial Governance

Are you taking care of all the below? Framework Areas Status Secure Coding Standards Cloud Configuration Standards Production Environment – Patterns & Practices Standard Operating Procedures/KB Account & Financial Governance ✓ X Done Not Done Status Markers

About Appsecco Pragmatic, holistic, business-focused approach Specialist Cloud and Application Security company Highly experienced and diverse team Assigned multiple CVEs Certified hackers OWASP chapter leads Cloud security experts Black Hat & Def Con speakers

Akash Mahajan [email protected] | @makash | @appseccouk | appsecco.com Black Hat Trainer Defcon Reviewer