Slide 1

Slide 1 text

@ramimacisabird Learning from AWS Customer Security Incidents OWASP DevSlop - May 14, 2022 Rami McCarthy

Slide 2

Slide 2 text

@ramimacisabird Hello! I’m Rami McCarthy 👋 • Security @ series d health-tech • Reformed security consultant • AWS Certified Security, Specialty & CCSKv4 • -creator & -contributor

Slide 3

Slide 3 text

@ramimacisabird Today This is what you’ll get! • A strong understanding of how AWS accounts are breached in the real world • Common initial access and escalation vectors in AWS • Actionable practices to prevent joining this esteemed cohort

Slide 4

Slide 4 text

@ramimacisabird

Slide 5

Slide 5 text

@ramimacisabird Disclaimers!

Slide 6

Slide 6 text

@ramimacisabird The Shared Responsibility Model

Slide 7

Slide 7 text

@ramimacisabird

Slide 8

Slide 8 text

@ramimacisabird "Postmortem Culture: Learning from Failure” by John Lunney and Sue Lueder ”Avoid Blame and Keep It Constructive” https://landing.google.com/sre/sre-book/chapters/postmortem-culture ”Blameless postmortems can be challenging to write, because the postmortem format clearly identifies the actions that led to the incident. Removing blame from a postmortem gives people the confidence to escalate issues without fear. It is also important not to stigmatize frequent production of postmortems by a person or team. An atmosphere of blame risks creating a culture in which incidents and issues are swept under the rug, leading to greater risk for the organization [Boy13]." Best Practice:

Slide 9

Slide 9 text

@ramimacisabird Survivorship Bias Infrequent disclosure

Slide 10

Slide 10 text

@ramimacisabird DisruptOps’ Top 10 Cloud Attack Killchains • Static API Credential Exposure to Account Hijack • Compromised Server via Exposed Remote Access Ports • Compromised Database via Inadvertent Exposure • Object Storage Public Data Exposure • Server Side Request Forgery ● Cryptomining ● Network Attack ● Compromised Secrets ● Novel Cloud Data Exposure and Exfiltration ● Subdomain Takeover

Slide 11

Slide 11 text

@ramimacisabird

Slide 12

Slide 12 text

@ramimacisabird The Common Cases

Slide 13

Slide 13 text

@ramimacisabird

Slide 14

Slide 14 text

@ramimacisabird Open S3 Buckets and Other Exposed Data Stores

Slide 15

Slide 15 text

@ramimacisabird Secure Defaults • 2017: prominent indicator next to each S3 bucket that is publicly accessible • 2017: Clarified UX (“Authenticated - Anyone with an AWS account”) • 2018: Trusted Advisor S3 Public Access rule • 2018: Block public access • 2019: Access Analyzer for S3 • 2020: Amazon GuardDuty to Protect Your S3 Buckets

Slide 16

Slide 16 text

@ramimacisabird Open S3 Buckets and Other Exposed Data Stores

Slide 17

Slide 17 text

@ramimacisabird Database Ransomware • AWS services or user managed • Generally, internet exposed with a weak password • BTC ransom • Examples: • https:/ /mangolassi.it/topic/19664/database-h eld-for-ransom-anyone-experience-this-befor e/16 • https:/ /forums.aws.amazon.com/thread.jspa?t hreadID=249445

Slide 18

Slide 18 text

@ramimacisabird “Through 2025, more than 99% of cloud breaches will have a root cause of preventable misconfigurations or mistakes by end users.” - Gartner. (H/T Anton Chuvakin)

Slide 19

Slide 19 text

@ramimacisabird Case Study Speed Run

Slide 20

Slide 20 text

@ramimacisabird

Slide 21

Slide 21 text

@ramimacisabird S3 Global Write: Magecart https://www.riskiq.com/blog/labs/magecart-amazon-s3-buckets/

Slide 22

Slide 22 text

@ramimacisabird S3 Global Write Politifact 2017 Initial Access: “Misconfigured cloud computing server” Impact: Coinhive cryptojacking LA Times 2018 Initial Access: S3 global write access Impact: Coinhive cryptojacking added to homicide.latimes.com Twilio 2020 Initial Access: S3 global write access Impact: Magecart

Slide 23

Slide 23 text

@ramimacisabird S3 Global Write AWS IAM Access Analyzer for S3 Cloud Security Posture Management Prevention: Infrastructure as code + SAST

Slide 24

Slide 24 text

@ramimacisabird Malicious AMI Cryptomining AMI 2018 Initial Access: Unknown AMI Impact: "mines cryptocurrencies, asks for ransom money, and tries to exploit things to spread” Cryptomining AMI 2020 Initial Access: Windows 2008 Server Community AMI Impact: Cryptojacking for Monero Subscription Scam 2020 Initial Access: CentOS AMI squatting Impact: $$$ subscription price

Slide 25

Slide 25 text

@ramimacisabird Malicious AMI Using random community AMIs Prevention:

Slide 26

Slide 26 text

@ramimacisabird Application Vulnerability Tesla 2018 Initial Access: Globally exposed Kubernetes console, pod with AWS credentials Impact: Cryptojacking Imperva 2018 Initial Access: “Internal compute instance” globally accessible, “contained” AWS API key Impact: RDS snapshot stolen JW Player 2019 Initial Access: Weave Scope (publicly exposed), RCE by design Impact: Cryptojacking 1/3

Slide 27

Slide 27 text

@ramimacisabird Capital One 2019 Initial Access: Misconfigured “firewall” (WAF), SSRF access to IMDS (metadata service) Impact: 100M+ credit card applications stored in S3 TeamTNT Worm 2020 Initial Access: Misconfigured Docker & k8s platforms Impact: Cryptojacking for Monero Uran Company 2021 Initial Access: Compromised Drupal with API keys Impact: Cryptomining Application Vulnerability 2/3

Slide 28

Slide 28 text

@ramimacisabird Application Vulnerability Onus 2021 Initial Access: Log4Shell vulnerability in Cyclos server Impact: 2 million ONUS users’ information including EKYC data, personal information, and password hash was leaked “Cloud Metadata Abuse by UNC2903” 2022 Initial Access: Adminer CVE Impact: Unknown Escalation/ Persistence 1. AmazonS3FullAccess creds (and DB creds) in Cyclos config 2. Steals AWS credentials from ~/.aws/* 3/3

Slide 29

Slide 29 text

@ramimacisabird Application Vulnerability:SSRF

Slide 30

Slide 30 text

@ramimacisabird Application Vulnerability Using IMDSv2 - check out SSDLC: Threat Model -> Design Review -> Code Review -> SAST -> Assessments Asset Inventory Prevention: Patch Management Putting internal applications on the internet

Slide 31

Slide 31 text

@ramimacisabird Abuse of Valid Credentials Malindo Air 2019 Initial Access: Former employees for a third party e-commerce provider abused their access Impact: 35 million customer records 1/2 Voova 2019 Initial Access: Stolen credentials by former employee Impact: Deleted 23 servers Cisco 2018 Initial Access: Former employee with AWS access 5 months post-resignation Impact: Deleted ~450 EC2 instances

Slide 32

Slide 32 text

@ramimacisabird Abuse of Valid Credentials Ubiquiti 2021 Initial Access: Compromised credentials from IT employee Lastpass (alleged former employee insider threat) Impact: root administrator access to all AWS accounts, extortion “Insider Threat Scenario” 2020 Initial Access: Fired employee uses credentials Impact: Deleted production databases 2/2 Escalation/ Persistence 1. Access CI/CD server, create a new user, steal credentials

Slide 33

Slide 33 text

@ramimacisabird Abuse of Valid Credentials Standardize and automate offboarding Manage third party risk Least privilege applications and services Prevention: Improve logging, monitoring, and detection: Time/Location/Activity heuristics

Slide 34

Slide 34 text

@ramimacisabird Abuse of Stolen Credentials Code Spaces 2014 Initial Access: AWS Console Credentials (Phishing?) Impact: Wiped S3 buckets, EC2 instances, AMIs, EBS snapshots Datadog 2016 Initial Access: CI/CD AWS access key and SSH private key leaked Impact: 3 EC2 instances and subset of S3 buckets Uber 2016 Initial Access: Private Github Repo with AWS credentials Impact: Names and driver’s license numbers of 600k drivers; PII of 57 million users 1/5

Slide 35

Slide 35 text

@ramimacisabird Abuse of Stolen Credentials OneLogin 2017 Initial Access: AWS keys Impact: Accessed database tables (with encrypted data) DXC Technologies 2017 Initial Access: Private AWS key exposed via Github Impact: 244 EC2 instances started Cameo 2020 Initial Access: Credentials in mobile app package Impact: Access to backend infrastructure, including user data 2/5 Open Exchange Rates 2020 Initial Access: Third-party compromise exposing access key Impact: User database

Slide 36

Slide 36 text

@ramimacisabird Abuse of Stolen Credentials Natures Basket 2020 Initial Access: Hard-coded root keys in source code exposed via public S3 bucket Impact: Responsible disclosure Animal Jam 2020 Initial Access: Slack compromise exposes AWS credentials Impact: User database Juspay 2021 Initial Access: Compromised old, unrecycled Amazon Web Services (AWS) access key Impact: Masked card data, email IDs and phone numbers 3/5

Slide 37

Slide 37 text

@ramimacisabird Abuse of Stolen Credentials 20/20 Network 2021 Initial Access: Compromised credential Impact: S3 buckets accessed then deleted LogicGate 2021 Initial Access: Compromised credentials Impact: Backup files in S3 stolen Kaspersky 2021 Initial Access: Compromised SES token from third party Impact: Phishing attacks 4/5

Slide 38

Slide 38 text

@ramimacisabird Abuse of Stolen Credentials “Alert-to-fix in AWS” 2020 Initial Access: Root IAM user access key compromised Impact: Cryptojacking “A key pair to remember” 2021 Initial Access: 8 IAM access keys compromised Impact: Command line access to EC2 instances “From CLI to console, chasing an attacker in AWS” 2021 Initial Access: Credentials in publicly available code repository Impact: Cryptomining (prevented) 5/5

Slide 39

Slide 39 text

@ramimacisabird Abuse of Stolen Credentials Escalation/ Persistence 1. Attacker created additional accounts/access keys 2. Attacker attempted to pivot with customer credentials 3. Attacker created EC2 instances 4. Attacker generated SSH keys for EC2 instances 5. Attacker backdoored security groups 6. Attacker used AttachUserPolicy for privilege escalation

Slide 40

Slide 40 text

@ramimacisabird Abuse of Stolen Credentials Follow IAM Best Practices: MFA, key rotation Audit and monitor privileging Prevention: Using IAM users Storing credentials in code

Slide 41

Slide 41 text

@ramimacisabird Unknown DNC Hack by the GRU 2016 Initial Access: Unknown, test clusters breached Impact: Tableau and Vertica Queries Flexbooker 2021 Initial Access: ??? Impact: 3.7M first and last names, email addresses, phone numbers, "encrypted" passwords Escalation/ Persistence 1. EC2 Snapshots copied to attacker AWS accounts

Slide 42

Slide 42 text

@ramimacisabird Trends

Slide 43

Slide 43 text

@ramimacisabird

Slide 44

Slide 44 text

@ramimacisabird Threat Actors 1. Monero mining is primary monitization a. RCE & Brute force passwords b. 8220 Mining Group (chinese speaking) • Docker and k8s targeting c. Rocke (chinese speaking) • JS backdoors d. Pacha Group (chinese speaking) • lot of evasion, advanced anti-analysis 2. Dark web market exists for public cloud access 3. Docker-focused malware (XoRDDOS, Groundhog and Tsunami) 4. Denonia (lambda targeting malware) The Usual Suspects: A Look at Threat Actors Targeting the Cloud and their Battle for Superiority 2021 IBM Security X-Force Cloud Threat Landscape Report https://www.cadosecurity.com/cado-discovers-denonia-the-first-malware-specifically-targeting-lambda/

Slide 45

Slide 45 text

@ramimacisabird Thank you! 👋 Key references: https:/ /blog.christophetd.fr/cloud-security-breaches-and-vulnerabilities-2021-in-review/ https:/ /tldrsec.com/blog/cloud-security-orienteering/ https:/ /summitroute.com/downloads/aws_security_maturity_roadmap-Summit_Route.pdf https:/ /www.marcolancini.it/2021/blog-cloud-security-roadmap/ Stop by Adrien Coquet from NounProject.com Starting by Luis Prado from NounProject.com Rami McCarthy, 2022 Slides: https:/ /speakerdeck.com/ramimac/learning-from-aws-custome r-security-incidents-2022

Slide 46

Slide 46 text

@ramimacisabird Subdomain Takeovers https:/ /0xpatrik.com/subdomain-takeover-basics/