Learning from AWS (Customer) Security Incidents

Presented at BSidesCT 2020

In light of the increasing adoption of cloud computing, there have has been broad coverage of the compromise of customer environments in the cloud. In both popular and technical literature however, there has been a focus on the most egregious, simplest breaches (i.e open S3 buckets). However, deeper analysis shows a much broader variety of tactics currently exploited by attackers and researchers to compromise cloud environments.

This talk will, with a focus on AWS, discuss over a dozen different public breaches. We'll walk through the technical details of these attacks, establish the common root causes, look at lessons learned, and establish how you can proactively secure your environment against these real world risks.

Transcript

1. Learning from AWS Customer
   Security Incidents
   Rami McCarthy @ramimacisabird
   

   View Slide

2. Rami McCarthy
   Product Security Engineer
   (Reformed Security Consultant)
   ● AWS Certified Security, Specialty
   ● CCSKv4
   Creator of sadcloud Contributor to ScoutSuite
   

   View Slide

3. ● Background
   ● Breaches
   ● Prior Art
   ● Case Studies
   ● Trends
   ● Recommendations
   

   View Slide

4. Photo by Kenrick Mills on Unsplash
   The Cloud
   

   View Slide

5. View Slide

6. View Slide

7. Breaches
   

   View Slide

8. Blameless Post-Mortems
   https://landing.google.com/sre/sre-book/chapters/postmortem-culture
   

   View Slide

9. The common cases - S3, ElasticSearch
   ● https://github.com/nagwww/s3-leaks
   ● https://www.upguard.com/breaches
   @ramimacisabird
   

   View Slide

10. Database Ransomware
    ● AWS services or user managed
    ● Generally, internet exposed with a weak password
    ● BTC ransom
    ● Some examples:
    ○ https://mangolassi.it/topic/19664/database-held-for-ransom-a
    nyone-experience-this-before/16
    ○ https://forums.aws.amazon.com/thread.jspa?threadID=249445
    @ramimacisabird
    

    View Slide

11. Prior Art
    

    View Slide

12. View Slide

13. View Slide

14. View Slide

15. View Slide

16. View Slide

17. Case Studies
    

    View Slide

18. Capital One
    2019
    Initial Access:
    ● Misconfigured “firewall” (WAF)
    ● SSRF -> Metadata
    Escalation/Persistence:
    ● Over-Privileged EC2 Role
    Impact:
    ● 100M+ Credit Card Applications
    stored in S3
    https://www.fugue.co/blog/a-technical-analysis-of-the-capital-one-cloud-misconfiguration-breach
    @ramimacisabird
    1/19
    

    View Slide

19. Code Spaces
    2014
    Initial Access:
    ● AWS Console Credentials
    (Phishing?)
    Escalation/Persistence:
    ● Attacker created additional
    accounts/access keys
    Impact:
    ● Wiped S3 buckets, EC2 instances,
    AMIs, EBS snapshots
    https://threatpost.com/hacker-puts-hosting-service-code-spaces-out-of-business/106761/
    @ramimacisabird
    2/19
    

    View Slide

20. DNC Hack by the
    GRU
    2016
    Initial Access:
    ● Unknown, test clusters breached
    Escalation/Persistence:
    ● EC2 Snapshots copied to attacker
    AWS accounts
    Impact:
    ● Tableau and Vertica Queries
    https://www.politico.com/f/?id=00000168-6161-de11-af7d-ef7327ea0000
    @ramimacisabird
    3/19
    

    View Slide

21. DataDog
    2016
    Initial Access:
    ● CI/CD AWS access key and SSH
    private key leaked
    Escalation/Persistence:
    ● Attacker attempted to pivot with
    customer credentials
    Impact:
    ● 3 EC2 instances and subset of S3
    buckets
    https://www.datadoghq.com/blog/2016-07-08-security-notice/
    @ramimacisabird
    4/19
    

    View Slide

22. Uber
    2016
    Initial Access:
    ● Private Github Repo with AWS
    credentials
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Names and driver’s license numbers
    of 600k drivers
    ● PII of 57 million users
    https://www.bloomberg.com/news/articles/2017-11-21/uber-concealed-cyberattack-that-exposed-57-million-pe
    ople-s-data
    https://www.commerce.senate.gov/services/files/7d70e53e-73e9-4336-a100-67b233084f12
    @ramimacisabird
    5/19
    

    View Slide

23. OneLogin
    2017
    Initial Access:
    ● AWS keys
    Escalation/Persistence:
    ● Created EC2 instances
    Impact:
    ● Accessed database tables (with
    encrypted data)
    https://www.onelogin.com/blog/may-31-2017-security-incident
    @ramimacisabird
    6/19
    

    View Slide

24. Politifact
    2017
    Initial Access:
    ● “Misconfigured cloud computing
    server”
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Coinhive cryptojacking
    https://www.washingtonpost.com/news/the-switch/wp/2017/10/13/hackers-have-turned-politifacts-website-in
    to-a-trap-for-your-pc/
    @ramimacisabird
    7/19
    

    View Slide

25. LA Times
    2018
    Initial Access:
    ● S3 global write access
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Coinhive cryptojacking added to
    homicide.latimes.com
    https://www.tripwire.com/state-of-security/security-data-protection/la-times-website-cryptojacking-attack/
    @ramimacisabird
    8/19
    

    View Slide

26. Tesla
    2018
    Initial Access:
    ● Globally exposed Kubernetes
    console
    ● Pod with AWS credentials
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Cryptojacking
    https://www.wired.com/story/cryptojacking-tesla-amazon-cloud/
    @ramimacisabird
    9/19
    

    View Slide

27. Imperva
    2018
    Initial Access:
    ● “Internal compute instance”
    globally accessible
    ● “Contained” AWS API key
    Escalation/Persistence:
    ● N/A
    Impact:
    ● RDS snapshot stolen
    https://www.imperva.com/blog/ceoblog/
    @ramimacisabird
    10/19
    

    View Slide

28. Cisco
    2018
    Initial Access:
    ● Former employee with AWS access 5
    months post-resignation
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Deleted ~450 EC2 instances
    https://www.zdnet.com/article/former-cisco-engineer-pleads-guilty-to-network-damage-wiping-16000-webex-tea
    ms-accounts/
    @ramimacisabird
    11/19
    

    View Slide

29. JW Player
    2019
    Initial Access:
    ● Weave Scope (publicly exposed),
    RCE by design
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Cryptojacking
    https://medium.com/jw-player-engineering/how-a-cryptocurrency-miner-made-its-way-onto-our-internal-kuber
    netes-clusters-9b09c4704205
    @ramimacisabird
    12/19
    

    View Slide

30. Malindo Air
    2019
    Initial Access:
    ● Former employees for an
    e-commerce provider abused their
    access
    Escalation/Persistence:
    ● N/A
    Impact:
    ● 35 million customer records
    https://www.infosecurity-magazine.com/news/malindo-air-data-breach-was-inside/
    @ramimacisabird
    13/19
    

    View Slide

31. Twilio
    2020
    Initial Access:
    ● S3 global write access
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Magecart
    https://www.twilio.com/blog/incident-report-taskrouter-js-sdk-july-2020
    @ramimacisabird
    14/19
    

    View Slide

32. Magecart and S3 Global Write … interlude
    As of July 2019:
    “the group has managed to compromise a vast collection
    of S3 buckets to impact well over 17,000 domains”
    - RiskIQ
    https://www.riskiq.com/blog/labs/magecart-amazon-s3-buckets/
    

    View Slide

33. “Behind the scenes
    in the Expel SOC:
    Alert-to-fix in
    AWS”
    2020
    Initial Access:
    ● Root IAM user access key
    compromised
    Escalation/Persistence:
    ● SSH keys generated for EC2
    instances
    Impact:
    ● Cryptojacking
    https://expel.io/blog/behind-the-scenes-expel-soc-alert-aws/
    @ramimacisabird
    15/19
    

    View Slide

34. 2020
    “Finding evil in
    AWS: A key pair to
    remember”
    Initial Access:
    ● 8 IAM access keys compromised
    Escalation/Persistence:
    ● Backdoored security groups
    Impact:
    ● Command line access to EC2
    instances
    https://expel.io/blog/finding-evil-in-aws/
    @ramimacisabird
    16/19
    

    View Slide

35. TeamTNT Worm
    2020
    Initial Access:
    ● Misconfigured Docker & k8s
    platforms
    Escalation/Persistence:
    ● Steals AWS credentials from
    ~/.aws/*
    Impact:
    ● Cryptojacking for Monero
    https://www.cadosecurity.com/post/team-tnt-the-first-crypto-mining-worm-to-steal-aws-credentials
    @ramimacisabird
    17/19
    

    View Slide

36. Cryptomining
    AMI
    2020
    Initial Access:
    ● Windows 2008 Server Community
    AMI
    Escalation/Persistence:
    ● N/A
    Impact:
    ● Monero miner
    https://www.darkreading.com/cloud/cryptominer-found-embedded-in-aws-community-ami/d/d-id/1338713
    @ramimacisabird
    18/19
    

    View Slide

37. 2020
    Mandiant: Insider
    Threat Scenario
    Initial Access:
    ● Fired employee uses credentials
    Escalation/Persistence:
    ● Access CI/CD server, create a new
    user, steal credentials
    Impact:
    ● Deleted production databases
    https://www.youtube.com/watch?v=rtEjI_5TPdw
    @ramimacisabird
    19/19
    

    View Slide

38. Trends
    

    View Slide

39. Mitre Att&ck AWS Matrix
    https://attack.mitre.org/matrices/enterprise/cloud/aws/
    

    View Slide

40. Mitre Att&ck AWS Matrix
    

    View Slide

41. Initial Vectors
    @ramimacisabird
    +
    

    View Slide

42. Initial Vectors
    @ramimacisabird
    +
    

    View Slide

43. Recommendations
    ● SSDLC
    ● Penetration Testing
    ● IMDSv2
    Metadata Access
    ● Access Analyzer for S3*
    https://docs.aws.amazon.com/AmazonS3
    /latest/user-guide/access-analyzer.html
    S3 Global Write
    ● Only use trusted
    sources
    Malicious AMIs
    @ramimacisabird
    

    View Slide

44. Recommendations
    ● Asset Inventory
    ● Patch Management
    ● Limit External
    Exposure
    Application Vulnerability
    ● Offboarding
    ● Third party risk
    ● Principle of Least
    Privilege
    ● Logging/Monitoring
    ○ Heuristics
    ■ Time
    ■ Location
    ■ Activity
    Valid Credential Abuse
    ● IAM Best Practices
    ○ MFA
    ○ Key Rotation
    ○ Avoid static
    credentials
    ● Principle of Least
    Privilege
    ○ Cloudsplaining
    Valid Credential Theft
    @ramimacisabird
    

    View Slide

45. https://opensource.salesforce.com/cloudsplaining/#/summary
    @ramimacisabird
    

    View Slide

46. @ramimacisabird
    Thank you!
    Feel free to track me
    down in Discord or
    on Twitter
    https://speakerdeck.com/ramimac
    Puppy tax
    

    View Slide

47. Top 10 Cloud Attack Killchains
    1. Static API Credential Exposure to Account Hijack
    2. Compromised Server via Exposed Remote Access Ports
    3. Compromised Database via Inadvertent Exposure
    4. Object Storage Public Data Exposure
    5. Server Side Request Forgery
    6. Cryptomining
    7. Network Attack
    8. Compromised Secrets
    9. Novel Cloud Data Exposure and Exfiltration
    10. Subdomain Takeover
    https://disruptops.com/stop-todays-top-10-cloud-attack-killchains/
    @ramimacisabird
    

    View Slide