Defending the Cloud from the Full Stack Hack | Source Boston 2016

Amazon Web Services (AWS) and other cloud services are billed as an amazingly secure and resilient cloud services provider, but what is the reality? Once you look past that pristine environment and the manicured forests and start to build on top of it you’ll find yourself very quickly in a dark jungle of your own creation.

With concrete examples this presentation will explore “full stack” vulnerabilities and their effect on security and how they create new pitfalls when migrating to and operating in a Cloud world. From the simple (checking in your Cloud credentials to github or embedding them in your app) the unexpected (XXE injection to expose Cloud metadata), to the unintended (data leakage and service exposure and 3rd party cloud management services). Many examples will be shared along side new techniques showing how easy it is to expose your applications and infrastructure to attack through misunderstanding, ignorance or bad actors.

Transcript

1. May/2016 Defending the Cloud from the Full Stack Hack Erik Peterson/Veracode/@silvexis
2. Hi, I’m Erik, thanks for coming! • Cloud Security Weapons Manufacturer @ Veracode • Researching the Cloud and architecting solutions in AWS since 2009 @silvexis
3. EVERYBODY HAS A PLAN UNTIL THEY GET PUNCHED IN THE FACE GO CLOUD
4. CLOUD
 IS THE NEW OPERATING SYSTEM
5. CLOUD INFRASTRUCTURE 
 IS CODE
6. A Fully Realized AWS Application
7. Traditional App represents small % of system • Java, .Net, etc… including 3rd party code in red
8. Majority of the system provided by AWS Using it all securely however is your responsibility All of this however is your application and securing it is your responsibility
9. What you get from AWS • The digital equivalent of infinite empty rack space • A friendly looking web interface • A mile long list of compliance certifications • The rest is up to you
10. …and yet after a few months this might be you
11. EMERGENT INSECURITY The individual components of your system may be secure but when deployed into the Cloud, the system becomes insecure AND IT GETS WORSE AT SCALE
12. THE 4 HORSEMEN OF EMERGENT INSECURITY INTERNET WEATHER SOFTWARE DEFINED EVERYTHING GUARANTEED FAILURE OUT-OF-BAND SERVICE API
13. What’s your real attack surface? API’s drive everything, existing security controls haven’t caught up For example, to secure a system on premise: Block all network traffic using host FW, throw away SSH keys/passwords, install network IDS, log all network traffic to/from system System is now inaccessible, any attempt to access would also be detected In the cloud however Use APIs to snapshot the disk, mount snapshots on different system, and extract everything without touching the network or system Zero indication from traditional controls that any access has taken place Same is true of cloud databases, I don’t need your passwords or even SQL Injection, I just need to snapshot your DB
14. API ACCESS
    
    
    
    
    PHYSICAL ACCESS IN THE CLOUD
15. API Credential Exposure Impact The attacker can: Sets up bitcoin mining operations within your cloud environment Alter your applications to spread malware Uses your environment as a means to launch additional attacks Download all your customers data
16. Destroy your entire Cloud
17. Getting Access to the API Checking your API keys into Github (Not so common anymore thankfully) Exploiting vulnerabilities to access Cloud Metadata
18. Oops Keys in code Not so common anymore
19. However….why are keys in code at all? 78,667 placeholder results?! There is no good reason to put API keys in code, it’s dangerous Use IAM roles or local AWS configuration files
21. What is Cloud Metadata? Based on RFC 3927 - Dynamic Configuration of IPv4 Link-Local Addresses Metadata contains all kinds of awesome things, like startup scripts and….your AWS access credentials (if you are using roles) It’s not just AWS, all cloud providers have it AWS: http://169.254.169.254/latest/user-data Google: http://169.254.169.254/computeMetadata/v1/ MS Azure, most is in a static file, but that’s changing http://169.254.169.254/ metadata/v1/ There is nothing wrong with any of this, as long as you are aware and protect it
22. $ wget -q -O - http://169.254.169.254/latest/meta-data/ ami-id ami-launch-index ami-manifest-path block-device-mapping/ hostname iam/ instance-action instance-id instance-type kernel-id local-hostname local-ipv4 mac metrics/ network/ placement/ profile public-hostname public-ipv4 public-keys/ reservation-id security-groups ACCESSING AWS METADATA AWS command line tool ec2-metadata will extract some (but not all!) of the metadata To get everything use wget or curl Each one of these represents meta data you can access $ wget -q -O - http://169.254.169.254/latest/meta-data/instance-id i-0496132e
23. METADATA USER-DATA User data is distributed in AWS via the metadata API Often will also contain sensitive information [ec2-user@ip-10-0-4-32 ~]$ wget -q -O - http://169.254.169.254/latest/user-data #!/bin/bash yum update -y yum install -y aws-cli wget vim python27-pip mkdir -p /etc/worker echo config > /etc/worker/environment_config_bucket echo “mysql://system:fsdkj4540@#$2-fV2@10.0.4.50:3306" > /etc/worker/mysql.conf
24. METADATA IAM CREDENTIALS [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/security- credentials/<YOUR PROFILE HERE> { "Code" : "Success", "LastUpdated" : "2014-08-13T16:43:54Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA", "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV", "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/iF94PTi8+kO5sxgboy4XPB +Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g+iyZLw8mT5ADz9zGUQTf +P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf +GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS+sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/ Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc +iFQaFBvCD3q66fBQ==", "Expiration" : "2014-08-13T23:05:19Z" }
25. METADATA CREDENTIAL EXTRACTION [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/security- credentials/<YOUR PROFILE HERE> { "Code" : "Success", "LastUpdated" : "2014-08-13T16:43:54Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA", "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV", "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/iF94PTi8+kO5sxgboy4XPB +Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g+iyZLw8mT5ADz9zGUQTf +P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf +GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS+sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/ Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc +iFQaFBvCD3q66fBQ==", "Expiration" : "2014-08-13T23:05:19Z" }
26. OLD VULNS NEW LIFE These vulnerabilities can result in a
 total data center compromise
 in AWS CWE-918: SSRF CWE-611: XXE CWE-441: Unintended Proxy or Intermediary CWE-77: Command Injection CWE-200: Information Exposure Why? All of these can lead to unintended exposure of metadata
27. Has this really happened? Real world Examples Prezi experienced this first hand http://engineering.prezi.com/blog/2014/03/24/prezi-got- pwned-a-tale-of-responsible-disclosure/ Andres Riancho demonstrated this in his BlackHat talk “Pivoting in Amazon Clouds” Exploited SSRF vuln to inject malicious AWS SQS messages to then exploit a celery/python vuln and take control
28. Is Your Cloud is Leaking? Leaking tags to 3rd party services EC2 Classic RFC 1918 (private IP range) networks are not private Unsecured S3 buckets accessible to anyone able to guess their name
29. The Full Stack Hack 3. SSRF or XXE vulnerability exposes EC2 Metadata, revealing AWS API Keys 1. AWS DNS Reveals Private IP of web server 2. Private IP of web server reveals detailed errors or admin interface 4. AWS API Key lets you create new instances where your clone and mount any existing EBS vol. 5. Cloned system gets you SSH keys to app servers and API key with IAM:* giving you access to everything 6. With new root credentials create trust relationship with external account and clone DB for quiet extraction
30. DEFENDING YOURSELF
31. The basics Encrypt all the things, turn on EBS and RDS encryption even if you are going to let AWS manage the keys, it will complicate exfiltration Turn on AWS Cloudtrail for all regions, use Cloudtrail logs to monitor Turn on AWS Config and use Config rules to enforce a basic defaults like Required tags, limit open ports, encryption use, open policies
32. AppSec is your next problem AppSec was already the #1 problem you were not paying attention too. Good news, now it’s worse! Application vulnerabilities can lead to complete infrastructure exploitation All of your systems need constant, automated review Automated Code security testing must be part of your development process Make continuous testing for known vulnerabilities part of your operating environment
33. Protecting the Metadata API Metadata API is a great service, don’t just block access to it outright, it can have unintended consequences 1. Ensure you are auditing your systems for vulnerabilities or “features” that could expose access to the metadata IP 169.254.169.254 2. Restrict API key permissions, aggressively practice least privilege
34. Limit API Access Restrict sensitive API Keys Lock to specific IP addresses Require MFA Eliminate * policies!! Ask yourself “If this key was compromised, what is the worst that could happen?” { "Version": "2012-10-17", "Statement":[{ "Effect":"Deny", "Action":"*", "Resource":"*", "Condition":{ "NotIpAddress":{ “aws:SourceIp”:[“VPC GATEWAY IP/32”] } } }] } { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "*", "Resource": "*" }] }
35. Plug Leaks Ensure tags do not contain sensitive data Lock down S3 buckets If you have been using AWS for a long time, migrate off of EC2 Classic
36. This is not your IDS Do not use your bill as your IDS! Implement API logging AWS: Cloudtrail Azure: Audit Logs Actually monitor your cloud logs Invest in a host IDS/monitoring solution that is cloud aware
37. @silvexis erik@silvexis.com THANK YOU May all your Clouds come with a Luck Dragon