Share
Embed
Transcript
45 slides
-
Hi, I’m Erik • AppSec guy for 17 years •
Director of Technology Strategy at Veracode • Have been researching the Cloud and building products on top of AWS since 2009 @silvexis -
AGENDA I II III IV INTRO GETTING ACCESS V VI
IV PROBLEMS MACHETE FULL STACK HACKS THE API IS KING EMERGENT INSECURITY -
Traditional App represents small % of system • Java, .Net,
etc… including 3rd party code in red -
Majority of the system provided by AWS Using it all
securely however is your responsibility -
What you get from AWS • The digital equivalent of
infinite empty rack space • A friendly looking web interface • A mile long list of compliance certifications • The rest is up to you -
Forklifting also Dangerous Forklifting is the process of taking legacy
data center applications and loading them into a cloud provider with little or no changes Often this is the most expensive way to use AWS It is also dangerous IDS and network access controls are unaware of AWS API activity Previously ignored or de-prioritized vulnerabilities can become critical The meaning of availability changes -
EMERGENT INSECURITY The individual components of your system may be
secure but when deployed into the Cloud, the system becomes insecure AND IT GETS WORSE AT SCALE -
EMERGENT INSECURITY FACTORS “Internet Weather” - AWS systems and API
calls are subject to unpredictable, non-persistent, network latency, system performance and connection interruption Guaranteed Failure - System availability is a factor of redundancy and automation, not the stability and performance of monolithic systems Software defined everything (SDe) - Virtualized networks, network interfaces, file systems, computing power and more can change independent of the underlying system Out-of-band management - AWS API’s operate outside of traditional security controls, can make all existing controls irrelevant -
EMBRACE AN EVENTUALLY CONSISTENT SECURITY MODEL Instead of trying to
enforce change control which creates brittle systems that are insecure and not survivable, design your systems to be eventually consistent with your security and operational goals If your system requires strict change control to maintain order, in the Cloud, you will eventually have chaos. -
ALL HAIL THE KING In the cloud, the king of
the jungle is the API -
What’s your Real Attack Surface? “If your security sucks now,
you’ll be pleasantly surprised by the lack of change when you move to Cloud.”* In reality is it gets worse AWS API endpoints are open by design, trumps all existing controls Private IP’s might be public, data leaks through 3rd party services Only thing standing between total compromise of your _entire_ datacenter is the secrecy of your API keys *Chris Hoff, @beaker -
API Credential Exposure Impact The attacker can: Sets up bitcoin
mining operations within your cloud environment Alter your applications to spread malware Uses your environment as a means to launch additional attacks Downloads all your customers data -
Bypassing traditional controls via the API Start with an EC2
system in AWS-VPC, block all network traffic, throw away SSH keys/ passwords, place IDS on VPC Gateway to detect intrusion attempts, log all network traffic to/from system In a traditional data center, system is now inaccessible, any attempt to access would be detected In AWS, use API to snapshot AMI/EBS vol’s, mount snapshots on different system, extract whatever I need/want. Zero indication from traditional controls that any access has taken place Same is true of RDS, I don’t need your passwords, I just need to snapshot your DB -
Getting Access to the API Checking your API keys into
Github Exploiting vulnerabilities to access Metadata -
API Honeypot? Curious how fast an API key can be
stolen from GitHub? Create a key, lock it to only one IP address and watch your logs You should see failed access attempts in 60 minutes or less -
What is Cloud Metadata? Based on RFC 3927 - Dynamic
Configuration of IPv4 Link-Local Addresses Metadata contains all kinds of awesome things, like startup scripts and your AWS access credentials It’s not just AWS, all cloud providers have it (except for MS Azure!) AWS: http://169.254.169.254/latest/user-data Google: http://169.254.169.254/computeMetadata/v1/ OpenStack/RackSpace: http://169.254.169.254/openstack HP Helion: http://169.254.169.254/2009-04-04/meta-data/ On Azure, metadata is not dynamic but is copied to /var/lib/waagent (linux) or %SYSTEMDRIVE%\AzureData\CustomData.bin (Windows) There is nothing wrong with any of this, as long as you are aware and protect it -
$ wget -q -O - http://169.254.169.254/latest/meta-data/ ami-id ami-launch-index ami-manifest-path block-device-mapping/
hostname iam/ instance-action instance-id instance-type kernel-id local-hostname local-ipv4 mac metrics/ network/ placement/ profile public-hostname public-ipv4 public-keys/ reservation-id security-groups Accessing AWS Metadata AWS command line tool ec2-metadata will extract some (but not all!) of the metadata To get everything use wget or curl Each one of these represents meta data you can access $ wget -q -O - http://169.254.169.254/latest/meta-data/instance-id i-0496132e -
METADATA CREDENTIAL EXTRACTION [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/security-
credentials/<YOUR PROFILE HERE> { "Code" : "Success", "LastUpdated" : "2014-08-13T16:43:54Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA", "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV", "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/iF94PTi8+kO5sxgboy4XPB +Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g+iyZLw8mT5ADz9zGUQTf +P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf +GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS+sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/ Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc +iFQaFBvCD3q66fBQ==", "Expiration" : "2014-08-13T23:05:19Z" } -
METADATA CREDENTIAL EXTRACTION [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/security-
credentials/<YOUR PROFILE HERE> { "Code" : "Success", "LastUpdated" : "2014-08-13T16:43:54Z", "Type" : "AWS-HMAC", "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA", "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV", "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/iF94PTi8+kO5sxgboy4XPB +Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g+iyZLw8mT5ADz9zGUQTf +P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf +GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS+sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/ Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc +iFQaFBvCD3q66fBQ==", "Expiration" : "2014-08-13T23:05:19Z" } -
OLD VULNS NEW LIFE These vulnerabilities can result in a
total data center compromise in AWS CWE-918: SSRF CWE-611: XXE CWE-441: Unintended Proxy or Intermediary CWE-77: Command Injection CWE-200: Information Exposure Why? All of these can lead to unintended exposure of metadata -
Remote Access to Metadata Real world Examples Prezi experienced this
first hand http://engineering.prezi.com/blog/2014/03/24/prezi-got- pwned-a-tale-of-responsible-disclosure/ Andres Riancho demonstrated this in his BlackHat talk “Pivoting in Amazon Clouds” Exploited SSRF vuln to inject malicious AWS SQS messages to then exploit a celery/python vuln and take control -
Controlling API Access IP Restrictions, they are not just for
security groups! Keys locked down to specific IP’s (like your VPC gateway) have less chance of damage Doesn’t fully eliminate the problem however { "Version": "2012-10-17", "Statement":[{ "Effect":"Deny", "Action":"*", "Resource":"*", "Condition":{ "NotIpAddress":{ “aws:SourceIp”:[“VPC GATEWAY IP/32”] } } }] } -
Your bill is not an IDS Many AWS customers today
use their bill as their IDS So far, most of these people have been pretty lucky -
Implement API Logging…please! Logging is off by default in AWS
In AWS, turn on CloudTrail for all regions Use LogStash, develop your own or buy a commercial solution to make API activity logs accessible -
EXCESSIVE LACK OF ACCESS CONTROL { "Version": "2012-10-17", "Statement": [
{ "Effect": "Allow", "Action": "*", "Resource": "*" } ] } -
DevOps Culture tends to “Fail Open” I love DevOps but….
Be wary of Developers building systems who are new to the Cloud because they might now be the ones configuring the firewall rules and API Access Policies as well Developers and Operations are often pre-disposed to “Just make it work” This is one of the reasons DevOps is such a powerful movement It’s on us as security people to adapt and keep DevOps powerful -
Leaking Tags Tags in AWS are one of the primary
means of keeping track of things Tags however often can contain private data that does not have strong access protections I’ve seen API keys and passwords in tags!!! If you use any 3rd party AWS management system like StackDriver, CloudHealth or others, your tag data is replicated to those systems -
PRIVATE DNS IS PUBLIC DNS AWS DNS from within AWS
will resolve EC2 Classic public DNS entries to RFC 1918 private address Good chance of Information Exposure (CWE-200) from web servers listening on private IP’s A casual poke around the EC2 10.0.0.0/8 address space reveals hundreds of web servers Remediation: Migrate EC2 Classic environments to VPC [ec2-user@heyzeus ~]$ host example.veracodelabs.com example.veracodelabs.com is an alias for ec2-107-22-197-187.compute-1.amazonaws.com. ec2-107-22-197-187.compute-1.amazonaws.com has address 10.72.187.67 -
Data and File Leakage AWS S3 1 in 7 buckets
left open to anyone attempting to access them This might very well be intentional but security through obscurity is a real problem AWS EBS If I have API access, I can clone your running systems and EBS volumes into an external account You can prevent this by encrypting your EBS volumes (you can’t encrypt root volumes however) -
The Full Stack Hack 3. SSRF or XXE vulnerability exposes
EC2 Metadata, revealing AWS API Keys 1. AWS DNS Reveals Private IP of web server 2. Private IP of web server reveals detailed errors or admin interface 4. AWS API Key lets you create new instances where your clone and mount any existing EBS vol. 5. Cloned system gets you SSH keys to app servers and API key with IAM:* giving you access to everything 6. With new root credentials create trust relationship with external account and clone DB for quiet extraction -
Capabilities Analyzes IAM users and permissions Requires only readonly API
access Records all services and objects in use Track changes, know what is in your account quickly Image Cracker Crack open AMI and EBS snapshots and extract access keys, user accounts, data Coming Soon Not quite ready yet, sorry! -
IAM Permissions Analysis epeterson@epeterson-mac:~/work/machete$ bin/machete -e 67d54268-dab1-4c73-a424-dbe75d466bb5 refresh Machete -
v0.1.5 - Connecting... ...connected Starting scan of environment 67d54268-dab1-4c73-a424-dbe75d466bb5 ================================================================================ id: 67d54268-dab1-4c73-a424-dbe75d466bb5|AROAIEMNXXXX3FXXXXXHM6 cust uuid: f4cec92f-9184-4394-a4d3-13b7d8708a13 env uuid: 67d54268-dab1-4c73-a424-dbe75d466bb5 First seen: 2014-10-11 13:38:20.335481+00:00 Last seen: 2014-10-17 10:04:00.754355+00:00 Parent: None Type: IAM_ROLE Tags: {} Metadata: -------------------------------------------------------------------------------- {u'arn': u'arn:aws:iam::034817543207:role/cloudatlas_node', u'assume_role_policy_document': {u'Statement': [{u'Action': u'sts:AssumeRole', u'Effect': u'Allow', u'Principal': {u'Service': u'ec2.amazonaws.com'}, u'Sid': u''}], u'Version': u'2012-10-17'}, u'create_date': u'2014-06-11T20:25:43Z', u'id': u'AXXXXXXNKYAIXXXXXXXX', u'name': u'cloudatlas_node', u'path': u'/', u'role_id': u'AXXXXXXNKYAIXXXXXXXX', u'role_name': u'cloudatlas_node'} -------------------------------------------------------------------------------- Versions: 1 -------------------------------------------------------------------------------- Change detected on: 2014-10-17 12:03:48 {u'added': {}, u'changed': {u'assume_role_policy_document': {u'Statement': [{u'Action': u'sts:AssumeRole', u'Effect': u'Allow', u'Principal': {u'Service': u'ec2.amazonaws.com'}, u'Sid': u''}], u'Version': u'2012-10-17'}}, u'removed': {}} -------------------------------------------------------------------------------- -
Resource tracking (machete)epeterson@epeterson-mac:~/work/machete/bin$ ./machete list -e 67d54268-dab1-4c73-a424-dbe75d466bb5 # : ID
Type State First Seen Last Seen Versions -------------------------------------------------------------------------------------------------------------------------------------------- 1 : 54.84.xxx.xxx ADDRESS None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:24.617682+00:00 No changes 2 : i-xxxxxxxx EC2 running 2014-08-21 15:34:16.964497+00:00 2014-09-02 18:23:59.619777+00:00 No changes 3 : i-xxxxxxxx EC2 stopped 2014-08-21 18:09:49.462541+00:00 2014-09-02 18:23:59.688451+00:00 {u'1408649760.41495': {u'added': {}, u'changed': {u'dns_name': u'', u'ip_address': None, u'public_dns_name': u'', u'reason': u'User initiated (2014-08-21 19:34:02 GMT)', u'state': [u'stopped', 80], u'state_reason': {u'code': u'Client.UserInitiatedShutdown', u'message': u'Client.UserInitiatedShutdown: User initiated shutdown'}}, u'removed': {}}} 4 : i-xxxxxxxx EC2 running 2014-08-29 19:41:35.097611+00:00 2014-09-02 18:23:59.682205+00:00 No changes 5 : i-xxxxxxxx EC2 running 2014-08-21 15:34:16.964497+00:00 2014-09-02 18:23:59.693405+00:00 No changes 6 : i-xxxxxxxx EC2 running 2014-08-21 15:34:16.964497+00:00 2014-09-02 18:23:59.678996+00:00 No changes 7 : igw-xxxxxxxx VPC_INTERNET_GATEWAY None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.219698+00:00 No changes 8 : igw-xxxxxxxx VPC_INTERNET_GATEWAY None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.936409+00:00 No changes 9 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:26.893741+00:00 No changes 10 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.240372+00:00 No changes 11 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:28.098728+00:00 No changes 12 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:27.484144+00:00 No changes 13 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.956196+00:00 No changes 14 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:28.691644+00:00 No changes 15 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.354823+00:00 No changes 16 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:24.590336+00:00 No changes 17 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:26.966074+00:00 No changes 18 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.913403+00:00 No changes 19 : sg-xxxxxxxx EC2_SECURITY_GROUP None 2014-09-02 18:12:16.600848+00:00 2014-09-02 22:39:30.547159+00:00 No changes -
Other Tools Netflix Security Monkey http://techblog.netflix.com/2014/06/announcing-security-monkey-aws-security.html …and a whole ton
of other OSS stuff you should check out AWS Trusted Advisor https://aws.amazon.com/premiumsupport/trustedadvisor/ NimboStratus https://andresriancho.github.io/nimbostratus/