Share
Embed
Bringing a Machete to the Amazon - AppSec USA 2014
So you are on AWS, now what? How do you cut through the jungle and secure your environment for the Cloud?
Erik Peterson PRO
September 19, 2014
Transcript
41 slides
-
AppSec USA 2014! Denver, Colorado Erik Peterson - @silvexis -
[email protected] Bringing a Machete to the Amazon -
! • AppSec guy for 17 years ! • Director
of Technology Strategy at Veracode ! • Have been researching the Cloud and building products on top of AWS since 2009 Hi, I’m Erik @silvexis -
AGENDA I II III IV INTRO OLD VULNS, NEW
LIFE V VI IV METADATA CLOUDATLAS PROJECT FULL STACK HACKS THE API IS KING EMERGENT INSECURITY -
• Java, .Net, etc… including 3rd party code in red
Traditional App represents small % of system -
• Using it all securely however is your responsibility Majority
of the system provided by AWS -
What you get from AWS • The digital equivalent of
infinite empty rack space • A friendly looking web interface • A mile long list of compliance certifications ! ! ! ! ! • The rest is up to you -
…and after a few months this might be you •
You have 39 security groups named “Launch-wizard-x”, half of which allow all from 0.0.0.0 • Your bill is 5x what you were expecting • 2 teams are asking you why their systems disappeared and 8 more are un-reachable • You have 176 instances running and you only know what 7 of them do -
• Treating AWS as just another hosting company is dangerous
• For starters, it’s likely the most expensive way to use AWS • IDS and network access controls are unaware of AWS API activity • Certain low priority vulnerabilities become critical Forklifting isn’t safe either -
• “If your security sucks now, you’ll be pleasantly surprised
by the lack of change when you move to Cloud.”* • In reality is it gets worse because of API access and unexpected information leakage – AWS API endpoints are open by default, trumps all existing controls – Private IP’s might be public – Only thing standing between total compromise of your _entire_ datacenter is the secrecy of your API keys What’s your Attack Surface? *Chris Hoff, @beaker -
The individual components of your system may be secure but
when deployed into the Cloud, the system becomes insecure EMERGENT INSECURITY AND IT GETS WORSE AT SCALE -
Instead of trying to enforce change control which creates brittle
systems that are insecure and not survivable focus on creating environments that are eventually consistent with your security and operational goals ! If your system requires strict change control to maintain order, in the cloud, you will eventually have chaos. EMERGENT INSECURITY -
In the cloud, the king of the jungle is the
API THE API IS KING -
• The following vulnerabilities can mean total data center compromise
in AWS • CWE-918: SSRF • CWE-611: XXE • CWE-441: Unintended Proxy or Intermediary • CWE-77: Command Injection • Why? • Metadata! OLD VULNS NEW LIFE -
• Based on RFC 3927 - Dynamic Configuration of IPv4
Link-Local Addresses • Metadata contains all kinds of awesome things, like startup scripts and your AWS access credentials • It’s not just AWS, all cloud providers have it – AWS: http://169.254.169.254/latest/user-data – Google: http://169.254.169.254/computeMetadata/v1/ – OpenStack/RackSpace: http://169.254.169.254/openstack – HP Helion: http://169.254.169.254/2009-04-04/meta-data/ – etc… • There is nothing wrong with any of this, as long as you are aware and protect it What is Cloud Metadata? -
• AWS command line tool ec2-metadata will extract some (but
not all!) of the metadata • To get everything use wget or curl Accessing AWS Metadata $ wget -q -O - http://169.254.169.254/latest/meta-data/! ami-id! ami-launch-index! ami-manifest-path! block-device-mapping/! hostname! iam/! instance-action! instance-id! instance-type! kernel-id! local-hostname! local-ipv4! mac! metrics/! network/! placement/! profile! public-hostname! public-ipv4! public-keys/! reservation-id! security-groups Each one of these represents meta data you can access $ wget -q -O - http://169.254.169.254/latest/meta-data/instance-id! i-0496132e -
METADATA CREDENTIAL EXTRACTION [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/
security-credentials/<YOUR PROFILE HERE>! {! "Code" : "Success",! "LastUpdated" : "2014-08-13T16:43:54Z",! "Type" : "AWS-HMAC",! "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA",! "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV",! "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/ iF94PTi8+kO5sxgboy4XPB+Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g +iyZLw8mT5ADz9zGUQTf+P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/ L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf+GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS +sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc +iFQaFBvCD3q66fBQ==",! "Expiration" : "2014-08-13T23:05:19Z"! } -
METADATA CREDENTIAL EXTRACTION [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/
security-credentials/<YOUR PROFILE HERE>! {! "Code" : "Success",! "LastUpdated" : "2014-08-13T16:43:54Z",! "Type" : "AWS-HMAC",! "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA",! "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV",! "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/ iF94PTi8+kO5sxgboy4XPB+Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g +iyZLw8mT5ADz9zGUQTf+P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/ L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf+GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS +sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc +iFQaFBvCD3q66fBQ==",! "Expiration" : "2014-08-13T23:05:19Z"! } -
• Prezi experienced this first hand • http://engineering.prezi.com/blog/ 2014/03/24/prezi-got-pwned-a-tale-of- responsible-disclosure/
• Andres Riancho demonstrated this in his BlackHat talk “Pivoting in Amazon Clouds” Remote Access to Metadata -
• IP Restrictions, they are not just for security groups!
• Keys locked down to specific IP’s could accidentally find their way in GitHub with less chance of damage • If you don’t need it or use it, you could also block it • route add -host 169.254.169.254 reject Controlling API Access {! "Version": "2012-10-17", ! "Statement":[{! "Effect":"Deny",! "Action":"*",! "Resource":"*",! "Condition":{! "NotIpAddress":{! “aws:SourceIp”:[“YOUR VPC NAT GATEWAY IP/32”]! }! }! }]! } -
• Curious how fast an API key can be stolen
from GitHub? • Create a key, lock it to only one IP address and watch your CloudTrail logs • You should see failed access attempts in 60 minutes or less API Honeypot Anyone? -
The Full Stack Hack 3. SSRF or XXE vulnerability
exposes EC2 Metadata, revealing AWS API Keys 1. AWS DNS Reveals Private IP of web server 2. Private IP of web server reveals detailed errors or admin interface 4. AWS API Key lets you create new instances where your clone and mount any existing EBS vol. 5. Cloned system gets you SSH keys to app servers and API key with IAM:* giving you access to everything 6. With new root credentials create trust relationship with external account and clone DB for quiet extraction -
RECOVERY DISASTER After a full compromise, the attacker can do
more than just kill a few systems to cover their tracks, they can nuke the entire datacenter Nuke the site from orbit, it’s the only way to be sure -
• Lets say I have read only to EBS but
no access to your servers – Actually, I have access to your servers – Clone the EBS vol’s, mount, extract whatever I need/want • Same is true of RDS, I don’t need your passwords, I just need to snapshot your DB Bypassing Auth with Clones -
• How many of you have accounts or API keys
with a policy that looks like this? EXCESSIVE LACK of ACCESS CONTROL {! "Version": "2012-10-17",! "Statement": [! {! "Effect": "Allow",! "Action": "*",! "Resource": "*"! }! ]! } -
• Tags in AWS are one of the primary means
of keeping track of things • Tags however often can contain private data that does not have strong access protections • If you use any 3rd party AWS management system like StackDriver, CloudHealth or others, your tag data is replicated to those systems TAGS -
• AWS DNS from within AWS will resolve EC2 Classic
public DNS entries to RFC 1918 private address • Good chance of Information Exposure (CWE-200) from web servers listening on private IP’s • A casual poke around the EC2 10.0.0.0/8 address space reveals hundreds of web servers • Remediation: Migrate EC2 Classic environments to VPC PRIVATE IS PUBLIC DNS [ec2-user@heyzeus ~]$ host example.veracodelabs.com! example.veracodelabs.com is an alias for ec2-107-22-197-187.compute-1.amazonaws.com.! ec2-107-22-197-187.compute-1.amazonaws.com has address 10.72.187.67 -
• Many AWS users use their bill as their IDS
• So far, most of these people have been pretty damn lucky ! • Don’t be one of these people, turn on CloudTrail • Use LogStash, develop your own or buy a commercial solution to keep track of API activity LACK OF AWARENESS -
• CloudAtlas analyzes the full stack of a cloud based
application • Requires only readonly API access • Free SaaS service for exploring your AWS environment • www.cloudatlas.veracodelabs.com ! • Coming Soon • Web Interface not ready yet, sorry! CloudAtlas -
• Things it does: • Enumerate all your services, user
accounts and permissions • Track and alert on changes in near real time • Map out your applications and services • Perform analysis on services in use CloudAtlas -
Service Discovery and Change Tracking (cloudatlas)epeterson@epeterson-mac:~/work/cloudatlas/bin$ ./cloudatlas list -e 67d54268-dab1-4c73-a424-dbe75d466bb5!
# : ID Type State First Seen Last Seen Versions! --------------------------------------------------------------------------------------------------------------------------------------------! 1 : 54.84.xxx.xxx ADDRESS None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:24.617682+00:00 No changes! 2 : i-xxxxxxxx EC2 running 2014-08-21 15:34:16.964497+00:00 2014-09-02 18:23:59.619777+00:00 No changes! 3 : i-xxxxxxxx EC2 stopped 2014-08-21 18:09:49.462541+00:00 2014-09-02 18:23:59.688451+00:00! {u'1408649760.41495': {u'added': {},! u'changed': {u'dns_name': u'',! u'ip_address': None,! u'public_dns_name': u'',! u'reason': u'User initiated (2014-08-21 19:34:02 GMT)',! u'state': [u'stopped', 80],! u'state_reason': {u'code': u'Client.UserInitiatedShutdown',! u'message': u'Client.UserInitiatedShutdown: User initiated shutdown'}},! u'removed': {}}}! ! 4 : i-xxxxxxxx EC2 running 2014-08-29 19:41:35.097611+00:00 2014-09-02 18:23:59.682205+00:00 No changes! 5 : i-xxxxxxxx EC2 running 2014-08-21 15:34:16.964497+00:00 2014-09-02 18:23:59.693405+00:00 No changes! 6 : i-xxxxxxxx EC2 running 2014-08-21 15:34:16.964497+00:00 2014-09-02 18:23:59.678996+00:00 No changes! 7 : igw-xxxxxxxx VPC_INTERNET_GATEWAY None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.219698+00:00 No changes! 8 : igw-xxxxxxxx VPC_INTERNET_GATEWAY None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.936409+00:00 No changes! 9 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:26.893741+00:00 No changes! 10 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.240372+00:00 No changes! 11 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:28.098728+00:00 No changes! 12 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:27.484144+00:00 No changes! 13 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.956196+00:00 No changes! 14 : r-xxxxxxxx EC2_RESERVATION None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:28.691644+00:00 No changes! 15 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.354823+00:00 No changes! 16 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:24.590336+00:00 No changes! 17 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:26.966074+00:00 No changes! 18 : rtb-xxxxxxxx VPC_ROUTE_TABLE None 2014-09-02 18:23:45.874644+00:00 2014-09-02 22:39:25.913403+00:00 No changes! 19 : sg-xxxxxxxx EC2_SECURITY_GROUP None 2014-09-02 18:12:16.600848+00:00 2014-09-02 22:39:30.547159+00:00 No changes -
• Image cracking: • Capable of cracking open AMI’s and
EBS volumes to enumerate applications and systems • IAM Permission Analysis • Complete analysis and reporting of who can do what to what, when and where from Adv. Capabilities -
• Netflix Security Monkey – http://techblog.netflix.com/2014/06/announcing-‐ security-‐monkey-‐aws-‐security.html –
…and a whole ton of other OSS • AWS Trusted Advisor – https://aws.amazon.com/premiumsupport/ trustedadvisor/ • NimboStratus – https://andresriancho.github.io/nimbostratus/ Other Tools -
THANK YOU May all your Clouds come with a Luck
Dragon AppSec USA 2014 @silvexis [email protected]