Defending the Cloud from the Full Stack Hack | Source Boston 2016

Amazon Web Services (AWS) and other cloud services are billed as an amazingly secure and resilient cloud services provider, but what is the reality? Once you look past that pristine environment and the manicured forests and start to build on top of it you’ll find yourself very quickly in a dark jungle of your own creation.

With concrete examples this presentation will explore “full stack” vulnerabilities and their effect on security and how they create new pitfalls when migrating to and operating in a Cloud world. From the simple (checking in your Cloud credentials to github or embedding them in your app) the unexpected (XXE injection to expose Cloud metadata), to the unintended (data leakage and service exposure and 3rd party cloud management services). Many examples will be shared along side new techniques showing how easy it is to expose your applications and infrastructure to attack through misunderstanding, ignorance or bad actors.

Transcript

1. May/2016
   Defending the Cloud
   from the Full Stack
   Hack
   Erik Peterson/Veracode/@silvexis
   

   View Slide

2. Hi, I’m Erik, thanks for coming!
   • Cloud Security Weapons
   Manufacturer @ Veracode
   • Researching the Cloud and
   architecting solutions in AWS
   since 2009
   @silvexis
   

   View Slide

3. EVERYBODY HAS A
   PLAN UNTIL THEY
   GET PUNCHED IN THE FACE
   GO CLOUD
   

   View Slide

4. CLOUD

   IS THE NEW
   OPERATING
   SYSTEM
   

   View Slide

5. CLOUD
   INFRASTRUCTURE 

   IS CODE
   

   View Slide

6. A Fully Realized AWS Application
   

   View Slide

7. Traditional App represents small % of system
   • Java, .Net, etc… including 3rd party code in red
   

   View Slide

8. Majority of the system provided by AWS
   Using it all securely however is your responsibility
   All of this however is your application and securing it is your responsibility
   

   View Slide

9. What you get from AWS
   • The digital equivalent of infinite empty rack
   space
   • A friendly looking web interface
   • A mile long list of compliance certifications
   • The rest is up to you
   

   View Slide

10. …and yet after a few months this might be you
    

    View Slide

11. EMERGENT INSECURITY
    The individual components of your
    system may be secure but when
    deployed into the Cloud, the system
    becomes insecure
    AND IT GETS WORSE AT SCALE
    

    View Slide

12. THE 4 HORSEMEN OF EMERGENT INSECURITY
    INTERNET
    WEATHER
    SOFTWARE
    DEFINED
    EVERYTHING
    GUARANTEED
    FAILURE
    OUT-OF-BAND
    SERVICE
    API
    

    View Slide

13. What’s your real attack surface?
    API’s drive everything, existing security controls haven’t caught up
    For example, to secure a system on premise:
    Block all network traffic using host FW, throw away SSH keys/passwords, install
    network IDS, log all network traffic to/from system
    System is now inaccessible, any attempt to access would also be detected
    In the cloud however
    Use APIs to snapshot the disk, mount snapshots on different system, and extract
    everything without touching the network or system
    Zero indication from traditional controls that any access has taken place
    Same is true of cloud databases, I don’t need your passwords or even SQL Injection, I
    just need to snapshot your DB
    

    View Slide

14. API ACCESS PHYSICAL ACCESS
    IN THE CLOUD
    

    View Slide

15. API Credential Exposure Impact
    The attacker can:
    Sets up bitcoin mining operations within your cloud
    environment
    Alter your applications to spread malware
    Uses your environment as a means to launch additional
    attacks
    Download all your customers data
    

    View Slide

16. Destroy your entire Cloud
    

    View Slide

17. Getting Access to the API
    Checking your API keys into Github
    (Not so common anymore thankfully)
    Exploiting vulnerabilities to access
    Cloud Metadata
    

    View Slide

18. Oops
    Keys in code
    Not so common anymore
    

    View Slide

19. However….why are keys in code at all?
    78,667 placeholder results?!
    There is no good reason to put
    API keys in code, it’s
    dangerous
    Use IAM roles or local AWS
    configuration files
    

    View Slide

20. View Slide

21. What is Cloud Metadata?
    Based on RFC 3927 - Dynamic Configuration of IPv4 Link-Local Addresses
    Metadata contains all kinds of awesome things, like startup scripts and….your
    AWS access credentials (if you are using roles)
    It’s not just AWS, all cloud providers have it
    AWS: http://169.254.169.254/latest/user-data
    Google: http://169.254.169.254/computeMetadata/v1/
    MS Azure, most is in a static file, but that’s changing http://169.254.169.254/
    metadata/v1/
    There is nothing wrong with any of this, as long as you are aware and protect it
    

    View Slide

22. $ wget -q -O - http://169.254.169.254/latest/meta-data/
    ami-id
    ami-launch-index
    ami-manifest-path
    block-device-mapping/
    hostname
    iam/
    instance-action
    instance-id
    instance-type
    kernel-id
    local-hostname
    local-ipv4
    mac
    metrics/
    network/
    placement/
    profile
    public-hostname
    public-ipv4
    public-keys/
    reservation-id
    security-groups
    ACCESSING AWS METADATA
    AWS command line tool ec2-metadata will extract some (but not all!)
    of the metadata
    To get everything use wget or curl
    Each one of these represents
    meta data you can access
    $ wget -q -O - http://169.254.169.254/latest/meta-data/instance-id
    i-0496132e
    

    View Slide

23. METADATA USER-DATA
    User data is distributed in AWS via the metadata API
    Often will also contain sensitive information
    [ec2-user@ip-10-0-4-32 ~]$ wget -q -O - http://169.254.169.254/latest/user-data
    #!/bin/bash
    yum update -y
    yum install -y aws-cli wget vim python27-pip
    mkdir -p /etc/worker
    echo config > /etc/worker/environment_config_bucket
    echo “mysql://system:fsdkj4540@#$2-fV2@10.0.4.50:3306" > /etc/worker/mysql.conf
    

    View Slide

24. METADATA IAM CREDENTIALS
    [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/security-
    credentials/
    {
    "Code" : "Success",
    "LastUpdated" : "2014-08-13T16:43:54Z",
    "Type" : "AWS-HMAC",
    "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA",
    "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV",
    "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD
    +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/iF94PTi8+kO5sxgboy4XPB
    +Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g+iyZLw8mT5ADz9zGUQTf
    +P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf
    +GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS+sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/
    Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia
    +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr
    +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt
    +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc
    +iFQaFBvCD3q66fBQ==",
    "Expiration" : "2014-08-13T23:05:19Z"
    }
    

    View Slide

25. METADATA CREDENTIAL EXTRACTION
    [ec2-user@ip-10-0-1-125 ~]$ wget -q -O - http://169.254.169.254/latest/meta-data/iam/security-
    credentials/
    {
    "Code" : "Success",
    "LastUpdated" : "2014-08-13T16:43:54Z",
    "Type" : "AWS-HMAC",
    "AccessKeyId" : "ASIAJ4YSTDWONSUFPHIA",
    "SecretAccessKey" : "iqFuJWcj9AUaBXe0tbuc6MC70oQW2wehWufZ9cQV",
    "Token" : "AQoDYXdzEGIa0AMskSUj1Ing9DHLT1QmD
    +vDimxTCnAbrNGcGPbV9jEPYO5LDMMLBAjVdklFo7vS8HnEDrH3ea0T7f8aXW9BGMSdc/iF94PTi8+kO5sxgboy4XPB
    +Bh44xHSKFV4WIrMKfMUwAftcieER7z6CakegOoe6Q/H0PsK9GpS1pO6g+iyZLw8mT5ADz9zGUQTf
    +P3anQ3dAl32SWYEiJR0fTQCuKqE8/dpLbnmdhOn3WyW8eF3TJFPd8/L0MQak3EMgo1pAxm+eWAMj1B5Crewy4sbvBzf
    +GcemFYiMClsY9gFxZCxOexV09j9nPos/d9VRpFakm1tWAS+sqHKz1zxLidWJewUfuhyLSxcR5xOeZYJ6/
    Pt6bQitf21ep6FJExEGE3Ho0A10z4tv9Yo5c2tPafEhWsACBOia
    +kpQExftmuIulmkRK9NugNuKcd0OzDkoftkpIFAj09oP2tgsDuImc0R3LScijbmhgLZsG1UuEpNpTnr
    +7DmbvmrJWihfTD2hJzFAzhvsvN8ytt
    +mWkSGAeBhA6UosCgj2VjBIkHG6kkTcL9FnEU7XPoKqVsiusRqwZYOOeITsL4dE38pcfHhmBPaCI6H5TbjO4FuRQc
    +iFQaFBvCD3q66fBQ==",
    "Expiration" : "2014-08-13T23:05:19Z"
    }
    

    View Slide

26. OLD VULNS NEW LIFE
    These vulnerabilities can result in a

    total data center compromise

    in AWS
    CWE-918: SSRF
    CWE-611: XXE
    CWE-441: Unintended Proxy or Intermediary
    CWE-77: Command Injection
    CWE-200: Information Exposure
    Why?
    All of these can lead to unintended exposure of metadata
    

    View Slide

27. Has this really happened?
    Real world Examples
    Prezi experienced this first hand
    http://engineering.prezi.com/blog/2014/03/24/prezi-got-
    pwned-a-tale-of-responsible-disclosure/
    Andres Riancho demonstrated this in his BlackHat talk
    “Pivoting in Amazon Clouds”
    Exploited SSRF vuln to inject malicious AWS SQS messages
    to then exploit a celery/python vuln and take control
    

    View Slide

28. Is Your Cloud is Leaking?
    Leaking tags to 3rd party services
    EC2 Classic RFC 1918 (private IP range) networks
    are not private
    Unsecured S3 buckets accessible to anyone able
    to guess their name
    

    View Slide

29. The Full Stack Hack
    3. SSRF or XXE
    vulnerability exposes
    EC2 Metadata, revealing
    AWS API Keys
    1. AWS DNS
    Reveals Private
    IP of web server
    2. Private IP of web
    server reveals
    detailed errors or
    admin interface
    4. AWS API Key lets
    you create new
    instances where your
    clone and mount any
    existing EBS vol.
    5. Cloned system
    gets you SSH keys
    to app servers and
    API key with IAM:*
    giving you access to
    everything
    6. With new root
    credentials create trust
    relationship with external
    account and clone DB for
    quiet extraction
    

    View Slide

30. DEFENDING YOURSELF
    

    View Slide

31. The basics
    Encrypt all the things, turn on EBS and RDS encryption even if you
    are going to let AWS manage the keys, it will complicate exfiltration
    Turn on AWS Cloudtrail for all regions, use Cloudtrail logs to
    monitor
    Turn on AWS Config and use Config rules to enforce a basic
    defaults like
    Required tags, limit open ports, encryption use, open policies
    

    View Slide

32. AppSec is your next problem
    AppSec was already the #1 problem you were not paying attention too.
    Good news, now it’s worse! Application vulnerabilities can lead to
    complete infrastructure exploitation
    All of your systems need constant, automated review
    Automated Code security testing must be part of your development
    process
    Make continuous testing for known vulnerabilities part of your
    operating environment
    

    View Slide

33. Protecting the Metadata API
    Metadata API is a great service, don’t just block access to it
    outright, it can have unintended consequences
    1. Ensure you are auditing your systems for vulnerabilities or
    “features” that could expose access to the metadata IP
    169.254.169.254
    2. Restrict API key permissions, aggressively practice least
    privilege
    

    View Slide

34. Limit API Access
    Restrict sensitive API Keys
    Lock to specific IP addresses
    Require MFA
    Eliminate * policies!!
    Ask yourself “If this key was
    compromised, what is the worst that
    could happen?”
    {
    "Version": "2012-10-17",
    "Statement":[{
    "Effect":"Deny",
    "Action":"*",
    "Resource":"*",
    "Condition":{
    "NotIpAddress":{
    “aws:SourceIp”:[“VPC GATEWAY IP/32”]
    }
    }
    }]
    }
    {
    "Version": "2012-10-17",
    "Statement": [
    {
    "Effect": "Allow",
    "Action": "*",
    "Resource": "*"
    }]
    }
    

    View Slide

35. Plug Leaks
    Ensure tags do not contain sensitive data
    Lock down S3 buckets
    If you have been using AWS for a long time,
    migrate off of EC2 Classic
    

    View Slide

36. This is not your IDS
    Do not use your bill as your
    IDS!
    Implement API logging
    AWS: Cloudtrail
    Azure: Audit Logs
    Actually monitor your cloud logs
    Invest in a host IDS/monitoring
    solution that is cloud aware
    

    View Slide

37. @silvexis erik@silvexis.com
    THANK YOU
    May all your Clouds come with a Luck Dragon
    

    View Slide