Security Incident Response in AWS - SF Loft 2015-09-10

Presentation on Security Incident Response in AWS presented at the AWS Pop-up Loft in SF

Transcript

1. Security Incident Response in AWS AWS Pop-up Loft San Francisco John Martinez Principal Solutions Architect September 10, 2015
2. About Me ▪Been doing DevOps and Cloud stuff for ~5 years ▪Helped architect and build Creative Cloud @ Adobe ▪Cut my teeth on “the cloud” at Netflix ▪UNIX and Linux throat beard for >20 years ▪Have been involved in countless security and production incidents, most at 2:00 AM ▪I now talk to people about security for a living 2
3. Tonight’s Agenda ✓AWS Shared Security Responsibility ✓Incident Response on AWS ✓Anatomy of an Attack ✓Engaging AWS Support ✓Logging and Data on your side ✓Mitigation Techniques ✓A Practitioner’s Toolchest ✓Q&A 3
4. AWS Shared Security Responsibility 4
5. Shared Security Responsibility Model 5 http://aws.amazon.com/compliance/shared-responsibility-model/
6. Shared Security Responsibility - AWS ▪AWS owns responsibility for all things physical ▪They build the services and APIs you need to deploy and run your applications on AWS ▪They deliver to their customers the ability to create identity credentials to access AWS services and resources ▪They secure the hypervisor and the network below layer 7 6
7. Shared Security Responsibility - You ▪Creating and managing IAM entities and their authentication tokens ▪Creating and managing IAM access policies ▪Creating and managing network access to EC2 instances and VPC networks ▪Deploying applications and creating virtual infrastructure ▪OS Patching and Maintenance ▪Monitoring activity and collecting data ▪Designing for failure 7
8. AWS Security Services - Identity ▪Identity and Access Management (IAM) ▪Secure Token Service (STS) ▪Directory Service 8
9. AWS Security Services - Encryption ▪Key Management Service ▪CloudHSM 9
10. AWS Security Services - EC2 ▪EC2 Security Groups ▪EC2 Keypairs (SSH) ▪VPC Subnet ACLs 10
11. AWS Security Services - Data/Log Analysis ▪CloudTrail ▪Config ▪CloudWatch Logs 11
12. AWS Security Services - Storage ▪S3 ACLs and Bucket Policies ▪S3 Lifecycle Rules ▪EBS Volume Encryption 12
13. Incident Response in AWS 13
14. Incident Response: Not very different ▪Have a plan ▪NOC and SOC roles ▪Communication is key ▪Your AWS/Cloud guru may be your best ally if you don’t have an explicit “Cloud Security” Architect/Engineer 14
15. Incident Response: somewhat different ▪Your AWS/Cloud guru may be your best ally if you don’t have an explicit “Cloud Security” Architect/Engineer ▪DevOps can play a major role ▪Engage AWS Support often and early (more on that later) ▪Come armed with data (also more on that later) ▪Download latest CSA Guidance, Domain 9 “Incident Response” https://cloudsecurityalliance.org/research/security-guidance/ 15
16. Triage the Incident Evaluate the situation ▪What’s going on right now? ▪No time to freak out 16
17. Triage the Incident Stop the bleeding ▪Secure the site/isolate the damage ▪May need to bring down the site temporarily 17
18. Triage the Incident Start the breathing ▪Get the business running again ▪Restart services when you’re ready 18
19. Triage the Incident Protect the wound ▪Investigate root cause ▪Data is king 19
20. Triage the Incident Treat for Shock ▪Make it better (so it does not happen again) ▪Learn from your mistakes ▪Monitor continuously 20
21. Security Incident Checklist ✓Have both a plan and tools (and have tested them) ✓Define what and when it happened ✓Define what and when it was not happening ✓Initial triage of the problem ✓The more detail you can collect, the better ✓Scale up - Scale out - Isolate the issue ✓Review CloudTrail Logs ✓Snapshot/dd effected resources ✓Open a support case with AWS early ✓Update Often 21
22. Anatomy of an Attack 22
23. What does an attack in AWS look like? ▪More than likely, you don’t even know it’s going on ▪Sometimes, an email from the AWS Security will come in that an attack is originating from one of your EC2 instances ▪Preventative tools like Evident.io can help reduce the amount of surprises ▪Dev accounts are just as important to secure as production 23
24. What does an attack in AWS look like? for EC2 ▪Most common type of attack ▪EC2 IP address space is public https://ip-ranges.amazonaws.com/ip-ranges.json ▪Security Group rules are too open ▪Vulnerable applications ▪Vulnerable operating systems ▪Instances in unused regions (look for micros) ▪AutoScaling gone wild 24
25. What does an attack in AWS look like? for IAM ▪Exposed and compromised API Access Keys ▪New users/roles ▪Deleted users/roles ▪IAM policy changes ▪Root user locked out ▪AWS resources being created or deleted ▪Costs going up trough the roof 25
26. What does an attack in AWS look like? for S3 ▪Wide open ACLs and/or Policies ▪Watch AuthenticatedUsers grantee in ACLs ▪Missing buckets and objects ▪Created buckets and objects 26
27. Engaging AWS Support 27
28. What Will AWS do WITH you? ▪Communicate via Support Ticket ▪Can provide you guidance on ▪Scaling up ▪Scaling out ▪Isolating the issue ▪May shift traffic to help ▪Can provide you forensic image for analysis in AWS 28
29. AWS Support is Part of YOUR Team ▪Always include AWS Support in your incident response program ▪If you are an Enterprise Support customer, include your TAM ▪On occasion, an AWS Support person can be on your call ▪Always report abuse and security issues to AWS Support, their support center is bigger than yours! 29
30. Penetration Testing ▪ALWAYS use the official AWS Pen Test Request form ▪Is not immediate, but you can request immediate via AWS Support http://aws.amazon.com/security/penetration-testing/ 30
31. Logging and Data on Your Side 31
32. Come Prepared ▪The more data you have, the better ▪Make sure you have data for ▪AWS infrastructure and resource state (Evident.io can help) ▪AWS Activity and events (CloudTrail) ▪OS and Application state (FIM) ▪S3 buckets (S3 Logs) ▪ELBs (ELB Logs) 32
33. CloudTrail FTW ▪CloudTrail provides a pretty comprehensive amount of data for events on AWS resources ▪CloudTrail produces tons of data, be prepared to sift through it ▪Send CloudTrail data to a centralized bucket for easy consumption into log aggregation systems ▪AWS may request CloudTrail data during an incident ▪Protect your CloudTrail buckets as if they had customer sensitive PII 33
34. Forensic Images ▪A great way to determine root cause ▪Law enforcement may want later ▪Create EBS snapshot or AMI from affected instance ▪Use an isolated VPC with no route to the internet to bring up and test 34
35. Mitigation Techniques 35
36. Preventative Medicine ▪Follow the Evident.io Top 10 Security Best Practices ▪Read and be familiar with the AWS Security Best Practices White Paper ▪Automate data gathering 36
37. Security + DevOps ▪Configuration Management can help get you to the last known good state ▪Immutable AMIs are also helpful in restoring operating systems and applications ▪CloudFormation templates for infrastructure management ▪Test your CI/CD pipeline 37
38. A Practitioner’s Tools 38
39. Some tools that can help you ▪Evident.io (infrastructure) ▪AWS Trusted Advisor (infrastructure) ▪Cloudability (cost) ▪Splunk (monitoring and logging) ▪AWS CloudWatch and CW Logs (monitoring and logging) ▪Qualys (app and vulnerability scanning) ▪nmap (network scanning) ▪Autopsy and SluethKit (forensics) ▪Kali Linux (pen testing) ▪…and many others… 39
40. john@evident.io • john@johnmartinez.com • @johnmartinez • LinkedIn 40 Thank You AWS Pop-up Loft SF!