Serverless and Dys-FUNctional Cloud Red Teaming

Research on Serverless and Dys-FUNctional Cloud Red Teaming presented at SaintCon in Utah on October 27th 2020.

Serverless Compute Technologies (e.g. AWS Lambda, Azure Functions, GCP Cloud Functions, etc.) enables the rapid development of Applications and APIs which can scale to epic proportions at minimal cost. In this session, we will cover the how to quickly develop serverless applications (e.g. AWS SAM) for the purposes of significantly improving and automating typical Red Teaming operations, including:

- Discovering semi-sensitive information (e.g. AWS Account IDs) via analyzing responses from various Cloud services.

- Elegantly Redirecting a Red Team’s Command & Control (C2) through Cloud services (e.g. AWS API Gateway & Lambda, Azure Functions, etc.) to camouflage C2.

- Disrupting Logging Services (e.g. AWS CloudTrail) with Serverless applications (e.g. Lambda) to remain undetected within compromised AWS environments.

Transcript

1. Version 0.0.1 Copyright 2020 by Stage 2 Security Serverless & Dys-FUNctional Cloud Red Teaming (AWS, Azure, GCP, etc...)
2. Copyright 2020 by Stage 2 Security Introduction
3. Copyright 2020 by Stage 2 Security About Me Defense DHS SOC Offense NSA Red Team Adobe DX Bryce Kunz @TweekFawkes
4. Copyright 2020 by Stage 2 Security
5. Copyright 2020 by Stage 2 Security
6. Copyright 2020 by Stage 2 Security Special Thanks! Michael Butler VP of Services & Pentest Lead Waylon Grange @Professor__Plum Research Lead & Creator of . Scott Pack Cloud Security Architect at Adobe
7. Copyright 2020 by Stage 2 Security Overview
8. Copyright 2020 by Stage 2 Security Overview of Serverless Technologies, Services, & Frameworks - API Gateway, Lambda, CloudFormation, etc. - Zappa, AWS Serverless Application Model (SAM), etc. Use Cases Applicable for Red Teamers, Penetration Testers, etc. - AWS Account ID Discovery - Command & Control through AWS Services - Leveraging Lambda to Disrupt AWS Logging Services Agenda
9. Copyright 2020 by Stage 2 Security AWS Services
10. Copyright 2020 by Stage 2 Security API Gateway API Proxy (HTTP or REST) as a managed service https:/ /aws.amazon.com/api-gateway/
11. Copyright 2020 by Stage 2 Security API Gateway Overview API Gateway is a managed service that creates APIs at scale • API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, ◦ Such as … code running on AWS Lambda, or any web application. API Gateway can be considered a backplane in the cloud to connect AWS services and other public or private websites. Provides consistent RESTful application programming interfaces (APIs) for web applications to access AWS services.
12. Copyright 2020 by Stage 2 Security HTTP APIs are designed for low-latency, cost-effective integrations with AWS services e.g. Lambda, and HTTP endpoints REST APIs currently offer more features, and full control over API requests and responses. API Gateway Types https:/ /docs.aws.amazon.com/apigateway/latest/developerguide/http-api-vs-rest.html
13. Copyright 2020 by Stage 2 Security Lambda AWS Scripts as a managed service https:/ /aws.amazon.com/lambda/
14. Copyright 2020 by Stage 2 Security Lambda Overview AWS Lambda is a serverless compute service • Runs your code in response to events and automatically manages the underlying compute resources for you. • Lambda can automatically run code in response to multiple events, such as: ◦ HTTP requests via Amazon API Gateway, ◦ PutObject calls to Amazon S3 Buckets, ◦ Table updates in Amazon DynamoDB, and ◦ State transitions in AWS Step Functions. • You can use AWS Lambda to extend other AWS services with custom logic
15. Copyright 2020 by Stage 2 Security Lambda Runtimes & Environment • Supports many native runtimes ◦ Python2.7 & 3, Java, Ruby, Go, C# + • AWS SDKs built in (e.g. boto3) • Role Credentials placed in Environment Variables • Max 15 Minute Runtime
16. Copyright 2020 by Stage 2 Security Lambda Common Web App Architecture https:/ /aws.amazon.com/lambda/
17. Copyright 2020 by Stage 2 Security CloudFormation Amazon Services Setup https:/ /aws.amazon.com/cloudformation/
18. Copyright 2020 by Stage 2 Security Administration Control Plane (APIs) Data Plane Management UI Cloud Admin (Dave)
19. Copyright 2020 by Stage 2 Security CloudFormation Overview Control Plane (APIs) Data Plane Management UI Cloud Admin Cloud Automation - CloudFormation - Terraform - Salt Cloud - Custom Scripts Automation
20. Copyright 2020 by Stage 2 Security Setup Resources Control Plane (APIs) Management UI Cloud Admin
21. Copyright 2020 by Stage 2 Security Application Control Plane (APIs) Management UI Cloud Admin
22. Copyright 2020 by Stage 2 Security Services Control Plane (APIs) ... ... Storage VM App LB
23. Copyright 2020 by Stage 2 Security Cloud-Aware Application Control Plane (APIs) Cloud Admin (Dave) Ext Cloud Automation - Terraform - Salt Cloud - Custom Storage VM App LB
24. Copyright 2020 by Stage 2 Security Open for Business Control Plane (APIs) Cloud Admin (Dave) Ext Cloud Automation - Terraform - Salt Cloud - Custom Storage VM App LB USERS
25. Copyright 2020 by Stage 2 Security Serverless Frameworks
26. Copyright 2020 by Stage 2 Security Zappa $ pip install zappa $ zappa init $ zappa deploy Zappa makes it super easy to build and deploy server-less, event-driven Python applications (including, but not limited to, WSGI web apps) on AWS Lambda + API Gateway (via CloudFormation). https:/ /github.com/Miserlou/Zappa
27. Copyright 2020 by Stage 2 Security AWS Serverless Application Model (SAM) https:/ /aws.amazon.com/serverless/build-a-web-app/
28. Copyright 2020 by Stage 2 Security AWS Serverless Application Model (SAM) Infrastructure as Code Based on CloudFormation, but much simpler to implement common Serverless Application Models... https:/ /aws.amazon.com/serverless/sam/
29. Copyright 2020 by Stage 2 Security AWS Serverless Application Model (SAM) The Serverless Application Model is commonly referred to as “SAM” in AWS documentation SAM is NOT an AWS Service, it’s more similar to Zappa or Terraform New Resource Types with SAM: • AWS::Serverless::Function -> AWS Lambda Functions • AWS::Serverless::Api -> AWS API Gateway APIs • AWS::Serverless::SimpleTable -> AWS Dynamo DB Tables https:/ /aws.amazon.com/serverless/sam/
30. Copyright 2020 by Stage 2 Security Use Cases
31. Copyright 2020 by Stage 2 Security Use Case # 1 Discover Account IDs Discovering semi-sensitive information (e.g. AWS Account IDs) via analyzing responses from various Cloud services.
32. Copyright 2020 by Stage 2 Security AWS Account IDs Every AWS account is given a unique ID Account IDs are 12 digit numbers e.g. 885264802853 Semi-Sensitive? Usually kept private... But historically, considered to be equivalent to a username disclosure
33. Copyright 2020 by Stage 2 Security AWS Account IDs - The Math Account IDs are 12 digit numbers • e.g. 885264802853 10 possible digits (0,1,2,3,4,5,6,7,8,9) • 12 digits long 10^12 means... • 1 Trillion! Possible Account IDs
34. Copyright 2020 by Stage 2 Security AWS Account IDs - Should They be Protected? Many vendors share their Account IDs: https://github.com/duo-labs/cloudmapper/blob/ main/vendor_accounts.yaml To enable clients to use their services Should They be Protected? Tech industry is saying… NO Including, but not limited to: • Rackspace • TrendMicro • Redlock • Sumo Logic • Bridgecrew • Onelogin • Rapid7 • Threat Stack • Lucidchart • Palo Alto Networks • Tenable etc...
35. Copyright 2020 by Stage 2 Security Known Attacks via Knowing Account IDs AWS error messages disclose whether a role exists or not, w/ a given Account ID Role Names can disclose: • AWS Services being used • Software & Technologies being used • Names of IAM users (social engineering) • 3rd party integrations being used (Okta, Datadog, Cloudsploit, etc.) Once roles are enumerated, one can try to assume any open roles and pilfer the role credentials. Same attack vector applies for IAM Usernames https:/ /rhinosecuritylabs.com/aws/assume-worst-aws-assume-role-enumeration/ https:/ /rhinosecuritylabs.com/aws/aws-iam-user-enumeration/ https:/ /github.com/dagrz/aws_pwn/blob/master/reconnaissance/validate_iam_principals.py
36. Copyright 2020 by Stage 2 Security Other Attacks via Knowing Account IDs Areas for Future Research: • Discovering overly privileged topics and/or queues with guessable names This is the way!
37. Copyright 2020 by Stage 2 Security Known Ways to Discover Account IDs Including, but not limited to: • Compromise a resource (e.g. Lambda, etc.) • Public resources (e.g. Snapshots, AMI, etc.) • Source Code Review (e.g. GitHub) • Error Messages from Services • Screenshots and/or Documentation • Forums and/or Discussion Boards • etc.
38. Copyright 2020 by Stage 2 Security Known Ways to Discover Account IDs https:/ /github.com/dagrz/aws_pwn/blob/master/miscellanea/Kiwicon%202016%20-%20Hacking%20AWS%20End%20to%20End.pdf By default, login URL for the web console, includes the Account ID as a subdomain. Subdomain has different HTTP response codes based on if login page exists NOTE: URL can be changed by the owner of the AWS, and is done so somewhat frequently
39. Copyright 2020 by Stage 2 Security Known Ways to Discover Account IDs https:/ /github.com/dagrz/aws_pwn/blob/master/miscellanea/Kiwicon%202016%20-%20Hacking%20AWS%20End%20to%20End.pdf VirusTotal Results:
40. Copyright 2020 by Stage 2 Security Known Ways to Discover Account IDs https:/ /github.com/Stage2Sec/CaptureTheCloud/blob/master/find_aws_account_ids_virustotal.py VirusTotal has updated the API now to v2, which requires a slightly different syntax to query
41. Copyright 2020 by Stage 2 Security Other Ways to Discover Account IDs Other OSINT Sources: • Passive Total <- This Works Better for Us than VirusTotal • Amass <- Always Awesome • etc.
42. Copyright 2020 by Stage 2 Security Leverage SAM to Create... SAM: • template.yaml • app.py • etc.
43. Copyright 2020 by Stage 2 Security SAM Template SAM Template: • API Gateway ◦ GET /hello • Lambda ◦ Python 3.7 • Attach Policy to Enable: ◦ S3 Bucket Access • Set Timeout to Max: ◦ 15 Minutes (900 Seconds)
44. Copyright 2020 by Stage 2 Security Lambda Application Python App Source: • Generate an ID • Secrets Library • Try w/ Web Console • Requests Library • Analyze Response • Write Result to S3
45. Copyright 2020 by Stage 2 Security SAM Deploy https:/ /aws.amazon.com/blogs/aws/new-aws-toolkits-for-pycharm-intellij-preview-and-visual-studio-code-preview/
46. Copyright 2020 by Stage 2 Security Leverage SAM to Create...
47. Copyright 2020 by Stage 2 Security Finding Accounts At every 3rd minute (*/3 * * * *) 33x ~40 Account IDs (302s) Per Hour 0-3 Second Delay w/ Max 15 Min
48. Copyright 2020 by Stage 2 Security Leverage SAM to Create... At every 3rd minute (*/3 * * * *) 33x ~40 Account IDs (302s) Per Hour 0-3 Second Delay w/ Max 15 Min
49. Copyright 2020 by Stage 2 Security Another Option: Cron via CloudWatch https:/ /docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-schedule.html
50. Copyright 2020 by Stage 2 Security Correlating Account ID w/ Resources S3 feature to validate correct bucket ownership! If we know of a publically available object within an S3 bucket And we think we may know • API Gateway ◦ GET /hello • Lambda ◦ Python 3.7 • Attach Policy to Enable: ◦ S3 Bucket Access • Set Timeout to Max: ◦ 15 Minutes (900 Seconds) https:/ /aws.amazon.com/about-aws/whats-new/2020/09/amazon-s3-bucket-owner-condition-helps-validate-correct-bucket-ownership/ https:/ /docs.aws.amazon.com/AmazonS3/latest/dev/bucket-owner-condition.html https:/ /boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html S3 feature to validate correct bucket ownership! If we know of a publically available object within an S3 bucket e.g. a static resource (image, javascript, etc.) for a public website We can check to see if an AWS Account ID is associated with it via this API!
51. Copyright 2020 by Stage 2 Security Use Case #2 Serverless C2 Elegantly Redirecting a Red Team’s Command & Control (C2) through Cloud services (e.g. AWS API Gateway & Lambda, Azure Functions, etc.) to camouflage C2.
52. Copyright 2020 by Stage 2 Security Azure: Domain Fronting https:/ /truneski.github.io/blog/2019/02/27/empire-domain-fronting-with-microsoft-azure/
53. Copyright 2020 by Stage 2 Security Azure: Functions - Serverless Redirection https:/ /fortynorthsecurity.com/blog/azure-functions-functional-redirection/ https:/ /medium.com/@rvrsh3ll/azure-app-services-for-offensive-operations-f1de99a83fa0
54. Copyright 2020 by Stage 2 Security AWS: API GW & Lambda - Serverless Redirection https:/ /blog.xpnsec.com/aws-lambda-redirector/
55. Copyright 2020 by Stage 2 Security ... Cross Platform Post Exploitation Toolkit
56. Copyright 2020 by Stage 2 Security Cross Platform (Linux, macOS, & Windows) Post Exploitation Red Team Toolkit Designed for Red Teams operating within the tech sector (macOS laptops w/ Linux production servers): • Initiative Multi-User Collaboration, • OPSEC from the ground up (e.g. Memory Only), • Pivoting (chaining callbacks & listeners), • Scripting (python in memory only), • & more!
57. Copyright 2020 by Stage 2 Security Voodoo Overview The overview Screen provides a quick look at what agents you have and how their comms are being routed. Agents shown in gray are dead agents which have stopped calling in. You can double click on any of the icons to jump right to that agent.
58. Copyright 2020 by Stage 2 Security Voodoo Boneyard Once an agent is presumed dead you can move it to the boneyard by clicking the X button on the top right of the agent (it is a check mark in the picture to the right). This allows you to keep you agent less cluttered while still keeping logs of previous activities. If at anytime an agent in the boneyard list suddenly becomes online it will automatically move back to the agents list.
59. Copyright 2020 by Stage 2 Security Voodoo Users Voodoo is a multi-operator concurrent system. By creating separate accounts for each of your team members you can log who issued which commands and each work independently or collaboratively on the same agents. Two Factor authentication is also available should you require tighter control on access to your operations.
60. Copyright 2020 by Stage 2 Security Voodoo Stager (communication options) • Name -- a name for your reference • Communication Style ◦ HTTPS call-back -- The agent will initiate connections to the Voodoo LP over a specified port to check for tasking ◦ TLS Call-in -- The agent will listen on a specified port for the Voodoo LP to connect into it and provide tasking • Call-out options ◦ Domain -- address where agent calls back ◦ Port -- port to use (typically 443) ◦ Interval how often to check-in • Call-in options ◦ Port -- port to listen on for inbound connections
61. Copyright 2020 by Stage 2 Security Voodoo Stager (target environment options) • Target -- OS Family you are targeting • Host Process -- An executable which the agent will be injected into ◦ This must be a valid executable on the target • Command Argument / Passphrase ◦ This is the command line argument that will show up in a ps list ◦ It is also used as a decryption key for the agents configuration • Update -- Click this button to save your changes
62. Copyright 2020 by Stage 2 Security Voodoo Stager (execution options) • Script tabs -- These are a number of different one lines you can use to execute the agent. ◦ Each is for a different environment ◦ On older Linux systems (or target where SYS_PTRACE is disabled) there is an “No Inject” option ◦ On windows there is also a powershell option • Download Executable -- Alternatively, you can download an executable transfer it to the target and run it manually. ◦ exename targetprocess passpharse
63. Copyright 2020 by Stage 2 Security Stagers (under the hood) • The one liners (stagers) bootstrap voodoo a. Downloads an encoded reflective library b. Decodes and loads the library into current process c. Calls library function with passphrase and name of target process ▪ The library then starts up the target process ▪ Hollows out target process ▪ Injects voodoo ▪ Cleans up any artifacts d. Original process then exits
64. Copyright 2020 by Stage 2 Security Injection • Voodoo uses a custom shellcode bootstrap that loads a dll into memory ◦ Doesn’t touch disk ◦ Avoids creating RWX memory sections ◦ Supports C++ exception handling ▪ Even in injected libraries ◦ After injection bootstrap wipes itself from memory
65. Copyright 2020 by Stage 2 Security Secure Communication Channels • Although Voodoo uses HTTPS/TLS for C2 communication it doesn’t rely on it for security ◦ Many SSL visibility products exist ◦ What about MitM? • Voodoo uses ChaCha20 and Poly1305 underneath to protect PII of target ◦ Less processor intensive than AES, but same level of security ◦ Uses a one time pad to encrypt data ◦ More difficult to signature compared to AES ◦ Each agent is keyed with the public key of the LP, no MitM ◦ Each Agent generates a unique private key ◦ If one agent were to be compromised, all other agent comms would still be secure.
66. Copyright 2020 by Stage 2 Security Public 10.0.1.0/24 Web App Private 10.0.2.0/24 #0 - Exploit
67. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 #1 - Callback
68. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 Database #2 - Connect #1 - Callback
69. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 Database #2 - Connect #1 - Callback
70. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 Database #2 - Connect #1 - Callback Monitoring
71. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 Database Monitoring #2 - Connect #1 - Callback #3 - Connect
72. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 Database Monitoring #2 - Connect #1 - Callback #3 - Connect
73. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 Database Monitoring Secrets! #2 - Connect #4 - Connect #1 - Callback #3 - Connect
74. Copyright 2020 by Stage 2 Security Public LP 10.0.1.0/24 Web App Private 10.0.2.0/24 Database Monitoring Secrets! #2 - Connect #4 - Connect #1 - Callback #3 - Connect
75. Copyright 2020 by Stage 2 Security API Gateway
76. Copyright 2020 by Stage 2 Security Flow API Gateway Redirector Ref: https:/ /aws.amazon.com/blogs/aws/identify-unintended-resource-access-with-aws-identity-and-access-management-iam-access-analyzer/
77. Copyright 2020 by Stage 2 Security Flow API Gateway Redirector Ref: https:/ /aws.amazon.com/blogs/aws/identify-unintended-resource-access-with-aws-identity-and-access-management-iam-access-analyzer/
78. Copyright 2020 by Stage 2 Security Flow API Gateway Ref: https:/ /aws.amazon.com/blogs/aws/identify-unintended-resource-access-with-aws-identity-and-access-management-iam-access-analyzer/ HTTPS HTTPS Agent LP
79. Copyright 2020 by Stage 2 Security API Gateway & Lambda
80. Copyright 2020 by Stage 2 Security Flow API Gateway Ref: https:/ /aws.amazon.com/blogs/aws/identify-unintended-resource-access-with-aws-identity-and-access-management-iam-access-analyzer/ Base64 HTTPS Py Urllib Lambda Agent LP
81. Copyright 2020 by Stage 2 Security Blog Post: AWS API Gateway fronting for Command and Control Ref: https:/ /medium.com/stage-2-security/aws-api-gateway-fronting-for-command-and-control-45d03dffb24e Waylon Grange @Professor__Plum Stage2Sec.com -> “About Us” -> “Blog” https://medium.com/stage-2-security Content-Type: application/octet-stream
82. Copyright 2020 by Stage 2 Security Use Case #3 Lambda Log Disruption Disrupting Logging Services (e.g. AWS CloudTrail) with Serverless applications (e.g. Lambda) to remain undetected within compromised AWS environments.
83. Copyright 2020 by Stage 2 Security AWS: Logging Overview Overview
84. Copyright 2020 by Stage 2 Security CloudTrail - Control Plane ( e.g. API ) Logging CloudTrail provides event history of your AWS account activity, including actions taken through the: • AWS Management Console, • AWS SDKs, • command line tools, • and other AWS services. This event history simplifies security analysis, resource change tracking, troubleshooting and Incident Response (IR).
85. Copyright 2020 by Stage 2 Security Data Plane Control Plane (APIs) Data Plane Management UI Cloud Admin Ext Cloud Automation - Terraform - Salt Cloud - Custom USERS
86. Copyright 2020 by Stage 2 Security Logging Disruption via Lambda! Overview
87. Copyright 2020 by Stage 2 Security CloudTrail Logging 10.0.1.x 10.0.2.x Private 10.0.2.0/24 Public 10.0.1.0/24 VPC 10.0.0.0/16 Control Plane Bucket CloudTrail SSH Logs CloudTrail provides event history of your AWS account activity, including actions taken through the: • AWS Management Console, • AWS SDKs, • command line tools, • and other AWS services. This event history simplifies security analysis, resource change tracking, and troubleshooting.
88. Copyright 2020 by Stage 2 Security 10.0.1.x 10.0.2.x Private 10.0.2.0/24 Public 10.0.1.0/24 VPC 10.0.0.0/16 Control Plane Bucket CloudTrail SSH Logs - S3 Write Triggers a Lambda function - Lambda function scrubs logs of • Attacker’s Actions • Attacker’s IP Addresses - Ideally, other logging systems collect the cleaned log files from S3 CloudTrail Logging Disruption
89. Copyright 2020 by Stage 2 Security AWS: Buffering Dominos Overview
90. Copyright 2020 by Stage 2 Security CloudTrail ... CloudTrail S3 Bucket Internet ... Users ... ...
91. Copyright 2020 by Stage 2 Security CloudTrail: Buffering ... CloudTrail S3 Bucket Internet Users ... Buffer ... ...
92. Copyright 2020 by Stage 2 Security CloudTrail: Suspicious Notable CloudTrail S3 Bucket Internet ... Buffer ... ... Suspicious?
93. Copyright 2020 by Stage 2 Security CloudTrail: DeleteTrail CloudTrail S3 Bucket Internet DeleteTrail Buffer DeleteTrail DeleteTrail Detected!
94. Copyright 2020 by Stage 2 Security CloudTrail: Delete & Create w/ Attacker Bucket ... CloudTrail S3 Bucket Internet S3 Bucket CreateTrail Buffer ... DeleteTrail DeleteTrail DeleteTrail
95. Copyright 2020 by Stage 2 Security CloudTrail: Delete & Create w/ Attacker Bucket ... aws sts get-caller-identity --profile target aws cloudtrail describe-trails --profile target … redteam017s3bucket001 ... aws cloudtrail delete-trail --name "redteam017trail001" --profile target aws cloudtrail create-trail --name "redteam017trail001" --s3-bucket-name "redteam017s3bucket101" --is-multi-region-trail --profile target aws cloudtrail start-logging --name "redteam017trail001" --profile target
96. Copyright 2020 by Stage 2 Security Blog: AWS CloudTrail Logging Logic Flaw Ref: https:/ /medium.com/stage-2-security/aws-api-gateway-fronting-for-command-and-control-45d03dffb24e Bryce Kunz @TweekFawkes Stage2Sec.com -> “About Us” -> “Blog” https://medium.com/stage-2-security
97. Copyright 2020 by Stage 2 Security Conclusion
98. Copyright 2020 by Stage 2 Security Thank You! Bryce@Stage2Sec.com Bryce Kunz @TweekFawkes Trainings: Hands-On Cloud Red Teaming Code: Github.com/Stage2Sec/CaptureTheCloud Slides: SpeakerDeck.com/TweekFawkes