Share
Embed
You Did What In My Bucket? Red Teaming GCP (Google Cloud Platform)
More research on Red Teaming GCP (Google Cloud Platform) & K8s presented at BSides Nashville on April 13th 2019.
TweekFawkes
April 13, 2019
Transcript
108 slides
-
Stage 2 Security Version 1.0 Copyright 2019 by Stage 2
Security You Did What In My Bucket? Red Teaming GCP (Google Cloud Platform) -
Copyright 2019 by Stage 2 Security Stage 2 Security Agenda
Bryce Kunz @TweekFawkes - Who Am I? - GCP Overview - Compute Engine - Storage - Kubernetes (K8s) - Persistence for GCP -
Copyright 2019 by Stage 2 Security Stage 2 Security Past
WhoAmI Defense DHS SOC Offense NSA Red Team Adobe Digital Exp. (DX) -
Copyright 2019 by Stage 2 Security Stage 2 Security WhoAmI
- The Present Services • Hack (Pentest) • Hunt (Splunk ES) • Teach (BlackHat) -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Cloud Overview Overview -
Copyright 2019 by Stage 2 Security Stage 2 Security Management
UI Control Plane (APIs) Data Plane Management UI Cloud Admin -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Management UI Web Management Console -> https://Console.Cloud.Google.com -
Copyright 2019 by Stage 2 Security Stage 2 Security Control
Plane (APIs) Control Plane (APIs) Data Plane Management UI Cloud Admin Ext Cloud Automation - Terraform - Salt Cloud - Custom -
Copyright 2019 by Stage 2 Security Stage 2 Security Data
Plane Control Plane (APIs) Data Plane Management UI Cloud Admin Ext Cloud Automation - Terraform - Salt Cloud - Custom USERS -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Roles GCP Roles are a collection of permissions • GCP Roles are similar to AWS IAM Policies Permissions enable you to take certain actions: • e.g. Compute.Instances.Start -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Roles Primitives Role Primitives: • Owner -> Billing & Access • Editor -> Changes & Updates • Viewer -> Read-Only Pre-Dates Cloud Identity & Access Management (IAM) service in GCP • Generally, overly permissive and/or too broad -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Roles Predefined Role Predefined: • Granular access to specific GCP resources (IAM) ◦ roles/pubsub.subscriber Role Custom: • Project and/or Organization level roles w/ granular permissions -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Cloud Identity & Access Management (IAM) Authorization for GCP Resources • Introduced in early 2016 “Member” is one of the following • user, group, domain, service account, or public Cloud IAM does NOT directly manage identities, hence these reference: • Individual google account, Google groups, G Suite / Cloud Identity Domain Every identity has a unique email address -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Cloud IAM Policies Policies bind Members to Roles at a specific hierarchy levels: • Org • Folder • Project • Resource Who can do what to which thing? -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Accounts Three General Types of Creds: • User Accounts • API Keys • Services Accounts -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Cloud Identity Identity as a Service • Users • Groups Similar to AWS IAM service or Active Directory Supports MFA and Security Key Enforcement (e.g. Hardware Device) Google Cloud Directory Sync -> LDAP and Active Directory Sync -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Cloud Resource Manager Manage & Secure organization’s projects Similar to AWS Organizations -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Cloud Audit Logging (Stackdriver) Audit Logs -> Who, What, When, and Where Logs • Admin Activity -> 400 days of retention for free • Data Access -> 7 days of retention for free, 30 days of retention for $ NOTE: GCP Services, so does not log apps running on GCE Similar to AWS CloudTrail -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Cloud KMS Encryption service (AES256) designed to protect secrets • Secrets out of code base and into the environment • Does not store secrets • Encrypts and/or Decrypts secrets stored elsewhere • Control access to keys for Encryption and/or Decryption Integrated with IAM for Authorization & Cloud Audit Logging • Key rotation and key versioning for decryption Similar to AWS KMS & Vault by HashiCorp -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Service Account Service Accounts are accounts for applications • Similar to AWS IAM Roles Service Accounts can be assumed by an application (or user, if authorized) • Should use least privilege -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
User Managed Keys Two Types of Keys: • user managed keys ◦ generate/download private keys ◦ (e.g. for AWS to access GCP) ◦ expire 10 years from creation • GCP managed keys Ref: https:/ /cloud.google.com/iam/docs/understanding-service-accounts -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
GCP Managed Keys Two Types of Keys: • GCP managed keys ◦ GCP native secrets ◦ prefered for GCP services ◦ (e.g. GCF, GAE, GCE, GKE, etc…) ◦ automatically rotated ◦ used for a maximum of two weeks Ref: https:/ /cloud.google.com/iam/docs/understanding-service-accounts -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
GCE New VM Instance Google Compute Engine (GCE) Default: • Identity and API Access • Firewall • Startup script (Optional) • Metadata (Optional) Defaults: • Block project-wide SSH keys (unchecked) • Disk Encryption (Google-managed key) ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
GCE Identity and API Access Access scopes: • read-only access to Storage and Service Management, • write access to Stackdriver Logging and Monitoring, • read/write access to Service Control. ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
GCE Startup Script GCE Startup Script: • runs under the root user ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
GCE Metadata Access scopes: • read-only access to Storage and Service Management, • write access to Stackdriver Logging and Monitoring, • read/write access to Service Control. ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Compute
Engine Overview -
Copyright 2019 by Stage 2 Security Stage 2 Security Data
Center Firewall Server Side Request Forgery (SSRF) ... Web App Database Monitoring 10.1.1.1 Images 10.1.1.2 Internet 1 GET /app?img=b.jpg 2 3 4 -
Copyright 2019 by Stage 2 Security Stage 2 Security Server
Side Request Forgery (SSRF) ... Web App Database Monitoring 10.1.1.1 Images Internet Data Center Firewall 1 GET /?img=http://10.1.1.1/... 2 3 4 0 -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Server Side Request Forgery (SSRF) ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Metadata Service ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Metadata Service ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Metadata Service HTTP Header ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Metadata Service HTTP Header ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
v1beta1 ? ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
v1beta1 ! ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
SSRF Demo Steps on macOS: • curl http://35.247.8.30/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://35.247.8.30/extimage?p=http://metadata.google.internal/computeMet adata/v1beta1/instance/service-accounts/default/token ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
SSRF Demo Steps on macOS: • curl http://34.73.197.205/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://34.73.197.205/extimage?p=http://metadata.google.internal/compute Metadata/v1beta1/instance/service-accounts/default/token ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
SSRF Demo Steps on macOS: • curl http://34.73.197.205/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://34.73.197.205/extimage?p=http://metadata.google.internal/compute Metadata/v1beta1/instance/service-accounts/default/token ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
SSRF Demo Steps on macOS: • curl http://35.247.8.30/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://35.247.8.30/extimage?p=http://metadata.google.internal/computeMet adata/v1beta1/instance/service-accounts/default/token ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
SSRF Demo Steps on macOS: • curl http://35.247.8.30/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://35.247.8.30/extimage?p=http://metadata.google.internal/computeMet adata/v1beta1/instance/service-accounts/default/token ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Validate User Tokens https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=ACCESS_T OKEN ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Default Access ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Management UI Web Management Console -> https://Console.Cloud.Google.com -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Compute Engine Identity and API Access -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Compute Engine Identity and API Access -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Full Access ... -
Copyright 2019 by Stage 2 Security Stage 2 Security SSH
Agents Overview -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
GCE SSH SSH Access to VM: ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
GCE SSH SSH Agents ... -
Copyright 2019 by Stage 2 Security Stage 2 Security 2017:
May-Oct 1. “Internal Accenture Data, Customer Information Exposed in Public Amazon S3 Bucket” 2. “Another Wide-Open Amazon S3 Bucket Exposes Verizon Customer Account Data” 3. “US voter info stored on wide-open cloud box, thanks to bungling Republican contractor” 4. “Researcher discovers classified Army intel app, data on open public AWS bucket” 5. “Millions of Time Warner Cable Customer Records Exposed in Third-Party Data Leak” 6. “Drone Manufacturer DJI Leaves SSL Key Exposed on Public Repository” 7. “Dow Jones becomes the latest organization to be affected by an AWS cloud data leakage due to misconfiguration and user error.” etc... -
Copyright 2019 by Stage 2 Security Stage 2 Security 2017:
May-Oct 1. “Internal Accenture Data, Customer Information Exposed in Public Amazon S3 Bucket” 2. “Another Wide-Open Amazon S3 Bucket Exposes Verizon Customer Account Data” 3. “US voter info stored on wide-open cloud box, thanks to bungling Republican contractor” 4. “Researcher discovers classified Army intel app, data on open public AWS bucket” 5. “Millions of Time Warner Cable Customer Records Exposed in Third-Party Data Leak” 6. “Drone Manufacturer DJI Leaves SSL Key Exposed on Public Repository” 7. “Dow Jones becomes the latest organization to be affected by an AWS cloud data leakage due to misconfiguration and user error.” etc... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Storage Public Buckets... ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Accessing Objects https://storage.googleapis.com/its_all_in_the_cloud/object001.jpg storage.googleapis.com -> GCP its_all_in_the_cloud -> Globally Unique Bucket Name object001.jpg -> Object Name ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
Listable Buckets ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GoBuster
- Finding Buckets & Objects https://storage.googleapis.com/its_all_in_the_cloud/object001.jpg gobuster -m dir -u “https://storage.googleapis.com” -i -t 100 -e -s 200,204 -w quickdir.txt -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: Master Node & API Overview: • VM / Instance running the following services: ◦ kube-apiserver, ◦ kube-controller-manager and ◦ kube-scheduler. Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: etcd Overview: • Holds state information for the cluster ◦ “Access to etcd is equivalent to root permission in the cluster so ideally only the API server should have access to it.” Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: kubectl Overview: • kubectl is a cli to admin the cluster Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin Cli kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: Dashboard Overview: • Dashboard is a web-based UI for K8s clusters ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin Browser Dashboard -
Copyright 2019 by Stage 2 Security Stage 2 Security Telsa
K8s hacked! Unsecure Admin Console... ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: Worker Nodes Overview: • Nodes are VMs / Instances w/ ◦ Container Runtime (e.g. Docker) ◦ Kube-proxy ◦ Kubelet ▪ Port: 10250/TCP ▪ Port: 10255/TCP ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: Pods Overview: • Pod contains 1 or more containers ◦ Smallest unit in K8s ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: Services Overview: • Services map K8s names to pod IPs ◦ When nodes get stop/started ◦ Services continue to route • Similar to a load balancer/proxy ◦ endpoint lookup... ◦ more magic be here ◦ ref: kube-proxy, etc... ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview: Load Balancers Overview: • Load Balancing via GCP Services ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
SSRF Demo Steps on macOS: • curl http://34.73.197.205/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://34.73.197.205/extimage?p=http://metadata.google.internal/compute Metadata/v1beta1/instance/service-accounts/default/token ... -
Copyright 2019 by Stage 2 Security Stage 2 Security GCP:
SSRF Demo ... .sh -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
Overview Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:
/bin/cat /proc/1/cgroup ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:
pid 1 is not init or launchd ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Default
Service Account Find secrets: • /var/run/secrets/kuberenetes.io/serviceaccount/token ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:
Python via Memory Only Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:
Python via Memory Only Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:
pyscript Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Container
Escapes Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:
pyscript to access_token via metadata Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl metadata.google.internal 169.254.169.254 -
Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:
pyscript to access_token via metadata Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ... -
Copyright 2019 by Stage 2 Security Stage 2 Security .../v1beta1/instance/attributes/kube-env
• Masquerading as the Kubelet • To the K8s API ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl metadata.google.internal 169.254.169.254 -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet
certificate and the Kubelet private key • Masquerading as the Kubelet • To the K8s API ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl metadata.google.internal 169.254.169.254 -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet
-> Port: 10250/TCP, 10255/TCP Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet
-> Port: 10250/TCP, 10255/TCP Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet
-> Port: 10250/TCP, 10255/TCP Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet
-> Port: 10250/TCP, 10255/TCP ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Container
Escapes Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc • CVE-2016-5195 -> Dirty Cow ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Container
Escapes Container Escape Techniques: • Run Container in Cluster ◦ With Root File System Mounted! ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes
API Kubernetes API Vulnerabilities: • CVE-2018-1002105 -> kubernetes: authentication/authorization bypass ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Docker:
2375/TCP (no auth.), 2376/TCP (TLS) Lateral Movement: • EDB-ID: 42356 -> Unprotected TCP Socket ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security K8s
Defense Overview -
Copyright 2019 by Stage 2 Security Stage 2 Security Network
Security Highly recommend… • Isovalent.com w/ Cilium ◦ To lockdown network traffic ◦ via namepsaces ▪ … alternatively istio ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Update
Kubernetes Cluster • Updates Frequently • New Security Features Regularly Added • Defaults Get Stronger ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet
-> Configure Authentication • Enable “NodeRestriction” • --anonymous-auth=false ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl -
Copyright 2019 by Stage 2 Security Stage 2 Security Client-Side
Vectors: • Remote Mac Exploitation Via Custom URL Schemes Ref: https:/ /objective-see.com/blog/blog_0x38.html -
Copyright 2019 by Stage 2 Security Stage 2 Security Browser
Cookies Client-Side: • cookie_crimes -> https://github.com/defaultnamehere/cookie_crimes GCP Ref: https:/ /wunderwuzzi23.github.io/blog/passthecookie.html -
Copyright 2019 by Stage 2 Security Stage 2 Security Cloud
Shell Client-Side: • cookie_crimes -> https://github.com/defaultnamehere/cookie_crimes GCP ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Cloud
Shell: .bashrc modification • ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Cloud
Shell -> .bashrc -> Voodoo • ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Cloud
Shell -> .bashrc -> Voodoo -> Private Key • ... -
Copyright 2019 by Stage 2 Security Stage 2 Security Trainings
@ BlackHat & On-Site! Thank You! [email protected] .sh @TweekFawkes