Upgrade to Pro — share decks privately, control downloads, hide ads and more …

UVU - Red Teaming GCP (Google Cloud Platform)

UVU - Red Teaming GCP (Google Cloud Platform)

Research on Red Teaming GCP (Google Cloud Platform) & K8s presented at UVU on May 29th 2019.

TweekFawkes

May 29, 2019
Tweet

More Decks by TweekFawkes

Other Decks in Technology

Transcript

  1. Stage 2 Security Version 1.0 Copyright 2019 by Stage 2

    Security UVU Red Teaming GCP (Google Cloud Platform)
  2. Copyright 2019 by Stage 2 Security Stage 2 Security Past

    WhoAmI Defense DHS SOC Offense NSA Red Team Adobe Digital Exp. (DX)
  3. Copyright 2019 by Stage 2 Security Stage 2 Security WhoAmI

    - The Present Services • Hack (Pentest) • Hunt (Splunk ES) • Teach (BlackHat)
  4. Copyright 2019 by Stage 2 Security Stage 2 Security Agenda

    Bryce Kunz @TweekFawkes - Who Am I? - GCP Overview - Compute Engine - Storage - Kubernetes (K8s) - Persistence for GCP
  5. Copyright 2019 by Stage 2 Security Stage 2 Security Management

    UI Control Plane (APIs) Data Plane Management UI Cloud Admin
  6. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Management UI Web Management Console -> https://Console.Cloud.Google.com
  7. Copyright 2019 by Stage 2 Security Stage 2 Security Control

    Plane (APIs) Control Plane (APIs) Data Plane Management UI Cloud Admin Ext Cloud Automation - Terraform - Salt Cloud - Custom
  8. Copyright 2019 by Stage 2 Security Stage 2 Security Data

    Plane Control Plane (APIs) Data Plane Management UI Cloud Admin Ext Cloud Automation - Terraform - Salt Cloud - Custom USERS
  9. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Roles GCP Roles are a collection of permissions • GCP Roles are similar to AWS IAM Policies Permissions enable you to take certain actions: • e.g. Compute.Instances.Start
  10. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Roles Primitives Role Primitives: • Owner -> Billing & Access • Editor -> Changes & Updates • Viewer -> Read-Only Pre-Dates Cloud Identity & Access Management (IAM) service in GCP • Generally, overly permissive and/or too broad
  11. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Roles Predefined Role Predefined: • Granular access to specific GCP resources (IAM) ◦ roles/pubsub.subscriber Role Custom: • Project and/or Organization level roles w/ granular permissions
  12. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Cloud Identity & Access Management (IAM) Authorization for GCP Resources • Introduced in early 2016 “Member” is one of the following • user, group, domain, service account, or public Cloud IAM does NOT directly manage identities, hence these reference: • Individual google account, Google groups, G Suite / Cloud Identity Domain Every identity has a unique email address
  13. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Cloud IAM Policies Policies bind Members to Roles at a specific hierarchy levels: • Org • Folder • Project • Resource Who can do what to which thing?
  14. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Accounts Three General Types of Creds: • User Accounts • API Keys • Services Accounts
  15. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Cloud Identity Identity as a Service • Users • Groups Similar to AWS IAM service or Active Directory Supports MFA and Security Key Enforcement (e.g. Hardware Device) Google Cloud Directory Sync -> LDAP and Active Directory Sync
  16. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Cloud Resource Manager Manage & Secure organization’s projects Similar to AWS Organizations
  17. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Cloud Audit Logging (Stackdriver) Audit Logs -> Who, What, When, and Where Logs • Admin Activity -> 400 days of retention for free • Data Access -> 7 days of retention for free, 30 days of retention for $ NOTE: GCP Services, so does not log apps running on GCE Similar to AWS CloudTrail
  18. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Cloud KMS Encryption service (AES256) designed to protect secrets • Secrets out of code base and into the environment • Does not store secrets • Encrypts and/or Decrypts secrets stored elsewhere • Control access to keys for Encryption and/or Decryption Integrated with IAM for Authorization & Cloud Audit Logging • Key rotation and key versioning for decryption Similar to AWS KMS & Vault by HashiCorp
  19. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Service Account Service Accounts are accounts for applications • Similar to AWS IAM Roles Service Accounts can be assumed by an application (or user, if authorized) • Should use least privilege
  20. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    User Managed Keys Two Types of Keys: • user managed keys ◦ generate/download private keys ◦ (e.g. for AWS to access GCP) ◦ expire 10 years from creation • GCP managed keys Ref: https:/ /cloud.google.com/iam/docs/understanding-service-accounts
  21. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    GCP Managed Keys Two Types of Keys: • GCP managed keys ◦ GCP native secrets ◦ prefered for GCP services ◦ (e.g. GCF, GAE, GCE, GKE, etc…) ◦ automatically rotated ◦ used for a maximum of two weeks Ref: https:/ /cloud.google.com/iam/docs/understanding-service-accounts
  22. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    GCE New VM Instance Google Compute Engine (GCE) Default: • Identity and API Access • Firewall • Startup script (Optional) • Metadata (Optional) Defaults: • Block project-wide SSH keys (unchecked) • Disk Encryption (Google-managed key) ...
  23. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    GCE Identity and API Access Access scopes: • read-only access to Storage and Service Management, • write access to Stackdriver Logging and Monitoring, • read/write access to Service Control. ...
  24. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    GCE Startup Script GCE Startup Script: • runs under the root user ...
  25. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    GCE Metadata Access scopes: • read-only access to Storage and Service Management, • write access to Stackdriver Logging and Monitoring, • read/write access to Service Control. ...
  26. Copyright 2019 by Stage 2 Security Stage 2 Security Data

    Center Firewall Server Side Request Forgery (SSRF) ... Web App Database Monitoring 10.1.1.1 Images 10.1.1.2 Internet 1 GET /app?img=b.jpg 2 3 4
  27. Copyright 2019 by Stage 2 Security Stage 2 Security Server

    Side Request Forgery (SSRF) ... Web App Database Monitoring 10.1.1.1 Images Internet Data Center Firewall 1 GET /?img=http://10.1.1.1/... 2 3 4 0
  28. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Server Side Request Forgery (SSRF) ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance
  29. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Metadata Service ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance
  30. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Metadata Service ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance
  31. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Metadata Service HTTP Header ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance
  32. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Metadata Service HTTP Header ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance
  33. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    v1beta1 ? ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance
  34. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    v1beta1 ! ... Web App Database metadata.google.internal 169.254.169.254 Images Internet GCP Firewall 1 GET /?img=http://metadata/.. 2 3 4 0 Instance
  35. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    SSRF Demo Steps on macOS: • curl http://35.247.8.30/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://35.247.8.30/extimage?p=http://metadata.google.internal/computeMet adata/v1beta1/instance/service-accounts/default/token ...
  36. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    SSRF Demo Steps on macOS: • curl http://34.73.197.205/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://34.73.197.205/extimage?p=http://metadata.google.internal/compute Metadata/v1beta1/instance/service-accounts/default/token ...
  37. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    SSRF Demo Steps on macOS: • curl http://34.73.197.205/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://34.73.197.205/extimage?p=http://metadata.google.internal/compute Metadata/v1beta1/instance/service-accounts/default/token ...
  38. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    SSRF Demo Steps on macOS: • curl http://35.247.8.30/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://35.247.8.30/extimage?p=http://metadata.google.internal/computeMet adata/v1beta1/instance/service-accounts/default/token ...
  39. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    SSRF Demo Steps on macOS: • curl http://35.247.8.30/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://35.247.8.30/extimage?p=http://metadata.google.internal/computeMet adata/v1beta1/instance/service-accounts/default/token ...
  40. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Validate User Tokens https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=ACCESS_T OKEN ...
  41. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Management UI Web Management Console -> https://Console.Cloud.Google.com
  42. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Compute Engine Identity and API Access
  43. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Compute Engine Identity and API Access
  44. Copyright 2019 by Stage 2 Security Stage 2 Security 2017:

    May-Oct 1. “Internal Accenture Data, Customer Information Exposed in Public Amazon S3 Bucket” 2. “Another Wide-Open Amazon S3 Bucket Exposes Verizon Customer Account Data” 3. “US voter info stored on wide-open cloud box, thanks to bungling Republican contractor” 4. “Researcher discovers classified Army intel app, data on open public AWS bucket” 5. “Millions of Time Warner Cable Customer Records Exposed in Third-Party Data Leak” 6. “Drone Manufacturer DJI Leaves SSL Key Exposed on Public Repository” 7. “Dow Jones becomes the latest organization to be affected by an AWS cloud data leakage due to misconfiguration and user error.” etc...
  45. Copyright 2019 by Stage 2 Security Stage 2 Security 2017:

    May-Oct 1. “Internal Accenture Data, Customer Information Exposed in Public Amazon S3 Bucket” 2. “Another Wide-Open Amazon S3 Bucket Exposes Verizon Customer Account Data” 3. “US voter info stored on wide-open cloud box, thanks to bungling Republican contractor” 4. “Researcher discovers classified Army intel app, data on open public AWS bucket” 5. “Millions of Time Warner Cable Customer Records Exposed in Third-Party Data Leak” 6. “Drone Manufacturer DJI Leaves SSL Key Exposed on Public Repository” 7. “Dow Jones becomes the latest organization to be affected by an AWS cloud data leakage due to misconfiguration and user error.” etc...
  46. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    Accessing Objects https://storage.googleapis.com/its_all_in_the_cloud/object001.jpg storage.googleapis.com -> GCP its_all_in_the_cloud -> Globally Unique Bucket Name object001.jpg -> Object Name ...
  47. Copyright 2019 by Stage 2 Security Stage 2 Security GoBuster

    - Finding Buckets & Objects https://storage.googleapis.com/its_all_in_the_cloud/object001.jpg gobuster -m dir -u “https://storage.googleapis.com” -i -t 100 -e -s 200,204 -w quickdir.txt
  48. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: Master Node & API Overview: • VM / Instance running the following services: ◦ kube-apiserver, ◦ kube-controller-manager and ◦ kube-scheduler. Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin
  49. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: etcd Overview: • Holds state information for the cluster ◦ “Access to etcd is equivalent to root permission in the cluster so ideally only the API server should have access to it.” Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin
  50. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: kubectl Overview: • kubectl is a cli to admin the cluster Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin Cli kubectl
  51. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: Dashboard Overview: • Dashboard is a web-based UI for K8s clusters ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Control Admin Browser Dashboard
  52. Copyright 2019 by Stage 2 Security Stage 2 Security Telsa

    K8s hacked! Unsecure Admin Console... ...
  53. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: Worker Nodes Overview: • Nodes are VMs / Instances w/ ◦ Container Runtime (e.g. Docker) ◦ Kube-proxy ◦ Kubelet ▪ Port: 10250/TCP ▪ Port: 10255/TCP ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin
  54. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: Pods Overview: • Pod contains 1 or more containers ◦ Smallest unit in K8s ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin
  55. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: Services Overview: • Services map K8s names to pod IPs ◦ When nodes get stop/started ◦ Services continue to route • Similar to a load balancer/proxy ◦ endpoint lookup... ◦ more magic be here ◦ ref: kube-proxy, etc... ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin
  56. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview: Load Balancers Overview: • Load Balancing via GCP Services ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control Admin
  57. Copyright 2019 by Stage 2 Security Stage 2 Security GCP:

    SSRF Demo Steps on macOS: • curl http://34.73.197.205/ • View Source ◦ /extimage?p=http%3A...%2Fsalamander.jpg curl http://34.73.197.205/extimage?p=http://metadata.google.internal/compute Metadata/v1beta1/instance/service-accounts/default/token ...
  58. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    Overview Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  59. Copyright 2019 by Stage 2 Security Stage 2 Security Default

    Service Account Find secrets: • /var/run/secrets/kuberenetes.io/serviceaccount/token ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  60. Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:

    Python via Memory Only Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ...
  61. Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:

    Python via Memory Only Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  62. Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:

    pyscript Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ...
  63. Copyright 2019 by Stage 2 Security Stage 2 Security Container

    Escapes Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ...
  64. Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:

    pyscript to access_token via metadata Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl metadata.google.internal 169.254.169.254
  65. Copyright 2019 by Stage 2 Security Stage 2 Security Voodoo:

    pyscript to access_token via metadata Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ...
  66. Copyright 2019 by Stage 2 Security Stage 2 Security .../v1beta1/instance/attributes/kube-env

    • Masquerading as the Kubelet • To the K8s API ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl metadata.google.internal 169.254.169.254
  67. Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet

    certificate and the Kubelet private key • Masquerading as the Kubelet • To the K8s API ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl metadata.google.internal 169.254.169.254
  68. Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet

    -> Port: 10250/TCP, 10255/TCP Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ...
  69. Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet

    -> Port: 10250/TCP, 10255/TCP Overview: • Pod contains 1 or more containers ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  70. Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet

    -> Port: 10250/TCP, 10255/TCP Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc ...
  71. Copyright 2019 by Stage 2 Security Stage 2 Security Container

    Escapes Container Escape Vulnerabilities: • CVE-2019-5736 -> Runc • CVE-2016-5195 -> Dirty Cow ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  72. Copyright 2019 by Stage 2 Security Stage 2 Security Container

    Escapes Container Escape Techniques: • Run Container in Cluster ◦ With Root File System Mounted! ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  73. Copyright 2019 by Stage 2 Security Stage 2 Security Kubernetes

    API Kubernetes API Vulnerabilities: • CVE-2018-1002105 -> kubernetes: authentication/authorization bypass ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  74. Copyright 2019 by Stage 2 Security Stage 2 Security Docker:

    2375/TCP (no auth.), 2376/TCP (TLS) Lateral Movement: • EDB-ID: 42356 -> Unprotected TCP Socket ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  75. Copyright 2019 by Stage 2 Security Stage 2 Security Network

    Security Highly recommend… • Isovalent.com w/ Cilium ◦ To lockdown network traffic ◦ via namepsaces ▪ … alternatively istio ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  76. Copyright 2019 by Stage 2 Security Stage 2 Security Update

    Kubernetes Cluster • Updates Frequently • New Security Features Regularly Added • Defaults Get Stronger ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  77. Copyright 2019 by Stage 2 Security Stage 2 Security Kubelet

    -> Configure Authentication • Enable “NodeRestriction” • --anonymous-auth=false ... Infrastructure Cloud (e.g. Instances) or On-Premises (e.g. VMs) Workers Control kubectl
  78. Copyright 2019 by Stage 2 Security Stage 2 Security Client-Side

    Vectors: • Remote Mac Exploitation Via Custom URL Schemes Ref: https:/ /objective-see.com/blog/blog_0x38.html
  79. Copyright 2019 by Stage 2 Security Stage 2 Security Browser

    Cookies Client-Side: • cookie_crimes -> https://github.com/defaultnamehere/cookie_crimes GCP Ref: https:/ /wunderwuzzi23.github.io/blog/passthecookie.html
  80. Copyright 2019 by Stage 2 Security Stage 2 Security Cloud

    Shell Client-Side: • cookie_crimes -> https://github.com/defaultnamehere/cookie_crimes GCP ...
  81. Copyright 2019 by Stage 2 Security Stage 2 Security Cloud

    Shell: .bashrc modification • ...
  82. Copyright 2019 by Stage 2 Security Stage 2 Security Cloud

    Shell -> .bashrc -> Voodoo -> Private Key • ...
  83. Copyright 2019 by Stage 2 Security Stage 2 Security Trainings

    @ BlackHat & On-Site! Thank You! [email protected] .sh @TweekFawkes