GKE Deep Dive

Look at GKE's features and what differentiates it from the other KaaS platforms out there @ Multi-Cloud Engineering | London | Feb 2020

Transcript

1. GKE Deep Dive Stepan Stipl | @stepanstipl Senior Cloud Architect // DoiT International
2. ToC - History & Versions - Operability - Limits - Securtiy - Networking - Integration - Recap
3. Basics K8s GKE EKS AKS GA Jul 2015 Aug 2015 Jun 2018 Jun 2018 SLA _ 99.95% / 99.5% 99.9% 99.5% pricing _ free $0.10/h ($72/m) free latest 1.17.3 1.15.9-gke.9 1.14.9-eks.8 1.15.7 OS COS, Ubuntu Amazon Linux 2 Ubuntu runtime Docker, CRI Docker, containerd Docker Moby Docker Talking about fully managed, out-of-the box setup.
4. 2018-04 2018-07 2018-10 2019-01 2019-04 2019-07 2019-10 2020-01 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 aks Aa eks Aa gke-ga Aa gke-rapid Aa gke-regular Aa kubernetes Aa date k8s_version Releases
5. Releases - patch 2019-10 2020-01 1.14.03 1.14.05 1.14.06 1.14.07 1.14.08 1.14.09 1.14.10 aks Aa eks Aa gke-ga Aa date k8s_version
6. Releases - CVEs CVE-2019-1002100 (DoS JSON patch) CVE-2019-11249 (crd namespace) CVE-2019-11253 (anon auth) CVE-2019-9512 and CVE-2019-9514 (Go http DoS) 0 5 10 15 20 25 30 35 40 GKE EKS AKS CVE Days
7. gcloud beta container clusters create ldn-2020-primary \ --region "${PRIMARY_REGION}" \ --release-channel rapid \ --identity-namespace "${PROJECT_ID}.svc.id.goog" \ --scopes cloud-platform \ --enable-private-nodes \ --master-ipv4-cidr 10.1.0.0/28 \ --no-enable-master-authorized-networks \ --enable-ip-alias \ --network ldn-2020-primary \ --subnetwork ldn-2020-primary \ --num-nodes 1 \ --machine-type e2-standard-2 \ --preemptible
8. Operability GKE EKS AKS Node mgmt. Yes Nov 2019* Yes Master upgrade Auto User + Manual User Node upgrade Auto + Surge upgrades User User Release channels yes no no Cluster Auto-scaling H no H Pod auto-scaling H + V no (metrics) H Node health- monitoring yes no (ASG) no (VMSS) Node auto-repair yes no (ASG) no (VMSS) Live migration yes no no
9. GKE is fast GKE EKS AKS Cluster cruation ~5m ~15-20m ~10m Node creation <3m <5m <10m Limits GKE EKS AKS Nodes 5000 (1000) 1000 1000 (100) Pods per node 110 depends on ENIs - 4-737 ((IP-1) * ENI) + 2 110 / 30 cni / 250 max (65k)
10. Felix v1
11. Security - Native RBAC - also groups (β) (ΔEKS - special maps, only roles & users) --security-group "gke-security-groups@${YOUR_DOMAIN}" - Workload Identity (ΔAKS) --identity-namespace "${PROJECT_ID}.svc.id.goog" \ kind: ServiceAccount ... annotations: iam.gke.io/gcp-service-account: ${SVC_ACCOUNT}@${PROJECT_ID}.iam.gserviceaccount.com
12. Security #2 - Private nodes (ΔEKS) --enable-private-nodes - Private master API – disable public endpoint or control CIDR --master-ipv4-cidr "10.1.0.0/28" --no-enable-master-authorized-networks
13. Security #3 - Network Policy (ΔEKS) - Read-only root FS (ΔAKS, ΔEKS) - CMEK boot & attached disks (ΔAKS, ΔEKS) kind: StorageClass parameters: disk-encryption-kms-key: projects/key-project-id/locations/location/keyRings/key-ring/cryptoKeys/key - Shielded VMs (ΔAKS, ΔEKS) - Sandbox - gVisor (ΔAKS, ΔEKS) - Application-layer Secrets Encryption (ΔAKS, ΔEKS)
14. Networking - Native IP (Alias IP) Improved security - visibility, firewall rules, anti-spoofing Routable on GCP network (incl. Interconnect/VPN) Better performance & scalability Note: Service (ClusterIP) addresses are only available from within the cluster. - Intranode visibility (ΔAKS, ΔEKS) - No CoreDNS (ΔAKS, ΔEKS) - NodeLocal DNSCache (β, ΔAKS, ΔEKS)
15. Load Balancing - External Ingress Load Balancing - global anycast IP (no DNS!) 1000 nodes limit - External/Internal Service Load Balancing - Internal Ingress Load Balancing (β) Managed SSL certificates (β, ΔAKS, ΔEKS - manual) apiVersion: networking.gke.io/v1beta1 kind: ManagedCertificate metadata: name: example-certificate spec: domains: - myapp.example.com
16. Felix v2
17. Container-native load balancing & NEGs (ΔAKS, ΔEKS) - Performance, visibility - Readiness gates cloud.google.com/neg: '{"ingress": true}' cloud.google.com/neg: '{"exposed_ports": {"80":{}}}'
18. Felix v3
19. Networking - multi-cluster - kubemci (β) - classic ingress networking, only Instance Groups - not so great story - Autoneg GKE controller https://github.com/GoogleCloudPlatform/gke-autoneg-controller anthos.cft.dev/autoneg: '{"name":"ldn-2020-primary", "max_rate_per_endpoint":10}'
20. Felix v4
21. Networking Integration - CDN (ΔAKS, ΔEKS) kind: BackendConfig ... cdn: enabled: true cachePolicy: includeHost: true includeProtocol: true includeQueryString: false - Cloud Armor (ΔAKS, ΔEKS) ... spec: securityPolicy: name: ca-how-to-security-policy
22. IAP (ΔAKS, ΔEKS) apiVersion: cloud.google.com/v1beta1 kind: BackendConfig metadata: name: config-default namespace: my-namespace spec: iap: enabled: true oauthclientCredentials: secretName: my-secret
23. Config Connector (manual install, ΔAKS - OSBA, ΔEKS - ASB) apiVersion: spanner.cnrm.cloud.google.com/v1beta1 kind: SpannerInstance metadata: labels: label-one: "value-one" name: spannerinstance-sample spec: config: regional-us-west1 displayName: Spanner Instance Sample numNodes: 1
24. Add-ons Istio (β, AKS, ΔEKS) - Service Mesh - Not so great story so far kBatch (β, manual, AKS, ΔEKS) - High performance, high throughput and technical batch computing Config Sync (AKS, ΔEKS) - GitOps for cluster management Application Delivery (β, AKS, ΔEKS) - GitOps for application deployment
25. Monitoring Stackdriver (Operations) - Logging, Monitoring & Alerting - Dashboard APIs - Kubernetes monitoring - SRE API - SLIs, SLOs and Error Budgets - Stackdriver Prometheus sidecar
26. Infra As Code Magic Modules https://github.com/GoogleCloudPlatform/magic-modules Terrafrom & Ansible
27. Recap GKE is great place for your workloads... ... but don't forget to design for cloud & containers.
28. Thank you & Q & A @stepanstipl // DoiT International