GCLB Traffic Management: envoy inside @sakajunquality Google Developers Expert, Cloud 19 April 2019 #gcpug

Traffic Management between Services - Canary Release - Disaster Recovery - Service Migration - etc...

Istio?

Istio(in functionality) https://istio.io/

In “some” cases Istio is too much

In “some” cases Istio is too much (For the record, I am Istio user in production)

In “some” cases Istio is too much (For the record, I am Istio user in production) (And I like Istio)

Istio: Deployment Model https://istio.io/docs/concepts/deployment-models/

Istio: Control Plane? or https://istio.io/docs/concepts/deployment-models/

And many things to be taken care of…

Traffic Director?

Traffic Director - Yes! Control Plane is managed by Google and outside cluster - should be reliable enough - Still need to deploy envoy for the data plane - And still too much? Maybe - Anyways another time!

https://twitter.com/kelseyhightower/status/1150158904900431873

https://twitter.com/kelseyhightower/status/1150175947665993728

No content

https://twitter.com/mattklein123/status/1156413081116798976

+

Envoy L7 ILB - Announced in Cloud Next ’19 SF Sessions - L7 Internal Load Balancer - Powered by fully-managed envoy - => Traffic Management Capability https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management

How it works https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management

URL Map? Target Proxy …? - Remember the GCE Ingress! apiVersion: extensions/v1beta1 kind: Ingress metadata: name: my-ingress spec: rules: - http: paths: - path: /* backend: serviceName: hello servicePort: 8080 - path: /hi backend: serviceName: hey servicePort: 8080

How it works https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management https://cloud.google.com/load-balancing/docs/url-map-concepts#url-map-components

How it works https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management Those components are extended w/

How it works https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management https://www.envoyproxy.io/docs/envoy/latest/api-v2/api/v2/route/route.proto Those components are extended w/

How it works https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management This is actually a same technology of Traffic Director’s route rules and traffic policies https://cloud.google.com/traffic-director/docs/traffic-control

Let’s try!

Demo

Demo - Frontend and Backend Services - Frontend is somewhere within VPC - Backend is spread into two GKE Clusters across zones - Control Backend traffic w/ internal L7 LB Frontend Services Backend ILB Cloud Load Balancing Primary Cluster Kubernetes Engine Secondary Cluster Kubernetes Engine Control Traffic Here!

Detailed Architecture: Backend zone-b zone-a Primary Cluster Kubernetes Engine Service Pods NEG Backend Service Secondary Cluster Kubernetes Engine Service Pods NEG Backend Service URL Map Target HTTP Proxy Forwarding Rule

Setup Check out the Gist https://gist.github.com/sakajunquality/1f258cbe20e0a509324cb5b2f45bb901

Demo: Primary 100% At first backend service is using only running on the primary cluster. Frontend Services Backend ILB Cloud Load Balancing Primary Cluster Kubernetes Engine 100%

Demo: Primary 100% // Test Traffic $ ./test.sh *** *** Results of load balancing to 10.100.1.199: *** 1000 This is Primary

Demo: Test Script // test.sh for LB_IP in 10.100.1.199; do RESULTS= for i in {1..1000}; do RESULTS="$RESULTS:`curl ${LB_IP}`"; done >/dev/null 2>&1 IFS=':' echo "***" echo "*** Results of load balancing to $LB_IP: " echo "***" for line in $RESULTS; do echo $line; done | grep -Ev "^$" | sort | uniq -c echo done

Demo: Primary 100% // export the current config gcloud beta compute url-maps export sakajun-map --region=us-west1 --destination=config.yaml defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service kind: compute#urlMap name: sakajun-map region: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1 selfLink: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/urlMaps/sakajun-map

Demo: Primary 90% / Secondary 10% Prepared the secondary cluster and sending traffic little by little: e.g. approx. 10% Frontend Services Backend ILB Cloud Load Balancing Primary Cluster Kubernetes Engine Secondary Cluster Kubernetes Engine 90% 10%

Demo: Primary 90% / Secondary 10% defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service hostRules: - description : '' hosts: - '*' pathMatcher : matcher1 kind: compute#urlMap name: sakajun-map pathMatchers : - defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service name: matcher1 routeRules : - matchRules : - prefixMatch : / routeAction : weightedBackendServices : - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service weight: 90 - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-secondary-service weight: 10 region: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1 selfLink: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/urlMaps/sakajun-map

Demo: Primary 90% / Secondary 10% defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service hostRules: - description : '' hosts: - '*' pathMatcher : matcher1 kind: compute#urlMap name: sakajun-map pathMatchers : - defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service name: matcher1 routeRules : - matchRules : - prefixMatch : / routeAction : weightedBackendServices : - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service weight: 90 - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-secondary-service weight: 10 region: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1 selfLink: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/urlMaps/sakajun-map - For any url starts from “/” - Send 90% traffic to primary backend - Send the rest of traffic to secondary backend

// Update the current config gcloud beta compute url-maps import sakajun-map --region=us-west1 --source=config.yaml Demo: Primary 90% / Secondary 10% // Check the traffic $ ./test.sh *** *** Results of load balancing to 10.100.1.199: *** 910 This is Primary 90 This is Secondary

Demo: Primary 50% / Secondary 50% Looks good! Now it’s time to increase traffic to the secondary cluster! Frontend Services Backend ILB Cloud Load Balancing Primary Cluster Kubernetes Engine Secondary Cluster Kubernetes Engine 50% 50%

Demo: Primary 50% / Secondary 50% defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service hostRules: - description : '' hosts: - '*' pathMatcher : matcher1 kind: compute#urlMap name: sakajun-map pathMatchers : - defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service name: matcher1 routeRules : - matchRules : - prefixMatch : / routeAction : weightedBackendServices : - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service weight: 50 - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-secondary-service weight: 50 region: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1 selfLink: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/urlMaps/sakajun-map Just changing the weight!

// Update the current config gcloud beta compute url-maps import sakajun-map --region=us-west1 --source=config.yaml Demo: Primary 50% / Secondary 50% // Check the traffic $ ./test.sh *** *** Results of load balancing to 10.100.1.199: *** 480 This is Primary 520 This is Secondary

Demo: Destroy a Cluster by mistake Oops! I just destroyed one of the clusters…. Let’s fix it. Frontend Services Backend ILB Cloud Load Balancing Primary Cluster Kubernetes Engine Secondary Cluster Kubernetes Engine 50% 50%

Demo: Destroy a Cluster by mistake defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service hostRules: - description : '' hosts: - '*' pathMatcher : matcher1 kind: compute#urlMap name: sakajun-map pathMatchers : - defaultService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service name: matcher1 routeRules : - matchRules : - prefixMatch : / routeAction : weightedBackendServices : - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-primary-service weight: 0 - backendService : https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/backendServices/sakajun-secondary-service weight: 100 region: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1 selfLink: https://www.googleapis.com/compute/beta/projects/sakajunlabs-0916/regions/us-west1/urlMaps/sakajun-map Stop sending to the primary

// Update the current config gcloud beta compute url-maps import sakajun-map --region=us-west1 --source=config.yaml Demo: Destroy a Cluster by mistake // Check the traffic $ ./test.sh *** *** Results of load balancing to 10.100.1.199: *** 1000 This is Secondary

Demo: Destroy a Cluster by mistake Now we have a plenty of time to recover the deleted cluster! And next time, we can stop traffic before destroying clusters. Demo Ends.

Limitations - Internal LB Limitations - e.g. Shared VPC, IAP... - https://cloud.google.com/load-balancing/docs/l7-internal/#limitations - URL Map Limitations - Some resource does not work as intended - https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management#limitations

Takeaways

Takeaways With Internal HTTP Load Balancer (and thanks to envoy proxy), you can control traffic - of service with several versions - running in different workloads: VMs or containers without a rich control plane!

Thanks

Resources - Internal HTTP(S) Load Balancing concepts - https://cloud.google.com/load-balancing/docs/l7-internal/ - Setting Up Internal HTTP(S) Load Balancing for GKE pods - https://cloud.google.com/load-balancing/docs/l7-internal/set-up-gke-pods - Traffic management with route rules and traffic policies - https://cloud.google.com/load-balancing/docs/l7-internal/traffic-management - Network endpoint groups in load balancing concepts - https://cloud.google.com/load-balancing/docs/negs/ - Proxy-only subnets for internal HTTP(S) load balancers - https://cloud.google.com/load-balancing/docs/l7-internal/proxy-only-subnets