Ingress-GCE and its problem

2018/11/08 Kubernetes Meetup Tokyo #14

Transcript

1. Ingress GCE & Problem
   Kubernetes Meetup Tokyo #14
   2018/11/08
   @tjun
   

   View Slide

2. Who am I?
   - @tjun
   - Merpay SRE, EM
   - (Sometimes) Working with Mercari Microservices Platform Team
   - We are hiring!
   

   View Slide

3. Ingress-GCE
   ● Ingress controller for Google Cloud
   ○ GCE L7 load balancer controller
   ○ Configured through the Kubernetes Ingress API
   

   View Slide

4. L7 Load balancing on Kubernetes
   GCP resources are created with
   $ kubectl apply -f ingress.yaml
   Global Forwarding Rule -> TargetHttpProxy -> URL Map ->
   Backend Service
   -> Instance Group or Network Endpoint Group
   

   View Slide

5. Ingress GCE or Ingress Nginx
   https://medium.com/omnius/kubernetes-ingress-gce-vs-nginx-controllers-1-3-d89d6dd3da73
   

   View Slide

6. Merpay architecture
   Clients
   Merpay
   API
   Gateway
   API
   ServiceA
   ServiceA
   ServiceA
   API
   

   View Slide

7. Ingress GCE problem on GKE v1.10.6
   GKE Release Notes
   

   View Slide

8. Ingress GCE problem on GKE v1.10.6
   When we recreate an Ingress:
   ● External IP is never attached
   ● Backend Services are not created
   What we did:
   ● kubectl describe ing xxxxx / kubectl get events
   ● Check TLS on secret and
   ● Check kube-apiserver logs and cert-manager logs
   ● Recreate ingress without static-ip -> OK
   ● Recreate ingress with 1 backend -> sometimes OK
   

   View Slide

9. Ingress GCE problem on GKE v1.10.6
   Ask GCP support
   

   View Slide

10. Our workaround until cluster upgrade
    $ kubectl apply -f merpay-ingress1.yaml
    $ kubectl apply -f merpay-ingress2.yaml
    $ kubectl apply -f merpay-ingress3.yaml -> OK
    $ kubectl apply -f merpay-ingress4.yaml
    

    View Slide

11. Summary
    今は直っているので、IngressGCEめちゃ便利です。GKE最高！
    

    View Slide