Custom Kubernetes Controllers at Mercari

Transcript

1. Custom Kubernetes Controllers at Mercari
   Yuki Ito (@mrno110)
   Kubernetes Casual Talk
   

   View Slide

2. Merpay / Mercoin
   
   
   Architect
   
   
   Mercari
   
   
   Microservices Platform CI/CD
   Yuki Ito
   
   
   @mrno110
   

   View Slide

3. Agenda
   ɾCustom Kubernetes Controllers
   
   
   ɾCode Generation
   

   View Slide

4. Agenda
   ɾCustom Kubernetes Controllers
   
   
   ɾCode Generation
   

   View Slide

5. spanner-autoscaler
   Autoscaler for the Google Cloud Spanner Instances
   

   View Slide

6. spanner-autoscaler
   apiVersion: spanner.mercari.com/v1alpha1
   
   
   kind: SpannerAutoscaler
   
   
   metadata:
   
   
   name: spannerautoscaler-sample
   
   
   namespace: your-namespace
   
   
   spec:
   
   
   scaleTargetRef:
   
   
   projectId: your-gcp-project-id
   
   
   instanceId: your-spanner-instance-id
   
   
   minNodes: 1
   
   
   maxNodes: 4
   
   
   maxScaleDownNodes: 1
   
   
   targetCPUUtilization:
   
   
   highPriority: 60
   Autoscaler for the Google Cloud Spanner Instances
   

   View Slide

7. spanner-autoscaler
   Google Cloud Day 2020
   https://cloudonair.withgoogle.com/events/google-cloud-day-digital?talk=d3-da07
   

   View Slide

8. Bootes
   Bootes
   Namespace: a Namespace: b
   app: foo app: bar app: foo app: bar app: foo app: bar
   Namespace: c
   xDS Control Plane Kubernetes Controller
   

   View Slide

9. Bootes
   xDS Control Plane Kubernetes Controller
   apiVersion: bootes.io/v1
   
   
   kind: Cluster
   
   
   metadata:
   
   
   name: example-cluster
   
   
   namespace: foo
   
   
   spec:
   
   
   config:
   
   
   name: example-cluster
   
   
   connect_timeout: 1s
   
   
   type: EDS
   
   
   lb_policy: ROUND_ROBIN
   
   
   http2_protocol_options: {}
   
   
   eds_cluster_config:
   
   
   eds_config:
   
   
   ads: {}
   

   View Slide

10. Bootes
    Envoy Meetup Tokyo #2
    https://envoytokyo.connpass.com/event/175256/
    

    View Slide

11. Pull Request Replication Controller
    

    View Slide

12. Service Routing Controller
    

    View Slide

13. Service Routing Controller
    Merpay Tech Fest 2021
    https://events.merpay.com/techfest-2021/
    

    View Slide

14. Agenda
    ɾCustom Kubernetes Controllers
    
    
    ɾCode Generation
    

    View Slide

15. Agenda
    ɾCustom Kubernetes Controllers
    
    
    ɾCode Generation
    

    View Slide

16. De
    fi
    ne CRDs as Protocol Buffers
    message Repository {
    string owner = 1;
    string repository = 2;
    }
    type Repository struct {
    state protoimpl.MessageState
    sizeCache protoimpl.SizeCache
    unknownFields protoimpl.UnknownFields
    Owner string `protobuf:...`
    Repository string `protobuf:...`
    }
    Protocol Buffers
    func (in *Repository) DeepCopyInto(out *Repository) {
    p := proto.Clone(in).(*Repository)
    *out = *p
    }
    func (in *Repository) DeepCopy() *Repository {
    if in == nil {
    return nil
    }
    out := new(Repository)
    in.DeepCopyInto(out)
    return out
    }
    protoc-gen-go protoc-gen-deepcopy
    

    View Slide

17. istio / tools / protoc-gen-deepcopy
    https://github.com/istio/tools/tree/master/cmd/protoc-gen-deepcopy
    

    View Slide

18. Generic protoc-gen-deepcopy
    https://github.com/protobuf-tools/protoc-gen-deepcopy
    

    View Slide

19. De
    fi
    ne CRDs as Protocol Buffers
    type Repository struct {
    state protoimpl.MessageState
    sizeCache protoimpl.SizeCache
    unknownFields protoimpl.UnknownFields
    Owner string `protobuf:...`
    Repository string `protobuf:...`
    }
    Protocol Buffers
    protoc-gen-go
    type Repository struct {
    v1.TypeMeta
    v1.ObjectMeta
    // Proto Buffer Struct
    Spec apisv1alpha1.Repository
    }
    kubetype-gen
    

    View Slide

20. istio / tools / kubetype-gen
    https://github.com/istio/tools/tree/master/cmd/kubetype-gen
    

    View Slide

21. Generic kubetype-gen
    https://github.com/110y/kubetype-gen
    

    View Slide

22. De
    fi
    ne CRDs as Protocol Buffers
    Protocol Buffers
    protoc-gen-go / protoc-gen-deepcopy
    Protocol Buffer Go Struct (+ DeepCopy)
    Kubernetes Go Types
    Custom Resource De
    fi
    nitions (YAML)
    kubetype-gen
    controller-gen
    

    View Slide

23. De
    fi
    ne CRDs as Protocol Buffers
    Protocol Buffers
    Advanced
    Custom Resource De
    fi
    nitions (YAML)
    istio /tools / cue-gen
    

    View Slide