Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Custom Kubernetes Controllers at Mercari

Yuki Ito
February 21, 2022

Custom Kubernetes Controllers at Mercari

Yuki Ito

February 21, 2022
Tweet

More Decks by Yuki Ito

Other Decks in Programming

Transcript

  1. Custom Kubernetes Controllers at Mercari
    Yuki Ito (@mrno110)
    Kubernetes Casual Talk

    View Slide

  2. Merpay / Mercoin


    Architect


    Mercari


    Microservices Platform CI/CD
    Yuki Ito


    @mrno110

    View Slide

  3. Agenda
    ɾCustom Kubernetes Controllers


    ɾCode Generation

    View Slide

  4. Agenda
    ɾCustom Kubernetes Controllers


    ɾCode Generation

    View Slide

  5. spanner-autoscaler
    Autoscaler for the Google Cloud Spanner Instances

    View Slide

  6. spanner-autoscaler
    apiVersion: spanner.mercari.com/v1alpha1


    kind: SpannerAutoscaler


    metadata:


    name: spannerautoscaler-sample


    namespace: your-namespace


    spec:


    scaleTargetRef:


    projectId: your-gcp-project-id


    instanceId: your-spanner-instance-id


    minNodes: 1


    maxNodes: 4


    maxScaleDownNodes: 1


    targetCPUUtilization:


    highPriority: 60
    Autoscaler for the Google Cloud Spanner Instances

    View Slide

  7. spanner-autoscaler
    Google Cloud Day 2020
    https://cloudonair.withgoogle.com/events/google-cloud-day-digital?talk=d3-da07

    View Slide

  8. Bootes
    Bootes
    Namespace: a Namespace: b
    app: foo app: bar app: foo app: bar app: foo app: bar
    Namespace: c
    xDS Control Plane Kubernetes Controller

    View Slide

  9. Bootes
    xDS Control Plane Kubernetes Controller
    apiVersion: bootes.io/v1


    kind: Cluster


    metadata:


    name: example-cluster


    namespace: foo


    spec:


    config:


    name: example-cluster


    connect_timeout: 1s


    type: EDS


    lb_policy: ROUND_ROBIN


    http2_protocol_options: {}


    eds_cluster_config:


    eds_config:


    ads: {}

    View Slide

  10. Bootes
    Envoy Meetup Tokyo #2
    https://envoytokyo.connpass.com/event/175256/

    View Slide

  11. Pull Request Replication Controller

    View Slide

  12. Service Routing Controller

    View Slide

  13. Service Routing Controller
    Merpay Tech Fest 2021
    https://events.merpay.com/techfest-2021/

    View Slide

  14. Agenda
    ɾCustom Kubernetes Controllers


    ɾCode Generation

    View Slide

  15. Agenda
    ɾCustom Kubernetes Controllers


    ɾCode Generation

    View Slide

  16. De
    fi
    ne CRDs as Protocol Buffers
    message Repository {
    string owner = 1;
    string repository = 2;
    }
    type Repository struct {
    state protoimpl.MessageState
    sizeCache protoimpl.SizeCache
    unknownFields protoimpl.UnknownFields
    Owner string `protobuf:...`
    Repository string `protobuf:...`
    }
    Protocol Buffers
    func (in *Repository) DeepCopyInto(out *Repository) {
    p := proto.Clone(in).(*Repository)
    *out = *p
    }
    func (in *Repository) DeepCopy() *Repository {
    if in == nil {
    return nil
    }
    out := new(Repository)
    in.DeepCopyInto(out)
    return out
    }
    protoc-gen-go protoc-gen-deepcopy

    View Slide

  17. istio / tools / protoc-gen-deepcopy
    https://github.com/istio/tools/tree/master/cmd/protoc-gen-deepcopy

    View Slide

  18. Generic protoc-gen-deepcopy
    https://github.com/protobuf-tools/protoc-gen-deepcopy

    View Slide

  19. De
    fi
    ne CRDs as Protocol Buffers
    type Repository struct {
    state protoimpl.MessageState
    sizeCache protoimpl.SizeCache
    unknownFields protoimpl.UnknownFields
    Owner string `protobuf:...`
    Repository string `protobuf:...`
    }
    Protocol Buffers
    protoc-gen-go
    type Repository struct {
    v1.TypeMeta
    v1.ObjectMeta
    // Proto Buffer Struct
    Spec apisv1alpha1.Repository
    }
    kubetype-gen

    View Slide

  20. istio / tools / kubetype-gen
    https://github.com/istio/tools/tree/master/cmd/kubetype-gen

    View Slide

  21. Generic kubetype-gen
    https://github.com/110y/kubetype-gen

    View Slide

  22. De
    fi
    ne CRDs as Protocol Buffers
    Protocol Buffers
    protoc-gen-go / protoc-gen-deepcopy
    Protocol Buffer Go Struct (+ DeepCopy)
    Kubernetes Go Types
    Custom Resource De
    fi
    nitions (YAML)
    kubetype-gen
    controller-gen

    View Slide

  23. De
    fi
    ne CRDs as Protocol Buffers
    Protocol Buffers
    Advanced
    Custom Resource De
    fi
    nitions (YAML)
    istio /tools / cue-gen

    View Slide