Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Running gRPC Services for Serving Legacy RESTful API on Kubernetes

Buzzvil
December 10, 2019
560

Running gRPC Services for Serving Legacy RESTful API on Kubernetes

Buzzvil

December 10, 2019
Tweet

Transcript

  1. Who We Are Sungwon Lee / Whale Buzzvil Chief Architect

    Hoseong Hwang / Liam Buzzvil DevOps Lead
  2. Why gRPC? Performance matters Multiple services increases network latency Ad

    request should be done within 100ms API-first Approach Need to support polyglot IDL(Interface Definition Language) required
  3. gRPC is great, but.. We still need to support legacy

    RESTful JSON API There are partners using the legacy APIs REST APIs: mostly beloved API protocol B2B business
  4. Support both gRPC/REST protocols Build a server for transcoding? gRPC

    Server REST JSON Transcoding server Client Expensive maintenance cost 1. Parse JSON request 2. Transform JSON to gRPC Request 3. Send request to gRPC Server 4. Transcode gRPC response to JSON format 5. Send a response to the client It seems familiar!
  5. Istio Service Mesh Moving to Microservices Micoservices grow in size

    and complexity Difficult to understand and manage Service Mesh Detach network logic from business logic Monolythic Microservice Microservice Proxy Microservice Proxy
  6. Let’s Try It ! Setup Protobuf for Custom Routes Setup

    Istio/EnvoyFilter for gRPC JSON transcoder
  7. Setup Protocol Buffers service CalendarApi { rpc ListEvents(ListEventsRequest) returns (ListEventsResponse);

    // ... } The easiest way: Do nothing. Transcoder will handle it automatically.
  8. Setup Protocol Buffers (Cont.) package buzzvil.calendar.v1; service CalendarApi { rpc

    ListEvents(ListEventsRequest) returns (ListEventsResponse); } $ curl -X POST https://host.here/buzzvil.calendar.v1.CalendarApi/ListEvents = POST /<package>.<service>/<method>
  9. Setup Protocol Buffers (Cont.) package buzzvil.calendar.v1; service CalendarApi { rpc

    ListEvents(ListEventsRequest) returns (ListEventsResponse) { option (google.api.http) = { get: "/v1/events" }; } } Custom routes: google.api.http annotation $ curl -X GET https://host.here/v1/events
  10. Setup Protocol Buffers (Cont.) Parameters service CalendarApi { rpc UpdateEvent(UpdateEventRequest)

    returns (Event) { option (google.api.http) = { put: "/v1/events/{event.id}", body: "event" }; } } message Event { int64 id = 1; string name = 2; } message UpdateEventRequest { Event event = 1; }
  11. Setup Envoy Proxy (Cont.) apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name:

    example-transcoder namespace: example-namespace spec: workloadLabels: ... filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor: “path/to/bin” match_incoming_request_route: True auto_mapping: False services: - buzzvil.calendar.v1.CalendarApi filterName: envoy.grpc_json_transcoder match_incoming_request_route : True Services: - buzzvil.calendar.v1.CalendarApi Apply grpc_json_transcoder filter to the services Use the specified route
  12. Setup Envoy Proxy (Cont.) apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name:

    example-transcoder namespace: example-namespace spec: workloadLabels: ... filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor: “path/to/bin” match_incoming_request_route: True auto_mapping: False services: - buzzvil.calendar.v1.CalendarApi Proto Descriptor Envoy has to know the proto descriptor of your gRPC service in order to the transcoding. $ protoc -I$(GOOGLEAPIS_DIR) -I. \ --include_imports \ --include_source_info \ --descriptor_set_out =proto.pb \ test/proto/bookstore.proto proto_descriptor: “path/to/bin”
  13. Setup Envoy Proxy (Cont.) Proto Descriptor Path proto_descriptor: “generated/file/path/proto.pb” proto_descriptor_bin:

    Cr15ChVnb29nbGUvYXBpL2h0dHAucHJvdG8SCm... Proto Descriptor Bin $ cat proto.pb | openssl base64 -A Encode proto descriptor using base64 encoding, and set the value in yaml file
  14. API Changes Over Time message Post { int32 id =

    1; string title = 2; string body = 3; repeated string tags = 4; google.protobuf.Timestamp created_at = 5; } Service spec changes over time • Additional field to message, new RPC • New package version • Beware of backward incompatible changes!(renaming, changing type, removing field, …) message Post { int32 id = 1; string title = 2; string body = 3; repeated string tags = 4; google.protobuf.Timestamp created_at = 5; string featured_image_url = 6; }
  15. Deploying EnvoyFilter proto_descriptor should be updated whenever protocol buffer is

    changed. • proto_descriptor: expect *.pb descriptor to exist on file system → volumeMount to istio-proxy(sidecar) container • proto_descriptor_bin: embed base64-encoded *.pb descriptor content directly into EnvoyFilter apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: blog-transcoder namespace: blog spec: workloadLabels: ... filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor: “blog.pb” match_incoming_request_route: True auto_mapping: False services: - blog.BlogService
  16. Deploying EnvoyFilter (Cont.) • proto_descriptor ◦ Ensuring *.pb descriptor file

    is mounted to istio-proxy container is not easy ◦ Automatic sidecar injection from istio-sidecar-injector is not dynamic enough(mounted volume contains all pb descriptors) ◦ Hard to control deployment timing • proto_descriptor_bin ◦ Lack of readability ◦ No volume dependency ◦ New transcoding configuration is applied as soon as EnvoyFilter is updated
  17. Protocol Buffer Artifact Pipeline Language-specific gRPC Artifacts should be created

    whenever API is updated $ protoc -I (GOOGLE_APIS_DIR) --python_out=.. --go_out=.. --js_out=.. ./blog.proto blog.proto Private repository Private pypi server Private npm registry + rubygems, maven, ... Lint, Breaking Changes warning, ... comment. proto user.proto
  18. ├── Jenkinsfile ├── Makefile └── packages ├── blog │ ├──

    blog.proto │ └── package.json └── user ├── user.proto └── package.json Mono Repository Approach • Create CI pipeline that generates & publishes language-specific artifacts • Common CI checks(lint, breaking changes warning) run for changed proto files • Reduced boilerplate! • Use lerna(node package) to bump versions of each API Generating protobuf descriptor artifact: protoc --include_imports --include_source_info --descriptor_set_out=blog.pb
  19. Helm as Configuration Management Tool Helm • Already using helm

    for managing cluster-level services (ingress controller, EFK, prometheus, grafana, …) • Hosting private registry is easy(chartmuseum, object storage, ..) • Easy CI pipeline setup • CRDs can also be handled(VirtualService, EnvoyFilter, ...)
  20. Helm Chart for gRPC Transcoding apiVersion: apps/v1 kind: Deployment metadata:

    ... spec: ... containers: - name: {{ .Chart.Name }} ... ports: - name: grpc containerPort: 9000 protocol: TCP apiVersion: v1 kind: Service metadata: ... spec: type: {{ .Values.service.type }} ports: - port: {{ .Values.service.port }} targetPort: grpc protocol: TCP name: grpc selector: {{- include "blog.matchLabels" . | nindent 4 }} Istio looks for port named grpc or any name prefixed with grpc-
  21. Helm Chart for gRPC Transcoding New Helm release will update

    proto_descriptor_bin apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: {{ include "blog.fullname" . }}-transcoder namespace: {{ .Release.Namespace }} spec: workloadLabels: {{- include "blog.matchLabels" . | nindent 4 }} filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor_bin: {{ .Values.protoDescriptorBin }} match_incoming_request_route: True auto_mapping: False services: - blog.v1.Blog print_options: always_print_primitive_fields: True preserve_proto_field_names: True
  22. Releasing New API Version Using Helm CLI: $ helm upgrade

    RELEASE -f values.prod.yaml -f proto_descriptor.yaml --set image.tag=IMAGE_TAG repo/blog Helm Chart values.prod.yaml proto_descriptor.yaml image: tag: dkr.registry/blog:954ade7 secret: DATABASE_URL: mysql://xx:yy@db:3306 WEB_CONCURRENCY: 5 proto_descriptor_bin: Cr15ChVnb29n bGUvYXBpL2h0dHAucHJvdG... *We internally use Spinnaker to deploy helm chart based services
  23. • 503 status code is returned when requested to non-mapped

    path • kubectl port-forward didn’t work for testing out REST APIs • Arbitrary(non-JSON) message can be also supported by using google.api.HttpBody • grpc-status, grpc-message headers in response are useful when debugging Pitfalls & Tips
  24. Conclusion • It takes initial effort to build pipeline, but

    after then it becomes easy to develop any gRPC service that supports JSON transcoding • You can develop gRPC services while allowing other teams time to transition • API-First approach becomes standard throughout the organization • Over time, initial setup cost pays off
  25. Resources Sample Repo - https://github.com/Buzzvil/grpc-json-transcoding-example/ • https://medium.com/@ssowonny/grpc를-쓰면-rest가-공짜-19e3a6bed4a9 • https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ grpc_json_transcoder_filter

    • https://blog.envoyproxy.io/envoy-and-grpc-web-a-fresh-new-alternative-to-res t-6504ce7eb880 • https://medium.com/google-cloud/grpc-transcoding-e101cc53d51d • https://blog.jdriven.com/2018/11/transcoding-grpc-to-http-json-using-envoy/ • https://medium.com/building-ibotta/building-a-scaleable-protocol-buffers-grp c-artifact-pipeline-5265c5118c9d