Running gRPC Services for Serving Legacy RESTful API on Kubernetes

Transcript

2. Running gRPC Services for Serving Legacy RESTful API on Kubernetes
3. Who We Are Sungwon Lee / Whale Buzzvil Chief Architect Hoseong Hwang / Liam Buzzvil DevOps Lead
4. Why? Why did we choose gRPC? Why do we still need to support REST API?
5. Why microservices architecture? There are so many domains and stakeholders in ad-tech industry.
6. Why gRPC? Performance matters Multiple services increases network latency Ad request should be done within 100ms API-first Approach Need to support polyglot IDL(Interface Definition Language) required
7. gRPC is great, but.. We still need to support legacy RESTful JSON API There are partners using the legacy APIs REST APIs: mostly beloved API protocol B2B business
8. Support both gRPC/REST protocols Build a server for transcoding? gRPC Server REST JSON Transcoding server Client Expensive maintenance cost 1. Parse JSON request 2. Transform JSON to gRPC Request 3. Send request to gRPC Server 4. Transcode gRPC response to JSON format 5. Send a response to the client It seems familiar!
9. Istio Service Mesh Moving to Microservices Micoservices grow in size and complexity Difficult to understand and manage Service Mesh Detach network logic from business logic Monolythic Microservice Microservice Proxy Microservice Proxy
10. Istio Service Mesh
12. Let’s Try It ! Setup Protobuf for Custom Routes Setup Istio/EnvoyFilter for gRPC JSON transcoder
13. Setup Protocol Buffers service CalendarApi { rpc ListEvents(ListEventsRequest) returns (ListEventsResponse); // ... } The easiest way: Do nothing. Transcoder will handle it automatically.
14. Setup Protocol Buffers (Cont.) package buzzvil.calendar.v1; service CalendarApi { rpc ListEvents(ListEventsRequest) returns (ListEventsResponse); } $ curl -X POST https://host.here/buzzvil.calendar.v1.CalendarApi/ListEvents = POST /<package>.<service>/<method>
15. Setup Protocol Buffers (Cont.) package buzzvil.calendar.v1; service CalendarApi { rpc ListEvents(ListEventsRequest) returns (ListEventsResponse) { option (google.api.http) = { get: "/v1/events" }; } } Custom routes: google.api.http annotation $ curl -X GET https://host.here/v1/events
16. Setup Protocol Buffers (Cont.) Parameters service CalendarApi { rpc UpdateEvent(UpdateEventRequest) returns (Event) { option (google.api.http) = { put: "/v1/events/{event.id}", body: "event" }; } } message Event { int64 id = 1; string name = 2; } message UpdateEventRequest { Event event = 1; }
17. Setup Envoy Proxy Let’s try Istio / EnvoyFilter to setup envoy sidecar proxy
18. Setup Envoy Proxy (Cont.) apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: example-transcoder namespace: example-namespace spec: workloadLabels: ... filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor: “path/to/bin” match_incoming_request_route: True auto_mapping: False services: - buzzvil.calendar.v1.CalendarApi filterName: envoy.grpc_json_transcoder match_incoming_request_route : True Services: - buzzvil.calendar.v1.CalendarApi Apply grpc_json_transcoder filter to the services Use the specified route
19. Setup Envoy Proxy (Cont.) apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: example-transcoder namespace: example-namespace spec: workloadLabels: ... filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor: “path/to/bin” match_incoming_request_route: True auto_mapping: False services: - buzzvil.calendar.v1.CalendarApi Proto Descriptor Envoy has to know the proto descriptor of your gRPC service in order to the transcoding. $ protoc -I$(GOOGLEAPIS_DIR) -I. \ --include_imports \ --include_source_info \ --descriptor_set_out =proto.pb \ test/proto/bookstore.proto proto_descriptor: “path/to/bin”
20. Setup Envoy Proxy (Cont.) Proto Descriptor Path proto_descriptor: “generated/file/path/proto.pb” proto_descriptor_bin: Cr15ChVnb29nbGUvYXBpL2h0dHAucHJvdG8SCm... Proto Descriptor Bin $ cat proto.pb | openssl base64 -A Encode proto descriptor using base64 encoding, and set the value in yaml file
21. Start the Service $ kubectl apply -f \ blog.yaml
22. Setup Gateway $ kubectl apply -f \ blog-gateway.yaml
23. Setup Transcoder $ kubectl apply -f \ blog-transcoder.yaml
24. Handle gRPC Request
25. Handle JSON Request
26. Deploying service using transcoding Protocol Buffer Artifact Pipeline Releasing new API version using Helm
27. API Changes Over Time message Post { int32 id = 1; string title = 2; string body = 3; repeated string tags = 4; google.protobuf.Timestamp created_at = 5; } Service spec changes over time • Additional field to message, new RPC • New package version • Beware of backward incompatible changes!(renaming, changing type, removing field, …) message Post { int32 id = 1; string title = 2; string body = 3; repeated string tags = 4; google.protobuf.Timestamp created_at = 5; string featured_image_url = 6; }
28. Deploying EnvoyFilter proto_descriptor should be updated whenever protocol buffer is changed. • proto_descriptor: expect *.pb descriptor to exist on file system → volumeMount to istio-proxy(sidecar) container • proto_descriptor_bin: embed base64-encoded *.pb descriptor content directly into EnvoyFilter apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: blog-transcoder namespace: blog spec: workloadLabels: ... filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor: “blog.pb” match_incoming_request_route: True auto_mapping: False services: - blog.BlogService
29. Deploying EnvoyFilter (Cont.) • proto_descriptor ◦ Ensuring *.pb descriptor file is mounted to istio-proxy container is not easy ◦ Automatic sidecar injection from istio-sidecar-injector is not dynamic enough(mounted volume contains all pb descriptors) ◦ Hard to control deployment timing • proto_descriptor_bin ◦ Lack of readability ◦ No volume dependency ◦ New transcoding configuration is applied as soon as EnvoyFilter is updated
30. Protocol Buffer Artifact Pipeline Language-specific gRPC Artifacts should be created whenever API is updated $ protoc -I (GOOGLE_APIS_DIR) --python_out=.. --go_out=.. --js_out=.. ./blog.proto blog.proto Private repository Private pypi server Private npm registry + rubygems, maven, ... Lint, Breaking Changes warning, ... comment. proto user.proto
31. ├── Jenkinsfile ├── Makefile └── packages ├── blog │ ├── blog.proto │ └── package.json └── user ├── user.proto └── package.json Mono Repository Approach • Create CI pipeline that generates & publishes language-specific artifacts • Common CI checks(lint, breaking changes warning) run for changed proto files • Reduced boilerplate! • Use lerna(node package) to bump versions of each API Generating protobuf descriptor artifact: protoc --include_imports --include_source_info --descriptor_set_out=blog.pb
32. Helm as Configuration Management Tool Helm • Already using helm for managing cluster-level services (ingress controller, EFK, prometheus, grafana, …) • Hosting private registry is easy(chartmuseum, object storage, ..) • Easy CI pipeline setup • CRDs can also be handled(VirtualService, EnvoyFilter, ...)
33. Helm Chart for gRPC Transcoding apiVersion: apps/v1 kind: Deployment metadata: ... spec: ... containers: - name: {{ .Chart.Name }} ... ports: - name: grpc containerPort: 9000 protocol: TCP apiVersion: v1 kind: Service metadata: ... spec: type: {{ .Values.service.type }} ports: - port: {{ .Values.service.port }} targetPort: grpc protocol: TCP name: grpc selector: {{- include "blog.matchLabels" . | nindent 4 }} Istio looks for port named grpc or any name prefixed with grpc-
34. Helm Chart for gRPC Transcoding New Helm release will update proto_descriptor_bin apiVersion: networking.istio.io/v1alpha3 kind: EnvoyFilter metadata: name: {{ include "blog.fullname" . }}-transcoder namespace: {{ .Release.Namespace }} spec: workloadLabels: {{- include "blog.matchLabels" . | nindent 4 }} filters: - listenerMatch: listenerType: SIDECAR_INBOUND filterName: envoy.grpc_json_transcoder filterType: HTTP filterConfig: proto_descriptor_bin: {{ .Values.protoDescriptorBin }} match_incoming_request_route: True auto_mapping: False services: - blog.v1.Blog print_options: always_print_primitive_fields: True preserve_proto_field_names: True
35. Releasing New API Version Using Helm CLI: $ helm upgrade RELEASE -f values.prod.yaml -f proto_descriptor.yaml --set image.tag=IMAGE_TAG repo/blog Helm Chart values.prod.yaml proto_descriptor.yaml image: tag: dkr.registry/blog:954ade7 secret: DATABASE_URL: mysql://xx:yy@db:3306 WEB_CONCURRENCY: 5 proto_descriptor_bin: Cr15ChVnb29n bGUvYXBpL2h0dHAucHJvdG... *We internally use Spinnaker to deploy helm chart based services
36. • 503 status code is returned when requested to non-mapped path • kubectl port-forward didn’t work for testing out REST APIs • Arbitrary(non-JSON) message can be also supported by using google.api.HttpBody • grpc-status, grpc-message headers in response are useful when debugging Pitfalls & Tips
37. Conclusion • It takes initial effort to build pipeline, but after then it becomes easy to develop any gRPC service that supports JSON transcoding • You can develop gRPC services while allowing other teams time to transition • API-First approach becomes standard throughout the organization • Over time, initial setup cost pays off
38. Resources Sample Repo - https://github.com/Buzzvil/grpc-json-transcoding-example/ • https://medium.com/@ssowonny/grpc를-쓰면-rest가-공짜-19e3a6bed4a9 • https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ grpc_json_transcoder_filter • https://blog.envoyproxy.io/envoy-and-grpc-web-a-fresh-new-alternative-to-res t-6504ce7eb880 • https://medium.com/google-cloud/grpc-transcoding-e101cc53d51d • https://blog.jdriven.com/2018/11/transcoding-grpc-to-http-json-using-envoy/ • https://medium.com/building-ibotta/building-a-scaleable-protocol-buffers-grp c-artifact-pipeline-5265c5118c9d
39. Thank You! Sungwon Lee (Whale) whale.lee@buzzvil.com GitHub & Twitter @ssowonny Hoseong Hwang (Liam) liam.hwang@buzzvil.com GitHub & Twitter @thefron