Transparent gRPC Gateway in Go

Building a transaprent gRPC proxy in Go with advanced use-cases of modifying protobuf messages without exact knowledge of their structure.

See Go library https://github.com/talos-systems/grpc-proxy

Transcript

1. Transparent gRPC Gateway in Go GopherCon Russia’21 Andrey Smirnov, Talos Systems
2. About Andrey Smirnov @smira github.com/smira Go developer since 2014 Working on Talos: revolutionary OS for clusters
3. Agenda Why gRPC? Why API gateway? Why Go? First iteration in Go, problems and solutions. Transparent proxying in Go using gRPC. Cutting and concatenating protobuf messages.
4. Why gRPC? API, but as easy as calling a function
5. API Gateway/Proxy gRPC gRPC backend gRPC backend API Gateway
6. Use Cases for gRPC API Gateway Migrating from monolith to smaller services (or vice versa) Migrating to new API version Common authentication or authorization layer Logging, traceability, … Non-trivial proxy logic: send one request to many backends, combine responses
7. gRPC API Gateway Implementation TCP loadbalancer HTTP reverse proxy (e.g. nginx) ... Implement our own in Go (!)
8. Ping-pong gRPC service message Ping { string value = 1; } message Pong { string value = 1; } service TestService { rpc PingPong(Ping) returns (Pong) {} }
9. Easy! (?) func (s *Proxy) connect() { s.conn = grpc.Dial(...) s.client = pb.NewTestServiceClient(conn) } func (s *Proxy) PingPong(ctx context.Context, ping *pb.Ping) (*pb.Pong, error) { return s.client.PingPong(ctx, ping) }
10. gRPC metadata gRPC metadata: headers, trailers Go gRPC Client Metadata: Metadata Go gRPC Server Metadata: Headers Trailers
11. Metadata handling md, _ := metadata.FromIncomingContext(ctx) outCtx := metadata.NewOutgoingContext(ctx, md) var header, trailer metadata.MD resp, err := s.client.Ping(outCtx, ping, grpc.Header(&header), grpc.Trailer(&trailer)) grpc.SendHeader(ctx, header) grpc.SetTrailer(ctx, trailer) return resp, err
12. Streaming Calls Unary calls Client streaming calls Server streaming calls Bi-directional streaming
13. Streaming Service message Counter { int32 counter = 1; } service TestService { rpc Counter(Empty) returns (stream Counter) {} }
14. Streaming Proxy (½) ctx, cancel := context.WithCancel(srv.Context()) defer cancel() cli, err := s.client.Counter(ctx, in) if err != nil { return err } ...
15. Streaming Proxy (½) for { msg, err := cli.Recv() switch { case err == io.EOF: return nil case err != nil: return err } err = srv.Send(msg) if err != nil { return err } }
17. Ways out Code generation Libraries, common code, ...
18. Protobuf Update (v2) message Ping { string value = 1; int counter = 2; } message Pong { string value = 1; int counter = 2; } service TestService { rpc PingPong(Ping) returns (Pong) {} }
19. Version Mismatch gRPC backend API Gateway v1 v2 Ping value: “foo” counter: 42 Ping value: “foo” counter: 42 Pong value: “bar” counter: 24 Pong value: “bar” counter: 24
20. Solution grpc.CustomCodec(grpc.Codec) grpc.UnknownServiceHandler(grpc.StreamHandler) grpc.NewClientStream(context.Context, *StreamDesc, *ClientConn, method string)
21. grpc.Codec type Codec interface { // Marshal returns the wire format of v. Marshal(v interface{}) ([]byte, error) // Unmarshal parses the wire format into v. Unmarshal(data []byte, v interface{}) error }
22. Raw Codec type frame struct { payload []byte } func (c *rawCodec) Marshal(v interface{}) ([]byte, error) { out, ok := v.(*frame) if !ok { return fmt.Errorf("expected frame") } return out.payload, nil } func (c *rawCodec) Unmarshal(data []byte, v interface{}) error { dst, ok := v.(*frame) if !ok { return fmt.Errorf("expected frame")} dst.payload = data return nil }
23. grpc.UnknownServiceHandler grpc.NewServer( grpc.CustomCodec(proxy.Codec()), grpc.UnknownServiceHandler(handler)) func handler(srv interface{}, serverStream grpc.ServerStream) error { fullMethodName, ok := grpc.MethodFromServerStream(serverStream) ... }
24. grpc.NewClientStream func handler(srv interface{}, serverStream grpc.ServerStream) error { conn, err = grpc.Dial(addr, grpc.WithCodec(proxy.Codec())) clientStream, err = grpc.NewClientStream( ctx, &grpc.StreamDesc{ ServerStreams: true, ClientStreams: true, }, conn, fullMethodName) // copy clientStream <> serverStream }
25. Transparent gRPC Proxy Flow API Gateway grpcServerStream grpcClientStream Recv() Recv() Send() Send()
26. Proxying one → many Aggregating responses Encoding errors Attributing result to a backend
27. Response Metadata message ResponseMetadata { string upstream_node = 1; string upstream_error = 2; } message Pong { ResponseMetadata metadata = 99; string value = 1; }
28. Enriching Response gRPC backend node-1 API Gateway Pong value: “bar” Pong value: “bar” metadata: upstream_node: node_1
29. Protobuf Glue Pong value: “bar” (1) Pong value: “bar” (1) metadata: (99) upstream_node: node_1 (1) Pong metadata: (99) upstream_node: node_1 (1) (type: bytes, field: 1, length: 3): “bar” (type: bytes, field: 99, length: N): [(type: bytes, field: 1, length: 6): “node_1”] (type: bytes, field: 1, length: 3): “bar” (type: bytes, field: 99, length: N): [(type: bytes, field: 1, length: 6): “node_1”] protobuf serialization: messages:
30. Embedding Errors gRPC backend API Gateway Ping value: “foo” Pong metadata: upstream_node: node_1 upstream_error: ECONNREFUSED connection refused
31. Server Streaming API Gateway Ping value: “foo” Pong metadata: upstream_node: node_1 upstream_error: ECONNREFUSED Pong value: “bar” metadata: upstream_node: node_2
32. Unary Calls API Gateway Ping value: “foo” Pong metadata: upstream_node: node_1 upstream_error: ECONNREFUSED Pong value: “bar” metadata: upstream_node: node_2
33. Unary Protobuf Definition message ResponseMetadata { string upstream_node = 99; string upstream_error = 100; } message Pong { ResponseMetadata metadata = 99; string value = 1; } message PongResponse { repeated Pong messages = 1; }
34. Protobuf Scissors PongResponse messages: (1) - Pong: value: “bar” (1) (type: bytes, field: 1, length: N): [(type: bytes, field: 1, length: 3): bar] PongResponse messages: (1) - Pong: value: “bar” (1) metadata: (99) u_node: node_1 (1) (type: bytes, field: 1, length: N’): [(type: bytes, field: 1, length: 3): bar (type: bytes: field: 99, length: K): [(type: bytes: field: 1, length: 6): node_1 ]
35. grpc-proxy library https://github.com/talos-systems/grpc-proxy https://pkg.go.dev/github.com/talos-systems/grpc-proxy Thank you! @smira (https://github.com/smira) Talos Systems
36. Q&A