Transparent gRPC Gateway in Go GopherCon Russia’21 Andrey Smirnov, Talos Systems

About Andrey Smirnov @smira github.com/smira Go developer since 2014 Working on Talos: revolutionary OS for clusters

Agenda Why gRPC? Why API gateway? Why Go? First iteration in Go, problems and solutions. Transparent proxying in Go using gRPC. Cutting and concatenating protobuf messages.

Why gRPC? API, but as easy as calling a function

API Gateway/Proxy gRPC gRPC backend gRPC backend API Gateway

Use Cases for gRPC API Gateway Migrating from monolith to smaller services (or vice versa) Migrating to new API version Common authentication or authorization layer Logging, traceability, … Non-trivial proxy logic: send one request to many backends, combine responses

gRPC API Gateway Implementation TCP loadbalancer HTTP reverse proxy (e.g. nginx) ... Implement our own in Go (!)

Ping-pong gRPC service message Ping { string value = 1; } message Pong { string value = 1; } service TestService { rpc PingPong(Ping) returns (Pong) {} }

Easy! (?) func (s *Proxy) connect() { s.conn = grpc.Dial(...) s.client = pb.NewTestServiceClient(conn) } func (s *Proxy) PingPong(ctx context.Context, ping *pb.Ping) (*pb.Pong, error) { return s.client.PingPong(ctx, ping) }

gRPC metadata gRPC metadata: headers, trailers Go gRPC Client Metadata: Metadata Go gRPC Server Metadata: Headers Trailers

Metadata handling md, _ := metadata.FromIncomingContext(ctx) outCtx := metadata.NewOutgoingContext(ctx, md) var header, trailer metadata.MD resp, err := s.client.Ping(outCtx, ping, grpc.Header(&header), grpc.Trailer(&trailer)) grpc.SendHeader(ctx, header) grpc.SetTrailer(ctx, trailer) return resp, err

Streaming Calls Unary calls Client streaming calls Server streaming calls Bi-directional streaming

Streaming Service message Counter { int32 counter = 1; } service TestService { rpc Counter(Empty) returns (stream Counter) {} }

Streaming Proxy (½) ctx, cancel := context.WithCancel(srv.Context()) defer cancel() cli, err := s.client.Counter(ctx, in) if err != nil { return err } ...

Streaming Proxy (½) for { msg, err := cli.Recv() switch { case err == io.EOF: return nil case err != nil: return err } err = srv.Send(msg) if err != nil { return err } }

No content

Ways out Code generation Libraries, common code, ...

Protobuf Update (v2) message Ping { string value = 1; int counter = 2; } message Pong { string value = 1; int counter = 2; } service TestService { rpc PingPong(Ping) returns (Pong) {} }

Version Mismatch gRPC backend API Gateway v1 v2 Ping value: “foo” counter: 42 Ping value: “foo” counter: 42 Pong value: “bar” counter: 24 Pong value: “bar” counter: 24

Solution grpc.CustomCodec(grpc.Codec) grpc.UnknownServiceHandler(grpc.StreamHandler) grpc.NewClientStream(context.Context, *StreamDesc, *ClientConn, method string)

grpc.Codec type Codec interface { // Marshal returns the wire format of v. Marshal(v interface{}) ([]byte, error) // Unmarshal parses the wire format into v. Unmarshal(data []byte, v interface{}) error }

Raw Codec type frame struct { payload []byte } func (c *rawCodec) Marshal(v interface{}) ([]byte, error) { out, ok := v.(*frame) if !ok { return fmt.Errorf("expected frame") } return out.payload, nil } func (c *rawCodec) Unmarshal(data []byte, v interface{}) error { dst, ok := v.(*frame) if !ok { return fmt.Errorf("expected frame")} dst.payload = data return nil }

grpc.UnknownServiceHandler grpc.NewServer( grpc.CustomCodec(proxy.Codec()), grpc.UnknownServiceHandler(handler)) func handler(srv interface{}, serverStream grpc.ServerStream) error { fullMethodName, ok := grpc.MethodFromServerStream(serverStream) ... }

grpc.NewClientStream func handler(srv interface{}, serverStream grpc.ServerStream) error { conn, err = grpc.Dial(addr, grpc.WithCodec(proxy.Codec())) clientStream, err = grpc.NewClientStream( ctx, &grpc.StreamDesc{ ServerStreams: true, ClientStreams: true, }, conn, fullMethodName) // copy clientStream <> serverStream }

Transparent gRPC Proxy Flow API Gateway grpcServerStream grpcClientStream Recv() Recv() Send() Send()

Proxying one → many Aggregating responses Encoding errors Attributing result to a backend

Response Metadata message ResponseMetadata { string upstream_node = 1; string upstream_error = 2; } message Pong { ResponseMetadata metadata = 99; string value = 1; }

Enriching Response gRPC backend node-1 API Gateway Pong value: “bar” Pong value: “bar” metadata: upstream_node: node_1

Protobuf Glue Pong value: “bar” (1) Pong value: “bar” (1) metadata: (99) upstream_node: node_1 (1) Pong metadata: (99) upstream_node: node_1 (1) (type: bytes, field: 1, length: 3): “bar” (type: bytes, field: 99, length: N): [(type: bytes, field: 1, length: 6): “node_1”] (type: bytes, field: 1, length: 3): “bar” (type: bytes, field: 99, length: N): [(type: bytes, field: 1, length: 6): “node_1”] protobuf serialization: messages:

Embedding Errors gRPC backend API Gateway Ping value: “foo” Pong metadata: upstream_node: node_1 upstream_error: ECONNREFUSED connection refused

Server Streaming API Gateway Ping value: “foo” Pong metadata: upstream_node: node_1 upstream_error: ECONNREFUSED Pong value: “bar” metadata: upstream_node: node_2

Unary Calls API Gateway Ping value: “foo” Pong metadata: upstream_node: node_1 upstream_error: ECONNREFUSED Pong value: “bar” metadata: upstream_node: node_2

Unary Protobuf Definition message ResponseMetadata { string upstream_node = 99; string upstream_error = 100; } message Pong { ResponseMetadata metadata = 99; string value = 1; } message PongResponse { repeated Pong messages = 1; }

Protobuf Scissors PongResponse messages: (1) - Pong: value: “bar” (1) (type: bytes, field: 1, length: N): [(type: bytes, field: 1, length: 3): bar] PongResponse messages: (1) - Pong: value: “bar” (1) metadata: (99) u_node: node_1 (1) (type: bytes, field: 1, length: N’): [(type: bytes, field: 1, length: 3): bar (type: bytes: field: 99, length: K): [(type: bytes: field: 1, length: 6): node_1 ]

grpc-proxy library https://github.com/talos-systems/grpc-proxy https://pkg.go.dev/github.com/talos-systems/grpc-proxy Thank you! @smira (https://github.com/smira) Talos Systems

Q&A