Rpc on Steroids with Go and grpc

Inter-process communication faces many challenges in nowadays distributed systems. Service discovery, retries, and such need to be addressed in order to cover all the needs of a distributed scenario. Grpc provides a valid alternative to rest calls, essentially being an easy to use cross language rpc framework. On top of that, Go has proved itself to be a suitable language for those kind of applications. What it is less know is the fact that it is really easy to extend the Grpc Go implementation with features such as authentication, service discovery, tracing, retry mechanisms and more.

During this talk I will explain how Gprc works, how to use it, and then will show how to build powerful interprocess communication by leveraging Grpc full potential. The audience will walk away with a solid understanding of how Grpc internally works and several useful techniques for taking full advantage of its powerful but lesser known features.

Transcript

1. Rpc on Steroids with Go and Grpc
2. About me • Red Hatter • Doing distributed systems for more than 10 years • KubeVirt networking team • Passionate about open source @fedepaol fedepaol@gmail.com fpaoline@redhat.com
3. Examples Examples of all the concepts described during this talk can be found at github.com/fedepaol/talks/tree/master/grpc/examples
4. Examples Examples of all the concepts described during this talk can be found at github.com/fedepaol/talks/tree/master/grpc/examples NO WARRANTY!
5. Let’s talk about RPC
6. Prior Art • Corba • WSDL • Java RMI • COM
7. The need for communication flickr.com/photos/joeshlabotnik/2230781841
8. Splitting the monolith
9. Splitting the monolith
10. What options do we have? • Rest • Other rpc frameworks (thrift, twirp, avro) • Websockets • Framed tcp
11. Enters Grpc
12. Grpc & Http2 Stream A Stream B Stream C HTTP2 Connection
13. Grpc & Http2 Stream A Stream B Stream C HTTP2 Connection
14. Grpc & Http2 Http2 allows multiplexing of requests (streams) over a single tcp connection Grpc introduces channels, rpcs and messages
15. Serialization
16. Protocol Buffers
17. Protocol Buffers message Movie { int32 mid = 1; string movie_name = 2; string movie_description = 3; string movie_style = 4; } Type Definitions
18. Protocol Buffers message Movie { int32 mid = 1; string movie_name = 2; string movie_description = 3; string movie_style = 4; } service MoviesService { rpc GetMovie(MovieID) returns (Movie); rpc QueryMovie(MovieQueryParams) returns (stream Movie); } Type Definitions Interface Definitions
19. Features • Code Generation • Binary Serialization • Type Safe • New fields can be added without breaking the compatibility
20. protoc movie.proto --go_out=plugins=grpc:pkg/movie
21. Generated Code
22. Generated Code type Movie struct { Mid int32 MovieName string MovieDescription string MovieStyle string } Data Structures
23. Generated Code func NewMoviesServiceClient(cc *grpc.ClientConn) MoviesServiceClient func (c *moviesServiceClient) GetMovie(ctx context.Context, in *MovieID, opts ...grpc.CallOption) (*Movie, error) Client Side type Movie struct { Mid int32 MovieName string MovieDescription string MovieStyle string } Data Structures
24. Generated Code func NewMoviesServiceClient(cc *grpc.ClientConn) MoviesServiceClient func (c *moviesServiceClient) GetMovie(ctx context.Context, in *MovieID, opts ...grpc.CallOption) (*Movie, error) Client Side type MoviesServiceServer interface { GetMovie(context.Context, *MovieID) (*Movie, error) QueryMovie(*MovieQueryParams, MoviesService_QueryMovieServer) error } Server Side type Movie struct { Mid int32 MovieName string MovieDescription string MovieStyle string } Data Structures
25. Establishing a connection
26. Establishing a connection conn, _ := grpc.Dial(serverAddr, grpc.WithInsecure()) client := movie.NewMoviesServiceClient(conn) Client Side
27. Establishing a connection conn, _ := grpc.Dial(serverAddr, grpc.WithInsecure()) client := movie.NewMoviesServiceClient(conn) Client Side lis, _ := net.Listen("tcp", port) server := grpc.NewServer() movie.RegisterMoviesServiceServer(server, &MovieServer{}) server.Serve(lis) Server Side
28. Keep Alive Not enabled by the fault, need to use google.golang.org/grpc/keepalive • Useful to detect disconnections where the endpoint hangs or dies • Relies on http2 ping frames Nice side effects: • Keeps a pool of healthy connections • Avoids killing idle connections
29. Server Side Implementation func (b *MovieServer) GetMovie(ctx context.Context, id *movie.MovieID) (*movie.Movie, error) { movieID := id.GetMid() movie, ok := b.movies[int(movieID)] if !ok { // from google.golang.org/grpc/status return nil, status.Error(codes.NotFound, "movie not found") } return &movie, nil }
30. Server Side Implementation (Streaming) func (b *MovieServer) QueryMovie(p *movie.MovieQueryParams, s movie.MoviesService_QueryMovieServer) error { for _, b := range b.movies { if strings.Contains(b.MovieDescription, p.GetQuery()) { s.Send(&b) } } return nil }
31. Service Discovery & Load Balancing flickr.com/photos/pmillera4/31944058051
32. Name Resolution You have a name (URI) and you want one (or more) addresses back to connect to. That’s done by resolver.Resolver Available out of the box: • Passthrough • Dns • Manual
33. Build your own Resolver Implement & register a resolver.Builder • Build() method to return the resolver for the given target • Scheme() to declare what scheme we are registering against Inside the resolver we can then pass the list of endpoints to the clientconnection: cc.UpdateState(resolver.State{Addresses: addrs}) A nice resolver example is the etcd name resolver
34. Load Balancing In grpc we have persistent connection. The balancing is done per call, while maintaining a pool of connections to choose from. Built in: • Pick First • Round Robin • Grpclb (external)
35. External Load Balancing
36. External Load Balancing The external load balancer is a GRPC endpoint itself: service LoadBalancer { // Bidirectional rpc to get a list of servers. rpc BalanceLoad(stream LoadBalanceRequest) returns (stream LoadBalanceResponse); }
37. Error Handling
38. Error Handling (status code) func (b *MovieServer) GetMovie(ctx context.Context, id *movie.MovieID) (*movie.Movie, error) { /* ... */ return nil, status.Error(codes.NotFound, "movie not found") } Server Side movie, err := client.GetMovie(ctx, id) if err != nil { st := status.Convert(err) if st != nil && st.Code() == codes.NotFound { // handle missing movie } } Client Side
39. Error Handling (strongly typed errors) movieErr := movie.MovieError{ Reason: movie.MovieErrorCode_MOVIE_NOT_FOUND, Message: fmt.Sprintf("Could not find movie %v", *id), } st, _ := status.New(codes.NotFound, "not found").WithDetails(&movieErr) return nil, st.Err() Server Side res, err := client.GetMovie(ctx, id) if err != nil { st := status.Convert(err) for _, detail := range st.Details() { switch t := detail.(type) { case *movie.MovieError: // handle t payload } } } Client Side
40. Grpc & Context flickr.com/photos/magnopere/120924833 The context package makes it easy to pass request-scoped values, cancellation signals and deadline across API boundaries to all the goroutines involved in handling a request.
41. Deadline Propagation ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) movie, err := client.GetMovie(ctx, id) 2 sec deadline 1 sec
42. Deadline Propagation ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) movie, err := client.GetMovie(ctx, id) 2 sec deadline 1 sec 2 sec
43. Deadline Propagation ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second) movie, err := client.GetMovie(ctx, id) 2 sec deadline 1 sec 2 sec
44. Cancellation Propagation ctx, cancel := context.WithCancel(context.Background()) movie, err := client.GetMovie(ctx, id)
45. Cancellation Propagation ctx, cancel := context.WithCancel(context.Background()) movie, err := client.GetMovie(ctx, id) cancel()
46. Cancellation Propagation ctx, cancel := context.WithCancel(context.Background()) movie, err := client.GetMovie(ctx, id) cancel()
47. Values Propagation Context values are not propagated, but godoc.org/google.golang.org/grpc/metadata is.
48. Values Propagation Context values are not propagated, but godoc.org/google.golang.org/grpc/metadata is. md := metadata.Pairs("key", "value") ctx := metadata.NewOutgoingContext(context.Background(), md) Client Side
49. Values Propagation Context values are not propagated, but godoc.org/google.golang.org/grpc/metadata is. md := metadata.Pairs("key", "value") ctx := metadata.NewOutgoingContext(context.Background(), md) Client Side Server Side meta, ok := metadata.FromIncomingContext(ctx) value : = meta.Get("key")[0]
50. Interceptors flickr.com/photos/trekker308
51. Interceptors Same as http middleware. Two types: • Unary interceptors • Stream interceptors Both client side / server side
52. Retry Unary Interceptor func RetryInterceptor(ctx context.Context, method string, req, reply interface{}, cc *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error { var lastErr error for attempt := uint(0); attempt < 10; attempt++ { lastErr = invoker(ctx, method, req, reply, cc, opts...) if lastErr == nil { return nil } } return lastErr }
53. Rate Limiter Interceptor func UnaryServerInterceptor(ctx context.Context, req interface{}, info *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) { if Limit() { return nil, status.Errorf(codes.ResourceExhausted, "%s is rejected by retry limiter.", info.FullMethod) } return handler(ctx, req) }
54. Authentication (client side) func jwtInterceptor (ctx context.Context, method string, req interface{}, reply interface{}, cc *grpc.ClientConn,invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error { authCtx := metadata.AppendToOutgoingContext(ctx, "jwt", "bearer "+jwt.token) err := invoker(authCtx, method, req, reply, cc, opts...) if err != nil && status.Code(err) == codes.Unauthenticated { // handle unauthenticated } if err != nil { return err } } }
55. Authentication (server side) func authInterceptor(ctx context.Context, req interface{}, _ *grpc.UnaryServerInfo, handler grpc.UnaryHandler) (interface{}, error) { token := fetchTokenFromContext(ctx) // handle token validation return handler(ctx, req) }
56. How to set interceptors conn, err := grpc.Dial(*addr, grpc.WithUnaryInterceptor(unaryInterceptor)) Client Side s := grpc.NewServer(grpc.UnaryInterceptor(unaryInterceptor)) Server Side
57. Interceptor Pitfalls • Once set, an interceptor is called for all the methods • We need to find a way to disable (some) interceptors on method basis
58. Other Interceptor Goodies • Tracing (opentracing, opencensus) • Logging • Monitoring • More at github.com/grpc-ecosystem/go-grpc-middleware
59. Opentracing + Jaeger
60. Honorable Mentions
61. Grpc Gateway service YourService { rpc Echo(StringMessage) returns (StringMessage) { option (google.api.http) = { post: "/v1/example/echo" body: "*" }; }
62. Reflection Opt-in feature: As seen in: • grpcurl • grpcui s := grpc.NewServer() pb.RegisterGreeterServer(s, &server{}) reflection.Register(s)
63. Beyond the “simple” rpc
64. Thanks! Any questions? @fedepaol fedepaol@gmail.com Slides at: speakerdeck.com/fedepaol fpaoline@redhat.com