Rpc on Steroids with Go and grpc

Inter-process communication faces many challenges in nowadays distributed systems. Service discovery, retries, and such need to be addressed in order to cover all the needs of a distributed scenario. Grpc provides a valid alternative to rest calls, essentially being an easy to use cross language rpc framework. On top of that, Go has proved itself to be a suitable language for those kind of applications. What it is less know is the fact that it is really easy to extend the Grpc Go implementation with features such as authentication, service discovery, tracing, retry mechanisms and more.

During this talk I will explain how Gprc works, how to use it, and then will show how to build powerful interprocess communication by leveraging Grpc full potential. The audience will walk away with a solid understanding of how Grpc internally works and several useful techniques for taking full advantage of its powerful but lesser known features.

Transcript

1. Rpc on Steroids
   with Go and Grpc
   

   View Slide

2. About me
   ●
   Red Hatter
   ●
   Doing distributed systems for more than 10 years
   ●
   KubeVirt networking team
   ●
   Passionate about open source
   @fedepaol
   fedepaol@gmail.com
   fpaoline@redhat.com
   

   View Slide

3. Examples
   Examples of all the concepts described during this talk can be found at
   github.com/fedepaol/talks/tree/master/grpc/examples
   

   View Slide

4. Examples
   Examples of all the concepts described during this talk can be found at
   github.com/fedepaol/talks/tree/master/grpc/examples
   NO WARRANTY!
   

   View Slide

5. Let’s talk about RPC
   

   View Slide

6. Prior Art
   ●
   Corba
   ●
   WSDL
   ●
   Java RMI
   ●
   COM
   

   View Slide

7. The need for communication
   flickr.com/photos/joeshlabotnik/2230781841
   

   View Slide

8. Splitting the monolith
   

   View Slide

9. Splitting the monolith
   

   View Slide

10. What options do we have?
    ●
    Rest
    ●
    Other rpc frameworks (thrift, twirp, avro)
    ●
    Websockets
    ●
    Framed tcp
    

    View Slide

11. Enters Grpc
    

    View Slide

12. Grpc & Http2
    Stream A
    Stream B
    Stream C
    HTTP2 Connection
    

    View Slide

13. Grpc & Http2
    Stream A
    Stream B
    Stream C
    HTTP2 Connection
    

    View Slide

14. Grpc & Http2
    Http2 allows multiplexing of requests
    (streams) over a single tcp connection
    Grpc introduces channels, rpcs and
    messages
    

    View Slide

15. Serialization
    

    View Slide

16. Protocol Buffers
    

    View Slide

17. Protocol Buffers
    message Movie {
    int32 mid = 1;
    string movie_name = 2;
    string movie_description = 3;
    string movie_style = 4;
    }
    Type Definitions
    

    View Slide

18. Protocol Buffers
    message Movie {
    int32 mid = 1;
    string movie_name = 2;
    string movie_description = 3;
    string movie_style = 4;
    }
    service MoviesService {
    rpc GetMovie(MovieID) returns (Movie);
    rpc QueryMovie(MovieQueryParams) returns (stream Movie);
    }
    Type Definitions
    Interface Definitions
    

    View Slide

19. Features
    ●
    Code Generation
    ●
    Binary Serialization
    ●
    Type Safe
    ●
    New fields can be added without breaking the compatibility
    

    View Slide

20. protoc movie.proto --go_out=plugins=grpc:pkg/movie
    

    View Slide

21. Generated Code
    

    View Slide

22. Generated Code
    type Movie struct {
    Mid int32
    MovieName string
    MovieDescription string
    MovieStyle string
    }
    Data Structures
    

    View Slide

23. Generated Code
    func NewMoviesServiceClient(cc *grpc.ClientConn) MoviesServiceClient
    func (c *moviesServiceClient) GetMovie(ctx context.Context,
    in *MovieID,
    opts ...grpc.CallOption) (*Movie, error)
    Client Side
    type Movie struct {
    Mid int32
    MovieName string
    MovieDescription string
    MovieStyle string
    }
    Data Structures
    

    View Slide

24. Generated Code
    func NewMoviesServiceClient(cc *grpc.ClientConn) MoviesServiceClient
    func (c *moviesServiceClient) GetMovie(ctx context.Context,
    in *MovieID,
    opts ...grpc.CallOption) (*Movie, error)
    Client Side
    type MoviesServiceServer interface {
    GetMovie(context.Context, *MovieID) (*Movie, error)
    QueryMovie(*MovieQueryParams, MoviesService_QueryMovieServer) error
    }
    Server Side
    type Movie struct {
    Mid int32
    MovieName string
    MovieDescription string
    MovieStyle string
    }
    Data Structures
    

    View Slide

25. Establishing a connection
    

    View Slide

26. Establishing a connection
    conn, _ := grpc.Dial(serverAddr, grpc.WithInsecure())
    client := movie.NewMoviesServiceClient(conn)
    Client Side
    

    View Slide

27. Establishing a connection
    conn, _ := grpc.Dial(serverAddr, grpc.WithInsecure())
    client := movie.NewMoviesServiceClient(conn)
    Client Side
    lis, _ := net.Listen("tcp", port)
    server := grpc.NewServer()
    movie.RegisterMoviesServiceServer(server, &MovieServer{})
    server.Serve(lis)
    Server Side
    

    View Slide

28. Keep Alive
    Not enabled by the fault, need to use google.golang.org/grpc/keepalive
    ●
    Useful to detect disconnections where the endpoint hangs or dies
    ●
    Relies on http2 ping frames
    Nice side effects:
    ●
    Keeps a pool of healthy connections
    ●
    Avoids killing idle connections
    

    View Slide

29. Server Side Implementation
    func (b *MovieServer) GetMovie(ctx context.Context, id *movie.MovieID) (*movie.Movie, error) {
    movieID := id.GetMid()
    movie, ok := b.movies[int(movieID)]
    if !ok {
    // from google.golang.org/grpc/status
    return nil, status.Error(codes.NotFound, "movie not found")
    }
    return &movie, nil
    }
    

    View Slide

30. Server Side Implementation (Streaming)
    func (b *MovieServer) QueryMovie(p *movie.MovieQueryParams,
    s movie.MoviesService_QueryMovieServer) error {
    for _, b := range b.movies {
    if strings.Contains(b.MovieDescription, p.GetQuery()) {
    s.Send(&b)
    }
    }
    return nil
    }
    

    View Slide

31. Service Discovery & Load Balancing
    flickr.com/photos/pmillera4/31944058051
    

    View Slide

32. Name Resolution
    You have a name (URI) and you want one (or more) addresses back to connect to.
    That’s done by resolver.Resolver
    Available out of the box:
    ●
    Passthrough
    ●
    Dns
    ●
    Manual
    

    View Slide

33. Build your own Resolver
    Implement & register a resolver.Builder
    ●
    Build() method to return the resolver for the given target
    ●
    Scheme() to declare what scheme we are registering against
    Inside the resolver we can then pass the list of endpoints to the clientconnection:
    cc.UpdateState(resolver.State{Addresses: addrs})
    A nice resolver example is the etcd name resolver
    

    View Slide

34. Load Balancing
    In grpc we have persistent connection. The balancing is done per call, while
    maintaining a pool of connections to choose from.
    Built in:
    ●
    Pick First
    ●
    Round Robin
    ●
    Grpclb (external)
    

    View Slide

35. External Load Balancing
    

    View Slide

36. External Load Balancing
    The external load balancer is a GRPC endpoint itself:
    service LoadBalancer {
    // Bidirectional rpc to get a list of servers.
    rpc BalanceLoad(stream LoadBalanceRequest) returns (stream LoadBalanceResponse);
    }
    

    View Slide

37. Error Handling
    

    View Slide

38. Error Handling (status code)
    func (b *MovieServer) GetMovie(ctx context.Context, id *movie.MovieID)
    (*movie.Movie, error) {
    /* ... */
    return nil, status.Error(codes.NotFound, "movie not found")
    }
    Server Side
    movie, err := client.GetMovie(ctx, id)
    if err != nil {
    st := status.Convert(err)
    if st != nil && st.Code() == codes.NotFound {
    // handle missing movie
    }
    }
    Client Side
    

    View Slide

39. Error Handling (strongly typed errors)
    movieErr := movie.MovieError{
    Reason: movie.MovieErrorCode_MOVIE_NOT_FOUND,
    Message: fmt.Sprintf("Could not find movie %v", *id),
    }
    st, _ := status.New(codes.NotFound, "not found").WithDetails(&movieErr)
    return nil, st.Err()
    Server Side
    res, err := client.GetMovie(ctx, id)
    if err != nil {
    st := status.Convert(err)
    for _, detail := range st.Details() {
    switch t := detail.(type) {
    case *movie.MovieError:
    // handle t payload
    }
    }
    }
    Client Side
    

    View Slide

40. Grpc & Context
    flickr.com/photos/magnopere/120924833
    The context package makes it easy to pass request-scoped
    values, cancellation signals and deadline across API
    boundaries to all the goroutines involved in handling a
    request.
    

    View Slide

41. Deadline Propagation
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    movie, err := client.GetMovie(ctx, id)
    2 sec
    deadline
    1 sec
    

    View Slide

42. Deadline Propagation
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    movie, err := client.GetMovie(ctx, id)
    2 sec
    deadline
    1 sec 2 sec
    

    View Slide

43. Deadline Propagation
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    movie, err := client.GetMovie(ctx, id)
    2 sec
    deadline
    1 sec 2 sec
    

    View Slide

44. Cancellation Propagation
    ctx, cancel := context.WithCancel(context.Background())
    movie, err := client.GetMovie(ctx, id)
    

    View Slide

45. Cancellation Propagation
    ctx, cancel := context.WithCancel(context.Background())
    movie, err := client.GetMovie(ctx, id)
    cancel()
    

    View Slide

46. Cancellation Propagation
    ctx, cancel := context.WithCancel(context.Background())
    movie, err := client.GetMovie(ctx, id)
    cancel()
    

    View Slide

47. Values Propagation
    Context values are not propagated, but godoc.org/google.golang.org/grpc/metadata is.
    

    View Slide

48. Values Propagation
    Context values are not propagated, but godoc.org/google.golang.org/grpc/metadata is.
    md := metadata.Pairs("key", "value")
    ctx := metadata.NewOutgoingContext(context.Background(), md)
    Client Side
    

    View Slide

49. Values Propagation
    Context values are not propagated, but godoc.org/google.golang.org/grpc/metadata is.
    md := metadata.Pairs("key", "value")
    ctx := metadata.NewOutgoingContext(context.Background(), md)
    Client Side
    Server Side
    meta, ok := metadata.FromIncomingContext(ctx)
    value : = meta.Get("key")[0]
    

    View Slide

50. Interceptors
    flickr.com/photos/trekker308
    

    View Slide

51. Interceptors
    Same as http middleware.
    Two types:
    ●
    Unary interceptors
    ●
    Stream interceptors
    Both client side / server side
    

    View Slide

52. Retry Unary Interceptor
    func RetryInterceptor(ctx context.Context,
    method string,
    req, reply interface{},
    cc *grpc.ClientConn,
    invoker grpc.UnaryInvoker,
    opts ...grpc.CallOption) error {
    var lastErr error
    for attempt := uint(0); attempt < 10; attempt++ {
    lastErr = invoker(ctx, method, req, reply, cc, opts...)
    if lastErr == nil {
    return nil
    }
    }
    return lastErr
    }
    

    View Slide

53. Rate Limiter Interceptor
    func UnaryServerInterceptor(ctx context.Context,
    req interface{},
    info *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler) (interface{}, error) {
    if Limit() {
    return nil, status.Errorf(codes.ResourceExhausted,
    "%s is rejected by retry limiter.", info.FullMethod)
    }
    return handler(ctx, req)
    }
    

    View Slide

54. Authentication (client side)
    func jwtInterceptor (ctx context.Context, method string, req interface{},
    reply interface{}, cc *grpc.ClientConn,invoker grpc.UnaryInvoker,
    opts ...grpc.CallOption) error {
    authCtx := metadata.AppendToOutgoingContext(ctx, "jwt", "bearer "+jwt.token)
    err := invoker(authCtx, method, req, reply, cc, opts...)
    if err != nil && status.Code(err) == codes.Unauthenticated {
    // handle unauthenticated
    }
    if err != nil {
    return err
    }
    }
    }
    

    View Slide

55. Authentication (server side)
    func authInterceptor(ctx context.Context,
    req interface{},
    _ *grpc.UnaryServerInfo,
    handler grpc.UnaryHandler) (interface{}, error) {
    token := fetchTokenFromContext(ctx)
    // handle token validation
    return handler(ctx, req)
    }
    

    View Slide

56. How to set interceptors
    conn, err := grpc.Dial(*addr, grpc.WithUnaryInterceptor(unaryInterceptor))
    Client Side
    s := grpc.NewServer(grpc.UnaryInterceptor(unaryInterceptor))
    Server Side
    

    View Slide

57. Interceptor Pitfalls
    ●
    Once set, an interceptor is called for all the methods
    ●
    We need to find a way to disable (some) interceptors on method basis
    

    View Slide

58. Other Interceptor Goodies
    ●
    Tracing (opentracing, opencensus)
    ●
    Logging
    ●
    Monitoring
    ●
    More at github.com/grpc-ecosystem/go-grpc-middleware
    

    View Slide

59. Opentracing + Jaeger
    

    View Slide

60. Honorable Mentions
    

    View Slide

61. Grpc Gateway
    service YourService {
    rpc Echo(StringMessage) returns (StringMessage) {
    option (google.api.http) = {
    post: "/v1/example/echo"
    body: "*"
    };
    }
    

    View Slide

62. Reflection
    Opt-in feature:
    As seen in:
    ●
    grpcurl
    ●
    grpcui
    s := grpc.NewServer()
    pb.RegisterGreeterServer(s, &server{})
    reflection.Register(s)
    

    View Slide

63. Beyond the “simple” rpc
    

    View Slide

64. Thanks!
    Any questions?
    @fedepaol
    fedepaol@gmail.com
    Slides at: speakerdeck.com/fedepaol fpaoline@redhat.com
    

    View Slide