ITT 2018 - Christopher Burnett - Types All the Way Down — gRPC and Go Infrastructure at Lyft

In this talk we'll discuss our experiences and lessons learned while introducing gRPC and Go into Lyft's ecosystem. We'll also cover how our Envoy proxy helps to bridge the type-safety gaps across polyglot systems and provides us with flexible and observable infrastructure solutions.

Transcript

1. TYPES ALL THE WAY DOWN
   GENERATING APIS WITH PROTOBUFS AND GRPC
   

   View Slide

2. TEŞEKKÜR EDERIM
   

   View Slide

3. HI!
   Christopher Burnett
   Lead Client/Server Networking Engineer NYC
   Formerly Twitter, VSCO, Posterous
   

   View Slide

4. ABOUT US
   • Ride sharing - US and Canada
   • Interesting Scale and Challenges
   • Polyglot with ~500 engineers
   

   View Slide

5. AGENDA
   • Why RPC?
   • Working with legacy systems
   • Disrupting workflows, nicely :)
   • What we’ve built
   

   View Slide

6. NETWORKING
   A little about us…
   • HTTP & gRPC Libraries
   • Async Streaming Infrastructure
   • Envoy Proxy
   

   View Slide

7. EVERY TEN YEARS…
   A furious bout of language and protocol design takes place and a new
   distributed computing paradigm is announced that is compliant with the
   latest programming model.
   - A Note On Distributed Computing, Waldo 1994
   

   View Slide

8. TRIGGER WARNING
   • CORBA
   • Thrift
   • SOAP
   • WDDX
   • JSON-RPC
   • XML-RPC
   • Avro
   • HyperMedia
   • REST
   • MessagePack
   

   View Slide

9. View Slide

10. View Slide

11. A LITTLE HISTORY
    Like any good story we begin with a PHP
    monolith…
    • Active decomposition efforts
    • 100s of Python Microservices
    - Flask HTTP/REST
    And to keep things interesting…
    • gRPC Core and Compositional Services
    

    View Slide

12. DEFINING A CORE SERVICE
    • Organizational Primitives
    - User, Rides, Vehicles
    • Zero (Service) Dependencies
    - Databases, caches, etc
    • Highly Performant
    

    View Slide

13. SO, WHY (G)RPC?
    

    View Slide

14. GRPC - THE BASICS
    • IDL (Interface Definition Language) Based
    • Protocol Buffer Wire Format
    • (Mostly) Full HTTP/2 Transport
    • Migration/Deprecation Options
    

    View Slide

15. BUT FIRST…
    

    View Slide

16. LET’S TALK ABOUT REST
    

    View Slide

17. RESTful
    

    View Slide

18. REST/JSON: S2S COMMUNICATION
    POST /api/updateUser HTTP/1.0
    Content-Type: application/json
    {
    "id": 18446744073709551615,
    "username": “chris"
    }
    

    View Slide

19. ALRIGHT, LET’S PAINT THAT SHED…
    PUT /api/users HTTP/1.0
    Content-Type: application/json
    {
    "id": 18446744073709551615,
    "username": “chris"
    }
    

    View Slide

20. PUTTING ON ANOTHER COAT…
    PUT /api/users/18446744073709551615 HTTP/1.0
    Content-Type: application/json
    {
    "username": “chris"
    }
    

    View Slide

21. FINISHING TOUCHES…
    PUT /api/v1/users/18446744073709551615 HTTP/1.0
    Content-Type: application/json
    {
    "username": “chris"
    }
    

    View Slide

22. RESTish
    

    View Slide

23. IDLs are pretty great :)
    

    View Slide

24. IDLS ARE PRETTY GREAT
    • Single Source of Truth
    -Primitive definitions
    • Code Generation
    -APIs, Clients, Servers, Data Models, Docs, Observability
    • Extensibility
    -Plugins for everything else
    

    View Slide

25. IDL SERVICE DEFINITION
    package lyft.service.users.v1
    service Users {
    rpc Update(UpdateRequest) UpdateResponse;
    }
    message UpdateRequest {
    uint64 id = 1;
    string name = 2;
    }
    

    View Slide

26. What about existing
    services?
    

    View Slide

27. IDL SERVICE DEFINITION — HTTP
    package lyft.service.users.v1
    service Users {
    option (http.http_server_options).isHttpServer = true;
    rpc Update(UpdateRequest) returns UpdateResponse {
    // Override `path` for legacy URL support
    option (http.http_options).path = "/api/v1/users/:id";
    option (http.http_options).method = "PUT";
    }
    }
    

    View Slide

28. TYPES ON THE WIRE
    • Simplified API I/O
    - Structs In, Structs Out
    • Safety
    - Big wins for dynamic languages
    • Transfer Cost
    - Improved latencies
    

    View Slide

29. Types on the wire eliminate
    an entire class of errors
    

    View Slide

30. TypeError: unsupported operand
    type(s) for -: 'NoneType' and 'float'
    

    View Slide

31. HTTP/2.0
    

    View Slide

32. HTTP/2.0
    • Full Duplex Streaming
    • Binary Transport
    • Push
    • Header Compression
    

    View Slide

33. What’s not so great?
    

    View Slide

34. New protocols or languages
    can be traumatic for teams.
    

    View Slide

35. “How do I cURL this?”
    

    View Slide

36. WHAT CAN MAKE THIS EASIER?
    • Incremental Adoption
    -Allow teams to opt-in to the new shiny things
    • Familiarity
    -Tooling that feels welcoming
    -Standardized framework patterns
    • Roll Forward
    -Wire format first, then the protocol and frameworks
    

    View Slide

37. How can we make infra
    changes flexibly?
    

    View Slide

38. View Slide

39. ENVOY PROXY - OVERVIEW
    • L4-L7 Proxy
    -Deployed at the edge or everywhere (sidecar)
    • Modern C++11 Codebase
    • Robust Metrics by Default
    • Highly Extensible
    -DB Monitoring, Protocol Upgrades
    

    View Slide

40. Service
    Mesh
    ENVOY TOPOLOGY
    Internet
    Clients
    “Front” Envoy
    Legacy PHP
    Monolith
    Envoy
    Go Services
    Python Services
    Envoy
    Envoy
    MongoDB /
    DynamoDB
    Ratelimiting
    Stats /
    Tracing
    Discovery
    

    View Slide

41. SIDECAR?
    

    View Slide

42. View Slide

43. ENVOY PROXY - USERS
    

    View Slide

44. Evolving clients and
    services independently.
    

    View Slide

45. ENVOY: GRPC BRIDGE FILTER
    Python HTTP
    Envoy
    Envoy
    HTTP/1
    POST /GetUser
    200 OK
    gRPC
    gRPC Bridge
    

    View Slide

46. ENVOY: JSON PROXY FILTER
    Python HTTP
    Envoy
    Envoy
    HTTP/1
    POST /GetUser
    200 OK
    gRPC
    JSON FILTER
    

    View Slide

47. How can we leverage
    IDLs beyond the API?
    

    View Slide

48. PROTOCOL BUFFER IDL (V3)
    • Service, Messages, and Types
    • Google blessed extensions
    - Well Known Types (WKTs), HTTP
    • Extension does become a necessity
    

    View Slide

49. How far can we take this?
    

    View Slide

50. CURRENTLY…
    • Data Modeling
    • “Thick” Clients
    • Analytics Events
    • Validation
    • Service Generation
    • Documentation
    

    View Slide

51. ODIE
    • DB Agnostic Object Document Mapper (ODM)
    -DynamoDB, Spanner, MongoDB
    • Protobuf IDL Schemas
    • Observability by Default
    -Distributed Tracing, Metrics, Logging
    • Highly Extensible
    -Decorator Pattern Extensions
    

    View Slide

52. ODIE: IDLS MEET THE DATASTORE
    Type-Safe Repository
    Driver-Agnostic Client
    Expression
    Engine
    Decorator
    Middleware
    Database
    Driver
    IDL-Based
    Model
    Lifecycle
    Events
    MongoDB
    DynamoDB
    Spanner
    BoltDB
    

    View Slide

53. ODIE: MODELS AS PROTOCOL BUFFERS
    message User {
    option (odie.mongo).enabled = true;
    string id = 1 [(odie.mongo).primary = true,
    (odie.type).object_id = true];
    string name = 2 [(odie.mongo).name = "username"];
    int64 date = 3 [(odie.type).datetime = true];
    uint32 vers = 4 [(odie.locking).revision = true];
    }
    

    View Slide

54. ODIE: MODELS AS PROTOCOL BUFFERS
    type UserModel struct {
    Id bson.ObjectId `bson:"_id"`
    Name string `bson:"username"`
    Date time.Time
    Vers uint32
    }
    func (pb *User) ToModel() *UserModel
    func (m *UserModel) ToProto() *User
    

    View Slide

55. ODIE: TYPE-SAFE REPOSITORIES
    type UserRepo interface {
    Events() *Events
    Get(ctx context.Context, id bson.ObjectId) *GetBuilder
    Put(ctx context.Context, m *UserModel) *PutBuilder
    Delete(ctx context.Context) *DeleteBuilder
    Update(ctx context.Context) *UpdateBuilder
    Query(ctx context.Context) *QueryBuilder
    }
    

    View Slide

56. EVENT INGEST
    • gRPC Streaming Analytics Ingestion
    • Compile Time Type Safety
    • Data Pipeline Agnostic
    • Tracing Abstraction
    

    View Slide

57. ANALYTICS EVENTS
    message SomethingHappened {
    common.EventBase event_base = 1
    [(validate.rules).message.required = true];
    string name = 2 [(validate.rules).string.required = true];
    }
    

    View Slide

58. PUSHER
    • gRPC Streaming API Proxy
    • Async Event Client Streams
    • Flexible API Infrastructure
    

    View Slide

59. PUSHER - OVERVIEW
    

    View Slide

60. PUSH EVENTS
    message DriverLocation {
    option (event.push) = {
    route: "/v1/rides/:ride_id/driver-location",
    topics: ["pb.events.push.v1.rides.:ride_id.driver_location"]
    };
    uint64 ride_id = 2 [(validate.rules).uint64.required = true];
    }
    

    View Slide

61. When all you have is a
    hammer…
    

    View Slide

62. IT’S A PRETTY GOOD HAMMER
    message CompositeThing {
    option (odie.mongo).enabled = true;
    option (odie.spanner).enabled = true;
    common.EventBase event_base = 1
    [(validate.rules).message.required = true];
    option (event.push) = {
    route: "/v1/rides/:ride_id/driver-location",
    topics: ["pb.events.push.v1.rides.:ride_id.driver_location"]
    };
    uint64 ride_id = 2 [(validate.rules).uint64.required = true];
    }
    

    View Slide

63. That’s an awful lot of
    codegen…
    

    View Slide

64. PROTOC-GEN-STAR (PG*)
    Code generation framework
    http://github.com/lyft/protoc-gen-star

    • AST of primitives
    • Simplifies code generation
    • Highly testable
    Package
    File
    Service
    Method Enum
    EnumValue
    Message
    OneOf
    Field
    

    View Slide

65. PG*: WALK THE AST
    type Visitor interface {
    VisitPackage(Package) (v Visitor, err error)
    VisitFile(File) (v Visitor, err error)
    VisitMessage(Message) (v Visitor, err error)
    VisitEnum(Enum) (v Visitor, err error)
    VisitEnumValue(EnumValue) (v Visitor, err error)
    VisitField(Field) (v Visitor, err error)
    VisitOneOf(OneOf) (v Visitor, err error)
    VisitService(Service) (v Visitor, err error)
    VisitMethod(Method) (v Visitor, err error)
    }
    

    View Slide

66. Takeaways
    

    View Slide

67. How do you change the
    wheels of a moving train?
    

    View Slide

68. Very carefully
    

    View Slide

69. Reduce trauma with
    transitional tooling
    

    View Slide

70. Listen to your customers
    

    View Slide

71. THANK YOU!
    Christopher Burnett
    Lead Client/Server Networking Engineer
    http://github.com/twoism
    cburnett@lyft.com
    

    View Slide