ITT 2018 - Christopher Burnett - Types All the Way Down — gRPC and Go Infrastructure at Lyft

In this talk we'll discuss our experiences and lessons learned while introducing gRPC and Go into Lyft's ecosystem. We'll also cover how our Envoy proxy helps to bridge the type-safety gaps across polyglot systems and provides us with flexible and observable infrastructure solutions.

Transcript

1. TYPES ALL THE WAY DOWN GENERATING APIS WITH PROTOBUFS AND GRPC
2. TEŞEKKÜR EDERIM
3. HI! Christopher Burnett Lead Client/Server Networking Engineer NYC Formerly Twitter, VSCO, Posterous
4. ABOUT US • Ride sharing - US and Canada • Interesting Scale and Challenges • Polyglot with ~500 engineers
5. AGENDA • Why RPC? • Working with legacy systems • Disrupting workflows, nicely :) • What we’ve built
6. NETWORKING A little about us… • HTTP & gRPC Libraries • Async Streaming Infrastructure • Envoy Proxy
7. EVERY TEN YEARS… A furious bout of language and protocol design takes place and a new distributed computing paradigm is announced that is compliant with the latest programming model. - A Note On Distributed Computing, Waldo 1994
8. TRIGGER WARNING • CORBA • Thrift • SOAP • WDDX • JSON-RPC • XML-RPC • Avro • HyperMedia • REST • MessagePack
11. A LITTLE HISTORY Like any good story we begin with a PHP monolith… • Active decomposition efforts • 100s of Python Microservices - Flask HTTP/REST And to keep things interesting… • gRPC Core and Compositional Services
12. DEFINING A CORE SERVICE • Organizational Primitives - User, Rides, Vehicles • Zero (Service) Dependencies - Databases, caches, etc • Highly Performant
13. SO, WHY (G)RPC?
14. GRPC - THE BASICS • IDL (Interface Definition Language) Based • Protocol Buffer Wire Format • (Mostly) Full HTTP/2 Transport • Migration/Deprecation Options
15. BUT FIRST…
16. LET’S TALK ABOUT REST
17. RESTful
18. REST/JSON: S2S COMMUNICATION POST /api/updateUser HTTP/1.0 Content-Type: application/json { "id": 18446744073709551615, "username": “chris" }
19. ALRIGHT, LET’S PAINT THAT SHED… PUT /api/users HTTP/1.0 Content-Type: application/json { "id": 18446744073709551615, "username": “chris" }
20. PUTTING ON ANOTHER COAT… PUT /api/users/18446744073709551615 HTTP/1.0 Content-Type: application/json { "username": “chris" }
21. FINISHING TOUCHES… PUT /api/v1/users/18446744073709551615 HTTP/1.0 Content-Type: application/json { "username": “chris" }
22. RESTish
23. IDLs are pretty great :)
24. IDLS ARE PRETTY GREAT • Single Source of Truth -Primitive definitions • Code Generation -APIs, Clients, Servers, Data Models, Docs, Observability • Extensibility -Plugins for everything else
25. IDL SERVICE DEFINITION package lyft.service.users.v1 service Users { rpc Update(UpdateRequest) UpdateResponse; } message UpdateRequest { uint64 id = 1; string name = 2; }
26. What about existing services?
27. IDL SERVICE DEFINITION — HTTP package lyft.service.users.v1 service Users { option (http.http_server_options).isHttpServer = true; rpc Update(UpdateRequest) returns UpdateResponse { // Override `path` for legacy URL support option (http.http_options).path = "/api/v1/users/:id"; option (http.http_options).method = "PUT"; } }
28. TYPES ON THE WIRE • Simplified API I/O - Structs In, Structs Out • Safety - Big wins for dynamic languages • Transfer Cost - Improved latencies
29. Types on the wire eliminate an entire class of errors
30. TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'
31. HTTP/2.0
32. HTTP/2.0 • Full Duplex Streaming • Binary Transport • Push • Header Compression
33. What’s not so great?
34. New protocols or languages can be traumatic for teams.
35. “How do I cURL this?”
36. WHAT CAN MAKE THIS EASIER? • Incremental Adoption -Allow teams to opt-in to the new shiny things • Familiarity -Tooling that feels welcoming -Standardized framework patterns • Roll Forward -Wire format first, then the protocol and frameworks
37. How can we make infra changes flexibly?
39. ENVOY PROXY - OVERVIEW • L4-L7 Proxy -Deployed at the edge or everywhere (sidecar) • Modern C++11 Codebase • Robust Metrics by Default • Highly Extensible -DB Monitoring, Protocol Upgrades
40. Service Mesh ENVOY TOPOLOGY Internet Clients “Front” Envoy Legacy PHP Monolith Envoy Go Services Python Services Envoy Envoy MongoDB / DynamoDB Ratelimiting Stats / Tracing Discovery
41. SIDECAR?
43. ENVOY PROXY - USERS
44. Evolving clients and services independently.
45. ENVOY: GRPC BRIDGE FILTER Python HTTP Envoy Envoy HTTP/1 POST /GetUser 200 OK gRPC gRPC Bridge
46. ENVOY: JSON PROXY FILTER Python HTTP Envoy Envoy HTTP/1 POST /GetUser 200 OK gRPC JSON FILTER
47. How can we leverage IDLs beyond the API?
48. PROTOCOL BUFFER IDL (V3) • Service, Messages, and Types • Google blessed extensions - Well Known Types (WKTs), HTTP • Extension does become a necessity
49. How far can we take this?
50. CURRENTLY… • Data Modeling • “Thick” Clients • Analytics Events • Validation • Service Generation • Documentation
51. ODIE • DB Agnostic Object Document Mapper (ODM) -DynamoDB, Spanner, MongoDB • Protobuf IDL Schemas • Observability by Default -Distributed Tracing, Metrics, Logging • Highly Extensible -Decorator Pattern Extensions
52. ODIE: IDLS MEET THE DATASTORE Type-Safe Repository Driver-Agnostic Client Expression Engine Decorator Middleware Database Driver IDL-Based Model Lifecycle Events MongoDB DynamoDB Spanner BoltDB
53. ODIE: MODELS AS PROTOCOL BUFFERS message User { option (odie.mongo).enabled = true; string id = 1 [(odie.mongo).primary = true, (odie.type).object_id = true]; string name = 2 [(odie.mongo).name = "username"]; int64 date = 3 [(odie.type).datetime = true]; uint32 vers = 4 [(odie.locking).revision = true]; }
54. ODIE: MODELS AS PROTOCOL BUFFERS type UserModel struct { Id bson.ObjectId `bson:"_id"` Name string `bson:"username"` Date time.Time Vers uint32 } func (pb *User) ToModel() *UserModel func (m *UserModel) ToProto() *User
55. ODIE: TYPE-SAFE REPOSITORIES type UserRepo interface { Events() *Events Get(ctx context.Context, id bson.ObjectId) *GetBuilder Put(ctx context.Context, m *UserModel) *PutBuilder Delete(ctx context.Context) *DeleteBuilder Update(ctx context.Context) *UpdateBuilder Query(ctx context.Context) *QueryBuilder }
56. EVENT INGEST • gRPC Streaming Analytics Ingestion • Compile Time Type Safety • Data Pipeline Agnostic • Tracing Abstraction
57. ANALYTICS EVENTS message SomethingHappened { common.EventBase event_base = 1 [(validate.rules).message.required = true]; string name = 2 [(validate.rules).string.required = true]; }
58. PUSHER • gRPC Streaming API Proxy • Async Event Client Streams • Flexible API Infrastructure
59. PUSHER - OVERVIEW
60. PUSH EVENTS message DriverLocation { option (event.push) = { route: "/v1/rides/:ride_id/driver-location", topics: ["pb.events.push.v1.rides.:ride_id.driver_location"] }; uint64 ride_id = 2 [(validate.rules).uint64.required = true]; }
61. When all you have is a hammer…
62. IT’S A PRETTY GOOD HAMMER message CompositeThing { option (odie.mongo).enabled = true; option (odie.spanner).enabled = true; common.EventBase event_base = 1 [(validate.rules).message.required = true]; option (event.push) = { route: "/v1/rides/:ride_id/driver-location", topics: ["pb.events.push.v1.rides.:ride_id.driver_location"] }; uint64 ride_id = 2 [(validate.rules).uint64.required = true]; }
63. That’s an awful lot of codegen…
64. PROTOC-GEN-STAR (PG*) Code generation framework http://github.com/lyft/protoc-gen-star
 • AST of primitives • Simplifies code generation • Highly testable Package File Service Method Enum EnumValue Message OneOf Field
65. PG*: WALK THE AST type Visitor interface { VisitPackage(Package) (v Visitor, err error) VisitFile(File) (v Visitor, err error) VisitMessage(Message) (v Visitor, err error) VisitEnum(Enum) (v Visitor, err error) VisitEnumValue(EnumValue) (v Visitor, err error) VisitField(Field) (v Visitor, err error) VisitOneOf(OneOf) (v Visitor, err error) VisitService(Service) (v Visitor, err error) VisitMethod(Method) (v Visitor, err error) }
66. Takeaways
67. How do you change the wheels of a moving train?
68. Very carefully
69. Reduce trauma with transitional tooling
70. Listen to your customers
71. THANK YOU! Christopher Burnett Lead Client/Server Networking Engineer http://github.com/twoism cburnett@lyft.com