Managing gRPC with Wire

At Cash App, mobile and server engineers collaborate on Protobuf and gRPC schemas to define APIs between their services. To improve our experience doing so, we built Wire. From a small in-house protobuf generation library, Wire has today matured into our favourite tool to manage and generate gRPC messages and services. Wire is backed by a versatile Gradle plugin and is used on our Android app, our iOS app, and our microservices at Cash App.
The talk will introduce Wire’s basic features before diving into:

- What Proto 3 support brings to the table in terms of new types and JSON serialisation, and the difference between Wire and protoc,
- How Wire handles options and provides comfortable APIs to consume them,
- Details about Wire Gradle plugin internals, what steps are processed and how to configure them,
- How Wire helps managing protobuf dependencies across microservices.

This presentation will provide simple entry points for people starting with Wire, and actionable improvements for experienced users alike.

Transcript

1. WIRE @oldergod
3. // service.proto message Request { optional Customer customer = 1; } message Customer { required string name = 1; optional int32 age = 2; } service CustomerService { rpc getCustomer(Request) returns (Response); }
4. - Proto3 - Options - Wire Gradle Plugin - Dependency Management Agenda
6. Identity if absent
7. syntax = "proto2"; message Message { required string a = 1; optional string b = 2; }
8. syntax = "proto2"; message Message { required string a = 1; optional string b = 2; }
9. syntax = "proto3"; message Message { string a = 1; }
10. Identity if absent Strings → "" Bytes → empty bytes Bools → false Numerics → 0, 0L, 0f, 0.0 Enums → first defined value (which must be 0) Messages → null
11. syntax = "proto3"; message Message { string a = 1; }
12. syntax = "proto3"; message Message { string a = 1; optional string b = 2; }
13. syntax = "proto2"; message MyMessage { required string a = 1; optional string b = 2; } class MyMessage( val a: String, val b: String? = null ) : Message<Message, Builder>() {} →
14. syntax = "proto3"; message MyMessage { string a = 1; optional string b = 2; } class MyMessage( val a: String = "", val b: String? = null ) : Message<Message, Builder>() {} →
15. New Types
16. Any → com.squareup.wire.AnyMessage Duration → java.time.Duration Timestamp → java.time.Instant Struct → map<String, *> Wrappers → Boxed types for primitives (String?) Empty → kotlin.Unit New Types google.protobuf. ~ Generic placeholder ~ same as java.time.Duration ~ same as java.time.Instant ~ JSON Object ~ JSON representation of primitive types + nullability ~ For parameter-less RPCs
17. message Request { // Only set when it is a Bitcoin request. optional BitcoinData bitcoin_data = 1; // Only set when it is a Stock request. optional StockData stock_data = 2; }
18. message Request { // Only set when it is a Bitcoin request. optional BitcoinData bitcoin_data = 1; // Only set when it is a Stock request. optional StockData stock_data = 2; } message Request { // Will be either BitcoinData or StockData. optional google.protobuf.Any request_data = 1; } ↓
19. Any → com.squareup.wire.AnyMessage Duration → java.time.Duration Timestamp → java.time.Instant Struct → map<String, *> Wrappers → Boxed types for primitives (String?) Empty → kotlin.Unit New Types google.protobuf.
20. Any → com.squareup.wire.AnyMessage Duration → java.time.Duration Timestamp → java.time.Instant Struct → map<String, *> Wrappers → Boxed primitives (e.g. String?) Empty → kotlin.Unit New Types google.protobuf.
21. syntax = "proto3"; import "google/protobuf/timestamp.proto"; message EndpointResponse { // ... google.protobuf.Timestamp valid_until = 6; }
22. val response: EndpointResponse if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
23. val protocResponse: Message.EndpointResponse if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
24. val millis = System.currentTimeMillis() val protocResponse: Message.EndpointResponse if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
25. val millis = System.currentTimeMillis() val protocResponse = Message.EndpointResponse.newBuilder() .setValidUntil( Timestamp... ) .build() if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
26. val millis = System.currentTimeMillis() val protocResponse = Message.EndpointResponse.newBuilder() .setValidUntil( Timestamp.newBuilder() .setSeconds(millis / 1000) .setNanos(((millis % 1000) * 1000000).toInt()) .build() ) .build() if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
27. val millis = System.currentTimeMillis() val protocResponse = Message.EndpointResponse.newBuilder().build() if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
28. val millis = System.currentTimeMillis() val protocResponse = Message.EndpointResponse.newBuilder().build() if (isResponseStale( Instant.ofEpochSecond( protocResponse.validUntil.seconds, protocResponse.validUntil.nanos.toLong() ) )) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean Protoc
29. Wire val wireResponse: EndpointResponse if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
30. Wire val millis = System.currentTimeMillis() val wireResponse: EndpointResponse if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
31. Wire val millis = System.currentTimeMillis() val wireResponse = EndpointResponse(Instant.ofEpochMilli(millis)) if (isResponseStale(...)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
32. Wire val millis = System.currentTimeMillis() val wireResponse = EndpointResponse(Instant.ofEpochMilli(millis)) if (isResponseStale(wireResponse.valid_until!!)) { fetch() } private fun isResponseStale(validUntil: Instant): Boolean
33. JSON
34. Why JSON?
35. JSON • Proto2 : Wire 㲗 Protoc communication is dangerous. • Proto3 : All good.
36. Proto2 㲗 Proto3 Compatibility
37. Proto2 㲗 Proto3 Compatibility • Cannot reference proto2 enums in proto3 message. • Because of identity-if-absent. • Anything else is good.
38. Protobuf Options
39. Protobuf Options message Money { optional int64 amount = 1 [(squareup.redacted) = true]; optional string title = 2 [deprecated = true]; optional Currency current = 3 [(whatever.opt) = 3]; }
40. Protobuf Options message Money { optional int64 amount = 1 [(squareup.redacted) = true]; optional string title = 2 [deprecated = true]; optional Currency current = 3 [(whatever.opt) = 3]; }
41. Protobuf Options • No intrinsic logic embedded • Wire handles a few • deprecated • default • json_name • packed • *.redacted • java_package • wire_package • "How about my option?"
42. extend google.protobuf.MessageOptions { optional string documentation_url = 22200; } message Octagon { option (documentation_url) = "https://en.wikipedia.org/wiki/Octagon"; optional bool stop = 1; }
43. wire { kotlin { emitDeclaredOptions = true emitAppliedOptions = true } } extend google.protobuf.MessageOptions { optional string documentation_url = 22200; } message Octagon { option (documentation_url) = "https://en.wikipedia.org/wiki/Octagon"; optional bool stop = 1; }
44. // emitDeclaredOptions = true @Retention(AnnotationRetention.RUNTIME) @Target(AnnotationTarget.CLASS) public annotation class DocumentationUrlOption( public val value: String ) // emitAppliedOptions = true @DocumentationUrlOption("https://en.wikipedia.org/wiki/Octagon") public class Octagon( ... ) : Message<Octagon, Octagon.Builder>(ADAPTER, unknownFields) {}
45. Wire Gradle Plugin
46. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate
47. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { sourcePath { srcDir 'src/main/protos' } sourcePath { srcJar 'lib/pizza-protos.jar' } protoPath { srcJar 'com.example.pizza:pizza-protos:1.0.0' } ... }
48. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
49. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
50. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
51. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { protoPath { srcJar 'com.squareup.protos:all-protos' } sourcePath { srcDir 'src/main/protos' include 'com/example/pizza/pizza_delivery.proto' include 'com/example/pizza/pizza.proto' } }
56. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { root 'com.example.store.Store' }
57. wire { root 'com.example.store.Store' }
58. wire { root 'com.example.store.Store' }
59. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { prune 'com.example.store.Store' prune 'com.example.geo.Country' }
60. wire { prune 'com.example.store.Store' prune 'com.example.geo.Country' }
61. wire { prune 'com.example.store.Store' prune 'com.example.geo.Country' }
62. wire { prune 'com.example.store.Store' prune 'com.example.geo.Country' }
63. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { kotlin { includes = ['com.example.pizza.*'] excludes = ['com.example.sales.*'] exclusive = false out "${buildDir}/custom" } }
64. wire { ... kotlin { // Kotlin emits the matched types only. includes = [‘com.example.pizza.*'] exclusive = true } java { // Java gets everything else! } }
65. Schema .proto Elements Java Kotlin Proto Schema Swift Parse Link Prune Generate wire { custom { schemaHandlerFactoryClass = "MyHandlerFactory" } } Schema Handler Coming soon™
66. interface SchemaHandler { fun handle(schema: Schema, context: Context) interface Factory : Serializable { fun create(): SchemaHandler } } Coming soon™
67. Dependency management
68. Common • common/money.proto • common/customer.proto Service Trades • trade/payment.proto Service Bank • bank/balance.proto Mobile
69. Common • common/money.proto • common/customer.proto // money.proto syntax = "proto3"; package common; message Money { int64 amount = 1; Current currency = 2; } enum Currency { EUR = 0; GBP = 1; USD = 2; CAD = 3; } // customer.proto syntax = "proto3"; package common; message Customer { string token = 1; string name = 2; }
70. Common • common/money.proto • common/customer.proto wire { sourcePath { srcDir("src/main/proto") } }
71. Common • common/money.proto • common/customer.proto wire { sourcePath { srcDir("src/main/proto") } kotlin { } }
72. Common • common/money.proto • common/customer.proto wire { sourcePath { srcDir("src/main/proto") } kotlin { } } sourceSets { val main by getting { resources.srcDir( project.file(“src/main/proto/") ) } }
73. Common • common/money.proto • common/customer.proto wire { protoLibrary = true sourcePath { srcDir("src/main/proto") } kotlin { } }
74. Common • common/money.proto • common/customer.proto wire { protoLibrary = true sourcePath { srcDir("src/main/proto") } kotlin { } } jar published internally to 'app.cash.common:common'
75. Common • common/money.proto • common/customer.proto Service Trades • trade/payment.proto Service Bank • bank/balance.proto Mobile
76. Service Trades • trade/payment.proto syntax = "proto3"; package trade; import "common/customer.proto"; import "common/money.proto"; message PaymentRequest { string common.Money money = 1; string common.Customer recipient = 2; } message PaymentResponse {} service PaymentService { rpc Pay(PaymentRequest) returns (PaymentResponse); }
77. Service Trades • trade/payment.proto wire { protoLibrary = true sourcePath { srcDir("src/main/proto") } kotlin { } }
78. Service Trades • trade/payment.proto wire { protoLibrary = true protoPath { srcJar("app.cash.common:common:<version>") } sourcePath { srcDir("src/main/proto") } kotlin { } }
79. Service Trades • trade/payment.proto wire { protoLibrary = true protoPath { srcJar("app.cash.common:common:<version>") } sourcePath { srcDir("src/main/proto") } kotlin { } } dependencies { implementation("app.cash.common:common:<version>") }
80. Service Trades • trade/payment.proto wire { protoLibrary = true protoPath { srcJar("app.cash.common:common:<version>") } sourcePath { srcDir("src/main/proto") } kotlin { } } dependencies { implementation("app.cash.common:common:<version>") } jar published internally to ‘app.cash.trade:trade'
81. Common • common/money.proto • common/customer.proto Service Trades • trade/payment.proto Service Bank • bank/balance.proto Mobile
82. Mobile wire { kotlin { } }
83. Mobile wire { kotlin { android = true } }
84. Mobile wire { sourcePath { srcJar("app.cash.trade:trade:<version>") } sourcePath { srcJar("app.cash.balance:balance:<version>") } kotlin { android = true } }
85. Mobile wire { sourcePath { srcJar("app.cash.common:common:<version>") } sourcePath { srcJar("app.cash.trade:trade:<version>") } sourcePath { srcJar("app.cash.balance:balance:<version>") } kotlin { android = true } }
86. Common • common/money.proto • common/customer.proto Service Trades • trade/payment.proto Service Bank • bank/balance.proto Mobile
87. WIRE FIN
88. References • Protocol Bu ff ers documentations • https://developers.google.com/protocol-bu ff ers/ • Wire documentations • https://square.github.io/wire/ • Protogram • https://github.com/mattprecious/protogram
89. Protogram CLI Web
90. WIRE FIN for real