Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Managing gRPC with Wire

Managing gRPC with Wire

At Cash App, mobile and server engineers collaborate on Protobuf and gRPC schemas to define APIs between their services. To improve our experience doing so, we built Wire. From a small in-house protobuf generation library, Wire has today matured into our favourite tool to manage and generate gRPC messages and services. Wire is backed by a versatile Gradle plugin and is used on our Android app, our iOS app, and our microservices at Cash App.
The talk will introduce Wire’s basic features before diving into:

- What Proto 3 support brings to the table in terms of new types and JSON serialisation, and the difference between Wire and protoc,
- How Wire handles options and provides comfortable APIs to consume them,
- Details about Wire Gradle plugin internals, what steps are processed and how to configure them,
- How Wire helps managing protobuf dependencies across microservices.

This presentation will provide simple entry points for people starting with Wire, and actionable improvements for experienced users alike.

Benoît Quenaudon

April 26, 2022
Tweet

More Decks by Benoît Quenaudon

Other Decks in Programming

Transcript

  1. WIRE
    @oldergod

    View Slide

  2. View Slide

  3. // service.proto


    message Request {


    optional Customer customer = 1;


    }


    message Customer {


    required string name = 1;


    optional int32 age = 2;


    }


    service CustomerService {


    rpc getCustomer(Request) returns (Response);


    }


    View Slide

  4. - Proto3


    - Options


    - Wire Gradle Plugin


    - Dependency Management
    Agenda

    View Slide

  5. View Slide

  6. Identity if absent

    View Slide

  7. syntax = "proto2";


    message Message {


    required string a = 1;


    optional string b = 2;


    }

    View Slide

  8. syntax = "proto2";


    message Message {


    required string a = 1;


    optional string b = 2;


    }

    View Slide

  9. syntax = "proto3";


    message Message {


    string a = 1;


    }

    View Slide

  10. Identity if absent
    Strings → ""
    Bytes → empty bytes
    Bools → false
    Numerics → 0, 0L, 0f, 0.0
    Enums → first defined value (which must be 0)
    Messages → null

    View Slide

  11. syntax = "proto3";


    message Message {


    string a = 1;


    }

    View Slide

  12. syntax = "proto3";


    message Message {


    string a = 1;


    optional string b = 2;


    }

    View Slide

  13. syntax = "proto2";


    message MyMessage {


    required string a = 1;


    optional string b = 2;


    }
    class MyMessage(


    val a: String,


    val b: String? = null


    ) : Message() {}

    View Slide

  14. syntax = "proto3";


    message MyMessage {


    string a = 1;


    optional string b = 2;


    }
    class MyMessage(


    val a: String = "",


    val b: String? = null


    ) : Message() {}

    View Slide

  15. New Types

    View Slide

  16. Any → com.squareup.wire.AnyMessage
    Duration → java.time.Duration
    Timestamp → java.time.Instant
    Struct → map
    Wrappers → Boxed types for primitives (String?)
    Empty → kotlin.Unit
    New Types
    google.protobuf.
    ~ Generic placeholder
    ~ same as java.time.Duration
    ~ same as java.time.Instant
    ~ JSON Object
    ~ JSON representation of primitive types + nullability
    ~ For parameter-less RPCs

    View Slide

  17. message Request {


    // Only set when it is a Bitcoin request.


    optional BitcoinData bitcoin_data = 1;


    // Only set when it is a Stock request.


    optional StockData stock_data = 2;


    }

    View Slide

  18. message Request {


    // Only set when it is a Bitcoin request.


    optional BitcoinData bitcoin_data = 1;


    // Only set when it is a Stock request.


    optional StockData stock_data = 2;


    }
    message Request {


    // Will be either BitcoinData or StockData.


    optional google.protobuf.Any request_data = 1;


    }

    View Slide

  19. Any → com.squareup.wire.AnyMessage
    Duration → java.time.Duration
    Timestamp → java.time.Instant
    Struct → map
    Wrappers → Boxed types for primitives (String?)
    Empty → kotlin.Unit
    New Types
    google.protobuf.

    View Slide

  20. Any → com.squareup.wire.AnyMessage
    Duration → java.time.Duration
    Timestamp → java.time.Instant
    Struct → map
    Wrappers → Boxed primitives (e.g. String?)
    Empty → kotlin.Unit
    New Types
    google.protobuf.

    View Slide

  21. syntax = "proto3";


    import "google/protobuf/timestamp.proto";


    message EndpointResponse {


    // ...


    google.protobuf.Timestamp valid_until = 6;


    }

    View Slide

  22. val response: EndpointResponse


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean

    View Slide

  23. val protocResponse: Message.EndpointResponse


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean
    Protoc

    View Slide

  24. val millis = System.currentTimeMillis()


    val protocResponse: Message.EndpointResponse


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean
    Protoc

    View Slide

  25. val millis = System.currentTimeMillis()


    val protocResponse = Message.EndpointResponse.newBuilder()


    .setValidUntil(


    Timestamp...


    )


    .build()


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean
    Protoc

    View Slide

  26. val millis = System.currentTimeMillis()


    val protocResponse = Message.EndpointResponse.newBuilder()


    .setValidUntil(


    Timestamp.newBuilder()


    .setSeconds(millis / 1000)


    .setNanos(((millis % 1000) * 1000000).toInt())


    .build()


    )


    .build()


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean
    Protoc

    View Slide

  27. val millis = System.currentTimeMillis()


    val protocResponse = Message.EndpointResponse.newBuilder().build()


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean
    Protoc

    View Slide

  28. val millis = System.currentTimeMillis()


    val protocResponse = Message.EndpointResponse.newBuilder().build()


    if (isResponseStale(


    Instant.ofEpochSecond(


    protocResponse.validUntil.seconds,


    protocResponse.validUntil.nanos.toLong()


    )


    )) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean
    Protoc

    View Slide

  29. Wire
    val wireResponse: EndpointResponse


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean

    View Slide

  30. Wire
    val millis = System.currentTimeMillis()


    val wireResponse: EndpointResponse


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean

    View Slide

  31. Wire
    val millis = System.currentTimeMillis()


    val wireResponse = EndpointResponse(Instant.ofEpochMilli(millis))


    if (isResponseStale(...)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean

    View Slide

  32. Wire
    val millis = System.currentTimeMillis()


    val wireResponse = EndpointResponse(Instant.ofEpochMilli(millis))


    if (isResponseStale(wireResponse.valid_until!!)) {


    fetch()


    }


    private fun isResponseStale(validUntil: Instant): Boolean

    View Slide

  33. JSON

    View Slide

  34. Why JSON?

    View Slide

  35. JSON
    • Proto2 : Wire 㲗 Protoc communication is dangerous.


    • Proto3 : All good.

    View Slide

  36. Proto2 㲗 Proto3


    Compatibility

    View Slide

  37. Proto2 㲗 Proto3 Compatibility
    • Cannot reference proto2 enums in proto3 message.


    • Because of identity-if-absent.


    • Anything else is good.

    View Slide

  38. Protobuf Options

    View Slide

  39. Protobuf Options
    message Money {


    optional int64 amount = 1 [(squareup.redacted) = true];


    optional string title = 2 [deprecated = true];


    optional Currency current = 3 [(whatever.opt) = 3];


    }

    View Slide

  40. Protobuf Options
    message Money {


    optional int64 amount = 1 [(squareup.redacted) = true];


    optional string title = 2 [deprecated = true];


    optional Currency current = 3 [(whatever.opt) = 3];


    }

    View Slide

  41. Protobuf Options
    • No intrinsic logic embedded


    • Wire handles a few


    • deprecated


    • default


    • json_name


    • packed


    • *.redacted


    • java_package


    • wire_package


    • "How about my option?"

    View Slide

  42. extend google.protobuf.MessageOptions {


    optional string documentation_url = 22200;


    }


    message Octagon {


    option (documentation_url) = "https://en.wikipedia.org/wiki/Octagon";


    optional bool stop = 1;


    }

    View Slide

  43. wire {


    kotlin {


    emitDeclaredOptions = true


    emitAppliedOptions = true


    }


    }
    extend google.protobuf.MessageOptions {


    optional string documentation_url = 22200;


    }


    message Octagon {


    option (documentation_url) = "https://en.wikipedia.org/wiki/Octagon";


    optional bool stop = 1;


    }

    View Slide

  44. // emitDeclaredOptions = true


    @Retention(AnnotationRetention.RUNTIME)


    @Target(AnnotationTarget.CLASS)


    public annotation class DocumentationUrlOption(


    public val value: String


    )
    // emitAppliedOptions = true


    @DocumentationUrlOption("https://en.wikipedia.org/wiki/Octagon")


    public class Octagon(


    ...


    ) : Message(ADAPTER, unknownFields) {}

    View Slide

  45. Wire Gradle Plugin

    View Slide

  46. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate

    View Slide

  47. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    sourcePath {


    srcDir 'src/main/protos'


    }


    sourcePath {


    srcJar 'lib/pizza-protos.jar'


    }


    protoPath {


    srcJar 'com.example.pizza:pizza-protos:1.0.0'


    }


    ...


    }

    View Slide

  48. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    sourcePath {


    srcDir 'src/main/protos'


    include 'com/example/pizza/pizza_delivery.proto'


    include 'com/example/pizza/pizza.proto'


    }


    }

    View Slide

  49. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    sourcePath {


    srcDir 'src/main/protos'


    include 'com/example/pizza/pizza_delivery.proto'


    include 'com/example/pizza/pizza.proto'


    }


    }

    View Slide

  50. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    sourcePath {


    srcDir 'src/main/protos'


    include 'com/example/pizza/pizza_delivery.proto'


    include 'com/example/pizza/pizza.proto'


    }


    }

    View Slide

  51. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    protoPath {


    srcJar 'com.squareup.protos:all-protos'


    }


    sourcePath {


    srcDir 'src/main/protos'


    include 'com/example/pizza/pizza_delivery.proto'


    include 'com/example/pizza/pizza.proto'


    }


    }

    View Slide

  52. View Slide

  53. View Slide

  54. View Slide

  55. View Slide

  56. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    root 'com.example.store.Store'


    }

    View Slide

  57. wire {


    root 'com.example.store.Store'


    }

    View Slide

  58. wire {


    root 'com.example.store.Store'


    }

    View Slide

  59. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    prune 'com.example.store.Store'


    prune 'com.example.geo.Country'


    }

    View Slide

  60. wire {


    prune 'com.example.store.Store'


    prune 'com.example.geo.Country'


    }

    View Slide

  61. wire {


    prune 'com.example.store.Store'


    prune 'com.example.geo.Country'


    }

    View Slide

  62. wire {


    prune 'com.example.store.Store'


    prune 'com.example.geo.Country'


    }

    View Slide

  63. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    kotlin {


    includes = ['com.example.pizza.*']


    excludes = ['com.example.sales.*']


    exclusive = false


    out "${buildDir}/custom"


    }


    }

    View Slide

  64. wire {


    ...


    kotlin {


    // Kotlin emits the matched types only.


    includes = [‘com.example.pizza.*']


    exclusive = true


    }


    java {


    // Java gets everything else!


    }


    }

    View Slide

  65. Schema
    .proto
    Elements
    Java Kotlin Proto
    Schema
    Swift
    Parse
    Link
    Prune
    Generate
    wire {


    custom {


    schemaHandlerFactoryClass = "MyHandlerFactory"


    }


    }
    Schema
    Handler
    Coming soon™

    View Slide

  66. interface SchemaHandler {


    fun handle(schema: Schema, context: Context)


    interface Factory : Serializable {


    fun create(): SchemaHandler


    }


    }
    Coming soon™

    View Slide

  67. Dependency management

    View Slide

  68. Common


    • common/money.proto


    • common/customer.proto
    Service Trades


    • trade/payment.proto
    Service Bank


    • bank/balance.proto
    Mobile

    View Slide

  69. Common


    • common/money.proto


    • common/customer.proto
    // money.proto


    syntax = "proto3";


    package common;


    message Money {


    int64 amount = 1;


    Current currency = 2;


    }


    enum Currency {


    EUR = 0;


    GBP = 1;


    USD = 2;


    CAD = 3;


    }
    // customer.proto


    syntax = "proto3";


    package common;


    message Customer {


    string token = 1;


    string name = 2;


    }

    View Slide

  70. Common


    • common/money.proto


    • common/customer.proto
    wire {


    sourcePath {


    srcDir("src/main/proto")


    }


    }

    View Slide

  71. Common


    • common/money.proto


    • common/customer.proto
    wire {


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }

    View Slide

  72. Common


    • common/money.proto


    • common/customer.proto
    wire {


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }


    sourceSets {


    val main by getting {


    resources.srcDir(


    project.file(“src/main/proto/")


    )


    }


    }

    View Slide

  73. Common


    • common/money.proto


    • common/customer.proto
    wire {


    protoLibrary = true


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }

    View Slide

  74. Common


    • common/money.proto


    • common/customer.proto
    wire {


    protoLibrary = true


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }
    jar published internally to 'app.cash.common:common'

    View Slide

  75. Common


    • common/money.proto


    • common/customer.proto
    Service Trades


    • trade/payment.proto
    Service Bank


    • bank/balance.proto
    Mobile

    View Slide

  76. Service Trades


    • trade/payment.proto
    syntax = "proto3";


    package trade;


    import "common/customer.proto";


    import "common/money.proto";


    message PaymentRequest {


    string common.Money money = 1;


    string common.Customer recipient = 2;


    }


    message PaymentResponse {}


    service PaymentService {


    rpc Pay(PaymentRequest) returns (PaymentResponse);


    }

    View Slide

  77. Service Trades


    • trade/payment.proto
    wire {


    protoLibrary = true


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }

    View Slide

  78. Service Trades


    • trade/payment.proto
    wire {


    protoLibrary = true


    protoPath {


    srcJar("app.cash.common:common:")


    }


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }

    View Slide

  79. Service Trades


    • trade/payment.proto
    wire {


    protoLibrary = true


    protoPath {


    srcJar("app.cash.common:common:")


    }


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }


    dependencies {


    implementation("app.cash.common:common:")


    }

    View Slide

  80. Service Trades


    • trade/payment.proto
    wire {


    protoLibrary = true


    protoPath {


    srcJar("app.cash.common:common:")


    }


    sourcePath {


    srcDir("src/main/proto")


    }


    kotlin {


    }


    }


    dependencies {


    implementation("app.cash.common:common:")


    }
    jar published internally to ‘app.cash.trade:trade'

    View Slide

  81. Common


    • common/money.proto


    • common/customer.proto
    Service Trades


    • trade/payment.proto
    Service Bank


    • bank/balance.proto
    Mobile

    View Slide

  82. Mobile
    wire {


    kotlin {


    }


    }

    View Slide

  83. Mobile
    wire {


    kotlin {


    android = true


    }


    }

    View Slide

  84. Mobile
    wire {


    sourcePath { srcJar("app.cash.trade:trade:") }


    sourcePath { srcJar("app.cash.balance:balance:") }


    kotlin {


    android = true


    }


    }

    View Slide

  85. Mobile
    wire {


    sourcePath { srcJar("app.cash.common:common:") }


    sourcePath { srcJar("app.cash.trade:trade:") }


    sourcePath { srcJar("app.cash.balance:balance:") }


    kotlin {


    android = true


    }


    }

    View Slide

  86. Common


    • common/money.proto


    • common/customer.proto
    Service Trades


    • trade/payment.proto
    Service Bank


    • bank/balance.proto
    Mobile

    View Slide

  87. WIRE
    FIN

    View Slide

  88. References
    • Protocol Bu
    ff
    ers documentations


    • https://developers.google.com/protocol-bu
    ff
    ers/


    • Wire documentations


    • https://square.github.io/wire/


    • Protogram


    • https://github.com/mattprecious/protogram

    View Slide

  89. Protogram
    CLI
    Web

    View Slide

  90. WIRE
    FIN
    for real

    View Slide