Open standards for building event-driven applications in the cloud

AsyncAPI is an open-source specification for describing and documenting asynchronous APIs, similar to OpenAPI specification for documenting RESTful APIs. CloudEvents is a specification for event data in the cloud. Together, they enable developers to design, document, and test event-driven APIs and to easily share and consume event data across different cloud platforms and ecosystems.

In this session, we will explore the benefits of using AsyncAPI and CloudEvents in your tech stack, and how they can help you build asynchronous, event-driven applications that are well-documented and easy to maintain.

Transcript

1. Open standards for building event-driven applications in the cloud Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel
2. Introduction 01
3. An example event-driven architecture (EDA) An event-driven order processing system Frontend App Engine Order request Payment Processor Cloud Run Authorize & charge CC Shipper Cloud Functions Prepare & ship items Notifier Cloud Run Notify user Message Broker / Event Bus OrderReceived OrderReceived Payment Processed Payment Processed Items Shipped Items Shipped
4. In EDAs, you need to decide and communicate: ❏ The envelope of events ❏ Which service publishes or subscribes to what events?
5. TL;DR CloudEvents defines an envelope for events AsyncAPI helps to document event formats and which service publishes or subscribes to what events
6. CloudEvents 02
7. Why CloudEvents? Middleware Event source Event source Event source Event destination Event destination Event destination Different protocols: Kafka, MQTT, Pub/Sub, … Different formats: JSON, Protobuf, Avro, … Different schemas Reading events and routing are difficult Kafka MQTT Google Pub/Sub
8. CloudEvents An open-source specification for describing events in a common way and SDKs to read and write CloudEvents Defines a minimal set of context attributes to route the event Data that is not intended for routing is placed within the data or data_base64 fields cloudevents.io
9. What does a CloudEvent look like? curl localhost:8080 -v -X POST \ -H "Content-Type: application/json" \ -H "ce-specversion: 1.0" \ -H "ce-type: com.mycompany.myapp.myservice.myevent" \ -H "ce-source: myservice/mysource" \ -H "ce-id: 1234-5678" \ -H "ce-time: 2023-01-02T12:34:56.789Z" \ -H "ce-subject: my-important-subject" \ -H "ce-extensionattr1: value" \ -H "ce-extensionattr2: 5" \ -d '{ "foo1": "bar1", "foo2": "bar2" }' Required Extension attributes Data Binary-mode: Good for receivers unaware of CloudEvents, as the metadata can be ignored Determines mode
10. What does a CloudEvent look like? curl localhost:8080 -v -X POST \ -H "Content-Type: application/cloudevents+json" \ -d '{ "specversion": "1.0", "type": "com.mycompany.myapp.myservice.myevent", "source": "myservice/mysource", "id": "1234-5678", "time": "2023-01-02T12:34:56.789Z", "subject": "my-important-subject", "datacontenttype": "application/json", "extensionattr1" : "value", "extensionattr2" : 5, "data": { "foo1": "bar1", "foo2": "bar2" } }' Required Extension attributes Data Structured-mode: Allows simple forwarding of the event across multiple routing hops Determines mode
11. What else CloudEvents provide? Event formats: JSON, Protobuf, Avro, XML (draft) Protocol bindings: HTTP, AMQP, Kafka, MQTT, NATS, WebSockets (draft) SDKs: Go, Javascript, Java, C#, Ruby, PHP, Python, Rust, Powershell Various CloudEvent libraries from vendors e.g. github.com/googleapis/google-cloudevents github.com/meteatamel/cloudevents-basics
12. However… There’s a batch-mode (multiple events encoded in a single message body) but not all protocol/SDK combinations support it e.g. Javascript SDK supports HTTP and Kafka batch mode but not MQTT batch mode Not all event formats are supported in each SDK e.g. Javascript SDK supports JSON but not Avro or Protobuf Not all protocol bindings are supported in each SDK e.g. Javascript SDK supports HTTP and Kafka but not AMQP, MQTT, NATS
13. Upcoming specs in CloudEvents Discovery API Subscriptions API Schema Registry
15. In EDAs, you need to decide and communicate: ✓ The envelope of events ❏ Which service publishes or subscribes to what events?
16. AsyncAPI 03
17. What is AsyncAPI? An open source specification to define an Async APIs, similar to what OpenAPI (aka Swagger) does for REST APIs Also provides tools to visualize and validate AsyncAPI definitions and generators to generate code from definitions in various languages and frameworks
18. Main concepts in AsyncAPI Application Server Channel Protocol Producer (Publisher) Consumer (Subscriber) Message Bindings for server, channel, operation, and message github.com/meteatamel/asyncapi-basics#concepts
19. OpenAPI AsyncAPI
20. What does an AsyncAPI definition look like? # The simplest AsyncAPI definition possible asyncapi: 2.6.0 info: title: Hello world application version: '0.1.0' channels: hello: publish: message: payload: type: string github.com/meteatamel/asyncapi-basics/tree/main/samples/hello-asyncapi
21. Publish/subscribe semantics in AsyncAPI In a channel, you can have publish and subscribe operations AsyncAPI Term WebSocket Term From Server Perspective From User Perspective Publish Send The server receives the message The user publishes/sends the message to the server Subscribe Receive The server publishes the message The user subscribes/receives the message from the server github.com/meteatamel/asyncapi-basics/tree/main/samples/account-email-services Most useful
22. Tools AsyncStudio Browser based tool to author and visualize and validate AsyncAPI files AsyncAPI CLI CLI based tool to work with AsyncAPI files AsyncAPI Generator Code generators for various languages & frameworks github.com/meteatamel/asyncapi-basics/tree/main/samples/quickstart
23. In EDAs, you need to decide and communicate: ✓ The envelope of events ✓ Which service publishes or subscribes to what events?
24. CloudEvents+AsyncAPI Google Pub/Sub+AsyncAPI 04
25. Can you use CloudEvents and AsyncAPI together? Yes! github.com/meteatamel/asyncapi-basics/tree/main/samples/account-service-cloudevents
26. Can you use Google Pub/Sub and AsyncAPI together? Yes! github.com/meteatamel/asyncapi-basics/tree/main/samples/google-pubsub
27. Thank you! Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel cloudevents.io github.com/meteatamel/cloudevents-basics asyncapi.com github.com/meteatamel/asyncapi-basics Feedback? bit.ly/atamel