Eventing with Knative and Cloud Run From Basics to Advanced

Knative Eventing provides composable primitives to connect event sources to event consumers on Kubernetes. Cloud Run is its managed version on Google Cloud. In this demo-driven talk, we'll start with basics of Knative and Cloud Run Eventing. We then continue with more advanced eventing topics. At the end of this talk, you'll have a solid understanding of Knative and Cloud Run Eventing and how they can help build event-driven architectures.

Transcript

1. Eventing with Knative and Cloud Run From Basics to Advanced Mete Atamel Developer Advocate at Google @meteatamel atamel.dev speakerdeck.com/meteatamel
2. Introduction
3. Operational Model Programming Model No Infra Management Managed Security Pay only for usage Service-based Event-driven Stateless Serverless
4. Containers Any language Any library Ecosystem around containers .js .rb .go .py .sh … 0 1 0 1 0 0 1 1 1
5. Containers Flexibility Serverless Velocity
6. Cloud Run Fully managed, deploy your workloads and don’t see the cluster. Cloud Run on Anthos Deploy into Anthos, run serverless side-by-side with your existing workloads. Knative Everywhere Use the same APIs and tooling anywhere you run Kubernetes with Knative. Serverless containers with Knative and Cloud Run
7. Confidential & Proprietary What is Knative? Kubernetes based open source building blocks for serverless github.com/knative
8. Knative Stack Serving Eventing Kubernetes Platform Products Components Cloud Run Cloud Run on Anthos Gateway Kourier Istio
9. Confidential & Proprietary Knative Serving What is it? Rapid deployment of serverless containers Automatic (0-n) scaling Configuration and revision management Traffic splitting between revisions Pluggable Connect to your own logging and monitoring platform, or use the built-in system Auto-scaler can be tuned or swapped out for custom code
10. Confidential & Proprietary Knative Serving Knative Service High level abstraction for the application Configuration Current/desired state of an application Code & configuration separated (a la 12-factor) Revision Point in time snapshots for your code and configuration Route Maps traffic to revisions
11. Confidential & Proprietary Knative Eventing What is it? For loosely coupled, event-driven services A number of different delivery methods Scales from just few events to live streams Uses standard CloudEvents Delivery Methods Event source(s) Event type(s) Event consumer(s)
12. Knative Eventing
13. Confidential & Proprietary Terminology of Knative Eventing CloudEvents → Format Event Source → Producer Service → Consumer Channel → Persistence layer Subscription → Interest in messages from channel Broker → Event mesh in the namespace Trigger → Interest in messages from Broker & filter
14. Confidential & Proprietary CloudEvents - cloudevents.io { "specversion": "1.0", "type": "com.github.pull.create", "source": "https://github.com/cloudevents/spec/pull/123", "id": "A234-1234-1234", "time": "2019-04-08T17:31:00Z", "datacontenttype": "application/json", "data": "{ GitHub Payload... }" } FTP GitHub GCS Broker FTP Receive Adapter GitHub Receive Adapter GCS Receive Adapter CloudEvent
15. Confidential & Proprietary Event Sources Name Description Apache Camel Allows to use Apache Camel components for pushing events into Knative Apache Kafka Brings Apache Kafka messages into Knative AWS SQS Brings AWS Simple Queue Service messages into Knative Cron Job Uses an in-memory timer to produce events on the specified Cron schedule. GCP PubSub Brings GCP PubSub messages into Knative GitHub Brings GitHub organization/repository events into Knative GitLab Brings GitLab repository events into Knative. Google Cloud Scheduler Google Cloud Scheduler events in Knative when jobs are triggered Google Cloud Storage Brings Google Cloud Storage bucket/object events into Knative Kubernetes Brings Kubernetes cluster/infrastructure events into Knative https://github.com/knative/docs/tree/master/docs/eventing/sources
16. Service Receives events Knative or Kubernetes Service Subscriber of a trigger or a sink of a source apiVersion: eventing.knative.dev/v1alpha1 kind: Trigger metadata: name: trigger-event-display spec: subscriber: ref: # apiVersion: v1 apiVersion: serving.knative.dev/v1 kind: Service name: event-display Knative Service
17. Channel Persistence layer In-memory, PubSub, Kafka implementations Default channel apiVersion: messaging.knative.dev/v1beta1 kind: InMemoryChannel metadata: name: channel apiVersion: messaging.knative.dev/v1alpha1 kind: KafkaChannel metadata: name: my-kafka-channel spec: numPartitions: 1 replicationFactor: 1 Channel
18. Subscription Subscribes Service to Channel Also defines the notion of event replies apiVersion: messaging.knative.dev/v1alpha1 kind: Subscription metadata: name: subscription1 spec: channel: apiVersion: messaging.knative.dev/v1alpha1 kind: InMemoryChannel name: channel subscriber: ref: apiVersion: serving.knative.dev/v1 kind: Service name: service1 Subscription
19. Broker Combines Channel, reply, and filter functionality into a single resource Typically injected one per namespace apiVersion: eventing.knative.dev/v1beta1 kind: Broker labels: eventing.knative.dev/namespaceInjected: "true" name: default namespace: default status: address: Url:http://default-broker.default.svc.cluster.local Broker
20. Trigger Subscribes a Service to Broker Filtering apiVersion: eventing.knative.dev/v1beta1 kind: Trigger metadata: name: trigger-filter spec: filter: attributes: type: com.google.cloud.storage.object.finalize subscriber: ref: apiVersion: serving.knative.dev/v1 kind: Service name: filter Trigger
21. Delivery Methods
22. Confidential & Proprietary Delivery Methods Simple Delivery Event Source → Service, 1:1 Complex Delivery with optional reply Event Source → Channels → Subscription → Services, 1:N Broker Trigger Delivery Event Source → Broker → Triggeer → Services, 1:N
23. Confidential & Proprietary Simple Delivery
24. Confidential & Proprietary Complex Delivery
25. Confidential & Proprietary Complex Delivery with reply
26. Confidential & Proprietary Broker Trigger Delivery
27. Other topics
28. Confidential & Proprietary Knative GCP Project - github.com/google/knative-gcp Easy configuration and consumption of Google Cloud Platform events in Knative Ready to use event sources: 1. CloudPubSubSource 2. CloudStorageSource 3. CloudSchedulerSource 4. CloudAuditLogsSource 5. CloudBuildSource
29. Confidential & Proprietary Image Processing Pipeline
30. Confidential & Proprietary BigQuery Processing Pipeline
31. Sequence Define a list of services that will be invoked in order End without a reply, with a reply to another Service or chained to another Sequence apiVersion: flows.knative.dev/v1alpha1 kind: Sequence metadata: name: sequence spec: channelTemplate: apiVersion: messaging.knative.dev/v1alpha1 kind: InMemoryChannel steps: - ref: apiVersion: serving.knative.dev/v1 kind: Service name: first - ref: apiVersion: serving.knative.dev/v1 kind: Service name: second Sequence
32. Parallel Define a list of services that will be invoked in parallel Each branch can be guarded by filters Parallel can end with an optional reply apiVersion: messaging.knative.dev/v1alpha1 kind: Parallel metadata: name: parallel spec: channelTemplate: apiVersion: messaging.knative.dev/v1alpha1 kind: InMemoryChannel branches: - subscriber: ref: apiVersion: serving.knative.dev/v1 kind: Service name: service1 - subscriber: ref: apiVersion: serving.knative.dev/v1 kind: Service name: service2 Parallel
33. @meteatamel speakerdeck.com/meteatamel github.com/meteatamel/knative-tutorial knative.dev github.com/meteatamel/cloudrun-tutorial cloud.google.com/run Thank you!