Slide 1

Slide 1 text

Comparing Kubernetes and Dapr: Overlaps and Differences Bilgin Ibryam @bibryam Product Manager, Diagrid

Slide 2

Slide 2 text

Learning by Comparison

Slide 3

Slide 3 text

Agenda ● K8s App lifecycle vs Dapr Deployments ● K8s Services vs. Dapr Service Invocation API ● K8s Health Probes vs. Dapr App Health Check ● K8s vs. Dapr for creating EDA ● K8s StatefulSet vs. Dapr Workflow ● K8s PVCs vs. Dapr State API ● K8s CronJob vs. Dapr Cron ● K8s Lease vs. Dapr Distributed Lock API ● K8s ConfigMap vs. Dapr Config API

Slide 4

Slide 4 text

Why Should I talk about this topic? ● Product Manager at Diagrid ● Former Architect/PM at Red Hat ● Former Committer at Apache Camel ● Blogger and Author ○ Camel Design Patterns ○ Kubernetes Patterns @bibryam https://k8spatterns.com

Slide 5

Slide 5 text

Dapr Concepts

Slide 6

Slide 6 text

Dapr APIs HTTP API gRPC API Microservices written in Application code Any code or framework… Any cloud or edge infrastructure Microservices written in Virtual or physical machines State Management Distributed Lock Publish and Subscribe Service Invocation Secrets Bindings (In/Out) Workflows Configuration Actors Cryptography Observability Security Resiliency

Slide 7

Slide 7 text

My App Over 115 Components (swappable YAML file) State Stores Distributed Lock Pub/Sub Brokers Secret Stores Bindings & Triggers Workflow Configuration Cryptography Firebase Cassandra Redis Azure CosmosDB AWS DynamoDB Redis AWS SQS Azure Service Bus RabbitMQ GCP Pub/Sub GCP Storage AWS S3 Kafka Azure Storage Twilio AWS Secrets Manager Azure KeyVault GCP Secret Manager HashiCorp Vault Kubernetes Secret Redis Redis Dapr Workflow Azure KeyVault Kubernetes Secret Dapr Components Azure App Config PostgreSQL Redis PostgreSQL Azure CosmosDB MongoDB AWS DynamoDB Actors

Slide 8

Slide 8 text

Dapr state API Save state POST /v1.0/state/orderstore Retrieve state GET /v1.0/state/orderstore/myorder1 Delete state DELETE /v1.0/state/orderstore/myorder1 Get bulk state POST /v1.0/state/orderstore/bulk Submit multiple state transactions POST /v1.0/state/orderstore/transaction Redis Component apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: orderstore spec: type: state.redis version: v1 metadata: - name: redisHost value: redis.default.svc.cluster.local:6379 - name: redisPassword value: supersecret

Slide 9

Slide 9 text

Dapr state API Save state POST /v1.0/state/orderstore Retrieve state GET /v1.0/state/orderstore/myorder1 Delete state DELETE /v1.0/state/orderstore/myorder1 Get bulk state POST /v1.0/state/orderstore/bulk Submit multiple state transactions POST /v1.0/state/orderstore/transaction CosmosDB Component apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: orderstore spec: type: state.azure.cosmosdb version: v1 metadata: - name: url value: corpdb.documents.azure.com - name: masterKey secretKeyRef: name: master-key key: cosmos-key - name: database value: orders - name: collection value: processed

Slide 10

Slide 10 text

1. Application Lifecycle

Slide 11

Slide 11 text

Kubernetes Workloads Stateless Service Stateful Service Serverless Workloads Batch/Periodic Job

Slide 12

Slide 12 text

Dapr Control Plane on Kubernetes Diagrid Conductor $ kubectl create -f https://operatorhub.io/install/dapr-k ubernetes-operator.yaml Dapr Operator from OperatorHub.io Helm $ dapr init -k Dapr CLI $ helm upgrade --install dapr dapr/dapr \ --version=1.13 https://www.diagrid.io/conductor

Slide 13

Slide 13 text

Dapr Data Plane Deployments My App Dapr API HTTP/gRPC Application Code Dapr Runtime

Slide 14

Slide 14 text

Sidecar Model kind: Deployment / StatefulSet … annotations: dapr.io/enabled: "true" dapr.io/app-id: "my-app" dapr.io/app-port: "8080" (for long-lived workloads)

Slide 15

Slide 15 text

Dapr-Shared Model Trade Offs Advantages • Faster startup times (Knative, OpenFunctions) • Reduced resource usage • No App restart on Dapr upgrade Disadvantages • Shared downtime • Noisy neighbor • Manual scaling DaemonSet / Deployment https://github.com/dapr/dapr-shared (for serverless workloads)

Slide 16

Slide 16 text

Job Model package main import ( "context" "log" "os" dapr "github.com/dapr/go-sdk/client" ) func main() { client, err := dapr.NewClient() if err != nil { log.Panic(err) } defer client.Close() defer client.Shutdown() } apiVersion: batch/v1 kind: Job / CronJob metadata: name: my-job spec: template: metadata: annotations: dapr.io/enabled: "true" dapr.io/app-id: "with-shutdown" spec: containers: - name: job image: alpine command: ["/bin/sh"] restartPolicy: Never (for short-lived tasks)

Slide 17

Slide 17 text

Deployment Types https://www.diagrid.io/blog/dapr-deployment-models

Slide 18

Slide 18 text

Application Lifecycle Kubernetes Kubernetes manages heterogeneous workloads regardless of the language, including stateless, stateful, serverless (through Knative), and short-lived jobs. Dapr Dapr doesn't manage the application lifecycle, but it adapts to all of these: stateless, serverless, stateful, and short-lived jobs.

Slide 19

Slide 19 text

2. Health Checks

Slide 20

Slide 20 text

Failure Detection and Recovery Process APIs Health Check APIs

Slide 21

Slide 21 text

Kubernetes Health Probes ● Startup Probe ○ Restarting containers if startup probes fails ● Readiness Probe ○ Removing from service endpoint if readiness probe fails ● Liveness Probe ○ Restarting containers if liveness probes fails ■ Probe methods ■ HTTP endpoint ■ gRPC endpoint ■ TCP socket endpoint ■ Unix command return value

Slide 22

Slide 22 text

Dapr Sidecar Health readinessProbe: httpGet: path: v1.0/healthz port: 3500 initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds : 5 failureThreshold: 3 livenessProbe: httpGet: path: v1.0/healthz port: 3500 initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds : 5 failureThreshold : 3 My App kubelet HTTP /healthz HTTP /healthz HTTP /healthz/outbound

Slide 23

Slide 23 text

Dapr App Health Checks kind: Deployment … annotations: dapr.io/enabled: "true" dapr.io/app-id: "my-app" dapr.io/app-port: "7001" dapr.io/app-protocol: "http" dapr.io/enable-app-health-check: "true" dapr.io/app-health-check-path: "/healthz” My App App health check (failed) Dapr stops accepting new work ● Short-circuiting all service-invocation requests ● Unsubscribing from all pub/sub subscriptions ● Stopping all input bindings requests ✓ X

Slide 24

Slide 24 text

Health Checks Kubernetes Reboots applications to rectify issues Redirects incoming traffic away from unhealthy app instances Dapr Stops outgoing connections on behalf of the app

Slide 25

Slide 25 text

3. Synchronous Interactions

Slide 26

Slide 26 text

Kubernetes Service Discovery Node Port Load Balancer Ingress

Slide 27

Slide 27 text

Kubernetes Service Discovery External Service Discovery Internal Service Discovery Advantages ● Simplifies access to pods through service discovery ● Provides load balancing across pods to distribute traffic ● Integrates with DNS for easy name resolution Limitations ● Lacks built-in resiliency features like retries, circuit breakers, and timeouts ● Does not offer tracing or network metrics ● Does not support traffic encryption

Slide 28

Slide 28 text

Dapr Service Invocation API Shopping cart Checkout POST http://checkout:5000/order

Slide 29

Slide 29 text

Dapr Service Invocation API POST http://localhost:3500/v1.0/invoke/checkout/method/order POST http://localhost:5100/order Shopping cart Checkout DNS Service discovery Access Control Resiliency mTLS encryption Observability token authentication token authentication

Slide 30

Slide 30 text

Dapr Service Invocation API Features ● Service discovery (Kubernetes, HashiCorp Consul, SQLite, mDNS) ● Mitigating request timeouts or transient failures. ● Built-in distributed tracing & metrics ● Access control policies ● End-to-end security and mTLS ● Chain pluggable middleware components const order = {orderId: 1}; const url = `localhost:3500/order` const response = await fetch(url, { method: "POST", headers: { "dapr-app-id": "checkout", }, body: JSON.stringify(order), });

Slide 31

Slide 31 text

Kubernetes - Network Policy kind: NetworkPolicy apiVersion: networking.k8s.io/v1 metadata: name: allow-database spec: podSelector: matchLabels: id: checkout ingress: - from: - podSelector: matchLabels: id: shopping-cart Features ● Pod-level network isolation ● Supports both IP address and namespace-based rules ● Defines inbound and outbound traffic rules ● Allows for TCP/UDP/SCTP and port-based filtering Notification services Pod Checkout Pod Shopping cart Pod

Slide 32

Slide 32 text

Service Invocation: Access Control Features ● Unique App Identity based on SPIFFE ID ● Granular access control (Layer 7) ● HTTP verbs / Path or gRPC operation ● Support for both HTTP and gRPC protocols apiVersion: dapr.io/v1alpha1 kind: Configuration metadata: name: checkout-config spec: accessControl: defaultAction: deny trustDomain: "public" policies: - appId: shopping-cart defaultAction: deny trustDomain: 'public' namespace: "default" operations: - name: /order httpVerb: ['POST'] action: allow

Slide 33

Slide 33 text

Is Dapr a Service Mesh? https://docs.dapr.io/concepts/service-mesh/

Slide 34

Slide 34 text

Synchronous Interactions Kubernetes ● Service discovery and load balancing across pods ● L3/L4 network policies Dapr ● Traffic encryption ● Resiliency policies ● Distributed tracing & metrics ● L7 granular access control ● Swappable service discovery ● Pluggable middleware

Slide 35

Slide 35 text

4. Event-driven Interactions

Slide 36

Slide 36 text

Event-Driven Apps on Kubernetes

Slide 37

Slide 37 text

Dapr Pub/Sub API Checkout Shipping Message broker

Slide 38

Slide 38 text

Dapr Pub/Sub API POST http://localhost:3500/v1.0/publish/mybroker/order-messages POST http://localhost:5100/orders Checkout Shipping Observability backend Message broker Cloudevents Access Control Resiliency token authentication token authentication Encryption

Slide 39

Slide 39 text

Dapr Pub/Sub API const client = new DaprClient({ daprHost, daprPort }); const order = { orderId: 1 }; await client.pubsub.publish(“mypubsub”, “new-orders”, order); Publish Subscribe (Dapr pushes each message to the app) const server = new DaprServer({ serverHost, serverPort, clientOptions: { daprHost, daprPort } }); server.pubsub.subscribe(“mypubsub”, “new-orders”, data => console.log("Received:" + JSON.stringify(data)) ); await server.start(); stop, err := cl.SubscribeWithHandler(context.Background(), client.SubscriptionOptions{ PubsubName: "pubsub", Topic: "orders", }, eventHandler, ) Streaming Subscription (Bidirectional stream between app/sidecar ) Features ● Integrates with many message brokers and queues ● Push or Pull mode for subscriptions ● Content-based message routing and filtering ● Aggregator/Splitter through message batches ● Dead-letter topics and resiliency policies ● Access control per topic ● Message expiration ● Delayed delivery (coming soon)

Slide 40

Slide 40 text

Event-driven Interactions Kubernetes ● Platform for other projects: Knative Eventing, Strimzi, KEDA, CloudEvents, Dapr.. Dapr ● Message broker abstraction ● Push or Pull mode for subscriptions ● Content-based message routing and filtering ● Aggregator/Splitter through message batches ● Dead-letter topics and resiliency policies ● Access control per topic ● Message expiration ● Delayed delivery (coming soon)

Slide 41

Slide 41 text

5. State Management

Slide 42

Slide 42 text

Kubernetes Storage Abstraction (PVs) PVC accessModes ● ReadWriteOnce a volume that can be mounted to a single node at a time. ● ReadOnlyMany The volume can be mounted to multiple nodes ● ReadWriteMany - the volume can be mounted by many nodes ● ReadWriteOncePod - guarantees that only a single Pod has access to a volume.

Slide 43

Slide 43 text

Dapr StateStore API (a.k.a Key/Value)

Slide 44

Slide 44 text

Dapr StateStore API GET http://localhost:3500/v1.0/state/mystatestore/order1 POST http://localhost:3500/v1.0/state/mystat estore [{ “key”: “order1”, “value”: “{orderId: 1}” }] Read Operation Write Operation Features ● Integrates with many state stores ● Concurrency: first-write / last-write; ● Consistency behaviors: strong / eventual ● Transactional operations ● Resiliency policies ● Bulk operations ● Time-to-Live (TTL) ● Data at rest encryption ● Limit access per app (not operation)

Slide 45

Slide 45 text

Transactional Outbox Pattern apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: mysql-outbox spec: type: state.mysql version: v1 metadata: - name: connectionString value: "" - name: outboxPublishPubsub # Required value: "mypubsub" - name: outboxPublishTopic # Required value: "newOrder"

Slide 46

Slide 46 text

State Management Kubernetes ● Provides durable storage abstraction over the file system interface Dapr ● Key/Value API with powerful features independent of the storage providers

Slide 47

Slide 47 text

6. Stateful Workloads

Slide 48

Slide 48 text

Stateful Workloads on Kubernetes Features ● Unique instance identity ● Stable network address ● Stable ordinality ● Non-shared persistent storage ● Minimal availability (quorum)

Slide 49

Slide 49 text

Stateful Workloads with Dapr Workflows Features ● Durable execution with persisted state ● Author workflows in C#, Java, Python or JavaScript ● Apply patterns such as: chaining, Split/Synchronize, external trigger, durable timers, child workflows.

Slide 50

Slide 50 text

Split/Syncrhonize with .NET SDK // Get a list of N work items to process in parallel. object[] workBatch = await context.CallActivityAsync("GetWorkBatch", null); // Schedule the parallel tasks, but don't wait for them to complete yet. var parallelTasks = new List>(workBatch.Length); for (int i = 0; i < workBatch.Length; i++) { Task task = context.CallActivityAsync("ProcessWorkItem", workBatch[i]); parallelTasks.Add(task); } // Everything is scheduled. Wait here until all parallel tasks have completed. await Task.WhenAll(parallelTasks); // Aggregate all N outputs and publish the result. int sum = parallelTasks.Sum(t => t.Result); await context.CallActivityAsync("PostResults", sum);

Slide 51

Slide 51 text

End Activity A Activity B Activity C Start End Activity A Activity B Activity C Start Activity CreateTimer Continue As New Start Wait with further execution Restart the workflow Wait for event Approval Event End Get Approval Not Approved Start Approved Sequence Split/Synchronize Timer External trigger

Slide 52

Slide 52 text

Choreography or Orchestration? https://www.diagrid.io/blog/choreography-or-orchestration-that-is-the-wrong-question/

Slide 53

Slide 53 text

Stateful Workloads Kubernetes ● Coarse-grained, process-level orchestration ● Start/stop processes in sequence ● Manage storage Dapr ● Define stateful business processes in code ● Durable execution guarantees ● Flexible storage options

Slide 54

Slide 54 text

7. Tasks

Slide 55

Slide 55 text

Jobs in Kubernetes Job - is an isolated atomic unit of work run until completion. apiVersion: batch/v1 kind: Job metadata: name: test-job spec: template: metadata: name: test-job spec: containers: - name: test image: alpine:latest command: - "bin/sh" - "-c" - "for i in 1 2 3 4 5 ; do echo $i ; done" restartPolicy: Never

Slide 56

Slide 56 text

Actors in Dapr public class HelloWorldActorImpl extends AbstractActor implements HelloWorldActor { public HelloWorldActorImpl(ActorRuntimeContext runtimeContext, String id) { super(runtimeContext, id); } @Override public CompletableFuture sayHello(String name) { return CompletableFuture.completedFuture("Hello " + name); } } Features ● No concurrency or threading required ● Built-in distribution and fail-over ● State is persisted in a configurable state store

Slide 57

Slide 57 text

Recurring Tasks in Kubernetes ● CronJob - allows the execution of a unit of work to be triggered by a temporal event. apiVersion: batch/v1 kind: CronJob metadata: name: hello-world-cronjob spec: schedule: "*/1 * * * *" # Runs every minute jobTemplate: spec: template: spec: containers: - name: hello image: busybox args: - /bin/sh - -c - echo "Hello, World!" restartPolicy: OnFailure

Slide 58

Slide 58 text

Cron Binding in Dapr (Jobs API in v1.14) My App HTTP POST Application Code Dapr runtime apiVersion: dapr.io/v1alpha1 kind: Component metadata: name: my-job spec: type: bindings.cron version: v1 metadata: - name: schedule value: "@every 15m"

Slide 59

Slide 59 text

Tasks Kubernetes ● Job: Finite unit of computation ○ Runs a container ● CronJob as temporal tasks Dapr ● Actors as a “unit of computation” ○ Runs a class/function ● Actor timers/reminders ● Cron Binding - stateless cron ● (New Jobs API) - stateful cron

Slide 60

Slide 60 text

8. Distributed Locks

Slide 61

Slide 61 text

Distributed Locks in Kubernetes ● Runs a single replica with StatefulSet ● Hold a lock on Kubernetes ConfigMaps ● Use Kubernetes Lease resource ● Custom distributed lock e.g.: ○ Zookeeper ○ Consul ○ Redis ○ etcd

Slide 62

Slide 62 text

Dapr Distributed Lock API ✖ Lock acquired Waiting for the lock curl -X POST http://localhost:3500/v1.0-alpha1/lock/lockstore -H 'Content-Type: application/json' -d ' {"resourceId":"my_file_name", "lockOwner":"abc123", "expiryInSeconds": 60}'

Slide 63

Slide 63 text

Distributed Locks Kubernetes ● StatefulSet - process lock ● Lease resource - for Kubernetes itself and custom controllers Dapr ● Simple HTTP based lock API

Slide 64

Slide 64 text

9. Configs and Secrets

Slide 65

Slide 65 text

ConfigMaps and Secrets in Kubernetes

Slide 66

Slide 66 text

Dapr Configuration API

Slide 67

Slide 67 text

Dapr Secrets API

Slide 68

Slide 68 text

Configurations and Secrets Kubernetes ● Requires pod restarts or app reloads for changes ● Scoped at the namespace level Dapr ● Supports dynamic updates without restarts ● Scoped to specific services or applications ● Supports multiple config and secret stores

Slide 69

Slide 69 text

10. More?

Slide 70

Slide 70 text

Bindings API for bi-directional connection to 3rd-party systems Resiliency: Retries/Back-offs, Timeouts, Circuit Breakers Observability: Distributed Tracing, Network Metrics, Logs. Crypto API for encrypting and decrypting messages Other Dapr APIs

Slide 71

Slide 71 text

Summary

Slide 72

Slide 72 text

Kubernetes or Dapr? Kubernetes enables Ops to operate heterogeneous applications on any infrastructure, with consistent abstractions. Dapr eables Devs to implement polyglot applications, using any backing infrastructure and interaction style, with consistent APIs.

Slide 73

Slide 73 text

No content

Slide 74

Slide 74 text

Resources @bibryam https://dapr.io https://www.diagrid.io/conductor https://www.diagrid.io/blog @diagridio

Slide 75

Slide 75 text

Thank You @bibryam