Role Based Access Control in Real-Time Streaming Data: What, Why and How Hojjat Jafarpour Founder & CEO @ DeltaStream, Inc. hojjat@deltastream.io @hojjat

Streaming Data is Everywhere 2 … IoT InfoSec Machine Generated Data Gaming Mobile ML/AI

Streaming Storage 3 ● Apache Kafka, AWS Kinesis, Apache Pulsar,... ● Backbone of streaming data ● Decouple producers and consumers ● Make data available in real-time (low latency) Streaming Storage (Kafka, Kinesis, …)

Access Control ● Access Control Lists (ACLs) ○ Good for smaller scale 4

Access Control ● Access Control Lists (ACLs) ○ Good for smaller scale ● Role-based Access Control (RBAC) ○ Access control privileges ○ Roles ○ Resources ○ Privileges determine which role can access and perform operations on specific resources. 5

Example1: Confluent Cloud ● Allows access control to ○ Organization, environment, cluster, granular Kafka resources(topics, consumer groups, and transactional IDs), SR and ksqlDB resources https://docs.confluent.io/cloud/current/access-management/access-control/cloud-rbac.html 6

Example1: Confluent Cloud ● Allows access control to ○ Organization, environment, cluster, granular Kafka resources(topics, consumer groups, and transactional IDs), SR and ksqlDB resources ● Roles are predefined ○ As of now there are 13 available roles ■ OrganizationAdmin, EnvironmentAdmin, CloudClusterAdmin, Operator,... https://docs.confluent.io/cloud/current/access-management/access-control/cloud-rbac.html 7

Example1: Confluent Cloud ● Allows access control to ○ Organization, environment, cluster, granular Kafka resources(topics, consumer groups, and transactional IDs), SR and ksqlDB resources ● Roles are predefined ○ As of now there are 13 available roles ■ OrganizationAdmin, EnvironmentAdmin, CloudClusterAdmin, Operator,... ● Each role has View Scope and Admin Scope https://docs.confluent.io/cloud/current/access-management/access-control/cloud-rbac.html 8

Example1: Confluent Cloud ● Role bindings ○ Permissions for principals ■ Which roles are assigned to a given user ■ e.g., grant permissions to a new user 9

Example1: Confluent Cloud ● Role bindings ○ Permissions for principals ■ Which roles are assigned to a given user ■ e.g., grant permissions to a new user ○ Permissions on resources ■ Which roles can access to a given resource ■ e.g., grant permissions on a resource such as a new cluster to roles 10

Example1: Confluent Cloud ● Example role bindings confluent iam rbac role-binding create --principal User:u-a03bcd --role CloudClusterAdmin --environment env-nx5jd --cloud-cluster lkc-xyxmz +--------------+-------------------+ | Principal | User:u-a03bcd | | Role | CloudClusterAdmin | | ResourceType | Cluster | +--------------+-------------------+ 11

Example1: Confluent Cloud ● Example role bindings confluent iam rbac role-binding create --principal User:u-e03vqq --role ResourceOwner \ --environment env-nx5jd --cloud-cluster lkc-xyxmz --kafka-cluster-id lkc-xyxmz \ --resource Topic:connect-config +----------------+----------------+ | Principal | User:u-e03vqq | | Email | | | Role | ResourceOwner | | Environment | | | CloudCluster | | | ClusterType | | | LogicalCluster | | | ResourceType | Topic | | Name | connect-config | | PatternType | LITERAL | +----------------+----------------+ 12

Example1: Confluent Cloud ● Need to use Confluent Cloud Console, CLI or API ● Confluent only commands ● Limited roles available 13

Example2: AWS MSK ● AWS MSK uses AWS Identity and Access Management(IAM) for authentication and authorization 14

Example2: AWS MSK ● AWS MSK uses AWS Identity and Access Management(IAM) for authentication and authorization ● Uses Authorization Policies ○ Specifies which actions to allow or deny on a resource for a role 15

Example2: AWS MSK ● There is a list of available actions ○ kafka-cluster:Connect, kafka-cluster:DescribeCluster,.. ○ Some actions depend on others ■ Actions with dependencies should include those dependencies 16

Example2: AWS MSK ● There is a list of available actions ○ kafka-cluster:Connect, kafka-cluster:DescribeCluster,.. ○ Some actions depend on others ■ Actions with dependencies should include those dependencies ● Four types of resources that can be used in authorization policy ○ Cluster, Topic, Group and Transactional ID 17

Example2: AWS MSK 18 { "Version": "2022-12-16", "Statement": [ { "Effect": "Allow", "Action": [ "kafka-cluster:Connect", "kafka-cluster:AlterCluster", "kafka-cluster:DescribeCluster" ], "Resource": [ "arn:aws:kafka:us-east-1:0123456789012:cluster/MyTestCluster/abcd1234-0123-abcd-5678-1234abcd-1" ] } ] }

Example2: AWS MSK ● Need to use AWS IAM ● AWS IAM requests only ● Can be used through AWS Management Console, the API, or the AWS CLI 19

20 Is there an easier way to have RBAC for streaming data?

RBAC in Relational Systems 21

RBAC in Relational Systems ● The most familiar for data at rest 22

RBAC in Relational Systems ● The most familiar for data at rest ● Widely used and large user base ○ From open source relational databases such as Postgres to commercial cloud data warehouses 23

RBAC in Relational Systems ● The most familiar for data at rest ● Widely used and large user base ○ From open source relational databases such as Postgres to commercial cloud data warehouses ● Why not use the same model for data in motion(streaming data)? 24

Relational Model for Streams 25

Relational Model for Streams 26 Relational model on streaming data

Relational Model for Streams 27 Relational model on streaming data Organize Process Secure

Example 28 CC_1 MSK_1 orders customers ● Two Kafka clusters, one on Confluent Cloud(CC_1) and one on AWS(MSK_1) ● Two topics, orders and customers

Hierarchical Namespacing ● Same as relational systems ○ Relation: static or dynamic set of tuples ■ STREAM ■ CHANGELOG ■ MATERIALIZED VIEW ■ TABLE 29

Hierarchical Namespacing ● Same as relational systems ○ Relation: static or dynamic set of tuples ■ STREAM ■ CHANGELOG ■ MATERIALIZED VIEW ■ TABLE ○ Schema: a logical grouping of relational objects such as Streams, Changelogs, Materialized Views, and Tables. 30

Hierarchical Namespacing ● Same as relational systems ○ Relation: static or dynamic set of tuples ■ STREAM ■ CHANGELOG ■ MATERIALIZED VIEW ■ TABLE ○ Schema: a logical grouping of relational objects such as Streams, Changelogs, Materialized Views, and Tables. ○ Database: a logical grouping of schemas. 31

Example 32 online_store(Database) public(Schema) orders(Stream) customers(Changelog)

Example 33 CC_1 MSK_1 orders customers ● Run continuous queries and build materialized views ○ Create enriched_orders by joining orders with customers. ○ Build a materialized view to compute daily order value per customer enriched_orders

Example 34 online_store(Database) public(Schema) orders(Stream) customers(Changelog) enriched_orders(Stream) daily_spend(Materialized View)

Key Concepts ● User: represents an authenticated identity. 35

Key Concepts ● User: represents an authenticated identity. ● Objects (Securable Objects): represents an entity to which access can be granted. 36

Key Concepts ● User: represents an authenticated identity. ● Objects (Securable Objects): represents an entity to which access can be granted. ● Privilege: is a defined level of access to an object. 37

Key Concepts ● User: represents an authenticated identity. ● Objects (Securable Objects): represents an entity to which access can be granted. ● Privilege: is a defined level of access to an object. ● Role: is an entity to which privileges can be granted. 38

RBAC for Streaming Data ● Use the same SQL syntax as the relational systems ○ CREATE/DROP/ALTER ○ GRANT/REVOKE 39

Example 40 ● Create a new role to access the enriched_orders stream and daily_spent materialized view CREATE ROLE test_role_1; GRANT ROLE test_role_1 TO USER user_1; GRANT SELECT ON RELATION online_store.public.enriched_orders TO ROLE test_role_1; GRANT SELECT ON RELATION online_store.public.daily_spent TO ROLE test_role_1;

Example 41 ● Create a new role to access all objects in online_store database CREATE ROLE test_role_2; GRANT ROLE test_role_2 TO USER user_2; GRANT USAGE ON DATABASE online_store TO ROLE test_role_2;

Relational RBAC for Streaming Data ● Works across streaming stores ○ Apache Kafka, AWS Kinesis, … 42

Relational RBAC for Streaming Data ● Works across streaming stores ○ Apache Kafka, AWS Kinesis, … ● Familiar syntax ○ No need to learn new syntax/concepts 43

Relational RBAC for Streaming Data ● Works across streaming stores ○ Apache Kafka, AWS Kinesis, … ● Familiar syntax ○ No need to learn new syntax/concepts ● Hierarchical namespacing 44

Relational RBAC for Streaming Data ● Works across streaming stores ○ Apache Kafka, AWS Kinesis, … ● Familiar syntax ○ No need to learn new syntax/concepts ● Hierarchical namespacing ● Unified view of all streaming data by abstracting the streaming stores 45

Relational RBAC for Streaming Data ● Works across streaming stores ○ Apache Kafka, AWS Kinesis, … ● Familiar syntax ○ No need to learn new syntax/concepts ● Hierarchical namespacing ● Unified view of all streaming data by abstracting the streaming stores ● No limitations on roles ○ Can build as many custom roles as needed 46

We have built exactly this at 47 Try all of these and much more for free at www.deltastream.io