@luxas @luxas Beyond RBAC: Avoid broken ACLs in control planes with declarative Relation-based Access Control Lucas Käldström, Jan 16, 2024 – , Helsinki 1

@luxas @luxas $ whoami Kubernetes Contributor, former maintainer => co-led kubeadm/cluster lifecycle work, ported Kubernetes to ARM CNCF Ambassador, meetup organizer 9-time KubeCon speaker Senior Software Engineer at formerly Weaveworks 1st-year MSc student at 2

@luxas @luxas 3 “All I want is a secure system where it's easy to do anything I want. Is that so much to ask?” – xkcd

@luxas @luxas 4 Source: xkcd

@luxas @luxas 5 Security landscape is rapidly changing Photo by Maksym Kaharlytskyi on Unsplash

@luxas @luxas 6 … it is becoming more dynamic Photo by Maksym Kaharlytskyi on Unsplash

@luxas @luxas 7 Photo by M Alazia on Unsplash … and fragmented

@luxas @luxas OWASP API #1: Broken Object-level Authorization “APIs tend to expose endpoints that handle object identifiers, creating a wide attack surface of Object Level Access Control issues. Object level authorization checks should be considered in every function that accesses a data source using an ID from the user.” – OWASP 8

@luxas @luxas OWASP API #3: Broken Property-level Authorization “This category combines API3:2019 Excessive Data Exposure and API6:2019 - Mass Assignment, focusing on the root cause: the lack of or improper authorization validation at the object property level. This leads to information exposure or manipulation by unauthorized parties.” – OWASP 9

@luxas @luxas Challenges with Access Control 10

@luxas @luxas Challenge#1: Uniform enforcement Separation of concerns: Clearly separate all computation of policy decisions (Policy Decision Point, PDP), from enforcement (Policy Enforcement Point, PEP). Even if “just” behind an interface. 11 User Policy Enforcement (PEP) Resource Access (RAP) Policy Decision (PDP) 1 2 3 4

@luxas @luxas Challenge#1: Uniform enforcement Separation of concerns: Clearly separate all computation of policy decisions (Policy Decision Point, PDP), from enforcement (Policy Enforcement Point, PEP). Even if “just” behind an interface. Policy Uniformity: Use a non-Turing complete* policy language like Open Policy Agent or Cedar for uniform enforcement and policy description across services in the organization. 12 *Or something that at least tries to be scoped down

@luxas @luxas Challenge#1: Uniform enforcement Separation of concerns: Clearly separate all computation of policy decisions (Policy Decision Point, PDP), from enforcement (Policy Enforcement Point, PEP). Even if “just” behind an interface. Policy Uniformity: Use a non-Turing complete* policy language like Open Policy Agent or Cedar for uniform enforcement and policy description across services in the organization. Uniform Attributes: Every request to the PDP consists of {principal, action, resource, (attributes)}** 13 **Sometimes you want to send the full payload as well, for ABAC

@luxas @luxas Challenge#2: Ensuring correctness Testing: Unit tests, static analysis, fuzzing, differential testing 14

@luxas @luxas Challenge#2: Ensuring correctness Testing: Unit tests, static analysis, fuzzing, differential testing Policy Equality: A lot bugs arise when you change/upgrade/refactor something, and you miss an edge case. Strive for a canonical representation of your ACL logic, so you can check for equality. 15

@luxas @luxas Challenge#2: Ensuring correctness Testing: Unit tests, static analysis, fuzzing, differential testing Policy Equality: A lot bugs arise when you change/upgrade/refactor something, and you miss an edge case. Strive for a canonical representation of your ACL logic, so you can check for equality. Shadow rollout: Run both new and old policy in production, enforce old, warn if new yields other results 16

@luxas @luxas Challenge#2: Ensuring correctness Testing: Unit tests, static analysis, fuzzing, differential testing Policy Equality: A lot bugs arise when you change/upgrade/refactor something, and you miss an edge case. Strive for a canonical representation of your ACL logic, so you can check for equality. Shadow rollout: Run both new and old policy in production, enforce old, warn if new yields other results Preventing Privilege Escalation: How can users be allowed to manage a subdivision of the authorization state, without escalating their, or someone else’s privileges? 17

@luxas @luxas Challenge#3: Consistency and Availability (CAP) A spectrum between “offline-only” and “online-only” modes. Offline-only: Rely only on attribute and role info in the credential (JWT or cert), static policy based on that. => Hard (if not impossible) to revoke access before expiration 18

@luxas @luxas Challenge#3: Consistency and Availability (CAP) A spectrum between “offline-only” and “online-only” modes. Offline-only: Rely only on attribute and role info in the credential (JWT or cert), static policy based on that. => Hard (if not impossible) to revoke access before expiration Online-only: Credentials ID-only, always lookup permissions from a central database. Authorization system “single point of failure”. => Is it acceptable to cache and return potentially stale results? 19

@luxas @luxas Challenge#3: Consistency and Availability (CAP) Authorization state size: How much data can you process in working memory or put in a credential? Dimensionality high => DB needed 20

@luxas @luxas Challenge#3: Consistency and Availability (CAP) Authorization state size: How much data can you process in working memory or put in a credential? Dimensionality high => DB needed Change Propagation Delay: What is the upper bound for change propagation delays? One millisecond? One second? One minute? One hour? One day? One year? 21

@luxas @luxas Challenge#3: Consistency and Availability (CAP) Authorization state size: How much data can you process in working memory or put in a credential? Dimensionality high => DB needed Change Propagation Delay: What is the upper bound for change propagation delays? One millisecond? One second? One minute? One hour? One day? One year? Casual Ordering: Google coined New Enemy Problem in a paper of theirs, defined as “when we fail to respect the ordering between ACL updates or when we apply old ACLs to new content.” 22

@luxas @luxas Challenge#3: Consistency and Availability (CAP) Authorization state size: How much data can you process in working memory or put in a credential? Dimensionality high => DB needed Change Propagation Delay: What is the upper bound for change propagation delays? One millisecond? One second? One minute? One hour? One day? One year? Casual Ordering: Google coined New Enemy Problem in a paper of theirs, defined as “when we fail to respect the ordering between ACL updates or when we apply old ACLs to new content.” Time Complexity: How does the latency scale? O(n)? O(n2)? O(n3)? 23

@luxas @luxas Challenge#4: Observability Explainability: The system’s ability to provide the “reasons” for a access decision in both human- and machine-friendly form 24

@luxas @luxas Challenge#4: Observability Explainability: The system’s ability to provide the “reasons” for a access decision in both human- and machine-friendly form Lookupability: List all the resources a user can access with a permission, or all the users with a given permission to a resource 25

@luxas @luxas Challenge#4: Observability Explainability: The system’s ability to provide the “reasons” for a access decision in both human- and machine-friendly form Lookupability: List all the resources a user can access with a permission, or all the users with a given permission to a resource Audit Logging: The system’s ability to log access control decisions to an append-only audit log 26

@luxas @luxas Relation-based Access Control (ReBAC) The Next Generation Access Control (NGAC) 2018 standard, developed by NIST, proposes a graph-based access control implementation that is a superset of RBAC, ABAC, MLS, SoD, etc. paradigms. 27

@luxas @luxas Relation-based Access Control (ReBAC) The Next Generation Access Control (NGAC) 2018 standard, developed by NIST, proposes a graph-based access control implementation that is a superset of RBAC, ABAC, MLS, SoD, etc. paradigms. Google Zanzibar paper in 2019 on how Google handles all their authorization concerns for Drive, Youtube, Gmail, etc. using one single graph with relations between users and resources. 28

@luxas @luxas Relation-based Access Control (ReBAC) The Next Generation Access Control (NGAC) 2018 standard, developed by NIST, proposes a graph-based access control implementation that is a superset of RBAC, ABAC, MLS, SoD, etc. paradigms. Google Zanzibar paper in 2019 on how Google handles all their authorization concerns for Drive, Youtube, Gmail, etc. using one single graph with relations between users and resources. => Core idea is to “compile” the access control problem down to: “is there a relation of type get in the graph between the user:lucas and document:todolist” 29

@luxas @luxas Relation-based Access Control 1. App developer specifies an authorization model (schema) a. “Users can read or write folders, folders contain documents” 30 user folder document write contains read

@luxas @luxas Relation-based Access Control 1. App developer specifies an authorization model (schema) a. “Users can read or write folders, folders contain documents” 2. Write or import relationship data a. “User Lucas can read folder ‘customers’ b. “folder ‘customers’ has documents ‘secret’ and ‘fooinc’” 31 user:lucas folder:customers document:secret document:fooinc contains read contains

@luxas @luxas Relation-based Access Control 1. App developer specifies an authorization model (schema) a. “Users can read or write folders, folders contain documents” 2. Write or import relationship data a. “User Lucas can read folder ‘customers’ b. “folder ‘customers’ has documents ‘secret’ and ‘fooinc’” 3. Query the graph a. “Can user Lucas read document ‘secret’?” => true b. “Can user Lucas write document ‘secret’?” => false c. “What documents can user Lucas read?” => [“document:secret”, “document:fooinc”] 32 user:lucas folder:customers document:secret document:fooinc contains read contains

@luxas @luxas ReBAC users or evaluators in the wild 33 Homegrown Zanzibar: Homegrown NGAC: OSS Zanzibar: Evaluating Zanzibar: Zanzibar Himeji AuthZ Source Source Source Source Source Source

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 34 user team member

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 35 user role assignee team member

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 36 user folder role assignee viewer editor team member admin

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 37 user folder role assignee viewer editor team member admin

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 38 user folder role assignee viewer editor team member admin parent

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 39 user folder role assignee viewer editor team member admin document viewer admin editor parent contains

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 40 user folder role assignee viewer editor team member admin document viewer admin editor parent contains

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar schema) 41 user folder role assignee viewer editor team member admin document viewer admin editor parent contains

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar data) 42 user:bob team:employees user:alice member

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar data) 43 user:bob team:employees user:alice role:admins folder:acmeco member assignee viewer editor admin

@luxas @luxas Implementing RBAC with ReBAC (Zanzibar data) 44 user:bob team:employees user:alice role:admins folder:acmeco member assignee viewer editor admin doc:salaryinfo viewer editor admin contains

@luxas @luxas parent Implementing RBAC with ReBAC (Zanzibar data) 45 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee viewer editor admin doc:salaryinfo viewer editor admin viewer admin editor contains

@luxas @luxas parent Implementing RBAC with ReBAC (Zanzibar data) 46 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas parent Implementing RBAC with ReBAC (Zanzibar data) 47 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains role:public-edit contains

@luxas @luxas parent Query: What documents does bob have access to? 48 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains role:public-edit contains

@luxas @luxas role:public-edit parent Query: What documents does bob have access to? 49 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas role:public-edit parent Query: What documents does bob have access to? 50 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas role:public-edit parent Query: What documents does bob have access to? 51 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas role:public-edit parent Query: What documents does bob have access to? 52 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas parent Query: What documents does alice have access to? 53 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains role:public-edit contains

@luxas @luxas role:public-edit parent Query: What documents does alice have access to? 54 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas role:public-edit parent Query: What documents does alice have access to? 55 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas role:public-edit parent Query: What documents does alice have access to? 56 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas role:public-edit parent Query: What documents does alice have access to? 57 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas role:public-edit parent Query: What documents does alice have access to? 58 user:bob team:employees user:alice role:admins folder:acmeco folder:public member assignee assignee viewer editor admin doc:templates viewer admin editor doc:salaryinfo viewer editor admin viewer admin editor contains contains

@luxas @luxas Emulating ABAC with ReBAC (Zanzibar) 59 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write Region definitions

@luxas @luxas Environment definitions Emulating ABAC with ReBAC (Zanzibar) 60 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write

@luxas @luxas Emulating ABAC with ReBAC (Zanzibar) 61 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write

@luxas @luxas Query: Which workloads can Lucas access? 62 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write

@luxas @luxas Query: Which workloads can Lucas access? 63 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write

@luxas @luxas Query: Which workloads can Lucas access? 64 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write

@luxas @luxas Query: Which workloads can Lucas access? 65 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write

@luxas @luxas Query: Which workloads can Lucas access? 66 workload:prod-backend-eu user:lucas read write attr-region:eu read write AND AND workload:prod-backend-us read write AND AND workload:dev-backend-eu read write AND AND attr-region:us attr-env:prod read write attr-env:dev read write

@luxas @luxas The “lingua franca” of authorization? Challenge: How do I map everything an employee has access to consistently throughout 10s of SaaS services, and 10s of internal platforms and tools, to see what they can do? 67

@luxas @luxas The “lingua franca” of authorization? Challenge: How do I map everything an employee has access to consistently throughout 10s of SaaS services, and 10s of internal platforms and tools, to see what they can do? 😱 There is no de-facto “standard” for comparing, merging or joining various authorization policies. 🔥 Can ReBAC do this? Maybe! 68

@luxas @luxas The “lingua franca” of authorization? Relation-based access control (ReBAC) provides for an intriguing way of “emulating” or “supporting” a multitude of access control paradigms, e.g. RBAC, ABAC, Multi-level security (MLS), Separation of Duty, and so on. 69

@luxas @luxas The “lingua franca” of authorization? Relation-based access control (ReBAC) provides for an intriguing way of “emulating” or “supporting” a multitude of access control paradigms, e.g. RBAC, ABAC, Multi-level security (MLS), Separation of Duty, and so on. A highly optimized index: The “compilation” to a graph - makes lookups extremely fast 70

@luxas @luxas The “lingua franca” of authorization? Relation-based access control (ReBAC) provides for an intriguing way of “emulating” or “supporting” a multitude of access control paradigms, e.g. RBAC, ABAC, Multi-level security (MLS), Separation of Duty, and so on. A highly optimized index: The “compilation” to a graph - makes lookups extremely fast - provides for a consistent data format 71

@luxas @luxas The “lingua franca” of authorization? Relation-based access control (ReBAC) provides for an intriguing way of “emulating” or “supporting” a multitude of access control paradigms, e.g. RBAC, ABAC, Multi-level security (MLS), Separation of Duty, and so on. A highly optimized index: The “compilation” to a graph - makes lookups extremely fast - provides for a consistent data format - allows for auditing of graph schema and data 72

@luxas @luxas Load test of Zanzibar OSS impl SpiceDB (similar for OpenFGA): Graph structure: Social-media-like (Pareto distribution) Graph size: 100,000,000,000 edges (100G) Authorization Graph is “Extremely Fast”? 73

@luxas @luxas Load test of Zanzibar OSS impl SpiceDB (similar for OpenFGA): Graph structure: Social-media-like (Pareto distribution) Graph size: 100,000,000,000 edges (100G) Read load: 1,000,000 permission checks/second (1M) Write load: 1,000 writes/second (1%) Authorization Graph is “Extremely Fast”? 74

@luxas @luxas Load test of Zanzibar OSS impl SpiceDB (similar for OpenFGA**): Graph structure: Social-media-like (Pareto distribution) Graph size: 100,000,000,000 edges (100G) Read load: 1,000,000 permission checks/second (1M) Write load: 1,000 writes/second (1%) Permission check latency*: ~3ms p50, ~6ms p95 (!) Permission write latency: ~16ms p50, ~50ms p95 Authorization Graph is “Extremely Fast”? 75 *Check result here can be cached for at most 10 seconds in this setup **Even without caching, OpenFGA records check latency of ~12ms/p50 & ~20ms/p95

@luxas @luxas ReBAC Datastores vs Policy Engines 76 ReBAC Datastore Policy Engine State Stateful Mostly stateless Response Allow/Deny Any Typing Strong Strong or Dynamic Topology Centralized Server Any (Client, Server, Embed) Policy Updates Via API Out of band Lookup resources Yes No Consistency Strong* Eventual Best for Hierarchical roles or attrs ABAC Preprocessing needed Yes No *If you make your app support it, and back the store with a consistent database

@luxas @luxas API Server Putting it all together 77 User Authenticator Flow Control Authorizer Audit Log

@luxas @luxas API Server Putting it all together 78 User Authenticator ReBAC Server Flow Control Authorizer ReBAC Database Audit Log Check

@luxas @luxas API Server Putting it all together 79 User Authenticator ReBAC Server Flow Control Authorizer Admission ReBAC Database Audit Log Policy Engine Check

@luxas @luxas API Server Putting it all together 80 User Authenticator ReBAC Server Flow Control Authorizer Admission Privilege Escalation Prevention Authorization State Update ReBAC Database Audit Log Policy Engine Check WriteRelations

@luxas @luxas API Server Putting it all together 81 User Authenticator ReBAC Server Flow Control Authorizer Admission Privilege Escalation Prevention Authorization State Update List Filter ReBAC Database Storage Audit Log Policy Engine Check ListResources WriteRelations

@luxas @luxas API Server Putting it all together 82 User Authenticator ReBAC Server Flow Control Authorizer Admission Privilege Escalation Prevention Authorization State Update List Filter ReBAC Database Storage Audit Log Policy Engine Check ListResources WriteRelations

@luxas @luxas API Server Putting it all together 83 User Authenticator ReBAC Server Flow Control Authorizer Admission Privilege Escalation Prevention Authorization State Update List Filter ReBAC Database Storage Audit Log Policy Engine Check ListResources WriteRelations Syncer SaaS/Partners Read data WriteRelations

@luxas @luxas Kubernetes as the generic control plane! Kubernetes provides everything needed for a generic control plane ✅ Extensible, uniform REST API, authn/z, audit, controllers, etc. 84

@luxas @luxas Kubernetes as the generic control plane! Kubernetes provides everything needed for a generic control plane ✅ Extensible, uniform REST API, authn/z, audit, controllers, etc. OPA, Cedar and CEL good implementations of policy engines SpiceDB and OpenFGA are good OSS ReBAC implementations 🙌 You don’t have to make it all yourself! 85

@luxas @luxas Kubernetes as the generic control plane! Kubernetes provides everything needed for a generic control plane ✅ Extensible, uniform REST API, authn/z, audit, controllers, etc. OPA, Cedar and CEL good implementations of policy engines SpiceDB and OpenFGA are good OSS ReBAC implementations 🙌 You don’t have to make it all yourself! Instead let’s put the ready-made components together. 🚀 Join us Upbounders in the OSS community to explore this area! 86

@luxas @luxas Thank you! [email protected] / [email protected] GitHub: @luxas Twitter: @kubernetesonarm LinkedIn: @luxas CNCF/Kubernetes/Crossplane Slack: @luxas 87

@luxas @luxas Other sources (not a fully exhaustive list) - Everything is a graph – Matt Rickard - OpenID Shared Signals Working Group - OpenID AuthZen Working Group - It’s time for Authorization Standards – AuthZEN – CNCF Blog – Omri Gazitt - API Improvement Proposals, on (Control/Data) Planes and Authorization – Google - The Authorization in Software Podcast – Damian Schenkelman - The Cedar Language and Policy Based Authorization – Emina Torlak - All things Rego, OPA and Styra – Tim Hinrichs - NGAC sample implementation (not production ready!) – NIST - Fine-Grained Policies RBAC with NGAC – José Carlos Chávez - Using Policy as Code for Enterprise wide Authorization - Demo – Ignasi Barerra - NGAC demo for Service Mesh – Tetrate - ABAC on SpiceDB: Enabling Netflix’s Complex Identity Types – Chris Wolfe, Joey Schorr, and Victor Roldan Betancort - Pitfalls of JWT Authorization – Jimmy Zelinskie - Policy-Based Access Control (PBAC) vs Google Zanzibar: When You Should Use One or the Other – Evan Cordell - Enforcing Causal Ordering in Distributed Systems: The Importance of Permissions Checking – Jake Moshenko - Higher-level OpenFGA Client – Canonical 88