Zero Trust for APIs: From Edge to Mesh with Istio

Securing modern infrastructure by assuming breach

Transcript

1. Zero Trust for APIs: From Edge to Mesh with Istio MOFESOLA BABALOLA Staff Site Reliability Engineer linkedin.com/in/mofesola Securing modern infrastructure by assuming breach
2. THE AGENDA 1. The SRE's Problem: The Flaw in the "Castle-and-Moat" 2. Anatomy of a Breach: A Real-World Case Study 3. The Solution, A new mindset: "Never Trust, Always Verify" 4. The Platform: Istio & The Rise of Ambient Mesh 5. The 3 Pillars: A Practical Implementation * Identity * Authentication * Authorization 6. The Payoff: Why This Makes Your Life Better
3. The SRE's Problem: The "Castle-and-Moat" 01 02 03 Firewalls, VPNs, Load Balancers. Strong Perimeter We focus all our effort on the "edge." Hard Shell, Soft Center Implicit Trust inside the perimeter The Critical Flaw
4. The Flaw: Unsecured "East-West" Traffic If a single non-critical service is breached, it has a trusted network path to all other services. Lateral Movement
5. Case Study: Anatomy of a Lateral Movement Breach ❖ Frontend-service: Public-facing web app. ❖ Reviews-service: A simple, non-critical service with a Log4Shell vulnerability. ❖ Customer-api: A high-value internal service. It should only be called by the frontend-service. Scenario
6. Case Study: The Breach 1. Foothold: Attacker gets RCE inside the reviews-service pod via Log4Shell. 2. Recon: Attacker finds 3. Lateral Movement: Attacker runs a simple curl: 4. Failure: In a "castle-and-moat" network, this works. DATA IS EXFILTRATED. http://customer-api.default.svc.cluster.local. [attacker@reviews-service]$ curl http://customer-api/api/v1/users/export
7. Breach Averted (Defense 1: L4 mTLS) Attacker tries the simple curl... [attacker@reviews-service]$ curl http://customer-api/api/v1/users/export curl: (56) Recv failure: Connection reset by peer ❖ What happened? The attacker's plain HTTP request was not using mTLS. ❖ Istio's Action: Our cluster-wide PeerAuthentication policy (Pillar 1) requires mTLS for all traffic. ❖ Result: The Ztunnel (L4) on the customer-api's node saw a non-mTLS request and immediately dropped the TCP connection. ❖ Key Point: The application was never touched. Explanation
8. The Sophisticated Attacker (Inside the Pod) Key Question: "What if the attacker uses the pod's own identity against us?" Key Point: The attacker has RCE. They can access everything the pod can. This includes its own cryptographic identity. Attacker's Recon: [attacker@reviews-service]$ ls -l /var/run/secrets/tokens/ total 0 lrwxrwxrwx 1 root root 15 Nov 17 20:30 istio-token -> ..data/istio-token [attacker@reviews-service]$ ls -l /var/run/secrets/istio/ total 0 lrwxrwxrwx 1 root root 17 Nov 17 20:30 cert-chain.pem -> ..data/cert-chain.pem lrwxrwxrwx 1 root root 11 Nov 17 20:30 key.pem -> ..data/key.pem lrwxrwxrwx 1 root root 15 Nov 17 20:30 root-ca.crt -> ..data/root-ca.crt
9. Sophisticated Attack (Step 1: Steal the Identity) The attacker reads the JWT token from the filesystem. # Attacker reads the token file and saves it [attacker@reviews-service]$ TOKEN=$(cat /var/run/secrets/tokens/istio-token) # This $TOKEN is a valid JWT representing the # 'reviews-service' identity Key Point: This token, combined with the mTLS certs, will bypass our L4 mTLS defense. The request will be valid mTLS.
10. Sophisticated Attack (Step 2: The curl) The attacker uses the pod's identity to make a valid mTLS request. What this command does: * --cacert, --key, --cert: Performs a valid mTLS handshake as the reviews-service. * -H "Authorization: Bearer $TOKEN": Passes the valid identity token. * This request will pass L4 (Ztunnel) and go to the L7 Waypoint Proxy. [attacker@reviews-service]$ curl \ --cacert /var/run/secrets/istio/root-ca.crt \ --key /var/run/secrets/istio/key.pem \ --cert /var/run/secrets/istio/cert-chain.pem \ -H "Authorization: Bearer $TOKEN" \ https://customer-api/api/v1/users/export
11. Sophisticated Attack (Step 3: The Response) What the attacker sees... RBAC: access denied Key Point: The server is istio-envoy. The customer-api app never saw the request. HTTP/1.1 403 Forbidden content-length: 19 content-type: text/plain date: Mon, 17 Nov 2025 20:50:00 GMT server: istio-envoy
12. How It Was Blocked: The AuthorizationPolicy This is the exact policy that generated that 403 Forbidden. apiVersion: security.istio.io/v1 kind: AuthorizationPolicy metadata: name: "customer-api-lockdown" spec: selector: matchLabels: istio.io/gateway-name: "customer-api" # Applied to the waypoint action: ALLOW rules: - from: - source: # ALLOWED IDENTITY: principals: - "cluster.local/ns/default/sa/frontend-service"
13. How It Was Blocked: The AuthorizationPolicy The Logic (Step-by-Step): * The Waypoint Proxy received the attacker's valid mTLS request. * It extracted the identity: cluster.local/ns/default/sa/reviews-service * It checked this policy. The policy only allows the frontend-service principal. * Identity did not match. * Result: Access Denied.
14. The Solution: A New Mindset "Never Trust, Always Verify." This is the core principle of Zero Trust. Key Points: * Assume the network is always hostile. * Assume breach. * Every single request must be verified.
15. Zero Trust: The 3 Practical Requirements For every API call, we must be able to answer * Verifiable Identity: Who are you? (Service-to-Service) * Verifiable Authentication: Who is the end-user? (User-to-Service) * Explicit Authorization: What are you allowed to do? (Policy)
16. The Platform Using Istio Ambient Mesh to Enforce Policy
17. The "Sidecar-less" Future: Ambient Mode * The Old Way: Sidecars (1 proxy per pod). Heavy & complex. * The New Way: Ambient Mesh (Sidecar-less) * Ztunnel (L4): A per-node agent. Handles Pillar 1 (Identity & mTLS) for all pods on that node. Very lightweight. * Waypoint (L7): An optional, per-service-account proxy. Handles Pillars 2 & 3 (Authn & Authz) only where you need it. * Best of both worlds: Secure-by-default, low-overhead, and incrementally adoptable.
18. Pillar 1: Verifiable Identity (mTLS) This is the code for our cluster-wide L4 defense apiVersion: security.istio.io/v1 kind: PeerAuthentication metadata: name: "default" namespace: "istio-system" spec: mtls: mode: STRICT
19. Pillar 1: Verifiable Identity (mTLS) Who: Enforced by the Ztunnel. What: Provides every service with a SPIFFE identity. Result: Encrypts all East-West traffic and stops basic IP spoofing.
20. Pillar 2: Verifiable Authentication (JWT) Offload end-user JWT validation to the platform. apiVersion: security.istio.io/v1 kind: RequestAuthentication metadata: name: "jwt-validator" spec: selector: matchLabels: istio.io/gateway-name: "customer-api" jwtRules: - issuer: "https://auth.your-company.com" jwksUri: "https://.../.well-known/jwks.json"
21. Pillar 2: Verifiable Authentication (JWT) * Who: Enforced by the Waypoint Proxy. * What: Checks the end-user's token (signature, expiration). * Result: App code no longer needs verify_jwt() boilerplate.
22. Pillar 3: Explicit Authorization This is the breach-stopping policy we just demonstrated. apiVersion: security.istio.io/v1 kind: AuthorizationPolicy metadata: name: "customer-api-lockdown" spec: selector: matchLabels: istio.io/gateway-name: "customer-api" action: ALLOW rules: - from: - source: principals: - "cluster.local/ns/default/sa/frontend-service"
23. Pillar 3: Explicit Authorization * Who: Enforced by the Waypoint Proxy. * What: Declarative, identity-aware access control. * Result: The blast radius is contained.
24. The Payoff Why does this Makes Your Life Better?
25. The Payoff (It's Not Just Security) ❖ For Security: Contain the blast radius. Move from "breach prevention" to "breach containment." ❖ For SREs: Uniform Observability. You can log every allowed and denied request. (A spike in 403s is a critical signal!) ❖ For Developers: Ship features faster. Stop writing security boilerplate. Security is now an enabling feature, not a bottleneck.
26. Your Zero Trust Journey ❏ Evaluate: Is a mesh right for you? (Read "To Mesh or Not to Mesh?") ❏ Crawl (L4): Install Istio Ambient. Enable PeerAuthentication in PERMISSIVE mode. Just observe. ❏ Walk (L4): Move critical services to STRICT mTLS. ❏ Run (L7): Deploy a WaypointProxy for one critical service and add a strict AuthorizationPolicy. ❏ Iterate and expand. This is a journey, not a switch.
27. linkedin.com/in/mofesola LinkedIn mofesola.me Website Thank You, Cloud Native Dublin! Any Questions? Scan the QR Code to read more of my full article on this