Isolated multiple trust domain mTLS in Envoy and Istio

@SPIFFE Meetup Tokyo #3 https://spiffe-jp.connpass.com/event/205213

1. Takeshi Yoneda, Software Engineer, Tetrate.io SPIFFE Meetup Tokyo #3 Isolated multiple trust domain mTLS in Envoy and Istio
2. • Takeshi Yoneda (Ϛελέ) / Twitter, Github: @mathetake • Software Engineer at Tetrate.io, California, US • OSS dev: Envoy, Istio, Proxy-Wasm, Wasm, TinyGo • C++ committer of Proxy-Wasm project • Contributor/Member of V8, Envoy, Istio, TinyGo, etc. whoami
3. 1. Introduction to SPIFFE 2. Introduction to Service Mesh 3. Introduction to mTLS 4. mTLS in Envoy / Istio 5. SPIFFE Certificate Validator in Envoy 6. Independent multiple trust domain support in Istio Agenda
4. 1. Introduction to SPIFFE
5. • SPIFFE = “Secure Production Identity Framework For Everyone” • Identityͱͦͷೝূʹؔ͢Δඪ४࢓༷ SPIFFE = Specification
6. • ࠓ೔ؔ܎͢Δ࢓༷ • SPIFFE Identity • SVID(SPIFFE Verifiable Identity Document) • x.509 SVID SPIFFE = Specification
7. • ࠓ೔ؔ܎͢Δ࢓༷ • SPIFFE Identity • SVID(SPIFFE Verifiable Identity Document) • x.509 SVID SPIFFE = Specification
8. • SPIFFE Identity = ݸʑͷWorkloadΛࣝผ͢ΔͨΊͷID • “spiffe://trust-domain-name/your/workload“ ͷܗͷURI • “trust-domain-name” = Trust Root • “your/workload” = Trust Root಺ͷWorkload • ྫ) spiffe://my-app.com/ns/kube-system/sa/my-service-account SPIFFE Identity
9. • SVID = ݸʑͷWorkload͕ࣗ਎Λূ໌͢ΔͨΊͷݕূՄೳͳυΩϡϝϯτ • ҎԼͷ3͔ͭΒߏ੒ • A SPIFFE ID • A Valid Signature • (Optional) Public key SVID(SPIFFE Verifiable Identity Document)
10. • x.509 SVID = SVIDͷ࣮૷(ܗࣜ)ͷҰͭ • x.509ূ໌ॻͷ֦ு࢓༷Λ༻͍Δ • URI SAN(Subject Alternative Name)ΛͨͩҰ͚ͭͩ࣋ͭ • URI SANͷ஋͕SVID (e.g. “spiffe://my-domain/my/workload”) ͷܗ x.509 SVID
11. • x.509 SVIDͷݕূ͸௨ৗͷPKIͱಉ͡ • URI SANʹ੍໿͕͋Δ͚ͩ • x.509 SVIDΛʹॺ໊ͨ͠Trust DomainͷRoot CAͰݕূ͢ΔͷΈ • طଘͷTLSͷΠϯϑϥʹ৐͔ͬΕΔ x.509 SVID
12. x.509 SVID https://thinkit.co.jp/sites/default/files/article_node/zl_kubernetes_07_04.png
13. 2. Introduction to Service Mesh
14. • Polyglot • Multiple Protocol • Observability • AuthN/Z Problems in Microservices https://blogs.vmware.com/networkvirtualization/2018/12/nsx-service-mesh.html/
15. • Service Mesh = Microservices؀ڥԼͷΞʔΩςΫνϟͷҰछ • αʔϏε΁ͷ ingress/egreeΛ͢΂ͯϓϩΩγܦ༝ʹ͢Δ Service Mesh = Architecture https://blog.envoyproxy.io/service-mesh-data-plane-vs-control-plane-2774e720f7fc
16. • τϥϑΟοΫ͕͢΂ͯproxyΛܦ༝͢Δ͜ͱͷԸܙ • AuthN/ZΛΞϓϦέʔγϣϯ͔Β෼཭ • ϓϩτίϧͷtranscodingΛΞϓϦέʔγϣϯ͔Β෼཭ • Retry/RatelimitͳͲΛΞϓϦέʔγϣϯ͔Β෼཭ • Ұ؏ͨ͠Metrics/Logͷऔಘ(Observability++) Service Mesh = Architecture
17. • τϥϑΟοΫ͕͢΂ͯproxyΛܦ༝͢Δ͜ͱͷԸܙ • AuthN/ZΛΞϓϦέʔγϣϯ͔Β෼཭ • ϓϩτίϧͷtranscodingΛΞϓϦέʔγϣϯ͔Β෼཭ • Retry/RatelimitͳͲΛΞϓϦέʔγϣϯ͔Β෼཭ • Ұ؏ͨ͠Metrics/Logͷऔಘ(Observability++) Service Mesh = Architecture ωοτϫʔΫϨΠϠʔͷ΋Ζ΋ΖΛશ෦ϓϩΩγͰ
18. • ֤αʔϏε͕όϥόϥʹProxyͷઃఆΛ͍ͯͯ͠͸ແବ͕ଟ͍ • Control Plane = தԝूݖతʹ֤αʔϏεͷProxyΛ؅ཧ͢Δਓ Control Plane in Service Mesh https://www.weave.works/blog/introduction-to-service-meshes-on-kubernetes-and-progressive-delivery
19. • Data Plane = Control Planeʹ؅ཧ͞ΕΔProxyୡ • ඞવతʹҰछྨͷProxy ServerʹͳΔ Data Plane in Service Mesh https://www.weave.works/blog/introduction-to-service-meshes-on-kubernetes-and-progressive-delivery
20. What is Envoy? • “Cloud-native high-performance edge/middle/service proxy” • CNCF Graduated Project, Github Star: 16,000+ • Written in C++
21. • ϦϞʔτͰಈతʹઃఆΛมߋ͢ΔxDSͱ͍͏ϓϩτίϧΛ࣋ͭ • EnvoyΛData Planeͱ͓ͯ͠खܰService Mesh͕࡞ΕΔ • xDS ServerΛ࣮૷͠Control Planeͱ͢Ε͹Α͍ Envoy as a Data Plane https://i-beam.org/2019/01/22/hello-envoy/
22. What is Istio? • xDS ServerΛ࣮૷ͨ͠Control PlaneͷҰͭ • GitHub Star 20k+ • ੈքதͷ໊ͩͨΔاۀ͕ຊ৔ӡ༻ https://www.suse.com/c/understanding-istio-and-its-installation/ https://github.com/istio/istio
23. 3. Introduction to mTLS
24. Problems in Zero-Trust env • Man in the Middle attack • ͢΂ͯͷ௨৴Λ҉߸Խ͍ͨ͠ • ʮ௨৴૬ख͸ຊ౰ʹαʔϏεhogeͳͷ͔?ʯ • ͔֬ΊΔखஈ͕ඞཁ: PKI?
25. TLS: Authenticating Servers • ௨ৗͷαʔό<->ΫϥΠΞϯτͷ௨৴ͷ৔߹ • αʔόʔͷূ໌ॻΛΫϥΠΞϯτͷखݩʹ͋Δϧʔτূ໌ॻͰݕূ • ݕূࣦഊ͢Ε͹ϋϯυγΣΠΫࣦഊ ূ໌ॻ͘Ε ূ໌ॻͰ͢ google.com ݕূ ͔֬ʹgoogleͬΆ͍͔Βݕࡧ݁Ռ͘Ε
26. Problems in Zero-Trust env • ௨ৗͷTLSͩͱ৺΋ͱͳ͍ • ࿩͔͚ͯ͘͠Δ૬ख͕୭ͳʹ͔Λݕূ͠ͳ͍ͱҙຯ͕ͳ͍ • ΫϥΠΞϯτ΋ೝূ͠ͳ͚Ε͹ͳΒͳ͍
27. mTLS = mutual TLS • ϋϯυγΣΠΫ࣌ʹΫϥΠΞϯτʹ΋ূ໌ॻͷఏࣔΛཁٻ • ΫϥΠΞϯτ͚ͩͰ͸ͳ͘αʔόʔ΋ূ໌ॻΛݕূ • OK: TLSηογϣϯཱ֬ • NG: ϋϯυγΣΠΫࣦഊ
28. Handshake in mTLS https://www.slideshare.net/lmeirosu/mtls-securing-microservice-architecture-with-mutual-tls-authentication
29. Handshake in mTLS https://www.slideshare.net/lmeirosu/mtls-securing-microservice-architecture-with-mutual-tls-authentication αʔόʔೝূ
30. Handshake in mTLS https://www.slideshare.net/lmeirosu/mtls-securing-microservice-architecture-with-mutual-tls-authentication ΫϥΠΞϯτೝূ
31. Mutual TLS in Service Mesh • Control Plane͔Β֤Data Plane(= ֤Workload/Service) ΁ূ໌ॻΛൃߦ • Control Plane͕Root CAΛ؅ཧͯ͠ॺ໊ • ূ໌ॻͷ಺༰ΛݩʹAuthN/ZΛ੍ޚ͢Δ https://speakerdeck.com/hannaprinz/service-mesh-fixing-microservice-architecture-for-good Control Plane
32. 4. mTLS AuthZ in Envoy / Istio
33. mTLS in Envoy • validation_contextͱ͍͏API͕ଘࡏ • ͲͷΑ͏ʹΫϥΠΞϯτূ໌ॻΛݕূ͢Δ͔ΛࢦఆͰ͖Δ
34. mTLS in Istio • Istiod(Control Plane)͕֤Envoyʹূ໌ॻΛ഑Δ • ಈతʹEnvoyͷvalidation_context͕ઃఆ͞ΕmTLS͕ୡ੒ • ֤ূ໌ॻ -> Workload Identity https://istio.io/latest/docs/concepts/security/#authorization-architecture
35. Istio as a SVID issuer • Istioͷ֤Workload Identity͸SPIFFE x.509 SVIDʹ४ڌ • URI SAN͕ͨͩҰͭଘࡏ(= SVID) • spiffe://<trust-domain>/ns/<namespace>/sa/<service-account> • SVIDͷத਎ʹService Account΍Namespaceؚ͕·Ε͍ͯΔ
36. SAN Matching in Envoy • ΫϥΠΞϯτূ໌ॻͷSANͷmatchingΛߦ͑Δ • Match͠ͳ͍৔߹͸ϋϯυγΣΠΫࣦഊͱ͍͏ڍಈ
37. mTLS + SAN Matching + SVID = ! • ֤Workload͸·ͣSVIDΛݕূ • ݕূޙSVIDͷத਎(=URI SAN)ΛऔΓग़͢ • Service Account΍Namespace͕Θ͔Δ • SAN MatchingΛk8sͷSA΍NSΛݩʹઃఆ • WorkloadϨϕϧͰࡉ͔͍ೝূೝՄ͕ୡ੒Ͱ͖Δ
38. Problems in Multi-cluster cases • Istio: k8s cluster = 1: 1ͱ͍͏ؔ܎ੑ • ෳ਺ͷIstioͷΫϥελ͕૬ޓʹmTLS͍ͨ͠৔߹…? • ྡͷΫϥελʔ͔ΒTrust Bundle(Root CA)Λऔಘ
39. Problems in Multi-cluster cases • ϋϯυγΣΠΫຖʹTrust BundleΛ੾Γସ͑ͳ͍ͱ͍͚ͳ͍ • ੾Γସ͑ͣʹTrust BundleΛࠞͥͨΒͲ͏ͳΔ͔ • ΫϥελʔA -> ΫϥελʔBʹ࿩͔͚͠Δέʔε • AͷWorkload͕Bͷ಺෦ͷWorkloadͷ;ΓΛ͢Δ͔΋͠Εͳ͍ • ೝՄΛ͢Γൈ͚ͯ͠·͏ • Trust domains are not isolated from each other.
40. Problems in Envoy listener for Multi-cluster • validation_context͕ෳ਺ͷTrust DomainΛαϙʔτ͍ͯ͠ͳ͔ͬͨ • Trust BundleΛϚʔδͯ͠Ұͭʹ·ͱΊͳ͍ͱ͍͚ͳ͍
41. 5. SPIFFE Certificate Validator in Envoy
42. SPIFFE Certificate Validator • ʮෳ਺ͷTrust domainΛಠཱͨ͠ܗͰݕূ͢Δ࢓૊Έʯ͕ඞཁ • SPIFFE Certificate ValidatorͱݺͿ͜ͱʹ • ઌ೔4ͭͷPRΛܦ࣮ͯ૷͠·ͨ͠ (߹Θͤͯ5000ߦ͙Β͍)
43. SPIFFE Certificate Validator
44. SPIFFE Certificate Validator: How it works 1. ΫϥΠΞϯτূ໌ॻͷSVIDΛऔΓग़͢ 2. SVID͔ΒTrust Domain (spiffe://ͷޙΖͷ෦෼) 3. Trust DomainʹରԠ͢ΔTrust BundleΛબ୒ 4. બ͹ΕͨTrust BundleΛݩʹূ໌ॻΛݕূ 5. ϋϯυγΣΠΫ׬ྃ
45. Reviewed by SPIFFE maintainers 🎉
46. 6. Independent multiple trust domain in Istio
47. Isolated cross-cluster mTLS in Istio • ݱঢ়αϙʔτ͍ͯ͠ͳ͍ • Root CA͸ඞͣෳ਺ͷΫϥελͰγΣΞ͞Ε͍ͯΔͱ͍͏લఏ • ΫϥελA͕ΫϥελBͷ;ΓΛग़དྷͯ͠·͏
48. Using SPIFFE Validator in Istio (WIP) • Envoyʹ࣮૷ͨ͠SPIFFE ValidatorΛ࢖͏͜ͱΛఏҊத
49. Using SPIFFE Validator in Istio (WIP) Root CAΛ௥Ճ͢ΔࡍʹTrust DomainΛׂΓ౰ͯΒΕΔΑ͏ʹ͢Δ
50. Independent multiple trust domain in Istio(WIP) Cluster B Bundle Endpoint Bundle Endpoint Cluster A mTLS GET Bundle mTLS
51. ·ͱΊ
52. ·ͱΊ • Service Mesh = αʔϏεؒ௨৴ΛϓϩΩγʹ೚ͤΔΞʔΩςΫνϟ • mTLS͸Service MeshͰॏཁ: SAN Matchingͱ߹ΘͤͯೝՄʹ΋࢖͑Δ • Istio͸x.509 SVIDͷissuer • Envoyͷ৽ػೳSPIFFE Validator • ෳ਺ͷTrust DomainΛҰͭͷListenerͰ҆શʹೝূͰ͖Δ • Cross-ClusterͳmTLSͷ࣮ݱͷͨΊͷجૅٕज़ • IstioͰ͜ΕΛԠ༻ͨ͠Cross-ClusterೝূೝՄͷ࢓૊ΈΛ࣮૷/ఏҊத
53. We are hiring! https://www.tetrate.io/careers/ Work Anytime and Anywhere + Unlimited paid time off