Yahoo! JAPANにおけるAthenzを用いたk8sアクセス制御管理 @JapanContainerDays

Transcript

1. Yahoo! JAPANʹ͓͚ΔAthenz Λ༻͍ͨk8sΞΫηε੍ޚ؅ཧ Japan Container Days, Dec 5, 2018
2. Yusuke Kato <yusukato@yahoo-corp.jp> @kpango Yahoo! JAPAN CTO Office SWAT Section / Language Support Go Team
3. About Yahoo! JAPAN
4. About Yahoo! JAPAN 1. On-premise Servers 2. ࣾ಺ʹ͸Kubernetes as a ServiceͱPaaS྆ํ͕͋Δ 3. k8sΫϥελͷ਺͸ࠓޙ΋૿Ճ͢ΔͱΈΒΕ͍ͯΔɻ 4. ࣾ಺ͷKubernetesΫϥελͷ਺͸175Ϋϥελ (2018/12/3࣌఺) 5. ࣾ಺ͷAPI΍PlatformͷೝূೝՄʹ͸
 Yahoo Inc. (ݱOath Inc.)͕OSSԽͨ͠AthenzΛར༻
5. Kubernetes Extensibility
6. Kubernetes Extensibility ▸ Policy API ▸ ResourceQuota, PodSecurityPolicies, NetworkPolicy, RBAC ▸ API Extension ▸ Custom Resource, Operator, 
 Authentication (Token Review API), 
 Authorization (Subject Access Review API),
 Admission Control ▸ Infrastructure Extension ▸ Network Plugin (CNI), Storage Plugin (CSI), 
 Device Plugin (GPU, FPGA, RDMA) , 
 Container Runtime (docker, cri-o, containerd, rkt) ▸ Kubectl Plugin
7. About k8s authorization mode
8. Kubernetes Authorization Mode ▸ AlwaysAllow / AlwaysDeny ▸ ৗʹAllow / Deny͢ΔΞΫηείϯτϩʔϧ ▸ Node ▸ kubelet͔ΒͷϦΫΤετʹରͯ͠ΞΫηείϯτϩʔϧΛߦ͏ ▸ ABAC ▸ Attribute-based Access Control, ϦΫΤετ͞ΕͨଐੑͱPolicyͷத਎Λൺֱ͠Ϛον͢Δ ΋ͷ͕͋Ε͹ೝՄɻ ▸ RBAC ▸ Role-based Access Control, Role΍ClusterRoleͱPolicyΛBind͠ΞΫηεͰ͖ΔResource ͳͲΛίϯτϩʔϧ͍ͯ͠Δɻ ▸ Webhook ▸ SubjectAccessReviewͷ಺༰͕WebhookͰඈΜͰ͘ΔͷͰೝՄͷϩδοΫΛಠࣗʹఆٛͰ ͖Δɻ
9. About k8s webhook authorization mode
10. Kubernetes Webhook Authorization Mode ▸ WebhookϦΫΤετϖΠϩʔυ͸ SubjectAccessReviewΦϒδΣΫτ ▸ ResponseϖΠϩʔυͷstatus.allowedʹ
 true / falseΛฦ͢͜ͱͰίϯτϩʔϧ͢Δ ▸ resourceAttributesϑΟʔϧυʹ͸ଞʹɺ
 ResourceName, SubResourceͳͲ΋͋Δ
11. About Athenz
12. About Athenz ▸ Service Authentication ▸ Ϟμϯͳ؀ڥͷ͋ΒΏΔWorkload/Serviceʹ͓͍ͯx.509 certificateΛ༻͍ͨ҆શͳidentity authenticationΛఏڙ ▸ Authorization ▸ Provides fine-grained Role Based Access Control (RBAC)
13. Why Athenz
14. Single source of truth ▸ AWS, GCP΍Kubernetes OpenStack ͳͲͷΫϥ΢υ؀ڥͷଟ ͘͸͢ͰʹݸʑʹΞΫηε੍ޚͷػೳΛ͍࣋ͬͯΔ ▸ Athenz͸༷ʑͳ؀ڥͰಈ࡞͠ΞΫηε੍ޚ৘ใͷSSoT ʢSingle Source of TruthʣΛ࣮ݱ͢Δࣄ͕Ͱ͖Δ Cloud computing environments OpenStack Kubernetes Screwdriver Amazon EC2 AWS ECS AWS Lambda
15. About Garm
16. About Garm ▸ k8s webhook authorizationΛར༻͢Δ͜ͱͰ
 AthenzʹauthorizationΛҕৡ͢Δ ▸ golang੡ ▸ yahoo/k8s-athenz-webhookͷϥΠϒϥϦΛར༻ ▸ yamlΛ༻͍ͨॊೈͳઃఆ ▸ k8s SubjectAccessReviewΛ
 AthenzͷData Modelʹॊೈʹม׵ ▸ ୯ମͰblack list / exclude black listػೳ ▸ Kubernetes as a Service޲͚ͷAuthorizationػೳ
17. About Garm 1. WebhookʹΑΓSubjectAccessReview͕ૹΒΕͯ͘Δɻ 2. SubjectAccessReviewΛconfig.yamlΛݩʹɺAthenz޲͚ʹม ׵Λߦ͍Athenzʹ໰͍߹ΘͤΔɻ
18. ▸ Kubernetes RBACͱAthenz DataModelͷൺֱ About Garm Kubernetes Athenz Verb get, list, watch, create, update Action get, list, watch, create, update Namespace kube-system, default, … SubDomain {Top Level Domain}.{namespace} ex. athenz.kube-system API Group apps, autoscaling, extensions … Resource {API Group}.{Resource}.{Name} ex. apps.deployments.garm Resource pods, configmaps, replicasets, deployments… Name garm, istio, prometheus …
19. Cluster Kube API Server About Garm Admin Domain Deny: kube-system’s secret User Domain KaaS Admin k8s user kube-system’s secret is kaas admin resource kubectl get secret -n kube-system kubectl get secret
20. Garm Overview
21. kube-api-server send webhook to Garm Garm maps SubjectAccessReview to Athenz Style Garm checks the Exclude White List Garm checks the Admin Access List About Garm Garm decides the Athenz Domain Garm Send AccessCheck Request to Athenz Athenz Domain checks Policy & Principal & Role If Exists In Black List If Exists In Exclude List Garm checks the Black List If Not Exists In Black List Garm API
22. Garm Future Work
23. Future Work ▸ Authenticationʹ΋ରԠ ▸ Garm <=> AthenzؒͷmTLSରԠ ▸ ෳ਺ͷUser DomainରԠ
24. Wrap up
25. Wrap up ▸ Yahoo! JAPANͰ͸਺ଟ͘ͷKubernetesΫϥελΛ؅ཧ͢Δඞ ཁ͕͋Δ ▸ Kubernetes ExtensibilityͷػೳͷதͷҰͭɺAuthorization WebhookΛར༻ ▸ GarmʹΑΓ਺ඦ୆ن໛ͷKubernetsΫϥελͷΞΫηεೝՄ ΛAthenzΛ༻͍ͯSSoT؅ཧ
26. One more thing
27. Yahoo! JAPAN @ Kubecon NA 2018 AthenzͱistioΛ࿈ܞͤͯ͞APIΞΫηεͷ ৄࡉͳίϯτϩʔϧΛ࣮ݱ Kubernetes ্Ͱಈ࡞͠ Service Discoverer ͱ
 L7 Loadbalancer ͷػೳΛఏڙ͢ΔGimbalͷ঺հ
28. Thank you!
29. About Athenz ▸ Athenz Website: http://athenz.io ▸ Athenz Github: https://github.com/yahoo/athenz ▸ Garm Github: https://github.com/yahoojapan/garm (ۙ೔ެ։) ▸ Athenz Slack: https://athenz.slack.com/ ▸ Questions or Comments: yusukato@yahoo-corp.jp
30. Appendix
31. Athenz Authorization
32. About Athenz ▸ Centralized Access Control ▸ Decentralized Access Control
33. Athenz Authorization Centralized Access Control
34. Athenz Authorization Decentralized Access Control
35. EOP