$30 off During Our Annual Pro Sale. View Details »

Yahoo! JAPANにおけるAthenzを用いたk8sアクセス制御管理 @JapanContainerDays

Yusuke Kato
December 05, 2018

Yahoo! JAPANにおけるAthenzを用いたk8sアクセス制御管理 @JapanContainerDays

Yusuke Kato

December 05, 2018
Tweet

More Decks by Yusuke Kato

Other Decks in Technology

Transcript

  1. Yahoo! JAPANʹ͓͚ΔAthenz Λ༻͍ͨk8sΞΫηε੍ޚ؅ཧ Japan Container Days, Dec 5, 2018

  2. Yusuke Kato <yusukato@yahoo-corp.jp> @kpango Yahoo! JAPAN CTO Office SWAT Section

    / Language Support Go Team
  3. About Yahoo! JAPAN

  4. About Yahoo! JAPAN 1. On-premise Servers 2. ࣾ಺ʹ͸Kubernetes as a

    ServiceͱPaaS྆ํ͕͋Δ 3. k8sΫϥελͷ਺͸ࠓޙ΋૿Ճ͢ΔͱΈΒΕ͍ͯΔɻ 4. ࣾ಺ͷKubernetesΫϥελͷ਺͸175Ϋϥελ (2018/12/3࣌఺) 5. ࣾ಺ͷAPI΍PlatformͷೝূೝՄʹ͸
 Yahoo Inc. (ݱOath Inc.)͕OSSԽͨ͠AthenzΛར༻
  5. Kubernetes Extensibility

  6. Kubernetes Extensibility ▸ Policy API ▸ ResourceQuota, PodSecurityPolicies, NetworkPolicy, RBAC

    ▸ API Extension ▸ Custom Resource, Operator, 
 Authentication (Token Review API), 
 Authorization (Subject Access Review API),
 Admission Control ▸ Infrastructure Extension ▸ Network Plugin (CNI), Storage Plugin (CSI), 
 Device Plugin (GPU, FPGA, RDMA) , 
 Container Runtime (docker, cri-o, containerd, rkt) ▸ Kubectl Plugin
  7. About k8s authorization mode

  8. Kubernetes Authorization Mode ▸ AlwaysAllow / AlwaysDeny ▸ ৗʹAllow /

    Deny͢ΔΞΫηείϯτϩʔϧ ▸ Node ▸ kubelet͔ΒͷϦΫΤετʹରͯ͠ΞΫηείϯτϩʔϧΛߦ͏ ▸ ABAC ▸ Attribute-based Access Control, ϦΫΤετ͞ΕͨଐੑͱPolicyͷத਎Λൺֱ͠Ϛον͢Δ ΋ͷ͕͋Ε͹ೝՄɻ ▸ RBAC ▸ Role-based Access Control, Role΍ClusterRoleͱPolicyΛBind͠ΞΫηεͰ͖ΔResource ͳͲΛίϯτϩʔϧ͍ͯ͠Δɻ ▸ Webhook ▸ SubjectAccessReviewͷ಺༰͕WebhookͰඈΜͰ͘ΔͷͰೝՄͷϩδοΫΛಠࣗʹఆٛͰ ͖Δɻ
  9. About k8s webhook authorization mode

  10. Kubernetes Webhook Authorization Mode ▸ WebhookϦΫΤετϖΠϩʔυ͸ SubjectAccessReviewΦϒδΣΫτ ▸ ResponseϖΠϩʔυͷstatus.allowedʹ
 true

    / falseΛฦ͢͜ͱͰίϯτϩʔϧ͢Δ ▸ resourceAttributesϑΟʔϧυʹ͸ଞʹɺ
 ResourceName, SubResourceͳͲ΋͋Δ
  11. About Athenz

  12. About Athenz ▸ Service Authentication ▸ Ϟμϯͳ؀ڥͷ͋ΒΏΔWorkload/Serviceʹ͓͍ͯx.509 certificateΛ༻͍ͨ҆શͳidentity authenticationΛఏڙ ▸

    Authorization ▸ Provides fine-grained Role Based Access Control (RBAC)
  13. Why Athenz

  14. Single source of truth ▸ AWS, GCP΍Kubernetes OpenStack ͳͲͷΫϥ΢υ؀ڥͷଟ ͘͸͢ͰʹݸʑʹΞΫηε੍ޚͷػೳΛ͍࣋ͬͯΔ

    ▸ Athenz͸༷ʑͳ؀ڥͰಈ࡞͠ΞΫηε੍ޚ৘ใͷSSoT ʢSingle Source of TruthʣΛ࣮ݱ͢Δࣄ͕Ͱ͖Δ Cloud computing environments OpenStack Kubernetes Screwdriver Amazon EC2 AWS ECS AWS Lambda
  15. About Garm

  16. About Garm ▸ k8s webhook authorizationΛར༻͢Δ͜ͱͰ
 AthenzʹauthorizationΛҕৡ͢Δ ▸ golang੡ ▸

    yahoo/k8s-athenz-webhookͷϥΠϒϥϦΛར༻ ▸ yamlΛ༻͍ͨॊೈͳઃఆ ▸ k8s SubjectAccessReviewΛ
 AthenzͷData Modelʹॊೈʹม׵ ▸ ୯ମͰblack list / exclude black listػೳ ▸ Kubernetes as a Service޲͚ͷAuthorizationػೳ
  17. About Garm 1. WebhookʹΑΓSubjectAccessReview͕ૹΒΕͯ͘Δɻ 2. SubjectAccessReviewΛconfig.yamlΛݩʹɺAthenz޲͚ʹม ׵Λߦ͍Athenzʹ໰͍߹ΘͤΔɻ

  18. ▸ Kubernetes RBACͱAthenz DataModelͷൺֱ About Garm Kubernetes Athenz Verb get,

    list, watch, create, update Action get, list, watch, create, update Namespace kube-system, default, … SubDomain {Top Level Domain}.{namespace} ex. athenz.kube-system API Group apps, autoscaling, extensions … Resource {API Group}.{Resource}.{Name} ex. apps.deployments.garm Resource pods, configmaps, replicasets, deployments… Name garm, istio, prometheus …
  19. Cluster Kube API Server About Garm Admin Domain Deny: kube-system’s

    secret User Domain KaaS Admin k8s user kube-system’s secret is kaas admin resource kubectl get secret -n kube-system kubectl get secret
  20. Garm Overview

  21. kube-api-server send webhook to Garm Garm maps SubjectAccessReview to Athenz

    Style Garm checks the Exclude White List Garm checks the Admin Access List About Garm Garm decides the Athenz Domain Garm Send AccessCheck Request to Athenz Athenz Domain checks Policy & Principal & Role If Exists In Black List If Exists In Exclude List Garm checks the Black List If Not Exists In Black List Garm API
  22. Garm Future Work

  23. Future Work ▸ Authenticationʹ΋ରԠ ▸ Garm <=> AthenzؒͷmTLSରԠ ▸ ෳ਺ͷUser

    DomainରԠ
  24. Wrap up

  25. Wrap up ▸ Yahoo! JAPANͰ͸਺ଟ͘ͷKubernetesΫϥελΛ؅ཧ͢Δඞ ཁ͕͋Δ ▸ Kubernetes ExtensibilityͷػೳͷதͷҰͭɺAuthorization WebhookΛར༻

    ▸ GarmʹΑΓ਺ඦ୆ن໛ͷKubernetsΫϥελͷΞΫηεೝՄ ΛAthenzΛ༻͍ͯSSoT؅ཧ
  26. One more thing

  27. Yahoo! JAPAN @ Kubecon NA 2018 AthenzͱistioΛ࿈ܞͤͯ͞APIΞΫηεͷ ৄࡉͳίϯτϩʔϧΛ࣮ݱ Kubernetes ্Ͱಈ࡞͠

    Service Discoverer ͱ
 L7 Loadbalancer ͷػೳΛఏڙ͢ΔGimbalͷ঺հ
  28. Thank you!

  29. About Athenz ▸ Athenz Website: http://athenz.io ▸ Athenz Github: https://github.com/yahoo/athenz

    ▸ Garm Github: https://github.com/yahoojapan/garm (ۙ೔ެ։) ▸ Athenz Slack: https://athenz.slack.com/ ▸ Questions or Comments: yusukato@yahoo-corp.jp
  30. Appendix

  31. Athenz Authorization

  32. About Athenz ▸ Centralized Access Control ▸ Decentralized Access Control

  33. Athenz Authorization Centralized Access Control

  34. Athenz Authorization Decentralized Access Control

  35. EOP