Securing Kubernetes Clusters on AWS (Japanese)

Kubernetes Meetup Tokyo #7
https://k8sjp.connpass.com/event/67092

Transcript

1. Securing Kubernetes Clusters on AWS @mumoshu
2. K8S on AWSͰӡ༻͍ͯ͠ΔαʔϏε͕߈ܸ͞Εͯ΋҆৺ ͳঢ়ଶʹ͍ͨ͠
3. ߈ܸͬͯ۩ମతʹ͸? ྫ͑͹ɺWebΞϓϦʹରͯ͠ - ո͛͠ͳΫΤϦετϦϯά - ո͛͠ͳHTTP POSTύϥϝʔλ - ruby΍shellεχϖοτΈ͍ͨͳͷ͕ࠞͬͯ͟Δ ϦΫΤετΛ౤͛ΒΕΔ
4. ߈ܸͬͯ۩ମతʹ͸? • ΞϓϦʹίʔυΠϯδΣΫγϣϯ੬ऑੑ͕͋ͬͯɺಛఆͷϦ ΫΤετΛૹΔͱ೚ҙͷ໋ྩΛ࣮ߦ͞ΕͨΓɺͻͲ͍ͱ͖͸ όοΫυΞΛ࢓ࠐ·ΕΔ • ʮΞϓϦͷಈ࡞؀ڥ͔ΒΞΫηεͰ͖Δ΋ͷ͢΂ͯʯʹΞΫ ηε͞Εͯ͠·͏ • !
5. ߈ܸର৅ͷ͜ͱ
6. ͓͞Β͍: K8SͰΞϓϦ͸Ͳ͜Ͱಈ͘? K8SʹσϓϩΠͨ͠ΞϓϦ͸ʮίϯςφʯͰಈ͘ɻ
7. ͓͞Β͍: ެ։͞ΕͨΞϓϦ͸Ͳ͏͍͏ܦ࿏ͰΞΫηε͞ΕΔ? -> LB (ELB, ALB, NLB) -> EC2Πϯελϯε (ServiceͷNodePortܦ༝) -> Pod (kube-proxy/iptables͕ServiceͷclusterIPΛpodIPʹม ׵) —> ίϯςφ (containerPortܦ༝) —> ΞϓϦέʔγϣϯϓϩηε ͱ͍͏ܦ࿏ͰΞΫηε͞ΕΔɻ
8. ίϯςφʹೖ͍ͬͯΔ΋ͷ • ίϚϯυ • bash, curl, etc. • ϛυϧ΢ΣΞ • apache, nginx, etc. • ΞϓϦέʔγϣϯ • όΠφϦ(Golang)ɺεΫϦϓτ(ruby, python, etc.)
9. ίϯςφʹೖ͍ͬͯΔ΋ͷ • ࣮ߦ؀ڥ • python, rubyϥϯλΠϜ, ruby gems, JDK, … • ڞ༗ϥΠϒϥϦ • glibc, libssl
10. ίϯςφʹೖ͍ͬͯΔ΋ͷ • Ͳ͜ʹ੬ऑੑ͕͋Δ͔Θ͔Βͳ͍Ͱ͢ΑͶ… • ͔ͩΒͦ͜ɺͰ͖Δ͚ͩK8SϨϕϧͰ๷ޚ͍ͨ͠
11. ྫ͑͹ɺόοΫυΞΛ࢓ࠐ·ΕͨΒԿΛ͞Εͯ͠·͏͔ • ίϯςφʹଘࡏ͢Δ೚ҙͷίϚϯυΛ࣮ߦͰ͖Δ • ྫ: ίϯςφʹcurl͕͋Ε͹೚ҙͷhttp(s)ϦΫΤετΛ஗ΕΔ
12. ೚ҙͷίϚϯυΛ࣮ߦͰ͖Δͱ… • KubernetesͳΒͰ͸ͷ΋ͷ • K8S API, Kubelet API(ඇެ։), Podʹcurlͱ͔ͰΞΫηε • ίϯςφΦʔέετϨʔγϣϯγεςϜશൠʹ͋ͯ͸·Δ΋ ͷ • ଞͷίϯςφʹΞΫηε • localhost΍ϩʔΧϧϘϦϡʔϜܦ༝
13. ରࡦ • όοΫυΞ࢓ࠐΊͳ͍Α͏ʹ͢Δ • όοΫυΞ࢓ࠐ·Εͨͱ͖ͷӨڹΛ࠷খݶʹ͢Δ
14. ۩ମతʹͲ͏ରࡦ͢Δ͔ • kube2iam • KubernetesͷNetwork Policies • RBAC • TLS Bootstrapping • ϊʔυϓʔϧɺΠϯελϯεϓʔϧ • coreos/clair
15. όοΫυΞ࢓ࠐΊͳ͍Α͏ʹ͢Δ (1) ੬ऑͳΞϓϦΛ࡞Βͳ͍ ! • ಛʹޠΕΔ͜ͱͳ͍ɾɾɾ • ϑϨʔϜϫʔΫ࢖͍·͠ΐ • ݴޠॲཧܥͷ੬ऑੑ͕ݟ͔ͭͬͨΒΞοϓσʔτ͠·͠ΐ͏ ʢGolang, RubyͳͲͷϨϕϧͰʣ • ϢʔβೖྗͷαχςΟνΣοΫ • ੬ऑͳίϚϯυ΍ϥΠϒϥϦΛؚΊͳ͍
16. όοΫυΞ࢓ࠐΊͳ͍Α͏ʹ͢Δ (2) CoreOSͷClairͰίϯςφΠϝʔδ੬ऑੑνΣοΫ ྫ: ੬ऑͳΠϝʔδ͕σϓϩΠ͞Εͳ͍Α͏ʹɺCIͰDocker Build & Pushͷޙʹ clair-scanner --clair https://myclaier.example.com --ip $YOUR_LOCAL_IP $IMAGE ͦͷ͋ͱK8SʹσϓϩΠ͢ΔΑ͏ʹ͢Δɻ ಉछͷπʔϧʹfuture-architect/vuls΋͋Γ·͢Ͷɻ
17. ࢓ࠐ·Εͨͱ͖ͷӨڹΛ࠷খݶʹ͢Δ kube2iam kube2iam: EC2ϝλσʔλαʔϏε(http://169.254.169.254)Λ ஔ͖׵͑ΔϦόʔεϓϩΩγ • EC2Πϯελϯεʹ͸ͦͦ͜͜ڧྗͳIAMϙϦγʔ͕͍ͭͯ Δ • ΞϓϦ͔ΒͦΕΛར༻Ͱ͖ͳ͍Α͏ʹ͢Δ • ௨ৗ͸EC2ϝλσʔλαʔϏεʹcurl͢ΔͱɺΠϯελϯε
18. Network Policies Podؒͷ௨৴ΛϧʔϧϕʔεͰڐՄɾېࢭ͢ΔͨΊͷ࢓૊Έ • PodΛෆཁͳΠϯλʔωοτ্ͷαʔϏεͱ௨৴ͤͨ͘͞ͳ ͍ • Pod΁ೖΔ: Ingress • Pod͔Βग़Δ: Egress (K8S 1.8͔Β࢖͑Δ) • EC2ϝλσʔλαʔϏε(169.254.169.254)͸σϑΥϧτϒ ϩοΫͰ͍͍ͷͰ͸
19. RBAC • K8S APIΛPod΍ϢʔβຖʹΞΫηείϯτϩʔϧ͢Δػೳ • K8S APIΛݺ΂ͳ͍PodʹόοΫυΞ࢓ࠐ·Εͯ΋҆৺ • K8S 1.8͔ΒGAʹͳΓ·ͨ͠Ͷ
20. TLS Bootstrapping • K8S API Server͕Kubelet΁ͷ௨৴ͷೝূɾ҉߸Խʹ࢖͏TLS ൿີ伴Λࣗಈੜ੒͢Δ • ྫ: kube-aws: 伴͕શKubelet=WorkerͰڞ௨ɻ1 Workerͷ伴 ͕࿙ΕΔͱଞͷWorkerʹ΋Өڹ͕… • TLS BootstrappingΛ࢖͏ͱɺWorkerผͷ伴ʹͳΔɻ΋Εͯ ΋ͦͷWorker͚ͩʹӨڹ
21. RotateKubeletClientCertificate • K8S 1.8Ͱalphaʹͳͬͨ • TLS BootstrappingͰɺ伴Λఆظతʹ(਺ϲ݄ɻͳ͕͍)ʹೖΕ ସ͑Δػೳ • 伴͕࿙Εͯ΋ظݶ੾ΕʢೖΕସ͑ʣޙͰ͋Ε͹Өڹͳ͠
22. ϊʔυϓʔϧɺΠϯελϯεϓʔϧ • GKE, kops, kube-awsͳͲʹ͋ΔɺNodeͷάϧʔϓʢlabel΍ αʔόεϖοΫɺGPUͷ༗ແͳͲ͕ҟͳΔʣΛ࢖͍෼͚Δػ ೳ • ྫ: ઀ଓՄೳͳݸਓ৘ใΛؚΉDBΠϯελϯεผʹϓʔϧΛ Θ͚Δ • ΍Γ͗͢ΔͱϦιʔεޮ཰͕Լ͕Δʀ
23. ͓·͚: K8S APIͷೝূ໰୊ • ΤϯδχΞ͕K8S APIʹΞΫηεͰ͖ΔτʔΫϯΛπΠʔτ͠ ͯ͠·ͬͨ • ΤϯδχΞ͕K8S APIʹΞΫηεͰ͖Δൿີ伴Λ͓࿙Βͯ͠͠ ͠·ͬͨ • !
24. ରࡦ: ωοτϫʔΫతʹ K8S API Endpoint (AWSͷ৔߹ଟ͘͸ELB)͸ • ࣾ಺ωοτϫʔΫ(IP੍ݶɺͰ͖Ε͹VPN) • VPC಺ͷϓϥΠϕʔτωοτϫʔΫ(ྫ: VPC಺ͷCIγεςϜ͔ ΒK8S APIΛར༻͢Δ) ͔ΒͷΈΞΫηεͰ͖ΔΑ͏ʹ͠·͠ΐ͏ɻ
25. ରࡦ: K8SΫϥελʹGitHubϩάΠϯ Using guard, you can log into your Kubernetes cluster using your Github or Google authentication token https://github.com/appscode/guard • ͨͩ͠ɺGitHubͷOAuthτʔΫϯΛK8S APIτʔΫϯ͕ΘΓ ʹ࢖͏ • τʔΫϯͷ༗ޮظݶ͕௕͍ͱͦΕ͸ͦΕͰةͳ͍͔΋ • GoogleϩάΠϯ΋Ͱ͖Δ
26. ରࡦ: K8SΫϥελʹAWS IAMͰϩάΠϯ kubernetes-aws-authenticator A tool for using AWS IAM credentials to authenticate to a Kubernetes cluster. https://github.com/heptiolabs/kubernetes-aws-authenticator • ͨͩ͠ɺAWSΫϨσϯγϟϧͷ༗ޮظݶ͕̍࣌ؒͳͷͰɺ࿙ ΕͨΒ̍࣌ؒ͸࢖ΘΕͯ͠·͏Մೳੑ • ωοτϫʔΫతͳରࡦ΋߹Θͤͯ
27. TL;DR; K8S on AWSΛηΩϡΞʹ͍ͨ͠ͳΒɺ࠷௿ݶ • kube2iam • Network Policies • coreos/clair • RBAC Ͱ͖Ε͹ • ϊʔυϓʔϧɺTLS Bootstrapping