Securing Kubernetes Clusters on AWS (Japanese)

Kubernetes Meetup Tokyo #7
https://k8sjp.connpass.com/event/67092

Transcript

1. Securing Kubernetes Clusters on AWS
   @mumoshu
   

   View Slide

2. K8S on AWSͰӡ༻͍ͯ͠ΔαʔϏε͕߈ܸ͞Εͯ΋҆৺
   ͳঢ়ଶʹ͍ͨ͠
   

   View Slide

3. ߈ܸͬͯ۩ମతʹ͸?
   ྫ͑͹ɺWebΞϓϦʹରͯ͠
   - ո͛͠ͳΫΤϦετϦϯά
   - ո͛͠ͳHTTP POSTύϥϝʔλ
   - ruby΍shellεχϖοτΈ͍ͨͳͷ͕ࠞͬͯ͟Δ
   ϦΫΤετΛ౤͛ΒΕΔ
   

   View Slide

4. ߈ܸͬͯ۩ମతʹ͸?
   • ΞϓϦʹίʔυΠϯδΣΫγϣϯ੬ऑੑ͕͋ͬͯɺಛఆͷϦ
   ΫΤετΛૹΔͱ೚ҙͷ໋ྩΛ࣮ߦ͞ΕͨΓɺͻͲ͍ͱ͖͸
   όοΫυΞΛ࢓ࠐ·ΕΔ
   • ʮΞϓϦͷಈ࡞؀ڥ͔ΒΞΫηεͰ͖Δ΋ͷ͢΂ͯʯʹΞΫ
   ηε͞Εͯ͠·͏
   • !
   

   View Slide

5. ߈ܸର৅ͷ͜ͱ
   

   View Slide

6. ͓͞Β͍: K8SͰΞϓϦ͸Ͳ͜Ͱಈ͘?
   K8SʹσϓϩΠͨ͠ΞϓϦ͸ʮίϯςφʯͰಈ͘ɻ
   

   View Slide

7. ͓͞Β͍: ެ։͞ΕͨΞϓϦ͸Ͳ͏͍͏ܦ࿏ͰΞΫηε͞ΕΔ?
   -> LB (ELB, ALB, NLB)
   -> EC2Πϯελϯε (ServiceͷNodePortܦ༝)
   -> Pod (kube-proxy/iptables͕ServiceͷclusterIPΛpodIPʹม
   ׵)
   —> ίϯςφ (containerPortܦ༝)
   —> ΞϓϦέʔγϣϯϓϩηε
   ͱ͍͏ܦ࿏ͰΞΫηε͞ΕΔɻ
   

   View Slide

8. ίϯςφʹೖ͍ͬͯΔ΋ͷ
   • ίϚϯυ
   • bash, curl, etc.
   • ϛυϧ΢ΣΞ
   • apache, nginx, etc.
   • ΞϓϦέʔγϣϯ
   • όΠφϦ(Golang)ɺεΫϦϓτ(ruby, python, etc.)
   

   View Slide

9. ίϯςφʹೖ͍ͬͯΔ΋ͷ
   • ࣮ߦ؀ڥ
   • python, rubyϥϯλΠϜ, ruby gems, JDK, …
   • ڞ༗ϥΠϒϥϦ
   • glibc, libssl
   

   View Slide

10. ίϯςφʹೖ͍ͬͯΔ΋ͷ
    • Ͳ͜ʹ੬ऑੑ͕͋Δ͔Θ͔Βͳ͍Ͱ͢ΑͶ…
    • ͔ͩΒͦ͜ɺͰ͖Δ͚ͩK8SϨϕϧͰ๷ޚ͍ͨ͠
    

    View Slide

11. ྫ͑͹ɺόοΫυΞΛ࢓ࠐ·ΕͨΒԿΛ͞Εͯ͠·͏͔
    • ίϯςφʹଘࡏ͢Δ೚ҙͷίϚϯυΛ࣮ߦͰ͖Δ
    • ྫ: ίϯςφʹcurl͕͋Ε͹೚ҙͷhttp(s)ϦΫΤετΛ஗ΕΔ
    

    View Slide

12. ೚ҙͷίϚϯυΛ࣮ߦͰ͖Δͱ…
    • KubernetesͳΒͰ͸ͷ΋ͷ
    • K8S API, Kubelet API(ඇެ։), Podʹcurlͱ͔ͰΞΫηε
    • ίϯςφΦʔέετϨʔγϣϯγεςϜશൠʹ͋ͯ͸·Δ΋
    ͷ
    • ଞͷίϯςφʹΞΫηε
    • localhost΍ϩʔΧϧϘϦϡʔϜܦ༝
    

    View Slide

13. ରࡦ
    • όοΫυΞ࢓ࠐΊͳ͍Α͏ʹ͢Δ
    • όοΫυΞ࢓ࠐ·Εͨͱ͖ͷӨڹΛ࠷খݶʹ͢Δ
    

    View Slide

14. ۩ମతʹͲ͏ରࡦ͢Δ͔
    • kube2iam
    • KubernetesͷNetwork Policies
    • RBAC
    • TLS Bootstrapping
    • ϊʔυϓʔϧɺΠϯελϯεϓʔϧ
    • coreos/clair
    

    View Slide

15. όοΫυΞ࢓ࠐΊͳ͍Α͏ʹ͢Δ
    (1) ੬ऑͳΞϓϦΛ࡞Βͳ͍ !
    • ಛʹޠΕΔ͜ͱͳ͍ɾɾɾ
    • ϑϨʔϜϫʔΫ࢖͍·͠ΐ
    • ݴޠॲཧܥͷ੬ऑੑ͕ݟ͔ͭͬͨΒΞοϓσʔτ͠·͠ΐ͏
    ʢGolang, RubyͳͲͷϨϕϧͰʣ
    • ϢʔβೖྗͷαχςΟνΣοΫ
    • ੬ऑͳίϚϯυ΍ϥΠϒϥϦΛؚΊͳ͍
    

    View Slide

16. όοΫυΞ࢓ࠐΊͳ͍Α͏ʹ͢Δ
    (2) CoreOSͷClairͰίϯςφΠϝʔδ੬ऑੑνΣοΫ
    ྫ: ੬ऑͳΠϝʔδ͕σϓϩΠ͞Εͳ͍Α͏ʹɺCIͰDocker
    Build & Pushͷޙʹ
    clair-scanner --clair https://myclaier.example.com --ip $YOUR_LOCAL_IP $IMAGE
    ͦͷ͋ͱK8SʹσϓϩΠ͢ΔΑ͏ʹ͢Δɻ
    ಉछͷπʔϧʹfuture-architect/vuls΋͋Γ·͢Ͷɻ
    

    View Slide

17. ࢓ࠐ·Εͨͱ͖ͷӨڹΛ࠷খݶʹ͢Δ
    kube2iam
    kube2iam: EC2ϝλσʔλαʔϏε(http://169.254.169.254)Λ
    ஔ͖׵͑ΔϦόʔεϓϩΩγ
    • EC2Πϯελϯεʹ͸ͦͦ͜͜ڧྗͳIAMϙϦγʔ͕͍ͭͯ
    Δ
    • ΞϓϦ͔ΒͦΕΛར༻Ͱ͖ͳ͍Α͏ʹ͢Δ
    • ௨ৗ͸EC2ϝλσʔλαʔϏεʹcurl͢ΔͱɺΠϯελϯε
    

    View Slide

18. Network Policies
    Podؒͷ௨৴ΛϧʔϧϕʔεͰڐՄɾېࢭ͢ΔͨΊͷ࢓૊Έ
    • PodΛෆཁͳΠϯλʔωοτ্ͷαʔϏεͱ௨৴ͤͨ͘͞ͳ
    ͍
    • Pod΁ೖΔ: Ingress
    • Pod͔Βग़Δ: Egress (K8S 1.8͔Β࢖͑Δ)
    • EC2ϝλσʔλαʔϏε(169.254.169.254)͸σϑΥϧτϒ
    ϩοΫͰ͍͍ͷͰ͸
    

    View Slide

19. RBAC
    • K8S APIΛPod΍ϢʔβຖʹΞΫηείϯτϩʔϧ͢Δػೳ
    • K8S APIΛݺ΂ͳ͍PodʹόοΫυΞ࢓ࠐ·Εͯ΋҆৺
    • K8S 1.8͔ΒGAʹͳΓ·ͨ͠Ͷ
    

    View Slide

20. TLS Bootstrapping
    • K8S API Server͕Kubelet΁ͷ௨৴ͷೝূɾ҉߸Խʹ࢖͏TLS
    ൿີ伴Λࣗಈੜ੒͢Δ
    • ྫ: kube-aws: 伴͕શKubelet=WorkerͰڞ௨ɻ1 Workerͷ伴
    ͕࿙ΕΔͱଞͷWorkerʹ΋Өڹ͕…
    • TLS BootstrappingΛ࢖͏ͱɺWorkerผͷ伴ʹͳΔɻ΋Εͯ
    ΋ͦͷWorker͚ͩʹӨڹ
    

    View Slide

21. RotateKubeletClientCertificate
    • K8S 1.8Ͱalphaʹͳͬͨ
    • TLS BootstrappingͰɺ伴Λఆظతʹ(਺ϲ݄ɻͳ͕͍)ʹೖΕ
    ସ͑Δػೳ
    • 伴͕࿙Εͯ΋ظݶ੾ΕʢೖΕସ͑ʣޙͰ͋Ε͹Өڹͳ͠
    

    View Slide

22. ϊʔυϓʔϧɺΠϯελϯεϓʔϧ
    • GKE, kops, kube-awsͳͲʹ͋ΔɺNodeͷάϧʔϓʢlabel΍
    αʔόεϖοΫɺGPUͷ༗ແͳͲ͕ҟͳΔʣΛ࢖͍෼͚Δػ
    ೳ
    • ྫ: ઀ଓՄೳͳݸਓ৘ใΛؚΉDBΠϯελϯεผʹϓʔϧΛ
    Θ͚Δ
    • ΍Γ͗͢ΔͱϦιʔεޮ཰͕Լ͕Δʀ
    

    View Slide

23. ͓·͚: K8S APIͷೝূ໰୊
    • ΤϯδχΞ͕K8S APIʹΞΫηεͰ͖ΔτʔΫϯΛπΠʔτ͠
    ͯ͠·ͬͨ
    • ΤϯδχΞ͕K8S APIʹΞΫηεͰ͖Δൿີ伴Λ͓࿙Βͯ͠͠
    ͠·ͬͨ
    • !
    

    View Slide

24. ରࡦ: ωοτϫʔΫతʹ
    K8S API Endpoint (AWSͷ৔߹ଟ͘͸ELB)͸
    • ࣾ಺ωοτϫʔΫ(IP੍ݶɺͰ͖Ε͹VPN)
    • VPC಺ͷϓϥΠϕʔτωοτϫʔΫ(ྫ: VPC಺ͷCIγεςϜ͔
    ΒK8S APIΛར༻͢Δ)
    ͔ΒͷΈΞΫηεͰ͖ΔΑ͏ʹ͠·͠ΐ͏ɻ
    

    View Slide

25. ରࡦ: K8SΫϥελʹGitHubϩάΠϯ
    Using guard, you can log into your Kubernetes cluster using your
    Github or Google authentication token
    https://github.com/appscode/guard
    • ͨͩ͠ɺGitHubͷOAuthτʔΫϯΛK8S APIτʔΫϯ͕ΘΓ
    ʹ࢖͏
    • τʔΫϯͷ༗ޮظݶ͕௕͍ͱͦΕ͸ͦΕͰةͳ͍͔΋
    • GoogleϩάΠϯ΋Ͱ͖Δ
    

    View Slide

26. ରࡦ: K8SΫϥελʹAWS IAMͰϩάΠϯ
    kubernetes-aws-authenticator
    A tool for using AWS IAM credentials to authenticate to a
    Kubernetes cluster.
    https://github.com/heptiolabs/kubernetes-aws-authenticator
    • ͨͩ͠ɺAWSΫϨσϯγϟϧͷ༗ޮظݶ͕̍࣌ؒͳͷͰɺ࿙
    ΕͨΒ̍࣌ؒ͸࢖ΘΕͯ͠·͏Մೳੑ
    • ωοτϫʔΫతͳରࡦ΋߹Θͤͯ
    

    View Slide

27. TL;DR;
    K8S on AWSΛηΩϡΞʹ͍ͨ͠ͳΒɺ࠷௿ݶ
    • kube2iam
    • Network Policies
    • coreos/clair
    • RBAC
    Ͱ͖Ε͹
    • ϊʔυϓʔϧɺTLS Bootstrapping
    

    View Slide