Upgrade to Pro — share decks privately, control downloads, hide ads and more …

EKS Cluster-based Canary Deploy implemented wit...

Tomoaki Nakagawa
February 10, 2021
570

EKS Cluster-based Canary Deploy implemented with Terraform Custom Provider

Tomoaki Nakagawa

February 10, 2021
Tweet

Transcript

  1. 2 த઒ ஐӯ (Tomoaki Nakagawa ) Site Reliability Engineering Team

    @ free e Wor k ➔ EC2 ͔Β EKS ΁ҠߦɺࣗಈԽɺվળ Twitte r ➔ https://twitter.com/tmnkgwa4 GitHu b ➔ https://github.com/naka-gawa
  2. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠓ೔࿩͢͜ͱ

    GSFFFͰ͸,VCFSOFUFT Ҏ߱&,4 ͸γϯάϧςφϯτઓུΛ औ͓ͬͯΓɺͦͷ݁Ռɺଟ਺ͷΫϥελΛ๊͑Δ͜ͱͱͳΓ· ͨ͠ɻͦͷӡ༻ͷதͰग़͖ͯͨ՝୊ײΛڞ༗͠ɺߴ͍Քಇ཰Λཁ ٻ͢ΔϓϩμΫτʹରͯ͠ɺͲͷ༷ʹ5FSSBGPSNΛ࢖͍ͭͭӡ༻Λ ࣗಈԽͨ͠ͷ͔Λ͝঺հ͠·͢ɻ
  3. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy freeeʹ͓͚ΔΠϯϑϥCI/CDਤ

    ALB&TG EKS ./xxx-pj sg.tf alb.tf route53.tf ./share-pj iam.tf terraform-repo manifest-repo ./xxx-cluster cluster.yaml ./xxx-service helmfile.yaml ci ci ci
  4. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ՝୊

    -1- ΫϥελͷόʔδϣϯΞοϓ ҆શʹ OPBMFSU ͰόʔδϣϯΛ্͛ऴ͑Δ·Ͱɺ13ඞཁʹͳ Δ SFQPTJUPSZΛԟ෮ͨ͠Γɺ࡞ͬͨϦιʔεΛผ13Ͱ࡞ͬͨ Γ ΍Δ͜ͱ͸ఆܕ࡞ۀͳͷʹ΋͔͔ΘΒͣɺ߆ଋ͕࣌ؒ௕͘ͳΔ पลϦιʔεͷ؅ཧSFQPTJUPSZͱ&,4؅ཧSFQPTJUPSZ͕෼அ͞Ε͍ͯΔ
  5. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ՝୊

    -2- ߴ͍Քಇ཰Λཁٻ͞ΕΔΫϥελαΠζͷେ͖͍QSPEVDUʹର͢Δ੾Γ໭࣌ؒ͠ͷ௕ظԽ ALB&TG EK S 1.14->1.18 JOQMBDFͰWFSTJPOVQͯ͠ɺ໰୊͕͋ͬͨ ͔Β੾Γ໭͍ͨ͠৔߹औΓಘΔબ୒ࢶ͸৽ͨ ʹΫϥελ࡞੒͔͠ͳ͍ɻ Ϋϥελʹ"QQΛσϓϩΠ͢Δ·Ͱૣͯ͘ ෼ऑɺ4DBMFͷ໰୊΋ࢴ͢Δͱ࣌ؒ௒͑ Δ͜ͱ΋ ࣌ظ࣍ୈͰ͸ݩʹ໭ͤͳ͍ͱ͍͏ࣄଶʹ ΋😇 EK S 1.14
  6. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ՝୊

    -3- େن໛ΫϥελʹͳΕ͹ͳΔ΄ͲɺXPSLFSOPEFͷೖΕସ͑౳Ͱ΋Ұۤ࿑ ࡉʑͱͨ͠ӡ༻ʹ͔͔Δ࣌ؒͷ௕ظԽ ۩ମྫΛ͋͛Δͱ 8PSLFSʹ͚ͭΔ*".1PMJDZ΍4FDVSJUZ(SPVQ෇ସ͕൒೔࡞ۀ /PEF%SBJO͢Δʹ΋Ұ޻෉ඞཁ
  7. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy Canary

    deploy per cluster ``্هͷΑ͏ͳมߋ͸௨ৗJOQMBDFͰߦ͏ͱࢥ͍·͕͢ɺલऀʹ͍ͭͯ͸$POUSPM 1MBOFͷ,TϚΠφʔόʔδϣϯΛҰ౓্͛ͯ͠·͏ͱԼ͛Δ͜ͱ͸Ͱ͖ͳ͍ɺޙऀʹ ͍ͭͯ͸໰୊ͷ͋ͬͨมߋΛ໭͚ͩ͢Ͱ΋਺ेඵఔ౓͕͔͔࣌ؒͬͯ͠·͏Մೳੑ͕͋ Γ·͢ɻ 4-"తʹͦΕ͕ڐ͞Εͳ͍৔߹ʹɺԿ͔Ͱ͖Δ͜ͱ͸ͳ͍ͷͰ͠ΐ͏͔ʁ Ұͭͷํ๏͸ɺ"-#΍/-#ͳͲͷޙΖʹෳ਺ͷ&,4ΫϥελΛ഑ஔ͠ɺҰํͷΫϥε λ΁ͷτϥϑΟοΫͷॏΈΛͳͲͷे෼ʹখ͍͞஋ʹͯ͠ɺͦͷॏΈͷখ͍͞Ϋϥ ελΛઌʹߋ৽͢Δ͜ͱͰ͢ɻͦͷޙɺಛʹΞϥʔτ౳্͕͕Βͳ͍ͷͰ͋Ε͹ɺͦͷม ߋ͸҆શͱΈͳͯ͠΋͏ҰํͷΫϥελʹશ͘ಉ͡มߋΛߦ͍·͢`` Ҿ༻ݩɿhttps://github.com/mumoshu/decouple-apps-and-eks-clusters-with-tf-and-gitops
  8. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ

    ALB&TG EK S old ./xxx-pj sg.tf alb.tf route53.tf ./share-pj iam.tf terraform-repo manifest-repo ./xxx-cluster cluster.yaml ./xxx-service helmfile.yaml ci ci ci custo m provider EK S new weight rule
  9. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ

    5FSSBGPSN$VTUPN1SPWJEFS ఏڙ͞Ε͍ͯΔGSBNFXPSLΛར༻ͨ͠ಠࣗͷQSPWJEFS UFSSBGPSNQSPWJEFSFLTDUM UFSSBGPSNQSPWJEFSIFMNGJMF UFSSBGPSNQSPWJEFSLVCFDUM
  10. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ

    UFSSBGPSNQSPWJEFSFLTDUM UFSSBGPSN͔ΒFLTDUMίϚϯ υΛ࣮ߦ͢Δத֩ͱͳΔ QSPWJEFS BMCSPVUFͷXFJHIUΛࣗಈ Ͱ੾Γସ͑ͯ͘ΕΔ΋ͷ provider "eksctl" { } resource "eksctl_cluster" "sample" { name = "sample " region = "ap-northeast-1 " spec = <<EO S nodeGroups : - name: ng 1 instanceType: m5.larg e desiredCapacity: 1 EO S }
  11. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ

    UFSSBGPSNQSPWJEFSIFMNGJMF UFSSBGPSN͔ΒIFMNGJMFίϚ ϯυΛ࣮ߦ͢ΔQSPWJEFS ಉ࣮࣌ߦͯ͠΄͘͠ͳ͍ $SPO+PCͳͲ͸͜͜Ͱ੍ޚ͢ Δ provider "helmfile" { } resource "helmfile_release_set" "sample" { working_directory = "mamifest-repo " content = file("./ helmfile.yaml" ) binary = "helmfile " helm_binary = "helm " environment = "default " kubeconfig = eksctl_cluster.sample.kubeconfig_pat h }
  12. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠾༻ͨ͠ιϦϡʔγϣϯ

    UFSSBGPSNQSPWJEFSLVCFDUM σϑΥϧτϦιʔεΛIFMN ؅ཧ͢ΔͨΊɺ BOOPUBUJPOMBCFMΛ෇༩͢Δ ͨΊͷQSPWJEFS DPSFEOTͷύϥϝʔλΛҰ෦ ม͍͑ͨ৔߹ʹ࢖͍ͬͯΔ provider "kubectl" { } resource "kubectl_ensure" "sample" { kubeconfig = eksctl_cluster.sample.kubeconfig_pat h namespace = "kube-system " resource = "configmap " name = "coredns " labels = { "app.kubernetes.io/managed-by" = "Helm " } annotations = { "meta.helm.sh/release-name" = "coredns " "meta.helm.sh/release-namespace" = "kube- system " } } https://www.slideshare.net/ssuser4fddce/ekscoredns
  13. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy Pros

    & Cons 1SPT ΫϥελόʔδϣϯΞοϓ࡞ۀʹ͓͍ͯΫϥελ࡞੒͔Β"QQ %FQMPZɺτϥϑΟοΫ੾Γସ͑·Ͱͷखॱ͕໿վળͨ͠ ࠓ·Ͱݫີʹίʔυ؅ཧͰ͖͍ͯͳ͔ͬͨࡉ͔͍&,4Ϧιʔε͕ ίʔυ؅ཧͰ͖ΔΑ͏ʹͳͬͨ FHBXTBVUI *34"ͳͲͰ࢖͏*".SPMFͳͲͷϦιʔεͷࢀর͕Մೳʹ ࠓ·Ͱ͸సه͍ͯͨ͠
  14. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy Pros

    & Cons $POT NBOJGFTUEFQMPZ DE ͸ͦͷ··։ൃऀ͕࢖͏ͨΊɺೋॏ؅ཧ ʹͳͬͯ͠·͏ 8PSLFS/PEF͚ͩมߋ͍ͨ͠ͷʹ"QQ%FQMPZ͕૸ͬͯ͠·͏ $*ͷ౎߹্ɺ%FQMPZ IFMNGJMFTZOD ͕஗͍ ΨνΨνʹݻΊΒΕͨ5FSSBGPSNNPEVMF͕བྷΉͱͭΒ͍ TUBHJOHQSPEVDUJPOͱஈ֊తʹ্͍͕͛ͨNPEVMFΛղ͘ ඞཁ͕͋Δ
  15. Terraform Custom Provider Ͱ࣮૷ͨ͠ EKS Cluster ϕʔεͷ Canary Deploy ࠓޙͷల๬

    ෼ք఺ͷௐ੔ Ϋϥελ࡞੒͔Βɺ"QQ%FQMPZͰ͖Δঢ়ଶʹ͢Δ·ͰΛ 5FSSBGPSNͰߦ͍ɺ"QQ%FQMPZΛ"SHP$%ͳͲͷ%FQMPZʹಛԽ ͨ͠UPPMʹҠৡ ΑΓ1SPHSFTTJWF%FMJWFSZʹ͚ۙͮΔ "QQ%FQMPZޙͷΫϥελਖ਼ৗςετΛॆ࣮ͤ͞ɺࣗಈͰ੾Γ ໭ͬͨΓɺΫϥελ࡟আ·Ͱ΍ͬͨΓͰ͖ΔΑ͏ʹ͢Δ 13Ͱ҆શ҆৺ʹΫϥελΛೖΕସ͑ΒΕΔੈք΁