Goでkubernetes operatorを実装してアプリのプレビュー環境を作る/go-kubernetes-operator

Transcript

1. 
   GVLVPLBHP (PͰLVCFSOFUFT
   PQFSBUPSΛ࣮૷ͯ͠
   ΞϓϦͷϓϨϏϡʔ؀ڥΛ࡞Δ

2. (.0ϖύϘגࣜձࣾ
   ϗεςΟϯάࣄۀ෦
   43&νʔϜ
   Ϋϥ΢υωΠςΟϒԽͷਪਐ ٱถ୓അ
   !UBLVNBLVNF

3. ໨࣍ wlLVCFSOFUFT
   PQFSBUPSΛ࡞Δz
   ͱ͸
   wࠓճ࣮૷ͨ͠ιϑτ΢ΣΞͷ঺հ
   w։ൃܦҢ
   w(PʹΑΔ0QFSBUPS࣮૷
   wॴײ

4. zLVCFSOFUFT
   PQFSBUPSΛ࡞Δz
   ͱ͸

5. zLVCFSOFUFT
   PQFSBUPSΛ࡞Δz
   ͱ͸ apiserver kubectl apply Control plane Data plane controller manager

   ReplicaSet Pod Pod ࢦఆ͞ΕͨPodͷ਺Λ อͱ͏ͱ͢Δ ReplicaSetͷྫ replicas:2 PodΛ1ͭ࡟আͯ͠΋ ίϯτϩʔϥʔ͕ݕ஌ͯ͠ ࠶࡞੒͞ΕΔ

6. zLVCFSOFUFT
   PQFSBUPSΛ࡞Δz
   ͱ͸ apiserver kubectl apply Control plane Data plane controller manager

   ReplicaSet Pod Pod ࢦఆ͞ΕͨPodͷ਺Λ อͱ͏ͱ͢Δ ReplicaSetͷྫ replicas:2 PodΛ1ͭ࡟আͯ͠΋ ίϯτϩʔϥʔ͕ݕ஌ͯ͠ ࠶࡞੒͞ΕΔ kubernetes͸ ͜ͷಈ͖Λ֦ுͰ͖Δ kubernetesΛ֦ு͢Δख๏ͷͻͱͭʹOperator͕͋Δ

7. zLVCFSOFUFT
   PQFSBUPSΛ࡞Δz
   ͱ͸ apiserver kubectl apply Control plane Data plane Custom Controller

   Custom Resource Custom Resource Definition (CRD) + ಠࣗͷϦιʔεఆٛ CRDͷఆٛʹج͍ͮͨ Ϧιʔε CRΛίϯτϩʔϧͯ͠ ఆٛ͞Εͨঢ়ଶʹอͭ kubernetes operator ͷ࣮ମ

8. ࠓճ࣮૷ͨ͠΋ͷ

9. apiVersion: service-expose.../v1alpha1 kind: ServiceExpose metadata: name: example namespace: ns1 spec:

   backend: service: name: example-svc port: number: 8080 domain: example.com path: / pathType: Prefix tlsEnable: true tlsSecretName: example-tls annotations: cert-manager.io/cluster-issuer: letsencrypt apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example namespace: ns1 annotations: cert-manager.io/cluster-issuer: letsencrypt spec: rules: - host: example-svc.ns1.example.com http: paths: - backend: service: name: example-svc port: number: 8080 path: / pathType: Prefix tls: - hosts: - example-svc.ns1.example.com secretName: example-tls backend: service: name: example-svc port: number: 8080 example-svc ns1 example.com example-svc ns1 example.com example-svc ns1 example.com ࢦఆͨ͠ServiceΛ IngressͰެ։͢Δ ެ։͢Δϗετ໊ͱͯ͠ αʔϏεσΟεΧόϦతʹ ࣗಈͰ෇༩͢Δ IUUQTHJUIVCDPNUBLVNBLVNFTFSWJDFFYQPTFPQFSBUPS αʔϏε໊ namespace υϝΠϯ Custom Resource

10. ։ൃܦҢ

11. ։ൃܦҢ w 8FCΞϓϦέʔγϣϯΛෳ਺ӡ༻͍ͯͯ͠ɺ։ൃؔ܎ऀ͸໊Ҏ্͍ Δ
    w 1VMM3FRVFTUຖͷϓϨϏϡʔ؀ڥΛLVCFSOFUFT্Ͱ࣮ߦ͍ͨ͠
    w 1VMM3FRVFTU͕࡞੒͞ΕͨΒɺઐ༻ͷ؀ڥ্ཱ͕͕ͪΔ
    w ݱࡏ͸ͭͷTUBHJOH؀ڥΛ։ൃऀͰڞ༗͍ͯ͠Δ
    

    w σϓϩΠͷखؒ
    w ར༻ऀͷڝ߹  എܠ

12. kubernetes cluster app repo system manifests repo Pull Request Github

    Actions ArgoCD Config ArgoCD Github Actions ArgoCD Config Pod Ingress Namespace: app-pr-XXX 1.PRͷ࡞੒ 2.PRͷϒϥϯνΛ kubernetesΫϥελʹ σϓϩΠ͢ΔͨΊͷ ArgoCDͷઃఆΛੜ੒ commit Service 3.ArgoCDͷ ઃఆ௥ՃΛݕ஌ 4.ArgoCDͷઃఆΛ σϓϩΠ 5.PRͷϒϥϯνͷ σϓϩΠΛ։࢝ dispatch argocd-util ίϚϯυͰੜ੒ ʲ1VMM3FRVFTUຖͷϓϨϏϡʔ؀ڥʳ ArgoCD΍Github ActionsΛ׆༻ͯ͠Pull ReqτϦΨʔͰGitOpsͰϓϨϏϡʔ؀ڥΛੜ੒͍ͯ͠Δ 6.PRͷϒϥϯνͷ ϓϨϏϡʔ؀ڥ͕࡞ΒΕΔ 7. external-dnsͰAϨίʔυ, cert-managerͰTLSূ໌ॻΛ ࣗಈઃఆ

13. !"" main.go #"" manifests !"" base $ !"" kustomization.yaml $

    !"" app.deployment.yaml $ #"" app.service.yaml #"" overlays !"" production $ !"" kustomization.yaml $ #"" app.ingress.yaml #"" staging !"" kustomization.yaml #"" app.ingress.yaml apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: app-pr-XXX spec: destination: namespace: app-pr-XXX server: https://kubernetes.default.svc source: path: manifests/overlays/staging repoURL: https://github.com/takumakume/app targetRevision: future-branch syncPolicy: syncOptions: - CreateNamespace=true ֤؀ڥͷmanifestsΛkustomizeͰ؅ཧ͍ͯ͠Δ #"" staging !"" kustomization.yaml #"" app.ingress.yaml Pull RequestຖʹNamespaceΛ੾ͬͯ staging؀ڥͷෳ੡Λ࡞͍ͬͯΔ app-pr-XXX ʲ1VMM3FRVFTUຖͷϓϨϏϡʔ؀ڥʳ app repo

14. ։ൃܦҢ  ٕज़త՝୊ apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: app-ingress

    namespace: app spec: rules: - host: staging-app.example.com http: paths: - backend: service: name: app-svc port: number: 8080 path: / pathType: Prefix tls: - hosts: - staging-app.example.com secretName: app-tls staging-app.example.com staging-app.example.com w ؀ڥͷෳ੡͸࡞Εͯ΋ɺ*OHSFTTͷϗετ ໊෦෼Λม͑Δ͜ͱ͕Ͱ͖ͳ͍ɻ
    w LVTUPNJ[Fͷ+40/
    1BUDIػೳ΍ɺZRίϚ ϯυͳͲͰஔ׵ͭͭ͠ద༻͢Δ͜ͱ͸Ͱ ͖Δ͕؅ཧ͕൥ࡶʹͳΔɻʢܦݧࡁʣ
    w ద༻௚લͰNBOJGFTUTΛॻ͖׵࣮͑ͭͭ ߦ͢Δͱ(JU0QTʹΑΔԸܙ͕ബΕΔɻ ίί ίί

15. ։ൃܦҢ  ࣮૷ํ਑ w (JU0QT͕Ͱ͖Δ͜ͱ
    w એݴతͰ͋Δ͜ͱ
    w ϓϨϏϡʔ؀ڥʹΞΫηε͢ΔͨΊͷϗετ໊ΛͲ͏͢Δ͔ʁ
    w

    LVCFSOFUFTͷ4FSWJDF
    %JTDPWFSZ
    w 4&37*$&@/".&/".&"1"$&TWDDMVTUFSMPDBM
    w *OHSFTTͰ࣮ݱͰ͖ΔͱศརͰ͸ͳ͍͔

16. apiVersion: service-expose.../v1alpha1 kind: ServiceExpose metadata: name: example namespace: ns1 spec:

    backend: service: name: example-svc port: number: 8080 domain: example.com path: / pathType: Prefix tlsEnable: true tlsSecretName: example-tls annotations: cert-manager.io/cluster-issuer: letsencrypt apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: example namespace: ns1 annotations: cert-manager.io/cluster-issuer: letsencrypt spec: rules: - host: example-svc.ns1.example.com http: paths: - backend: service: name: example-svc port: number: 8080 path: / pathType: Prefix tls: - hosts: - example-svc.ns1.example.com secretName: example-tls backend: service: name: example-svc port: number: 8080 example-svc ns1 example.com example-svc ns1 example.com example-svc ns1 example.com ެ։͢Δ Service αʔϏεσΟεΧόϦతͳ ϗετ໊ΛࣗಈͰ෇༩ αʔϏε໊ namespace υϝΠϯ ࠶ܝ

17. kubernetes cluster app repo system manifests repo Pull Request Github

    Actions ArgoCD Config ArgoCD Github Actions ArgoCD Config Pod Namespace: app-pr-XXX 1.PRͷ࡞੒ 2.PRͷϒϥϯνΛ kubernetesΫϥελʹ σϓϩΠ͢ΔͨΊͷ ArgoCDͷઃఆΛੜ੒ commit Service 3.ArgoCDͷ ઃఆ௥ՃΛݕ஌ 4.ArgoCDͷઃఆΛ σϓϩΠ 5.PRͷϒϥϯνͷ σϓϩΠΛ։࢝ 6.PRͷϒϥϯνͷ ϓϨϏϡʔ؀ڥ͕࡞ΒΕΔ dispatch argocd-util ίϚϯυͰੜ੒ Service Expose ੜ੒ app.app-pr-XXX.example.com Ingress 7. external-dnsͰAϨίʔυ, cert-managerͰTLSূ໌ॻΛ ࣗಈઃఆ ServiceExposeͷΈσϓϩΠ͢Δ͜ͱͰIngress͸ࣗಈੜ੒͞ΕΔ

18. (PʹΑΔ0QFSBUPSͷ࣮૷

19. (PʹΑΔ0QFSBUPS࣮૷  w ࣮૷खஈ
    w IUUQTLVCFSOFUFTJPEPDTDPODFQUTFYUFOELVCFSOFUFTPQFSBUPS w 0QFSBUPS
    'SBNFXPSL
    w $/$'
    *ODVCBUJOH
    1SPKFDU
    

    w (PΛ༻͍࣮ͨ૷ʹ͓͍ͯ͸಺෦ͰLVCFCVJMEFSΛར༻͍ͯ͠Δ

20. 0QFSBUPS
    'SBNFXPSL  w 0QFSBUPS
    4%,
    w LVCFSOFUFT
    "1*ʹਂ͍஌͕ࣝͳͯ͘΋ɺϩδοΫʹूதͰ͖ΔΑ͏ ʹӅณͯ͘͠Ε͍ͯΔ
    w ίʔυδΣωϨʔλʔ
    w

    ςετ
    w ύοέʔδϯά

21. 0QFSBUPS
    'SBNFXPSL  w ࣮૷ʹ͋ͨͬͯ΍ͬͨ͜ͱ
    w 0QFSBUPS
    'SBNFXPSLͷެࣜυΩϡϝϯτ͕ॆ࣮͍ͯ͠ΔͷͰɺج ຊతʹ͸ͦ͜Λࢀর͢Δ IUUQTTELPQFSBUPSGSBNFXPSLJPEPDTCVJMEJOHPQFSBUPSTHPMBOH
    w

    ͨ·ʹࡌ͍ͬͯͳ͍৔߹͕͋ΔͷͰɺLVCFCVJMEFSͷެࣜυΩϡϝ ϯτͰิ׬͢Δ IUUQTCPPLLVCFCVJMEFSJP
    w ϕετϓϥΫςΟεͷ࣮ફ
    IUUQTTELPQFSBUPSGSBNFXPSLJPEPDTCFTUQSBDUJDFTCFTUQSBDUJDFT
    

22.  w 3FDPODJMFS
    -PPQͷ࣮૷͕ϝΠϯ
    w ྫɿʮ1PEΛݸ࣮ߦ͢Δʯͱఆٛ͢Ε͹ͦͷঢ়ଶʹऩଋ͢Δಈ͖ (PʹΑΔ0QFSBUPS࣮૷ Observe Diff Action ঢ়ଶΛऔಘ

    ࠩ෼Λݕग़ ࠩ෼ͷमਖ਼

23.  w TFSWJDFFYQPTFPQFSBUPSͷ3FDPODJMFS
    -PPQ (PʹΑΔ0QFSBUPS࣮૷ Observe Diff Action - ੜ੒͢΂͖IngressͷSpec͸ʁ -

    ੜ੒͢΂͖Ingressͱݱࡏͷ Ingressͷࠩ෼͸ͳʹ͔ʁ - ࠩ෼ΛຒΊΔͨΊʹIngressͷ Create/Update/DeleteΛ࣮ߦ

24.  w ςετ (PʹΑΔ0QFSBUPS࣮૷ ServiceExpose ঢ়ଶऔಘ Ingress͸ ଘࡏ͢Δ͔ʁ Ingressͷ Ξοϓσʔτ͸

    ඞཁ͔ʁ Ingress ੜ੒ Ingress Ξοϓσʔτ Y Y N N Reconciler Loop w ্هͷΑ͏ʹ0QFSBUPS͸ঢ়ଶભҠ͕ൃੜ͢Δɻ
    w ͋ΒΏΔύλʔϯͰ3FDPODJMFS
    -PPQ͕ႈ౳ʹͳΔΑ͏ʹςετ͠ ͍ͨɻ

25.  w 0QFSBUPS
    4%,͕ҎԼͷπʔϧΛ༻͍ͯৼΔ෣͍ςετͷ࣮ߦ؀ڥΛ ఏڙ͍ͯ͠Δ
    w FOWUFTU
    DPOUSPMMFSSVOUJNFͷύοέʔδͰɺςετ༻ͷ LVCFSOFUFT
    DPOUSPM
    QMBOFΛఏڙ͢Δ
    w HJOLHP
    (PMBOHͷ#%%ςετϑϨʔϜϫʔΫ
    w

    HPNFHB
    (PMBOHͷ.BUDIFS
    -JCSBSZ
    HJOLHPͱηοτͰ࢖͏ (PʹΑΔ0QFSBUPS࣮૷

26.  w ྫ͑͹ɺ4FSWJDF&YQPTF
    $VTUPN
    3FTPVSDF͕σϓϩΠ͞Εͨ͋ͱʹɺ 4UBUVT͕3FBEZʹભҠ͢ΔͷΛ଴ͪɺ*OHSFTT͕ੜ੒Ͱ͖͍ͯΔ͔ͱ͍ ͏ςετ͕ॻ͚Δɻ (PʹΑΔ0QFSBUPS࣮૷

27. ॴײ

28.  w (JU0QTͰΧόʔͰ͖ͳ͔ͬͨҰ෦ͷ໰୊Λ੾Γग़ͯ͠0QFSBUPSͱ͍ ͏ख๏ͰղܾͰ͖ͨɻ
    w ϓϨϏϡʔ؀ڥͷੜ੒શମΛ0QFSBUPSͱ࣮ͯ͠૷͢Δ͜ͱ΋ߟ͑ ͕ͨɺιϑτ΢ΣΞΛγϯϓϧʹอͭ΄͏͕ྑ͍ͱߟ͑ͨɻ
    w (PͰ0QFSBUPSΛ࣮૷͢Δ্Ͱ0QFSBUPS
    4%,Λ࢖͕ͬͨɺϩδοΫ ʹूதͰ͖ͯศརͩͬͨɻ
    

    w (Pͱ͍͑͹ςʔϒϧۦಈςετΛΑ͘࢖͏͕ɺঢ়ଶભҠΛςετ͢ Δ্Ͱ#%%͸ศརͩͬͨɻ ॴײ