$30 off During Our Annual Pro Sale. View Details »

Kubernetes基盤を自律的に支えるController化の実装Tips / forkwell-202303-amsy810-k8s

Kubernetes基盤を自律的に支えるController化の実装Tips / forkwell-202303-amsy810-k8s

青山 真也 氏(@amsy810)
CyberAgent Software Engineer / CloudNative Days Tokyo Co-chair

Kubernetes Meetup Tokyo/Cloud Native Meetup Tokyo Organizer としても活動。
著書に「Kubernetes完全ガイド」「みんなのDocker/Kubernetes」等。

Kubernetesには、様々な拡張ポイントが用意されています。Kubernetes Controllerを利用することで、Kubernetesに対して様々な任意の機能を簡単に拡張していくことが可能です。

CyberAgent group Ingfrastructure Unitでは自分たちの運用業務知識をKubernetesに組み込み、クラスタの運用を人が毎回考えずに・人を介さずに自律的に行えるように、運用に関する作業は KRM に従って Kubernetes Controller などの拡張性を活かし、ロジックをプログラムコード化しています。
一連の運用の処理がコード化されているため、運用フローや障害時対応の誤りを継続的に改善し続けることもできます。

本セッションでは、KaaS で実現している Controller の実装例とともに実装Tipsを紹介し、皆さんがController化することができるパターンについてイメージが湧くことを目指します。

@ Forkwell エンジニア文化祭 2023

Masaya Aoyama (@amsy810)

March 03, 2023
Tweet

More Decks by Masaya Aoyama (@amsy810)

Other Decks in Programming

Transcript

  1. Masaya Aoyama
    CyberAgent
    Kubernetesج൫Λࣗ཯తʹࢧ͑ΔControllerԽͷ࣮૷Tips
    ΤϯδχΞจԽࡇ 2023
    amsy810 @amsy810

    View Slide

  2. ੨ࢁਅ໵ !BNTZ

    גࣜձࣾαΠόʔΤʔδΣϯτ
    $ZCFS"HFOUHSPVQ*OGSBTUSVDUVSF6OJU $*6

    ɹ,BB41SPEVDU0XOFS
    ɹ%FWFMPQFS&YQFSUT ,VCFSOFUFT$MPVE/BUJWFྖҬ

    $MPVE/BUJWF%BZT5PLZP$PDIBJS
    ,VCFSOFUFT.FFUVQ5PLZP0SHBOJ[FS
    - Co-chair
    ⻘⼭真也
    + CREATIONLINE - 技術アドバイザ
    + SAKURA Internet Research Center – 客員研究員
    + 3-shake 技術顧問
    + PLAID
    - Organizer
    - KaaS Product Owner - Publications
    Twitter: @amsy810

    View Slide

  3. গਓ਺Ͱޮ཰తʹӡ༻͢ΔͨΊʹ
    ιϑτ΢ΣΞʹΑΔࣗ཯ԽΛਪਐ
    ,BB4.-1MBUGPSN։ൃνʔϜ
    !MZKFF ![VJVST
    !BNTZ !,,BXBCF
    !DIF[@TIBOQV !"PLJ5FO[FO
    IPUUFS !DPUBQPO SUPNJOBHB

    View Slide


  4. ίϯςφΦʔέετϨʔλʔ
    ௐࠪɾٕज़બఆ։࢝

    0QFO4UBDL)FBU$POTVMϕʔεͷ
    ,VCFSOFUFTBTB4FSWJDFΛϦϦʔε
    ʢ",&Wʣ
    $MVTUFS"1*ϕʔεͷ
    ,BB4ͷઃܭɾ࣮૷΁
    $MVTUFS"1*ϕʔεͷ
    ,BB4ͷϦϦʔε
    ʢ",&Wʣ
    ਺ଟ͘ͷ,VCFSOFUFT$POUSPMMFSͷ࣮૷

    View Slide

  5. ",&WXJUI$MVTUFS"1*
    ؅ཧΫϥελʹσϓϩΠ͞Εͨ$MVTUFSϦιʔεΛݩʹϓϥΠϕʔτΫϥ΢υͷ"1*ͳͲΛૢ࡞
    ,VCFSOFUFTΫϥελΛߏஙɾ؅ཧ͢Δ࢓૊Έ
    ؅ཧΫϥελ
    W
    ϢʔβʔΫϥελ
    W
    ϢʔβʔΫϥελ
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec:


    topology:


    version: v1.20.7


    controlPlane:


    replicas: 3


    workers:


    machineDeployments:


    - class: type1-workers


    name: cpu-nodepool-01


    replicas: 3
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec:


    topology:


    version: v1.20.7


    controlPlane:


    replicas: 3


    workers:


    machineDeployments:


    - class: type1-workers


    name: cpu-nodepool-01


    replicas: 3
    CVJMENBOBHF
    CVJMENBOBHF
    $VTUPN$POUSPMMFST
    BOE
    $VTUPN8FCIPPLT
    $VTUPN$POUSPMMFST
    BOE
    $VTUPN8FCIPPLT
    ϚϧνϓϥΠϕʔτΫϥ΢υ΋ରԠ

    View Slide

  6. ෳ਺ͷΫϥελʹର͢Δ
    $VTUPN$POUSPMMFSΛ༰қʹ࣮૷Մೳ
    ໿೥ͷܦݧΛ౿·͑ͯ
    ͞Βʹ$POUSPMMFSԽΛਪਐ

    View Slide

  7. &YUFOEJOH,VCFSOFUFTجૅ

    View Slide

  8. ,VCFSOFUFT$POUSPMMFS
    ಛఆͷϦιʔεͷঢ়ଶΛએݴ͞Εͨঢ়ଶʹௐ੔͢ΔϓϩάϥϜ
    ,VCFSOFUFT"1*Λհͯ͠Ϧιʔεͷૢ࡞΍೚ҙͷॲཧΛߦ͏

    ,VCFSOFUFTΫϥελ
    3FQMJDB4FU$POUSPMMFS
    kind: ReplicaSet


    spec:


    replicas: 3


    template:


    spec:


    containers:


    - name: app


    image: nginx:1.19
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    XBUDI
    3FQMJDB4FUͷఆٛΛݩʹ
    1PEͷ࡞੒࡟আΛߦ͍ɺ
    ϨϓϦΧ਺Λҡ࣋
    $SFBUF1PE
    %FMFUF1PE
    ྫʣ3FQMJDB4FU$POUSPMMFSɿࢦఆ͞ΕͨϨϓϦΧ਺Ͱ1PEΛҡ࣋

    View Slide

  9. ,VCFSOFUFT$POUSPMMFS
    ಛఆͷϦιʔεͷঢ়ଶΛએݴ͞Εͨঢ়ଶʹௐ੔͢ΔϓϩάϥϜ
    ,VCFSOFUFT"1*Λհͯ͠Ϧιʔεͷૢ࡞΍೚ҙͷॲཧΛߦ͏

    ྫʣ3FQMJDB4FU$POUSPMMFSɿࢦఆ͞ΕͨϨϓϦΧ਺Ͱ1PEΛҡ࣋
    ,VCFSOFUFTΫϥελ
    3FQMJDB4FU$POUSPMMFS
    kind: ReplicaSet


    spec:


    replicas: 3


    template:


    spec:


    containers:


    - name: app


    image: nginx:1.19
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    XBUDI
    $SFBUF1PE
    %FMFUF1PE 3FQMJDB4FUͷఆٛΛݩʹ
    1PEͷ࡞੒࡟আΛߦ͍ɺ
    ϨϓϦΧ਺Λҡ࣋
    XBUDI

    View Slide

  10. 3FQMJDB4FU$POUSPMMFSͷ࣮૷
    RS=$(kubectl -n NAMESPACE get rs NAME -o json)


    FPods=$(kubectl get pods -l SELECTOR | wc -l)


    DIFF=$(( $FPods – RS.SPEC.REPLICAS ))


    if $FPods -lt 0; then


    kubectl -n NAMESPACE


    create pod –f RS.SPEC.TEMPLATE


    else if $FPods –gt 0; then


    kubectl -n NAMESPACE delete pods TARGET_POD


    fi


    rs, err := rsc.rsLister.ReplicaSets(namespace).Get(name)


    allPods, err :=


    rsc.podLister.Pods(rs.Namespace).List(labels.Everything())


    filteredPods := controller.FilterActivePods(allPods)


    filteredPods, err = rsc.claimPods(rs, selector, filteredPods)


    diff := len(filteredPods) - int(*(rs.Spec.Replicas))


    if diff < 0 {


    rsc.podControl.CreatePodsWithControllerRef(

    rs.Namespace, &rs.Spec.Template, ...


    )


    } else if diff > 0 {


    rsc.podControl.DeletePod(rs.Namespace, targetPod.Name, rs)


    }


    https://github.com/kubernetes/kubernetes/blob/release-1.21/pkg/controller/replicaset/replica_set.go ΑΓվมͯ͠ൈਮ
    3FQMJDB4FU$POUSPMMFS
    XBUDI $SFBUF1PE
    %FMFUF1PE
    ,VCFSOFUFTΫϥελ
    (P
    4IFMM
    3FDPODJMJBUJPO-PPQ

    View Slide

  11. ,VCFSOFUFT$POUSPMMFST
    ,VCFSOFUFT͸ɺ༷ʑͳϦιʔεͱ$POUSPMMFS͕

    ඇಉظʹڠௐಈ࡞͢Δ෼ࢄγεςϜ
    %FQMPZNFOU$POUSPMMFS
    ,VCFSOFUFTΫϥελ
    3FQMJDB4FU$POUSPMMFS
    1PE$POUSPMMFS LVCFMFU

    4FSWJDF$POUSPMMFS

    View Slide

  12. $VTUPN$POUSPMMFS
    ಛఆͷϦιʔεͷఆٛʹԠͯ͡ɺ೚ҙͷॲཧΛߦ͏$POUSPMMFS
    %FQMPZNFOU$POUSPMMFS
    ,VCFSOFUFTΫϥελ
    3FQMJDB4FU$POUSPMMFS
    1PE$POUSPMMFS LVCFMFU

    4FSWJDF$POUSPMMFS
    .Z$VTUPN$POUSPMMFS
    ྫʣ
    w %FQMPZNFOU࡞੒࣌ʹ4FSWJDF΋ಉ࣌ʹ࡞੒
    w 1PE࡞੒࣌ʹ4MBDL௨஌

    View Slide

  13. $VTUPN3FTPVSDF
    ಛఆͷϦιʔεͷఆٛʹԠͯ͡ɺ೚ҙͷॲཧΛߦ͏$POUSPMMFS
    ɹɹ೚ҙͷϑΟʔϧυΛ࣋ͭ৽ͨͳϦιʔεΛಠࣗʹఆٛՄೳ
    kind: MySQL


    spec:


    replicas: 3


    version: 8.0.23
    kind: StatefulSet


    spec: {...(লུ)...}
    kind: Service


    spec: {...(লུ)...}
    ,VCFSOFUFTΫϥελ
    .Z42-$POUSPMMFS
    XBUDI
    .BOBHF
    4UBUFGVM4FU4FSWJDF

    View Slide

  14. "ENJTTJPO8FCIPPL
    ,VCFSOFUFT"1*ʹϦΫΤετ͕ೖ͖ͬͯͨλΠϛϯάͰ
    ϦιʔεͷݕূɾมߋΛ8FCIPPLͰ࣮ߦ͢Δ࢓૊Έ
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19


    - name: sidecar


    image: envoy:v1.21.0
    .VUBUJOH
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:latest
    kind: Pod


    spec:


    containers:


    - name: app


    image: nginx:1.19
    7BMJEBUJOH

    View Slide

  15. LVCFCVJMEFSͱؔ࿈ϥΠϒϥϦɾπʔϧ܈
    LVCFCVJMEFS0QFSBUPS4%,
    ,VCFSOFUFT$POUSPMMFSΛ࣮૷͢ΔͨΊͷϑϨʔϜϫʔΫʢ#VJMEJOϦιʔε΋ରԠՄೳʣ
    DMJFOUHP
    $POUSPMMFSUPPMT
    LVCFCVJMEFS
    DPOUSPMMFSSVOUJNF
    DPOUSPMMFSUPPMT
    DPOUSPMMFSͷ਽ܗDPEF΍$3%ͷϚχϑΣετΛੜ੒͢Δπʔϧ
    DPOUSPMMFSSVOUJNF
    .BOBHFSɾ3FDPODJMFSΛ࣮ݱ͢ΔϥΠϒϥϦ
    DMJFOUHP
    ,VCFSOFUFT"1*Λૢ࡞͢ΔϥΠϒϥϦ
    ֦ுʹ͸ϑϨʔϜϫʔΫLVCFCVJMEFSͳͲͷϑϨʔϜϫʔΫɾϥΠϒϥϦ͕ར༻Մೳ

    View Slide

  16. .BOBHFSBOE$POUSPMMFST
    ෳ਺ͷ$POUSPMMFSΛ·ͱΊͯ.BOBHFS͕άϧʔϓԽͯ͠؅ཧ
    .BOBHFS୯ҐͰ*OGPSNFSΛڞ༗͢ΔͨΊɺ"1*ͷෛՙܰݮ
    .Z$VTUPN$POUSPMMFS"
    ,VCFSOFUFTΫϥελ
    .Z$VTUPN$POUSPMMFS#
    $POUSPMMFS.BOBHFS
    4IBSFE*OGPSNFS
    4DIFNF
    ϦιʔεͷΩϟογϡ

    View Slide

  17. $POUSPMMFSͷར༻ύλʔϯ

    View Slide

  18. $POUSPMMFSͷར༻ύλʔϯ
    $POUSPMMFS͕8BUDI͢ΔϦιʔε
    w ৽ͨʹఆٛͨ͠$VTUPN3FTPVSDF
    w طଘͷ$VTUPN3FTPVSDF
    w #VJMEJO3FTPVSDF
    $POUSPMMFS͕ૢ࡞͢Δର৅
    w ,VCFSOFUFT3FTPVSDF
    w ֎෦ͷγεςϜ
    º

    View Slide

  19. ৽ͨʹఆٛͨ͠$VTUPN3FTPVSDF
    ʙ$MVTUFS"1*ʙ
    ؅ཧΫϥελʹσϓϩΠ͞Εͨ$MVTUFSϦιʔεΛݩʹ
    ,VCFSOFUFTΫϥελΛߏஙɾ؅ཧ͢Δ࢓૊Έ
    ؅ཧΫϥελ
    W
    ϢʔβʔΫϥελ
    W
    ϢʔβʔΫϥελ
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec:


    topology:


    version: v1.20.7


    controlPlane:


    replicas: 3


    workers:


    machineDeployments:


    - class: type1-workers


    name: cpu-nodepool-01


    replicas: 3
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec:


    topology:


    version: v1.20.7


    controlPlane:


    replicas: 3


    workers:


    machineDeployments:


    - class: type1-workers


    name: cpu-nodepool-01


    replicas: 3
    CVJMENBOBHF
    CVJMENBOBHF
    ˞ݫີʹ͸0QFO4UBDL$MVTUFS΍0QFO4UBDL.BDIJOFΛར༻ɻ

    View Slide

  20. طଘͷ$VTUPN3FTPVSDFʹର͢Δ$POUSPMMFS
    ʙ"SHP$%$MVTUFS4FDSFUͷੜ੒ʙ
    ؅ཧΫϥελ্ͷ"SHP$%͕ෳ਺ΫϥελͷΞυΦϯΛ؅ཧ
    ɹFH1SPNFUIFVTɺ$MPVE$POUSPMMFS.BOBHFSɺ*OHSFTT$POUSPMMFSɺFUD
    ؅ཧର৅ͷ,VCFSOFUFTΫϥελͷೝূ৘ใΛ"SHP$%ʹొ࿥͢Δඞཁ͕͋Δ
    ؅ཧΫϥελ
    W
    ϢʔβʔΫϥελ
    W
    ϢʔβʔΫϥελ
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec: {...(লུ)...}
    NBOBHFBEEPOT
    NBOBHFBEEPOT
    "SHP$%
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec: {...(লུ)...}
    ʁ

    View Slide

  21. $MVTUFS"1*͕ੜ੒͢Δೝূ৘ใΛ΋ͱʹ"SHP$%༻ͷΫϥελೝূ৘ใͷ4FDSFUϦιʔεΛੜ੒
    ؅ཧΫϥελ
    W
    ϢʔβʔΫϥελ
    W
    ϢʔβʔΫϥελ
    NBOBHFBEEPOT
    NBOBHFBEEPOT
    "SHP$%
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec: {...(লུ)...}
    kind: Secret


    metadata:


    name: test-cluster-01


    namespace: argocd


    labels:


    argocd.argoproj.io/secret-type: cluster


    type: Opaque


    stringData:


    name: test-cluster-01


    server: https://10.0.0.1:6443


    clusterResources: true


    config: ʢArgo CD ಛ༗ͷkubeconfig૬౰ͷϑΝΠϧʣ
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec: {...(লུ)...}
    ",&"EEPO.BOBHFS
    طଘͷ$VTUPN3FTPVSDFʹର͢Δ$POUSPMMFS
    ʙ"SHP$%$MVTUFS4FDSFUͷੜ੒ʙ
    kind: Secret


    metadata:


    name: test-cluster-01


    namespace: argocd


    labels:


    argocd.argoproj.io/secret-type: cluster


    type: Opaque


    stringData:


    name: test-cluster-01


    server: https://10.0.0.1:6443


    clusterResources: true


    config: ʢArgo CD ಛ༗ͷkubeconfig૬౰ͷϑΝΠϧʣ

    View Slide

  22. #VJMEJO3FTPVSDFʹର͢Δ$POUSPMMFS
    ʙ*OHSFTT$POUSPMMFSʙ
    ʮ*OHSFTTϦιʔεͷઃఆมߋʯʮ/PEFͷঢ়ଶ΍਺ͷมߋʯΛ΋ͱʹɺ
    ϓϥΠϕʔτΫϥ΢υ্ͷϩʔυόϥϯαΛૢ࡞
    *OHSFTT$POUSPMMFS
    kind: Ingress


    metadata:


    name: my-http-loadbalancer


    spec: {...(লུ)...}
    XBUDI
    *OHSFTTͷઃఆ΍/PEFͷঢ়ଶΛ΋ͱʹ
    ϩʔυόϥϯαͷઃఆΛมߋ
    ϩʔυόϥϯα"1*
    ʢϓϥΠϕʔτΫϥ΢υ"1*ʣ
    ,VCFSOFUFTΫϥελ
    kind: Node


    metadata:


    name: node01
    kind: Node


    metadata:


    name: node01
    kind: Node


    metadata:


    name: node01
    XBUDI

    View Slide

  23. $POUSPMMFSͷར༻ύλʔϯ
    $POUSPMMFS͕8BUDI͢ΔϦιʔε
    w ৽ͨʹఆٛͨ͠$VTUPN3FTPVSDF
    w طଘͷ$VTUPN3FTPVSDF
    w #VJMEJO3FTPVSDF
    $POUSPMMFS͕ૢ࡞͢Δର৅
    w ,VCFSOFUFT3FTPVSDF
    w ֎෦ͷγεςϜ
    º

    View Slide

  24. $VTUPN3FTPVSDFͷར༻ύλʔϯ

    View Slide

  25. $VTUPN3FTPVSDFͷར༻ύλʔϯ
    wಛఆͷΞϓϦέʔγϣϯͷந৅Խ
    w֎෦Ϧιʔεͷঢ়ଶఆٛ
    wෳࡶͳఆٛΛѻ͍΍͍͢ܗʹ·ͱΊΔ
    wσʔλετΞͱͯ͠ར༻͢Δ

    View Slide

  26. ಛఆͷΞϓϦέʔγϣϯͷந৅Խ
    ΞϓϦέʔγϣϯͷઃఆ஋Λந৅Խͨ͠ϦιʔεΛఆٛ
    ɹࢀߟɿ0QFSBUPS8IJUFQBQFS
    FH.Z42-0QFSBUPS
    ग़యɿIUUQTHJUIVCDPNDODGUBHBQQEFMJWFSZCMPCNBJOPQFSBUPSXHXIJUFQBQFS
    kind: MySQL


    metadata:


    name: sample


    spec:


    replicas: 3


    version: 8.0.23


    password: somepass
    kind: StatefulSet


    metadata:


    name: sample


    spec:


    replicas: 3


    template:


    spec:


    containers:


    - name: mysql


    image: mysql:8.0.23


    env:


    - name: MYSQL_ROOT_PASSWORD


    valueFrom:


    secretKeyRef:


    name: sample


    key: MYSQL_ROOT_PASSWORD
    kind: Service


    spec: {...(লུ)...}
    kind: Secret


    metadata:


    name: sample


    stringData:


    MYSQL_ROOT_PASSWORD : somepass
    $POUSPMMFS

    View Slide

  27. ֎෦Ϧιʔεͷঢ়ଶఆٛ
    ֎෦ϦιʔεΛૢ࡞͢ΔͨΊͷઃఆΛఆٛ
    ɹFH"$.&Λར༻ͨ͠ূ໌ॻൃߦͷઃఆʢ$FSU.BOBHFSʣ
    %/4
    "$.&%/4$IBMMFOHF
    *OHSFTT
    "$.&)551$IBMMFOHF
    $POUSPMMFS
    kind: Certificate


    metadata:


    name: tls-sample


    spec:


    dnsNames:


    - sample.example.com


    issuerRef:


    group: cert-manager.io


    kind: ClusterIssuer


    name: letsencrypt-production


    secretName: tls-sample-secret
    kind: ClusterIssuer


    metadata:


    name: letsencrypt-production


    spec:


    acme:


    server: https://acme-v02...


    solvers:


    - dns01:


    route53: {...}

    View Slide

  28. ෳࡶͳఆٛΛѻ͍΍͍͢ܗʹ·ͱΊΔ
    ෳࡶԽ͢ΔઃఆΛ$VTUPN3FTPVSDFͱͯ͠੾Γग़ͯ͠ར༻
    ɹFH4JEFDBSͷઃఆΛ$VTUPN3FTPVSDFͰઃఆՄೳʹ͢Δ
    kind: SidecarProfile


    spec:


    selector:


    inject-envoy: true


    containers:


    - image: envoy


    env:


    - name: MY_KEY


    Value: MY_VALUE
    kind: Pod


    metadata:


    name: my-pod


    annotations:


    sidecar/profile: inject-envoy


    spec: {...(লུ)...}
    kind: SidecarProfile


    metadata:


    name: inject-envoy


    spec:


    containers:


    - image: envoy


    env:


    - name: MY_KEY


    Value: MY_VALUE
    kind: Pod


    metadata:


    name: my-pod


    labels:


    inject-envoy: true


    spec: {...(লུ)...}
    kind: Pod


    metadata:


    name: my-pod


    spec:


    containers:


    - {...(লུ)...}


    - image: envoy


    env:


    - name: MY_KEY


    value: MY_VALUE
    03
    ˞આ໌ྫͰ͢

    View Slide

  29. $VTUPN3FTPVSDFͷσʔλετΞར༻
    $VTUPN3FTPVSDFΛεΩʔϚΛ࣋ͬͨ؆қతͳσʔλετΞͱͯ͠ར༻
    *NQFSTPOBUF6TFSSFRVFTUTʹΑΓɺ,VCFSOFUFT3#"$ͷݖݶൣғͰϦΫΤετΛॲཧ
    8FC$POTPMF
    (16BB4"1*4FSWFS
    *NBHF$VTUPN3FTPVSDFͷ$36%ʢ*NQFSTPOBUF6TFSSFRVFTUTʣ
    apiVersion: ml.cyberagent.io/v1


    kind: Image


    metadata:


    name: asia.gcr.io-GCP_PROJECT-base-notebook-notebook-6.1.5


    spec:


    name: asia.gcr.io/GCP_PROJECT/base-notebook


    tag: notebook-6.1.5


    enable: true


    imagePullSecretRef: gcr-GCP_PROJECT
    apiVersion: ml.cyberagent.io/v1


    kind: Image


    metadata:


    name: asia.gcr.io-GCP_PROJECT-base-notebook-notebook-6.1.5


    spec:


    name: asia.gcr.io/GCP_PROJECT/base-notebook


    tag: notebook-6.1.5


    enable: true


    imagePullSecretRef: gcr-GCP_PROJECT

    View Slide

  30. $POUSPMMFSઃܭ࣌ͷϑϩʔ

    View Slide

  31. 3FDPODJMFؔ਺ʹΑΔௐ੔
    0QFO"1*W4DIFNB7BMJEBUJPO
    ه࿥
    ௐ੔
    ݕূɾมߋ
    $&-&YQSFTTJPO
    7BMJEBUJOH8FCIPPL
    .VUBUJOH8FCIPPL
    $POWFSTJPO8FCIPPL
    ݕূɾมߋ
    $POUSPMMFSىಈ࣌
    4ZOD1FSJPEͷ࣮ߦ࣌
    Ϧιʔεͷมߋ࣌
    Πϕϯτൃੜ࣌
    ࠶࣮ߦ࣌
    ࣮ߦ੍ޚ
    ه࿥
    ௐ੔
    ࣮ߦ੍ޚ
    ϦΫΤετΛड͚෇͚ͯ
    ϦιʔεΛొ࿥͢Δ·Ͱͷલॲཧ
    Ϧιʔεొ࿥ޙɺ࣮ࡍʹॲཧ͢Δ಺༰
    ॲཧޙͷ݁ՌΛه࿥͢Δ
    ͋Δ΂͖ঢ়ଶʹऩଋͤ͞ଓ͚Δ࣮ߦ੍ޚ
    'JOBMJ[FST
    0XOFS3FGFSFODFT
    ผϦιʔεͷ࡞੒ɾ࡟আ
    $VTUPN.FUSJDTͷߋ৽
    4UBUVTϑΟʔϧυͷมߋ
    "OOPUBUJPOTͷมߋ
    &WFOUϦιʔεͷ࡞੒
    ϩάग़ྗ
    /PUJ
    fi
    DBUJPO&OHJOF
    ֎෦௨஌

    View Slide

  32. 0QFO"1*W4DIFNB7BMJEBUJPO
    ه࿥
    ௐ੔
    ݕূɾมߋ
    $&-&YQSFTTJPO
    7BMJEBUJOH8FCIPPL
    .VUBUJOH8FCIPPL
    $POWFSTJPO8FCIPPL
    ݕূɾมߋ
    $POUSPMMFSىಈ࣌
    4ZOD1FSJPEͷ࣮ߦ࣌
    Ϧιʔεͷมߋ࣌
    Πϕϯτൃੜ࣌
    ࠶࣮ߦ࣌
    ࣮ߦ੍ޚ
    ه࿥
    ௐ੔
    ࣮ߦ੍ޚ
    ϦΫΤετΛड͚෇͚ͯ
    ϦιʔεΛొ࿥͢Δ·Ͱͷલॲཧ
    Ϧιʔεొ࿥ޙɺ࣮ࡍʹॲཧ͢Δ಺༰
    ॲཧޙͷ݁ՌΛه࿥͢Δ
    ͋Δ΂͖ঢ়ଶʹऩଋͤ͞ଓ͚Δ࣮ߦ੍ޚ
    3FDPODJMFؔ਺ʹΑΔௐ੔
    'JOBMJ[FST
    0XOFS3FGFSFODFT
    ผϦιʔεͷ࡞੒ɾ࡟আ
    $VTUPN.FUSJDTͷߋ৽
    4UBUVTϑΟʔϧυͷมߋ
    "OOPUBUJPOTͷมߋ
    &WFOUϦιʔεͷ࡞੒
    ϩάग़ྗ
    /PUJ
    fi
    DBUJPO&OHJOF
    ֎෦௨஌

    View Slide

  33. $VTUPN3FTPVSDFͷ7BMJEBUJPO
    $VTUPN3FTPVSDFʹର͢Δ؆қతͳ7BMJEBUJPO
    w 0QFO"1*Wͷ4DIFNB7BMJEBUJPO
    w $&-ʢ$PNNPO&YQSFTTJPO-BOHVBHFʣ7BMJEBUJPO
    ग़యɿIUUQTTXBHHFSJPTQFDJ
    fi
    DBUJPOTDIFNBPCKFDU
    ग़యɿIUUQTUPPMTJFUGPSHIUNMESBGUXSJHIUKTPOTDIFNBWBMJEBUJPO
    ग़యɿIUUQTPQFOTPVSDFHPPHMFQSPKFDUTDFM
    ग़యɿIUUQTHJUIVCDPNLVCFSOFUFTFOIBODFNFOUTJTTVFT
    openAPIV3Schema:


    type: object


    properties:


    spec:


    type: object


    properties:


    cronSpec:


    type: string


    pattern: '^(\d+|\*)(/\d+)?(\s+(\d+|\*)(/\d+)?){4}$'


    replicas:


    type: integer


    minimum: 0


    maxReplicas:


    type: integer


    minimum: 1


    x-kubernetes-validation-rules:


    - rule: "self.replicas <= self.maxReplicas"


    message: "replicas should be smaller than or equal to maxReplicas."
    e.g.ʣ


    spec:


    cronSpec: "* * * * */5"


    replicas: 10


    maxReplicas: 20

    View Slide

  34. 7BMJEBUJOH8FCIPPL
    γεςϜͰར༻͢Δ"OOPUBUJPOTʹ੍໿Λ͔͚Δ
    w FH֘౰ͷ",&7FSTJPO͕ଘࡏ͢Δ͔
    w FH,VCFSOFUFT7FSTJPO͕",&7FSTJPOʹରԠ͍ͯ͠Δ͔
    w FH$POUSPM1MBOFͷόʔδϣϯ8PSLFSͷόʔδϣϯҎ্͕੒ཱ͍ͯ͠Δ͔
    kind: Cluster


    metadata:


    name: test-cluster-01


    metadata:


    annotations:


    ake.cycloud.io/version: v1.23-ake.220201


    spec:


    topology:


    version: v1.23.1


    controlPlane:


    replicas: 3


    workers:


    machineDeployments:


    - class: type1-workers


    name: cpu-nodepool-01


    replicas: 3
    ಛఆͷϑΟʔϧυʹରͯ͠
    ೚ҙͷνΣοΫ
    7BMJEBUJOH
    ˞(BUFLFFQFS3FHPͰ΋୅ସՄೳ

    View Slide

  35. .VUBUJOH8FCIPPL
    طଘͷϑΟʔϧυͷઃఆ஋Λ΋ͱʹϑΟʔϧυΛઃఆ
    w FH,VCFSOFUFTόʔδϣϯʹରԠͨ͠ϚΠφʔϦϦʔε"OOPUBUJPOΛ෇༩
    kind: Cluster


    metadata:


    name: test-cluster-01


    metadata:


    annotations:


    ake.cycloud.io/kubernetes-release: v1.23


    spec:


    topology:


    version: v1.23.1


    controlPlane:


    replicas: 3


    workers:


    machineDeployments:


    - class: type1-workers


    name: cpu-nodepool-01


    replicas: 3
    ಛఆͷϑΟʔϧυΛ௥Ճɾमਖ਼
    .VUBUJOH
    ˞(BUFLFFQFSͰ΋͋Δఔ౓͸୅ସՄೳ

    View Slide

  36. $POWFSTJPO8FCIPPL
    $VTUPN3FTPVSDFͷ"1*7FSTJPO௥ैͷରԠ
    εΩʔϚͷมߋʹ൐͍૬ޓʹม׵Λߦ͏8FCIPPLTFSWFS
    ,VCFSOFUFT಺෦Ͱ͸ͲͪΒ͔ҰํͷܗࣜͰอ࣋
    $POWFSTJPO
    apiVersion: example.com/v1


    kind: CronJob


    annotations:


    example.com/suspend: true


    spec:


    schedule: "*/1 * * * *"
    apiVersion: example.com/v2


    kind: CronJob


    spec:


    schedule:


    minute: "*/1"


    hour: "*"


    dayOfMonth: "*"


    month: "*"


    dayOfWeek: "*"


    suspend: true

    View Slide

  37. 0QFO"1*W4DIFNB
    ه࿥
    ௐ੔
    ݕূɾมߋ
    $&-&YQSFTTJPO
    7BMJEBUJOH8FCIPPL
    .VUBUJOH8FCIPPL
    $POWFSTJPO8FCIPPL
    ݕূɾมߋ
    $POUSPMMFSىಈ࣌
    4ZOD1FSJPEͷ࣮ߦ࣌
    Ϧιʔεͷมߋ࣌
    Πϕϯτൃੜ࣌
    ࠶࣮ߦ࣌
    ࣮ߦ੍ޚ
    ه࿥
    ௐ੔
    ࣮ߦ੍ޚ
    ϦΫΤετΛड͚෇͚ͯ
    ϦιʔεΛొ࿥͢Δ·Ͱͷલॲཧ
    Ϧιʔεొ࿥ޙɺ࣮ࡍʹॲཧ͢Δ಺༰
    ॲཧޙͷ݁ՌΛه࿥͢Δ
    ͋Δ΂͖ঢ়ଶʹऩଋͤ͞ଓ͚Δ࣮ߦ੍ޚ
    3FDPODJMFؔ਺ʹΑΔௐ੔
    'JOBMJ[FST
    0XOFS3FGFSFODFT
    ผϦιʔεͷ࡞੒ɾ࡟আ
    $VTUPN.FUSJDTͷߋ৽
    4UBUVTϑΟʔϧυͷมߋ
    "OOPUBUJPOTͷมߋ
    &WFOUϦιʔεͷ࡞੒
    ϩάग़ྗ
    /PUJ
    fi
    DBUJPO&OHJOF
    ֎෦௨஌

    View Slide

  38. 3FDPODJMFؔ਺ͱ8PSLRVFVF
    0CKFDU͝ͱʹ3FDPODJMFؔ਺Λ࣮ߦ͠ɺએݴ͞Εͨঢ়ଶ΁ௐ੔
    func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request)

    (ctrl.Result, error) {


    // Reconcile ର৅ͷ Object ͷऔಘ


    obj := &clusterv1.Cluster{}


    err := r.Get(ctx, req.NamespacedName, obj)


    if err != nil {


    return ctrl.Result{}, client.IgnoreNotFound(err)


    }


    ...ʢႈ౳ʹͳΔΑ͏ͳॲཧʣ...


    return ctrl.Result{}, nil


    }
    8PSLRVFVF
    3FDPODJMF

    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNFQLHSFDPODJMF3FRVFTU
    ग़యɿIUUQTHJUIVCDPNLVCFSOFUFTTBNQMFDPOUSPMMFSCMPCNBTUFSEPDTDPOUSPMMFSDMJFOUHPNE
    NamespacedName {


    Namespace: default


    Name: my-res


    }

    View Slide

  39. err := r.Get(ctx, req.NamespacedName, obj)


    if apierrors.IsNotFound(err) {


    // already deleted


    return ctrl.Result{}, nil


    }


    if !obj.ObjectMeta.DeletionTimestamp.IsZero() {


    // delete in progress


    }
    w ࡟আࡁΈɾ࡟আதͷ൑ఆ
    ࡟আΠϕϯτͷऔΓѻ͍
    3FDPODJMF
    ؔ਺ʹ͸$3&"5&61%"5&%&-&5&

    ͢΂ͯͷΠϕϯτ͕౉ͬͯ͘Δ
    ˞ࢠϦιʔε࡞੒࣌ʹ$SFBUF͢Δ͔6QEBUF͢Δ͔ͷ

    ൑ผ࣌ʹ΋ར༻Մೳͳύλʔϯ
    ˞ ݫີʹ͸'JMUFSJOHͯ͠%&-&5&ΠϕϯτΛࣺͯΔ͜ͱ΋Մೳ

    View Slide

  40. ࠩ෼ͷൺֱͱߋ৽
    0CTFSWFɿݱঢ়ͷ؍ଌ
    "OBMZ[Fɿঢ়ଶͷ෼ੳ
    "DUɿ ཧ૝ঢ়ଶʹ͚ۙͮΔͨΊͷॲཧ
    FHࢠϦιʔεΛ࡞੒͢Δ$POUSPMMFSͷ৔߹
    ʢݱࡏ͸4FSWFSTJEFBQQMZ΁ͷա౉ظͳͨΊɺৄࡉ͸লུʣ
    $SFBUF

    ࢠϦιʔε͕ଘࡏ
    'BMTF
    5SVF
    ࠩ෼͕ଘࡏ 6QEBUF

    5SVF
    %FFQ&RVBMT
    ؔ਺Λར༻

    View Slide

  41. 0XOFS3FGFSFODFT
    ಛఆͷϦιʔεʹґଘ͢ΔϦιʔεͷ࡞੒࣌ʹ਌ࢠؔ܎Λఆٛ
    ɹ਌Ϧιʔε͕࡟আ͞Εͨ৔߹ɺࢠϦιʔε͸࡟আ
    w FH3FQMJDB4FUϦιʔεͱ1PEϦιʔε
    w FH$MVTUFSϦιʔεͱͦΕΛݩʹ࡞੒ͨ͠4FDSFU
    "SHP$%޲͚$MVTUFS઀ଓ༻4FDSFU
    kind: Cluster


    metadata:


    name: test-cluster-01


    spec: {...(লུ)...}
    0XOFS
    kind: Secret


    metadata:


    name: test-cluster-01


    namespace: argocd


    labels:


    argocd.argoproj.io/secret-type: cluster


    ownerReferences:


    - kind: Cluster


    name: test-cluster-01


    blockOwnerDeletion: true


    controller: true


    type: Opaque


    stringData:


    name: test-cluster-01


    server: https://10.0.0.1:6443


    clusterResources: true


    config: ʢArgo CD ಛ༗ͷkubeconfig૬౰ͷϑΝΠϧʣ
    $IJME

    View Slide

  42. 0XOFS3FGFSFODFTͷ෇༩ؔ਺
    setControllerReference(owner, child, scheme)


    w $IJME͸୯Ұͷ0XOFSΛ࣋ͭΑ͏ʹ੍ݶ
    w DPOUSPMMFSUSVF
    w 0XOFSϦιʔε͸$IJMEϦιʔεͷ࡟আ׬ྃΛ଴ػ
    w CMPDL0XOFS%FMFUJPOGBMTF
    0XOFS
    $IJME
    setOwnerReference(owner, child, scheme)


    w શ0XOFSϦιʔε࡟আޙʹ$IJMEϦιʔε͸ඇಉظʹ࡟আ
    w CMPDL0XOFS%FMFUJPOGBMTF
    0XOFS 0XOFS
    $IJME
    0XOFS 0XOFS
    $IJME
    0XOFS
    $IJME
    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNFQLHDPOUSPMMFSDPOUSPMMFSVUJM
    metadata:


    ownerReferences:


    - kind: Cluster


    name: test-cluster-01


    blockOwnerDeletion: true


    controller: true
    0XOFS
    $IJME

    View Slide

  43. 'JOBMJ[FST
    Ϧιʔεͷ࡟আલʹɺಛఆͷॲཧͷ׬ྃ·Ͱ଴ػ͢Δ࢓૊Έ
    w 4FSWJDF࡟আ࣌ʹ֎෦ͷϩʔυόϥϯαʔΛ࡟আ͢Δ
    w $MVTUFSϦιʔε࡟আ࣌ʹ"SHP$%ͷ"QQ1SPKFDU͔ΒΫϥελϨίʔυΛ࡟আ͢Δ
    "QQ1SPKFDU
    ࡟আࡁ $MVTUFS"
    $MVTUFS#
    $MVTUFS"
    "QQ1SPKFDU
    ࡟আલ
    $MVTUFS"
    $MVTUFS#
    ࡟আΠϕϯτ
    8PSLRVFVF
    $MVTUFS"
    "QQ1SPKFDU
    ࡟আࡁ
    $MVTUFS"
    $MVTUFS#
    3FDPODJMF

    $MVTUFS"
    ࡟আॲཧ
    "QQ1SPKFDU
    ࡟আࡁ
    $MVTUFS"
    $MVTUFS#
    $MVTUFS"
    'BJMBOE3FRVFVF
    $POUSPMMFS࠶ىಈʢϝϞϦ্ͷ8PSLRVFVF͸ΫϦΞʣ
    ࡟আ͞ΕͨΠϕϯτ͸ফࣦ
    3FDPODJMFؔ਺͸ೋ౓ͱ࣮ߦ͞Εͳ͍
    ॲཧͷྲྀΕ

    View Slide

  44. 'JOBMJ[FST
    Ϧιʔεͷ࡟আલʹɺಛఆͷॲཧͷ׬ྃ·Ͱ଴ػ͢Δ࢓૊Έ
    w 4FSWJDF࡟আ࣌ʹ֎෦ͷϩʔυόϥϯαʔΛ࡟আ͢Δ
    w $MVTUFSϦιʔε࡟আ࣌ʹ"SHP$%ͷ"QQ1SPKFDU͔ΒΫϥελϨίʔυΛ࡟আ͢Δ
    ॲཧͷྲྀΕ
    "QQ1SPKFDU
    ࡟আ଴ػ $MVTUFS"
    $MVTUFS#
    $MVTUFS"
    "QQ1SPKFDU
    ࡟আલ
    $MVTUFS"
    $MVTUFS#
    ࡟আΠϕϯτ
    8PSLRVFVF
    $MVTUFS"
    "QQ1SPKFDU
    ࡟আ଴ػ
    $MVTUFS"
    $MVTUFS#
    3FDPODJMF

    $MVTUFS"
    ࡟আॲཧ
    "QQ1SPKFDU
    ࡟আ଴ػ
    $MVTUFS"
    $MVTUFS#
    $MVTUFS"
    'BJMBOE3FRVFVF
    $POUSPMMFS࠶ىಈʢϝϞϦ্ͷ8PSLRVFVF͸ΫϦΞʣ
    ࠶౓࡟আΠϕϯτΛੜ੒
    ࡟আΠϕϯτ

    View Slide

  45. 'JOBMJ[FSTͷॲཧϑϩʔ
    ґଘ͢Δࣄલॲཧ͕׬͔ྃͨ͠͸'JOBMJ[FSʢ೚ҙͷ஋ͷϑϥάʣͰ൑அ
    w ॳճ3FDPODJMF࣌ͷং൫ɿɹ'JOBMJ[FSΛ෇༩
    w Ϧιʔε࡟আ࣌ɿɹɹɹɹɹґଘ͢ΔॲཧΛߦ͔ͬͯΒ'JOBMJ[FSΛ࡟আ
    ɹɹɹɹɹɹɹɹɹɹɹɹɹɹࣦഊͨ͠৔߹3FDPODJMFؔ਺Λ࠶࣮ߦ
    'JOBMJ[FS௥Ճ ओॲཧ
    Ϧιʔε࡟আத
    'BMTF
    5SVF
    ґଘ͢Δ
    ࣄલॲཧ
    'JOBMJ[FS࡟আ Ϧιʔε࡟আ
    ࣦഊ
    kind: Cluster


    metadata:


    finalizers:


    - ake.cycloud.io/unregister-argocd


    name: my-cluster


    spec: {}

    View Slide

  46. ʢࢀߟʣ'JOBMJ[FSTͷ࣮૷ྫ
    Ϧιʔε͕࡟আத͔͸NFUBEBUBEFMFUJPO5JNFTUBNQΛ΋ͱʹ൑அ
    if obj.ObjectMeta.DeletionTimestamp.IsZero() {


    if !controllerutil.ContainsFinalizer(obj, "ake.cycloud.io/unregister-argocd"){


    controllerutil.AddFinalizer(obj, "ake.cycloud.io/unregister-argocd")


    if err := r.Update(ctx, obj); err != nil {


    return ctrl.Result{}. err


    }


    }


    } else {


    if controllerutil.ContainsFinalizer(obj, "ake.cycloud.io/unregister-argocd"){


    if err :=ʢ࡟আલͷґଘॲཧʣ; err != nil {


    return ctrl.Result{}, err


    }


    controllerutil.RemoveFinalizer(obj, "ake.cycloud.io/unregister-argocd")


    if err := r.Update(ctx, obj); err != nil {


    return ctrl.Result{}. err


    }


    }


    return ctrl.Result{}, nil


    }
    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNFQLHDPOUSPMMFSDPOUSPMMFSVUJM

    View Slide

  47. 0QFO"1*W4DIFNB
    ه࿥
    ผϦιʔεͷ࡞੒ɾ࡟আ
    $VTUPN.FUSJDTͷߋ৽
    4UBUVTϑΟʔϧυͷมߋ
    "OOPUBUJPOTͷมߋ
    &WFOUϦιʔεͷ࡞੒
    ϩάग़ྗ
    ௐ੔
    ݕূɾมߋ
    $&-&YQSFTTJPO
    7BMJEBUJOH8FCIPPL
    .VUBUJOH8FCIPPL
    $POWFSTJPO8FCIPPL
    ݕূɾมߋ
    $POUSPMMFSىಈ࣌
    4ZOD1FSJPEͷ࣮ߦ࣌
    Ϧιʔεͷมߋ࣌
    Πϕϯτൃੜ࣌
    ࠶࣮ߦ࣌
    ࣮ߦ੍ޚ
    ه࿥
    ௐ੔
    ࣮ߦ੍ޚ
    ϦΫΤετΛड͚෇͚ͯ
    ϦιʔεΛొ࿥͢Δ·Ͱͷલॲཧ
    Ϧιʔεొ࿥ޙɺ࣮ࡍʹॲཧ͢Δ಺༰
    ॲཧޙͷ݁ՌΛه࿥͢Δ
    ͋Δ΂͖ঢ়ଶʹऩଋͤ͞ଓ͚Δ࣮ߦ੍ޚ
    3FDPODJMFؔ਺ʹΑΔௐ੔
    'JOBMJ[FST
    0XOFS3FGFSFODFT
    /PUJ
    fi
    DBUJPO&OHJOF
    ֎෦௨஌

    View Slide

  48. ผϦιʔεͷ࡞੒ɾ࡟আ
    ग़యɿIUUQTHJUIVCDPNDODGUBHBQQEFMJWFSZCMPCNBJOPQFSBUPSXHXIJUFQBQFS
    kind: MySQL


    metadata:


    name: sample


    spec:


    replicas: 3


    version: 8.0.23


    password: somepass
    kind: StatefulSet


    metadata:


    name: sample


    spec:


    replicas: 3


    template:


    spec:


    containers:


    - name: mysql


    image: mysql:8.0.23


    env:


    - name: MYSQL_ROOT_PASSWORD


    valueFrom:


    secretKeyRef:


    name: sample


    key: MYSQL_ROOT_PASSWORD
    kind: Service


    spec: {...(লུ)...}
    kind: Secret


    metadata:


    name: sample


    stringData:


    MYSQL_ROOT_PASSWORD : somepass
    $POUSPMMFS
    $POUSPMMFS͕3FDPODJMFதʹผϦιʔεΛ࡞੒
    ɹ㱺࡞੒͞ΕͨϦιʔεͷঢ়ଶࣗମ͕ه࿥

    View Slide

  49. 4UBUVTϑΟʔϧυͷมߋ
    3FTPVSDFͷTUBUVTϑΟʔϧυͷߋ৽ʢ#VJMEJOSFTPVSDFؚΉʣ
    ɹશମͷঢ়گɺ઀ଓ৘ใɺFUD
    kind: MySQL


    metadata:


    name: sample


    spec:


    replicas: 3


    version: 8.0.23


    password: somepass


    status:


    primaryNode: sample-mysql-0


    updatedNodes: 2


    updateStatus:


    - node: sample-mysql-0


    version: 8.0.11


    - node: sample-mysql-1


    version: 8.0.23


    - node: sample-mysql-2


    version: 8.0.23


    connectionInfo:


    ipAddress: x.x.x.x


    username: mysql


    Password: somepass


    conditions: []
    obj.Status = api.MySQLStatus{...}


    err := r.Status().Update(context.TODO(), obj)
    // +kubebuilder:rbac:groups=db.example.com,

    resources=mysql/status,

    verbs=get;update;patch
    w ϚʔΧʔͷઃஔ
    w 4UBUVTϑΟʔϧυͷߋ৽

    View Slide

  50. "OOPUBUJPOTͷมߋ
    4UBUVTϑΟʔϧυͷ୅ΘΓʹ"OOPUBUJPOTΛར༻
    w ؔ࿈͢Δ༷ʑͳϦιʔεʹ෇༩Մೳ
    kind: MySQL


    metadata:


    name: sample


    annotations:


    hoge: fuga


    spec:


    replicas: 3


    version: 8.0.23


    password: somepass
    kind: StatefulSet


    metadata:


    name: sample


    annotations:


    hoge: fuga


    spec:


    replicas: 3


    template:


    spec:


    containers:


    - name: mysql


    image: mysql:8.0.23


    env:


    - name: MYSQL_ROOT_PASSWORD


    valueFrom:


    secretKeyRef:


    name: sample


    key: MYSQL_ROOT_PASSWORD
    kind: Secret


    metadata:


    name: sample


    annotations:


    hoge: fuga


    stringData:


    MYSQL_ROOT_PASSWORD : somepass
    $POUSPMMFS

    View Slide

  51. &WFOUϦιʔεͷ࡞੒
    ಛఆͷΠϕϯτ͝ͱʹ&WFOUϦιʔεΛൃߦ
    LVCFDUMίϚϯυͰ֬ೝ
    PQTHFOJFLVCFSOFUFTFWFOUFYQPSUFSͰ௨஌
    Recorder.Eventf(


    &obj, # Object


    corev1.EventTypeWarning, # EventType


    "SomethingFailed", # Reason


    ”resource %s has error", # message


    obj.Name. # message args


    )
    &WFOU
    8BSOJOH
    ௚ۙ࣌ؒͷ&WFOUϦιʔεͷΈอ࣋ʢσϑΥϧτʣ
    var Recorder = mgr.GetEventRecorderFor("my-cont")
    ग़యɿIUUQTQLHHPEFWLTJPDMJFOUHPUPPMTSFDPSE&WFOU3FDPSEFS
    ग़యɿIUUQTQLHHPEFWLTJPBQJDPSFWQLHDPOTUBOUT
    w .BOBHFS͔Β&WFOU3FDPSEFSͷऔಘ
    w &WFOUϦιʔεͷ࡞੒
    kubectl get events


    kubectl describe RESOURCE
    &WFOU
    8BSOJOH
    3FTPVSDF
    &WFOU
    /PSNBM
    &WFOU
    /PSNBM
    ʜ &WFOU
    8BSOJOH
    &WFOUϦιʔε͸ͭͷϦιʔεʹඥ෇͚

    View Slide

  52. $VTUPN.FUSJDTͷߋ৽
    DPOUSPMMFSSVOUJNFͰ͸.BOBHFS͕NFUSJDTFOEQPJOUΛอ࣋ɾެ։
    ొ࿥ͨ͠ϝτϦΫε͸3FDPODJMF
    ؔ਺͔Βߋ৽
    ग़యɿIUUQTQLHHPEFWHJUIVCDPNQSPNFUIFVTDMJFOU@HPMBOHQSPNFUIFVT
    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNF!WQLHNFUSJDT
    .BOBHFS
    NFUSJDT

    4DSBQF
    SFHJTUFS SFHJTUFS
    .Z$VTUPN.FUSJDT" .Z$VTUPN.FUSJDT#
    6QEBUFGSPN3FDPODJMF

    var MyCustomMetricsA = prometheus.NewGaugeVec(


    prometheus.GaugeOpts{


    Name: "my_custom_metrics_a",


    Help: "my custom metrics related A",


    },


    []string{


    "resource_name",


    "resource_namespace"


    },


    )
    w ϝτϦΫεͷఆٛ
    MyCustomMetricsA.With(


    prometheus.Labels{


    "resource_name": obj.Name,


    "resource_namespace": obj.Namespace,


    }).Set(0.5)
    w 3FDPODJMF
    ؔ਺Ͱͷߋ৽
    func init() {


    metrics.Registry.MustRegister(MyCustomMetricsA)


    }
    w .BOBHFS΁ͷొ࿥

    View Slide

  53. ֎෦௨஌
    3FTPVSDFʹ௨஌ઃఆ༻ͷϑΟʔϧυΛ༻ҙ͠ɺͦͷ৘ใΛ༻͍ͯ֎෦௨஌Λߦ͏
    w 4QFDϑΟʔϧυɺ"OOPUBUJPOT
    w ࣮ࡍʹ͸5PLFO͸4FDSFUͱͯ࣋ͪ͠ɺ0CKFDU3FGFSFODFͳͲͰࢀর͕ਪ঑
    w IUUQTHJUIVCDPNLVCFSOFUFTLVCFSOFUFTCMPCSFMFBTFQLHBQJTDPSFUZQFTHP--
    ॲཧ͕࣌ؒ௕͍৔߹͸ɺඇಉظԽͳͲ΋ݕ౼
    func (r *MyReconciler) Reconcile(...) (...) {


    ...


    api := slack.New(obj.Spec.Notification.Slack.Token)


    _, _, err := api.PostMessage(


    obj.Spec.Notification.Slack.Channel,


    slack.MsgOptionText("Some message", false)


    )


    ...


    }
    kind: MyCustomResource


    metadata:


    name: sample


    spec:


    notification:


    slack:


    token: xxx


    channel: my-channel


    ...
    ग़యɿIUUQTQLHHPEFWHJUIVCDPNTMBDLHPTMBDL

    View Slide

  54. /PUJ
    fi
    DBUJPO&OHJOF
    Ϧιʔεͷঢ়ଶʹԠͯ͡௨஌Λߦ͏"SHP1SPKFDUൃ঵ͷϥΠϒϥϦʢ4MBDLɺ5FBNTɺ&NBJMɺFUDʣ
    ग़యɿIUUQTHJUIVCDPNBSHPQSPKOPUJ
    fi
    DBUJPOTFOHJOF
    ग़యɿIUUQT[FOOEFWLTQFSBSUJDMFTCFGCC
    !,,BXBCF
    kind: ConfigMap


    metadata:


    name: cert-manager-notifications-cm


    data:


    trigger.on-cert-ready: |


    - when: any(cert.status.conditions,

    {.reason == 'Ready' && .status == 'True'})


    send: [cert-ready]


    template.cert-ready: |


    message: |


    Certificate {{.cert.metadata.name}} is ready!
    ௨஌ઃఆͷ࡞੒
    w ௨஌৚݅ʢUSJHHFSʣ
    w ௨஌ϑΥʔϚοτʢUFNQMBUFʣ
    w ௨஌ઌʢTFSWJDFʣ
    ௨஌ର৅ͷઃఆ
    ར༻
    kind: Certificate


    metadata:


    annotations:


    notifications.argoproj.io/subscribe.on-cert-ready.slack: my-channel


    spec: {...}
    ৄ͘͠͸

    View Slide

  55. ϩάग़ྗ
    ىಈ࣌ʹ ctrl.SetLogger()Ͱొ࿥
    3FDPODJMFؔ਺಺Ͱ͸log.FromContext()Ͱऔಘͯ͠ར༻
    ˞LVCFCVJMEFSར༻࣌͸ొ࿥ࡁΈ
    import "sigs.k8s.io/controller-runtime/pkg/log"


    func (r *Reconciler) Reconcile(ctx context.Context, req ctrl.Request)


    (ctrl.Result, error) {


    log := log.FromContext(ctx)


    log.Error(err,


    "something went wrong..."


    "req", req.NamespacedName


    )


    }
    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNFQLHMPH

    View Slide

  56. 0QFO"1*W4DIFNB
    ه࿥
    ௐ੔
    ݕূɾมߋ
    $&-&YQSFTTJPO
    7BMJEBUJOH8FCIPPL
    .VUBUJOH8FCIPPL
    $POWFSTJPO8FCIPPL
    ݕূɾมߋ
    $POUSPMMFSىಈ࣌
    4ZOD1FSJPEͷ࣮ߦ࣌
    Ϧιʔεͷมߋ࣌
    Πϕϯτൃੜ࣌
    ࠶࣮ߦ࣌
    ࣮ߦ੍ޚ
    ه࿥
    ௐ੔
    ࣮ߦ੍ޚ
    ϦΫΤετΛड͚෇͚ͯ
    ϦιʔεΛొ࿥͢Δ·Ͱͷલॲཧ
    Ϧιʔεొ࿥ޙɺ࣮ࡍʹॲཧ͢Δ಺༰
    ॲཧޙͷ݁ՌΛه࿥͢Δ
    ͋Δ΂͖ঢ়ଶʹऩଋͤ͞ଓ͚Δ࣮ߦ੍ޚ
    3FDPODJMFؔ਺ʹΑΔௐ੔
    'JOBMJ[FST
    0XOFS3FGFSFODFT
    ผϦιʔεͷ࡞੒ɾ࡟আ
    $VTUPN.FUSJDTͷߋ৽
    4UBUVTϑΟʔϧυͷมߋ
    "OOPUBUJPOTͷมߋ
    &WFOUϦιʔεͷ࡞੒
    ϩάग़ྗ
    /PUJ
    fi
    DBUJPO&OHJOF
    ֎෦௨஌

    View Slide

  57. $POUSPMMFSͷىಈ࣌
    ىಈ࣌͸Ϧιʔεͷঢ়ଶΛ$POUSPMMFS͕೺ѲͰ͖͍ͯͳ͍ͨΊɺ
    ͢΂ͯͷ0CKFDUʹରͯ͠Ұ౓3FDPODJMFؔ਺͕࣮ߦ͞ΕΔ
    ˞.BOBHFSʹ͸-FBEFS&MFDUJPOͷػೳ͕ଘࡏ
    ɹSFQMJDBTͷ৔߹͸ಛఆͷ୆ͷΈ͕ಈ࡞ͯ͠ڝ߹ճආ
    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{


    LeaderElection: true,


    LeaderElectionID: "d2e52d4b.example.com",


    })
    NBOBHFS
    NBOBHFS
    $PO
    fi
    H.BQ
    EFECFYBNQMFDPN
    -FBEFS
    w -FBEFS&MFDUJPOͷ༗ޮԽ

    View Slide

  58. Ϧιʔεͷมߋ࣌
    3FDPODJMF
    ؔ਺Λݺͼग़͢৚݅ΛࢦఆՄೳ
    'PS $POUSPMMFSͷ3FDPODJMFର৅ͷϦιʔεΛࢦఆ
    0XOT Ϧιʔε͕࡞੒͢ΔࢠϦιʔεΛࢦఆ
    ࢠϦιʔεͷঢ়ଶมԽ࣌ʹ΋3FDPODJMF͞ΕΔ
    ࣄલʹ0XOFS3FGFSFODFͷ෇༩͕ඞཁ
    8JUI&WFOU'JMUFS 3FDPODJMFର৅ͷ&WFOUΛ੍ޚ
    func (r *MyDeploymentReconciler) SetupWithManager(...) error {


    return ctrl.NewControllerManagedBy(mgr).


    For(&myappsv1.MyDeployment{}).


    Owns(&appsv1.ReplicaSet{}).


    Complete(r)


    }
    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNFQLHCVJMEFS

    View Slide

  59. Ϧιʔεͷมߋ࣌
    3FDPODJMFؔ਺ΛൃՐͤ͞ΔϦιʔεΛࢦఆ
    'PS
    .Z%FQMPZNFOU"͕มߋ͞Εͨ৔߹
    .Z%FQMPZNFOU"ʹରͯ͠ൃՐ
    0XOT
    3FQMJDB4FU#͕มߋ͞Εͨ৔߹
    .Z%FQMPZNFOU#ʹରͯ͠ൃՐ
    func (r *MyDeploymentReconciler) SetupWithManager(mgr ctrl.Manager) error {


    return ctrl.NewControllerManagedBy(mgr).


    For(&myappsv1.MyDeployment{}).


    Owns(&appsv1.ReplicaSet{}).


    Complete(r)


    }
    3FQMJDB4FU"
    3FQMJDB4FU#
    .Z%FQMPZNFOU"
    .Z%FQMPZNFOU#
    PXOFS3FGFSFODF
    PXOFS3FGFSFODF
    'PS
    0XOT

    View Slide

  60. Ϧιʔεͷมߋ࣌
    ,JOEɿ ಛఆͷLJOEͷϦιʔεͷมߋΠϕϯτ
    $IBOOFMɿ (PDIBOOFM΁ͷ௨஌Πϕϯτʢ֎෦ΠϕϯτͳͲʣ
    *OGPSNFSɿJOGPSNFSܦ༝ͰͷΠϕϯτ
    4PVSDF
    )BOEMFS
    8PSL2VFVFʹ௥Ճ͠ɺ3FDPODJMFͷ࣮ߦ଴ػ΁
    &ORVFVF3FRVFTU'PS0CKFDUɿ ड͚औͬͨϦιʔεͷΠϕϯτΛൃՐ
    &ORVFVF3FRVFTU'PS0XOFSɿ ड͚औͬͨϦιʔεͷPXOFSͷΠϕϯτΛൃՐ
    &ORVFVF3FRVFTUT'SPN.BQ'VODɿ
    ड͚औͬͨϦιʔεͷΠϕϯτΛݩʹ
    Ϛοϐϯά͢Δؔ਺ʹ͔͚ͯΠϕϯτΛൃՐ
    8BUDI
    ؔ਺Λར༻͠ɺݺͼग़͢৚݅Λ͞ΒʹৄࡉࢦఆՄೳ
    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNF!WQLHTPVSDF
    ग़యɿIUUQTQLHHPEFWTJHTLTJPDPOUSPMMFSSVOUJNF!WQLHIBOEMFS

    View Slide

  61. ࠶࣮ߦ࣌
    3FDPODJMFؔ਺ͷฦΓ஋ʹΑͬͯ3FDPODJMFؔ਺Λ࠶࣮ߦ
    w ΤϥʔΛؚΉ৔߹
    w ໌ࣔతʹ3FRVFVF͞Εͨ৔߹
    ඇಉظͰ͕͔͔࣌ؒΔॲཧͳͲ͸3FDPODJMFؔ਺಺Ͱͷ଴ػΛආ͚3FRVFVF͢Δ
    ˞ޙड़͢Δ4ZOD1FSJPEͷ΄͏͕୹͍৔߹͸3FRVFVF"GUFSͷ஋Λ௕ظؒʹͯ͠΋ແࢹ
    func (r *MyReconciler) Reconcile(...) (...) {


    ...ʢলུʣ...


    if err != nil {


    return ctrl.Result{}, error


    }


    return ctrl.Result{RequeueAfter: time.Hour * 2}, nil


    }

    View Slide

  62. 4ZOD1FSJPEͷܦա࣌
    w όάʹΑΓ3FRVFVF͞Εͳ͍͜ͱΛ๷͙ͨΊɺ

    Ұఆظؒຖʢ%FGBVMUIʣʹ࠶౓3FDPODJMFؔ਺Λ࣮ߦ

    w .BOBHFS഑Լͷ͢΂ͯͷ$POUSPMMFS͕ಉ࣌ʹ3FDPODJMFΛ࣮ߦ࢝͠Ίͳ͍Α͏ʹ
    ͷδολʔΛؚΉ
    w $POUSPMMFS.BOBHFSຖʹઃఆՄೳ
    sp := 10 * time.Hour


    mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{


    SyncPeriod: &sp,


    })

    View Slide

  63. ·ͱΊ

    View Slide

  64. ·ͱΊ
    ,VCFSOFUFTBTB4FSWJDFͷ೥ۙ͘ӡ༻ܦݧΛݩʹ
    গਓ਺Ͱӡ༻͢ΔͨΊʹιϑτ΢ΣΞʹΑΔࣗ཯ԽΛ͞Βʹਪਐ
    ӡ༻ϩδοΫΛ$POUSPMMFSԽ͢Δ͜ͱͰܧଓతͳվળ΁
    ,VCFSOFUFT$POUSPMMFSઃܭ࣌ͷϑϩʔͱߟྀ఺Λ঺հ
    ,VCFSOFUFT$POUSPMMFST FUD
    CVJMENBOBHF
    CVJMENBOBHF

    View Slide

  65. IUUQTTQFBLFSEFDLDPNNBTBZBBPZBNBDBJOGSBBNTZLVCFSOFUFT

    View Slide

  66. 5IBOLZPVGPSZPVSBUUFOUJPO
    5XJUUFS!BNTZ

    View Slide