Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Kubernetes Admission Webhook Deep Dive

Kubernetes Admission Webhook Deep Dive

CloudNative Days Tokyo 2022のセッション「Kubernetes Admission Webhook Deep Dive」のサンプルプログラムです。

セッション情報
https://event.cloudnativedays.jp/cndt2022/talks/1579
補足記事
https://zenn.dev/zoetro/articles/admission-webhook-deep-dive
サンプルプログラム
https://github.com/zoetrope/sample-webhook

Akihiro Ikezoe

November 22, 2022
Tweet

More Decks by Akihiro Ikezoe

Other Decks in Programming

Transcript

  1. ࣗݾ঺հ u ஑ఴ ໌޺ ʢUXJUUFS ![PFUSPʣ u αΠϘ΢ζʹͯ,VCFSOFUFTΛϕʔεͱͨ͠৽͍͠ Πϯϑϥج൫ͷ։ൃͱӡ༻ʹैࣄɻ u

    ,VCFSOFUFTؔ࿈ͷهࣄ u IUUQT[FOOEFW[PFUSP u IUUQT[PFUSPQFHJUIVCJPLVCFCVJMEFSUSBJOJOH u ֶͭͬͯ͘Ϳ,VCFCVJMEFSΧελϜίϯτϩʔϥʔͷͭ͘ΓํΛղઆ u IUUQT[FOOEFWQDZCP[V@OFDP u /FDP8FFLMZ,VCFSOFUFT΍$MPVE/BUJWFؔ࿈ͷؾʹͳΔωλΛຖि঺հ 
  2. ຊ೔ͷൃද಺༰ u "ENJTTJPO 8FCIPPLͷগ͚ͩ͠σΟʔϓͳ࿩Λ͠·͢ɻ u ϕετϓϥΫςΟεʹ͍ͭͯ͸ҎԼͷࢿྉ͕͓͢͢Ί n ެࣜυΩϡϝϯτ%ZOBNJD"ENJTTJPO$POUSPM l IUUQTLVCFSOFUFTJPEPDTSFGFSFODFBDDFTTBVUIOBVUI[FYUFOTJCMF

    BENJTTJPODPOUSPMMFST n ,VCF$PO $MPVE/BUJWF$PO /""ENJTTJPO8FCIPPLT$POGJHVSBUJPO BOE%FCVHHJOH#FTU1SBDUJDFT )BPXFJ $BJ (PPHMF l IUUQTLDDODOBTDIFEDPNFWFOU6B7UBENJTTJPOXFCIPPLTDPOGJHVSBUJPO BOEEFCVHHJOHCFTUQSBDUJDFTIBPXFJDBJHPPHMF 
  3. "ENJTTJPO3FWJFX u "1*4FSWFSͱ8FCIPPLؒ Ͱ΍ΓͱΓ͢Δ+40/ܗࣜ u 3FRVFTUʹ͸ɺ8FCIPPLͷ ର৅ͱͳΔΦϒδΣΫτ΍ɺૢ ࡞Λ͓͜ͳͬͨϢʔβʔͷ৘ใ ͳͲؚ͕·ΕΔɻ u

    .VUBUJOHͷ3FTQPOTF͸ɺ +40/1BUDIܗࣜͰมߋ಺༰ Λදݱ͢Δɻ  { "request": { "uid": "XXXX-YYYY", "name": "sample", "namespace": "default", "operation": "UPDATE", "userInfo": { // 省略 }, "object": { "kind": "Deployment", "apiVersion": "apps/v1", "metadata": { "name": "sample", "namespace": "default" }, "spec": { // 省略 } }, "oldObject": { // 省略 }, "dryRun": false, "options": { "kind": "UpdateOptions", "apiVersion": "meta.k8s.io/v1" } } } Request { "response": { "uid": "XXXX-YYYY", "allowed": true } } Response (Validating) Response (Mutating) { "response": { "uid": "XXXX-YYYY", "allowed": true, "patchType": "JSONPatch", "patch": "W3sib3AiOiAiYWRkIiwgInBhdGgiOiAiL3NwZWM vcmVwbGljYXMiLCAidmFsdWUiOiAzfV0=" } } [{ "op": "add", "path": "/spec/replicas", "value": 3 }] base64
  4. 8FCIPPL$POGJHVSBUJPO u "1*4FSWFSʹ఻͑Δ8FCIPPLͷઃఆ ʢ,VCFSOFUFTͷΫϥελʔϦιʔεʣ n 8FCIPPLͷݺͼग़͠ํ๏ n 8FCIPPL͕ݺͼग़ͤͳ͔ͬͨ৔߹ͷڍಈ n 8FCIPPLͷର৅ͱ͢ΔϦιʔεͷछྨ΍

    /BNFTQBDFͷߜΓࠐΈ u छྨͷϦιʔε͕͋Δ n 7BMJEBUJOH8FCIPPL$POGJHVSBUJPO n .VUBUJOH8FCIPPL$POGJHVSBUJPO  apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: name: validating-webhook-configuration webhooks: - admissionReviewVersions: - v1 clientConfig: service: name: webhook-service namespace: system path: /validate-apps-v1-deployment caBundle: LS0tLS1CRUdJ failurePolicy: Fail name: vdeployment.kb.io namespaceSelector: matchExpressions: - key: kubernetes.io/metadata.name operator: NotIn values: - kube-system rules: - apiGroups: - apps apiVersions: - v1 operations: - CREATE - UPDATE resources: - deployments sideEffects: None
  5.  Kubernetes API Server Mutating Webhooks etcd Mutating Webhooks Mutating

    Webhooks Validating Webhooks Authentication Authorization OpenAPI Schema Validation Persist to etcd par loop ෳ਺ͷ8FCIPPLΛॱʹݺͼग़͢ɻ 3FJOWPDBUJPO1PMJDZʹै͍ ܁Γฦ͠ݺͼग़͢৔߹͕͋Δɻ ෳ਺ͷ8FCIPPLΛ ฒྻʹݺͼग़͢ Create, Update, Delete
  6. DPOUSPMMFSSVOUJNFʹΑΔ࣮૷ u "ENJTTJPO8FCIPPLͷ࣮૷ํ͕ࣜछྨ༻ҙ͞Ε͍ͯΔ n %FGBVMUFS7BMJEBUPS n $VTUPN%FGBVMUFS$VTUPN7BMJEBUPS n )BOEMFS u

    ৄࡉͳ࣮૷ํ๏͸ҎԼͷهࣄΛࢀর n IUUQT[PFUSPQFHJUIVCJPLVCFCVJMEFSUSBJOJOH n IUUQT[FOOEFW[PFUSPBSUJDMFTBENJTTJPOXFCIPPLEFFQEJWF 
  7. ͲͷํࣜΛ࢖͑͹͍͍ͷʁ %FGBVMUFS7BMJEBUPS $VTUPN%FGBVMUFS $VTUPN7BMJEBUPS )BOEMFS ѻ͑ΔϦιʔεͷछྨ ࣗ࡞ͷΧελϜϦιʔε ͷΈ ඪ४ϦιʔεͱΧελϜ Ϧιʔε

    ඪ४ϦιʔεͱΧελϜ Ϧιʔε "ENJTTJPO3FWJFX 3FRVFTU΁ͷΞΫηε ʷ ˓ ˓ 3FRVFTUͷ%FDPEFॲཧ +40/1BUDIͷ࡞੒ ࣗಈ ࣗಈ ࣗ෼Ͱ࣮૷͢Δඞཁ͋Γ ͭͷ8FCIPPLͰෳ਺छ ྨͷϦιʔεΛॲཧ ʷ ʷ ˓ 8BSOJOHΛฦ͢ ʷ ʷ ˓  u جຊ͸$VTUPN%FGBVMUFS$VTUPN7BMJEBUPS͕͓͢͢Ίɻ u ΑΓΧελϚΠζΛ͍ͨ͠৔߹͸)BOEMFSΛར༻͢Δɻ
  8. ࣄྫ)/$Ͱͷڝ߹  Namespace1 SubNamespace B $POGMJDU NamespaceB Namespace2 SubNamespace B

    u ҟͳΔ/BNFTQBDFʹಉ͡λΠϛ ϯάͰಉ໊ͷ4VC/BNFTQBDF ͕࡞੒͞ΕΔͱʜ u 8FCIPPL͸ฒྻͰݺͼग़͞ΕΔ ͨΊɺͦΕͧΕνΣοΫͨ࣌͠఺ Ͱ͸ର৅ͷ/BNFTQBDF͕ଘࡏͤ ͣɺνΣοΫΛ͢Γൈ͚ͯ͠·͏ɻ
  9. ࣄྫ)/$ʹ͓͚Δղܾࡦ u "ENJTTJPO 8FCIPPLʹΑΔνΣοΫΛ·Εʹ͢Γൈ͚Δ͜ͱ͸ڐ༰͢Δɻ u Ϧιʔεͷεςʔλεʹঢ়ଶҟৗΛه࿥͠Ϣʔβʔʹؾ͔ͮͤΔɻ  ʠ /PUF5IFSFBSFTPNFSBSFDPSOFSDBTFTUIBUDPVMESFTVMUJOBDZDMFCFJOH GPSNFE

    EFTQJUFUIFQSFTFODFTPGUIFWBMJEBUJOHBENJTTJPODPOUSPMMFST'PS FYBNQMF UXPEJGGFSFOUVTFSTNJHIUNBLFOBNFTQBDFT"BOE#QBSFOUTPGFBDI PUIFSBUFYBDUMZUIFTBNF UJNFUIFBENJTTJPODPOUSPMMFSXPVMEBMMPXUIJT TJODF OFJUIFSJTZFUUIFQBSFOUPGUIFPUIFS MFBEJOHUPBDZDMF"MUFSOBUJWFMZ BOBENJO NJHIUTJNQMZBDDJEFOUBMMZEJTBCMFUIFBENJTTJPODPOUSPMMFST*OTVDIDBTFT )/$ XJMMQVUBO"DUJWJUJFT)BMUFE DPOEJUJPOPOUIFOBNFTQBDFTVOUJMUIFDZDMFJT SFTPMWFE IUUQTHJUIVCDPNLVCFSOFUFTTJHTIJFSBSDIJDBMOBNFTQBDFTCMPCNBTUFSEPDTVTFSHVJEFDPODFQUTNE
  10. ࣄྫ3FTPVSDF2VPUB u 3FTPVSDF2VPUBͱ͸ɺ/BNFTQBDF͝ͱͷ૯Ϧιʔεফඅྔʢྫ͑͹ɺ 1PEʹׂΓ౰ͯΔ$16΍ϝϞϦʣΛ੍ݶ͢ΔͨΊͷػೳ u "ENJTTJPO8FCIPPLͰ͸ͳ͘,VCFSOFUFTඪ४ͷ"ENJTTJPO $POUSPMMFSͱ࣮ͯ͠૷͞Ε͍ͯΔ  Namespace1 ResouceQuota

    hard: limits.memory: 10Gi used: limits.memory: 8Gi Pod A limits.memory: 5Gi Pod B limits.memory: 3Gi Pod C limits.memory: 3Gi 2VPUBͷ੍ݶΛ௒͑ͯ 1PEΛ࡞੒͢Δ͜ͱ͸ Ͱ͖ͳ͍
  11. API Server ࣄྫ3FTPVSDF2VPUBʹ͓͚Δղܾࡦ  Workers Workers Workers Workers Workers Queue

    PodA PodB NS4 PodP PodQ 2VFVF͔Β/BNFTQBDFΛͭऔ Γग़͢ɻ ͦͷ/BNFTQBDFͷϦιʔε੍͕ ݶΛ௒͍͑ͯͳ͍͜ͱΛνΣοΫ͠ɺ 2VPUBͷ4UBUVTΛߋ৽͢Δɻ PodF NS3 1PEͷ࡞੒ 1PEͷ࡞੒ ͭͷHPSPVUJOF͕ ඵपظͰ࣮ߦ PodX PodA PodB NS2 NS1 ࡞੒͢ΔϦιʔεΛ /BNFTQBDF୯ҐͰ2VFVF ʹొ࿥͢Δɻ ॲཧ͕׬ྃ͢Δ·Ͱ଴ͭɻ /BNFTQBDF୯ҐͰܭࢉ͢Δ͜ͱͰɺ িಥΛݮΒ͢ ˞ ͜ͱ͕Ͱ͖Δɻ ͞Βʹɺෳ਺ͷϦιʔεͷܭࢉ݁ՌΛ Ұ౓Ͱॻ͖ࠐΊΔͨΊɺεϧʔϓοτ ͕޲্͢Δɻ ˞"1*4FSWFS͕৑௕Խ͞Ε͍ͯΔͱॻ͖ࠐΈ࣌ʹিಥ͕ൃੜ͢Δ৔߹͕͋Δɻͦͷ࣌͸ϦτϥΠ͢Δɻ
  12. ڝ߹ରࡦͷݸਓతݟղ u ՄೳͰ͋Ε͹ɺ"ENJTTJPO8FCIPPLͰଞͷϦιʔε΍֎෦ͷঢ়ଶʹґଘ͠ ͳ͍Α͏ʹ͢Δ u ڝ߹ঢ়ଶΛڐ༰Ͱ͖Δ৔߹ n λΠϛϯάʹΑͬͯݕূΛ͢Γൈ͚ΔέʔεΛ೺Ѳ͓ͯ͘͠ n )/$ͷΑ͏ʹɺڝ߹ঢ়ଶΛఆظతʹνΣοΫ͠Ϣʔβʔʹؾ͔ͮͤΔΑ͏ʹ͢Δ

    u ڝ߹Λݫີʹݕূ͍ͨ͠৔߹ n "ENJTTJPO8FCIPPLͰ͸ੑೳ໘΍ɺ࣮ߦॱং੍͕ޚͰ͖ͳ͍͜ͱ͕՝୊ͱͳΔ n 3FTPVSDF2VPUBͷΑ͏ͳ࣮૷Λ͍ͨ͠৔߹͸ɺ$VTUPN"1*4FSWFSͳͲΛݕ౼͢Δ 
  13. ᶄ ηΩϡϦςΟରࡦ u ,VCFSOFUFT"ENJTTJPO$POUSPM5ISFBU.PEFM n IUUQTHJUIVCDPNLVCFSOFUFTTJHTFDVSJUZCMPCNBJOTJHTFDVSJUZ EPDTQBQFSTBENJTTJPODPOUSPMLVCFSOFUFTBENJTTJPODPOUSPMUISFBU NPEFMNE u ओͳ.JUJHBUJPOT

    n ڧ͍ݖݶΛ࣋ͨͤͳ͍Α͏ʹ͠Α͏ʢ3#"$ QSJWJMFHFEͳͲʣ n 'BJM$MPTFEʹ͠Α͏ʢGBJMVSF1PMJDZΛ'BJMʹઃఆ͢Δʣ n ௨৴ͷ5-4҉߸Խ ΫϥΠΞϯτೝূ N5-4ͳͲΛ࠾༻͠Α͏ n ઃఆͷϨϏϡʔ΍ςετΛ࣮ࢪ͠Α͏ 
  14. ରࡦ'BJM$MPTFE u 8FCIPPL͕ར༻Ͱ͖ͳ͍ͱ͖ʹɺϦιʔεͷมߋૢ࡞Λࣦഊͤ͞Δ͜ͱ n ϙϦγʔΛແࢹͨ͠ϫʔΫϩʔυ͕࡞੒͞ΕΔ͜ͱΛ๷͙͜ͱ͕Ͱ͖Δ n Ұํɺ8FCIPPLͷো֐͕,VCFSOFUFTΫϥελʔʹӨڹΛ༩͑Δ͜ͱʹͳΔ u ো֐ͷര෩൒ܘʢ#MBTU3BEJVTʣΛ࠷খԽ͢Δ n

    γεςϜͷՔಇʹؔΘΔॏཁͳϫʔΫϩʔυΛ"ENJTTJPO 8FCIPPLͷର৅֎ʹ͢Δ n 8FCIPPLͷର৅ͱͳΔϦιʔε͸ɺ8FCIPPL಺ͷϩδοΫͰ൑ผ͢ΔͷͰ͸ͳ͘ɺ 8FCIPPL$POGJHVSBUJPOͷઃఆͰߜΓࠐΉɻ 
  15. DFSUNBOBHFSʹΑΔূ໌ॻͷ؅ཧ  Kubernetes API Server Admission Webhook Webhook Configuration HTTPS

    cert manager Certificate Secret $"ূ໌ॻ αʔόʔ ূ໌ॻ ൿີݤ Inject Generate DFSUXBUDIFS͕ϑΝΠ ϧͷมߋΛ؂ࢹ͓ͯ͠Γɺ ূ໌ॻ͕࠶ൃߦ͞ΕΔͱ ϑΝΠϧΛಡΈࠐΈ௚͢ ༗ޮظݶ͕ۙ͘ͳΔͱ ࣗಈతʹূ໌ॻΛ࠶ൃߦ 😊DFSUNBOBHFSʹ͓·͔ͤ
  16. kubeconfig ΫϥΠΞϯτূ໌ॻͷઃఆ  Kubernetes API Server Admission Webhook Admission Configuration

    HTTPS Secret $"ূ໌ॻ ΫϥΠΞϯτ ূ໌ॻ ൿີݤ Generate "1*4FSWFSͷىಈ࣌ʹ ίϚϯυϥΠϯΦϓγϣϯ Ͱࢦఆ͢Δඞཁ͕͋Δɻ Generate ,VCFSOFUFTΫϥελʔ Λ্ཱͪ͛Δલʹࣄલʹ ূ໌ॻΛ࡞੒͓ͯ͘͠ ઃఆΛมߋͨ͠৔߹ɺ "1*4FSWFSΛ࠶ىಈ ͠ͳ͚Ε͹ͳΒͳ͍ɻ 😥ؾܰʹઃఆมߋͰ͖ͳ͍ ˞ৄࡉͳઃఆํ๏͸ิ଍هࣄࢀর
  17. ରࡦαϒϦιʔεͷ8FCIPPL u ,VCFSOFUFTʹ͸ɺαϒϦιʔεͱ͍͏ϦιʔεͷҰ෦ͷϑΟʔϧυͷΈΛ ૢ࡞͢Δ"1* &OEQPJOU͕༻ҙ͞Ε͍ͯΔ u 7BMJEBUJOH8FCIPPLͰαϒϦιʔεͷνΣοΫ͕࿙ΕΔͱɺηΩϡϦςΟ ࣄނʹͭͳ͕ΔՄೳੑ͕͋Δɻ n 1PEϦιʔεͷ8FCIPPLͰಛݖίϯςφΛ࡞੒Ͱ͖ͳ͍Α͏ʹ੍ݶ͍͕ͯͨ͠ɺ

    QIFNFSBMDPOUBJOFSTαϒϦιʔεΛ8FCIPPLͷର৅ʹ͍ͯ͠ͳ͔ͬͨͨΊɺ ಛݖΛ࣋ͬͨ&QIFNFSBM$POUBJOFSΛ࡞੒Ͱ͖ͯ͠·͏ɻ n %FQMPZNFOUϦιʔεͷ8FCIPPLͰSFQMJDBTͷ੍ݶΛ͍͕ͯͨ͠ɺTDBMFαϒϦιʔε Λ8FCIPPLͷର৅ʹ͍ͯ͠ͳ͔ͬͨͨΊɺSFQMJDBTͷ੍ݶΛճආͰ͖ͯ͠·͏ɻ 
  18. αϒϦιʔεΛ8FCIPPLͷର৅ʹ͢Δ u ϦΫΤετͰ౉ͬͯ͘ΔPCKFDUͷܕ͕ɺα ϒϦιʔεʹΑͬͯҟͳΔͷͰ஫ҙɻ n TUBUVT͸ɺ਌Ϧιʔε͕ͦͷ··౉ͬͯ͘Δɻ n TDBMF͸ɺBVUPTDBMJOH4DBMFܕ͕౉ͬͯ͘Δɻ  apiVersion:

    admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration metadata: name: validating-webhook-configuration webhooks: - rules: - apiGroups: - "" apiVersions: - v1 operations: - CREATE - UPDATE resources: - pods - pods/ephemeralcontainers SFTPVSDFTʹαϒϦιʔεͷύεΛؚΊͯࢦఆ ͢Δ͜ͱ͕Ͱ͖Δɻ ͨͩ͠ Λࢦఆͯ͠΋αϒϦιʔε͸ର৅ͱͳ Βͳ͍ͷͰ஫ҙɻ શαϒϦιʔεΛࢦఆ͍ͨ͠৔߹͸ QPET ͷ Α͏ʹࢦఆ͢Δ ! "#$%&$'(")*! "+,-$.(")*! "/012")*"3.45$"6 "4708$#'0+1")*"4&(+'.45019:;<"6 "=$(424(4")*! "14=$")*"'4=75$"6 "14=$'74.$")*"2$>4&5("6 ?6 "'7$.")*! "#$750.4'")*@ ? ? ? ?
  19. Ϛϧνςφϯτ؀ڥ΁ͷରԠ u ϙϦγʔΤϯδϯʢ,ZWFSOP ,VCFXBSEFOʣ n ҰൠϢʔβʔ͕/BNFTQBDF୯ҐͷϙϦγʔΛͭ͘Δ͜ͱ͕Ͱ͖Δɻ u Ծ૝Ϋϥελʔ n 7$MVTUFS΍,$1ͳͲΛར༻ͯ͠ɺԾ૝Ϋϥελʔ্Ͱ"ENJTTJPO8FCIPPLΛར༻͢Δɻ

    u /BNFTQBDFE "ENJTTJPOʢ&YQFSJNFOUBMʣ n IUUQTHJUIVCDPN[PFUSPQFOBNFTQBDFEBENJTTJPO n νʔϜ͝ͱʹ"ENJTTJPO8FCIPPLΛ্ཱͪ͛ΔͨΊͷΧελϜίϯτϩʔϥʔ 
  20. Team X /BNFTQBDFE "ENJTTJPO  Namespace B Namespaced Admission Controller

    Namespaced Webhook Configuration Admission Webhook Webhook Configuration ҰൠϢʔβʔ Service Account σϓϩΠ ੜ੒ ର৅ͱͳΔνʔϜʹଐ͢Δ /BNFTQBDF͚͕ͩ 8FCIPPLͷର৅ͱͳΔΑ͏ʹ OBNFTQBDF4FMFDUPSΛઃఆ Namespace A 4FSWJDF"DDPVOU͕ΞΫηε ՄೳͳϦιʔεͷΈ͕8FCIPPL ͷର৅ͱͳ͍ͬͯΔ͜ͱΛ֬ೝ
  21. ᶆ ୤"ENJTTJPO8FCIPPL u "ENJTTJPO8FCIPPLͷ໰୊఺ n 8FCαʔόʔΛ্ཱͪ͛Δඞཁ͕͋Γɺӡ༻ͷෛ୲͕େ͖͍ɻ n "ENJTTJPO8FCIPPLͷՄ༻ੑ͕ɺ,VCFSOFUFTΫϥελʔશମͷՄ༻ੑʹӨڹ͢Δɻ n ֎෦ͷ8FCαʔόʔΛݺͼग़ͨ͢Ίɺ"1*

    4FSWFSͷϨΠςϯγʔ͕ѱԽ͢Δɻ u "ENJTTJPO8FCIPPLҎ֎ͷํ๏ n -JOUπʔϧΛར༻ͯ͠ɺ,VCFSOFUFTΫϥελʔ΁ͷద༻લʹνΣοΫ n ΧελϜϦιʔε͸0QFO"1* W4DIFNBͰνΣοΫͰ͖Δ͜ͱ΋ଟ͍ n কདྷతʹ͸$&-GPS"ENJTTJPO$POUSPMΛ׆༻ 
  22. $&-GPS "ENJTTJPO $POUSPM u $&-ʢ$PNNPO&YQSFTTJPO-BOHVBHFʣܗࣜͰ7BMJEBUJPOϧʔϧ͕ه ड़Ͱ͖Δ,VCFSOFUFTͷඪ४ػೳ n ,VCFSOFUFTWͰΧελϜϦιʔεͷ7BMJEBUJPOػೳ͕Ќ൛Ͱ࢖͑ΔΑ͏ʹͳͬͨɻ n ,VCFSOFUFTWͰ೚ҙͷϦιʔεʹରͯ͠7BMJEBUJPO͕͓͜ͳ͑Δػೳͷ։ൃ͕ਐ

    ΊΒΕ͍ͯΔɻʢ,&1ʣ u ಛ௃ n "ENJTTJPO8FCIPPLͷΑ͏ʹผαʔόʔΛཱͯΔඞཁ͕ͳ͍ͷͰӡ༻ͷख͕ؒෆཁɻ n "1*4FSWFSͷΠϯϓϩηεͰ࣮ߦ͞ΕΔͷͰɺϨΠςϯγʔΛ௿͘཈͑ΒΕΔɻ 
  23. $&-ʹΑΔϧʔϧͷهड़ u ϑΟʔϧυΛมߋͰ͖ͳ͍Α͏ʹ͢Δ u NBQ΁ͷΩʔͷ௥ՃΛڐՄ͢Δ͕ɺ࡟আ΍มߋΛېࢭ͢Δ u ίϯςφ໊͕YZ[͔Β࢝·Δ͜ͱΛνΣοΫ͢Δ  x-kubernetes-validations: -

    message: Keys may not be removed and their values must stay the same rule: oldSelf.all(key, key in self && self[key] == oldSelf[key]) validations: - scopes: [ "spec.containers[*]", "initContainers[*]", "spec.ephemeralContainers[*]" ] expression: "scope.name.startsWith('xyz-')" messageExpression: "scope.name + ' does not start with ¥'xyz¥''" x-kubernetes-validations: - message: Value is immutable rule: self == oldSelf