Yahoo! JAPAN の マネージド Kubernetes サービスを支える技術 / The Technology Behind Managed Kubernetes Service of Yahoo! JAPAN

Yahoo! JAPAN の マネージド Kubernetes サービスを支える技術 / The Technology Behind Managed Kubernetes Service of Yahoo! JAPAN

ヤフー名古屋 Tech Meetup #2 - https://yahoo-nagoya.connpass.com/event/129398/

大規模環境でのコンテナの利用において、Kubernetes がデファクトスタンダードになりました。 Yahoo! JAPAN は、2017年10月頃から一部のサービスが本番環境で Kubernetes の利用を開始し、現在では多くのサービスが利用しています。 その結果、Yahoo! JAPAN では数百の Kubernetes クラスタが運用されています。 本セッションでは、Kubernetes の概要から、数百の Kubernetes クラスタの運用を可能にする Yahoo! JAPAN のプライベートマネージド Kubernetes サービスの紹介とそれを支える技術についてお話します。

9f9df80ab6551776b49c4ad9432ba1b7?s=128

Kazuki Suda

May 28, 2019
Tweet

Transcript

  1. َؿ٦せ〢㾊5FDI.FFUVQ   ,B[VLJ4VEBLTVEB![MBCDPKQ !TVQFSCSPUIFST :BIPP+"1"/ך وط٦آس,VCFSOFUFT؟٦ؽأ׾佄ִ׷䪮遭

  2. ,B[VLJ4VEB!TVQFSCSPUIFST Ӝ ئحزٓن吳䒭⠓爡اؿزؐؑ،ؒٝآص، Ӝ 䎃חَؿ٦吳䒭⠓爡ח倜⼼Ⰵ爡ծ痥➿랲䌒։,VCFSOFUFT։ Ӝ 䎃ئحزٓن吳䒭⠓爡ח⳿ぢ Ӝ ,VCFSOFUFT.FFUVQ5PLZP $MPVE/BUJWF%FFQ%JWFⰟず⚺⪵罏

    Ӝ $/$'"NCBTTBEPS Ӝ չ,VCFSOFUFT㹋騧ⰅꟌպծչ׫׿זך%PDLFS,VCFSOFUFTպⰟ衼剅 Ӝ չⰅꟌ1SPNFUIFVTպ湊鏬剅
  3. ئحزٓن吳䒭⠓爡;-BC$PSQPSBUJPO Ӝ 䎃ח鏣甧ׁ׸׋َؿ٦吳䒭⠓爡ך㶨⠓爡 Ӝ ؎ٝؿٓ㛇湍䪮遭ך锃叨٥灇瑔Ꟛ涪 Ӝ َؿ٦吳䒭⠓爡ぢֽךوط٦آس,VCFSOFUFT؟٦ؽأךꟚ涪 Ӝ IUUQT[MBCDPKQ

  4. ،آؑٝت  ,VCFSOFUFTהכזחַ  َؿ٦ה,VCFSOFUFT  ז׈,VCFSOFUFTזךַ  ,VCFSOFUFTך،٦ؗذؙثٍהؔـآؙؑز 

    :BIPP+"1"/ךوط٦آس,VCFSOFUFT؟٦ؽأ׾佄ִ׷䪮遭  ״׶鑫׃ֻ濼׷חכ  תה׭
  5. ,VCFSOFUFTהכזחַ

  6. ,VCFSOFUFT Ӝ ؝ٝذشؔ٦؛أزٖ٦ءّٝخ٦ٕ 醱侧ךوءٝظ٦سד圓䧭ׁ׸׷ؙٓأةח㼎׃ג
 ؝ٝذش،فٔ؛٦ءّٝךꂁ⪒ծ鏣㹀ծ盖椚׾遤ֲ Ӝ κυβερνήτηςؘٔءٍ铂ד乼菭㡦 Ӝ (PPHMFך爡ⰻءأذيַ׵؎ٝأػ؎،ׁ׸׋ b

    a xm M kLf “Kubernetes is open source—a contrast to Borg and Omega,
 which were developed as purely Google-internal systems. “ Borg, Omega, and Kubernetes
  7. acL 3 3A g A3 2 M dT ,VCFSOFUFT Ӝ

    䎃剢ח剑ⴱך؝ىحزָ֮׶ծ䎃剢ח׾ٔٔ٦أ 䎃剢儗挿ך剑倜غ٦آّٝכ Ӝ ⴱ劍כ(PPHMFךاؿزؐؑ،׌׏׋ָծ׉ך䖓
 $MPVE/BUJWF$PNQVUJOH'PVOEBUJPO$/$'ח隁床ׁ׸ծ
 植㖈כ$/$'ָمأزׅ׷ؔ٦فٝا٦أفٗآؙؑز Ӝ ؔٝفٖכ׮׍׹׿ծ⚺銲זؙٓؐسفٗغ؎تדوط٦آس؟٦ؽأָ䲿⣘ׁ׸גְ׷ (PPHMF,VCFSOFUFT&OHJOF "NB[PO&,4 "[VSF,VCFSOFUFT4FSJWJDFT Ӝ չ䎃⟃♳ח床׶؝ٝذش׾劤殢橆㞮ד麊欽׃גֹ׋(PPHMFך穗꿀ה
 㢳ֻך⟰噟ח״׷؝ىُصذ؍ך⮚׸׋،؎ر،ה䩛岀ָ穈׫鴥ת׸גְ׷պ
  8. https://twitter.com/brendandburns/status/585479466648018944 psp FP b KT u o Fu

  9. َؿ٦ה,VCFSOFUFT

  10. َؿ٦ה,VCFSOFUFT 䎃剢갦ַ׵♧鿇؟٦ؽأך劤殢橆㞮ה׃ג,VCFSOFUFTךⵃ欽׾Ꟛ㨣կ植㖈דכծ㢳ֻך ؟٦ؽأדⵃ欽׃גְ׷կ Ӝ ر٦ةإٝة Ӝ   ؟٦غ Ӝ

     0QFO4UBDLؙٓأة Ӝ  ,VCFSOFUFTؙٓأة ˟䎃剢植㖈 n
  11. َؿ٦ך,VCFSOFUFTBTB4FSWJDF َؿ٦吳䒭⠓爡ぢֽחئحزٓنָꟚ涪ׅ׷وط٦آس,VCFSOFUFT؟٦ؽأկ
 ؔٝفٖىأך0QFO4UBDL橆㞮ח,VCFSOFUFTؙٓأة׾⡲䧭ծ盖椚ׅ׷կ䎃剢갦ַ׵Ꟛ涪 ׾㨣׭ծَؿ٦ך,VCFSOFUFTךⵃ欽׾佄ִ׷կ Ӝ إٕؿ؟٦ؽأꟚ涪罏ְָאד׮㥨ֹזהֹח Ӝ وط٦آس孡ָ⸬ֹתׅ Ӝ أ؛٦ٓـٕ㼭ְׁ׮ךַ׵㣐ֹז׮ךתד

    Ӝ ءؚٕٝذشٝز杝׶⽑׭ m SdT
  12. ז׈,VCFSOFUFTזךַ cL f ka

  13. ,VCFSOFUFTכؔ٦فٝד֮׷ Ӝ ؔ٦فٝا٦أاؿزؐؑ، HJUIVCDPNLVCFSOFUFTLVCFSOFUFT "QBDIF-JDFOTF  Ӝ ؔ٦فٝرؠ؎ٝ HJUIVCDPNLVCFSOFUFTDPNNVOJUZ Ӝ

    ؔ٦فٝ؝ىُصذ؍ $MPVE/BUJWF$PNQVUJOH'PVOEBUJPO 4QFDJBM*OUFSFTU(SPVQT 4*(T  Ӝ 4MBDL 4UBDL0WFSPX Ⰻ⚅歲ד⟃♳ךى٦ز،حف ,VCFSOFUFT4MBDLٙ٦ؙأل٦أTMBDLLTJPKQVTFST bF A03 3 3 )33 A A A03 m S dT
  14. None
  15. ,VCFSOFUFTכه٦ةـٕד֮׷ Ӝ ٗ٦سغٓٝ؟ծأزٖ٦آזוך*5؎ٝؿٓך乼⡲׾䬄韋⻉ DMPVEDPOUSPMMFSNBOBHFSؙٓؐسך乼⡲ ($1ծ"84ծ"[VSFծ0QFO4UBDLזו Ӝ ؝ٝذشطحزٙ٦ؙך垥彊⻉ $/*$POUBJOFS/FUXPSL*OUFSGBDF $BMJDPծ'MBOOFMծ8FBWF/FUזו Ӝ

    ؝ٝذش乼⡲ך垥彊⻉ $3*$POUBJOFS3VOUJNF*OUFSGBDF %PDLFSDPOUBJOFSEծDSJPծSLU t f uu f Kk
  16. ,VCFSOFUFTכ䧭ꞿ׃גְ׷ ,VCFSOFUFT %PDLFS4XBSN "QBDIF.FTPT Ⰻ窟鎘 $POUSJCVUPST    

    $PNNJUT       麓⿠٠剢 $POUSJCVUPST    $PNNJUT      https://www.openhub.net/p/_compare?project_0=Kubernetes&project_1=docker+swarm&project_2=Apache+Mesos 䎃剢傈儗挿
  17. b a Y dT https://www.flickr.com/photos/143247548@N03/46985430585/in/album-72157707188120301/ KubeCon + CloudNativeCon Europe 2019

    Barcelona, Spain (May 20 - 23, 2019)
  18. ז׈,VCFSOFUFTזךַ Ӝ ,VCFSOFUFTכؔ٦فٝד֮׷ Ӝ ,VCFSOFUFTכه٦ةـٕד֮׷ Ӝ ,VCFSOFUFTכ䧭ꞿ׃גְ׷ t F uu

    F t f K F oM k
  19. ז׈,VCFSOFUFTזךַ

  20. ז׈,VCFSOFUFTזךַ מ׿ם׿זرفٗ؎ח罣ִ׵׸׷ꬊ䌢ח⮚׸׋堣腉׾䭯א Ӝ ֿ׸תדךꟚ涪כծ،فٔ؛٦ءّٝךرفٗ؎ח侧傈ծ侧鹈꟦ַַ׷ֿה׮ Ӝ 儗꟦ךַַ׷رفٗ؎כծ㉏겗ָ֮׏׋הֹך⥜姻׮鹼ֻז׷ Ӝ 鷞חְֲהמ׿ם׿זرفٗ؎כ㉏겗׾ּׅח⥜姻דֹծٔأؙ׾《׸׷ אמהאךرفٗ؎ָ鯪ֻז׶نُٔ٦يծٔأؙ ꟦麩ְַ׵鴼鸞ח䗁傊דֹ׷״ֲחז׶

    ״׶㢳ֻךٔأؙ׾ה׷ֿהָדֹ׷ S m L bgO PilkLM ea Y k
  21. מ׿ם׿זرفٗ؎ח罣ִ׵׸׷ꬊ䌢ח⮚׸׋堣腉 Ӝ מ׿ם׿זرفٗ؎׾〳腉חׅ׷չ㹑鎉涸鏣㹀պ Ӝ 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ Ӝ 7.⟎䟝وءٝ⚥䗰דכזֻչ؝ٝذش⚥䗰պך؎ٝؿٓ

  22. מ׿ם׿זرفٗ؎׾〳腉חׅ׷չ㹑鎉涸鏣㹀պ ֿ׸תדך"׾㹋遤׃גַ׵#׾㹋遤ׅ׷״ֲז♧鸬ך䩛竲ֹח״׷رفٗ؎כծぐ䩛竲ָֹ⟃⵸ ך朐䡾ח⣛㶷ׅ׷կ Ӝ ♧鿇ךمأزָ✲⵸ח㢌刿ׁ׸גְג׮رفٗ؎ח䧭⸆ׅ׷ Ӝ ،فٔ؛٦ءّٝח㉏겗ָ֮׏׋הֹח然㹋חٗ٦ٕغحؙדֹ׷

  23. ♧鸬ך䩛竲ֹח״׷رفٗ؎ B A C ⋆ ♥ 乼⡲ך갫䎷ָ㢌׻׏ג׮䗳׆$ך朐䡾תד׋ו׶滠ֽ׷ "תד姻然חٗ٦ٕغحؙׅ׷חכ "ַ׵#ך㢌刿 #ַ׵$ך㢌刿

  24. 㹑鎉涸鏣㹀ח״׷رفٗ؎ رفٗ؎׃׋穠卓ך،فٔ؛٦ءّٝךչ劄ת׃ְ朐䡾պ׾鎸鶢׃׋鏣㹀ؿ؋؎ٕ׾⿾僥ׅ׷կ Ӝ ✲⵸חו׿ז㢌刿ָ⸇ִ׵׸גְג׮ծئַٗ׵ך圓眠ד֮׏ג׮䗳׆鏣㹀ؿ؋؎ٕח鎸鶢ׁ׸ ׋չ劄ת׃ְ朐䡾պחז׷ Ӝ ٗ٦ٕغحؙכ⟃⵸ך䨱׃׋ְ儗挿דך鏣㹀ؿ؋؎ٕ׾⿾僥ׅ׷׌ֽחז׷ 3 f T

  25. 㹑鎉涸鏣㹀ח״׷رفٗ؎ A C C 劄ת׃ְ朐䡾ָ鎸鶢ׁ׸׋鏣㹀ؿ؋؎ٕ "ַ׵$חז׷״ֲח荈䖒涸ח⹛⡲ׅ׷

  26. 㹑鎉涸鏣㹀ח״׷ٗ٦ٕغحؙ C A A ٗ٦ٕغحؙכ⟃⵸ך朐䡾ך鏣㹀ؿ؋؎ٕ׾⿾僥ׅ׷׌ֽ $ַ׵"חז׷״ֲח荈䖒涸ח⹛⡲ׅ׷

  27. apiVersion: apps/v1 kind: Deployment metadata: name: nginx spec: replicas: 3

    selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.13.3 ports: - containerPort: 80 ,VCFSOFUFTחֶֽ׷㹑鎉涸鏣㹀,VCFSOFUFTوصؿؑأز ٖفٔؕ侧 㹋遤ׁ׸גְ׷ץֹ؝ٝذش侧 㹋遤ׁ׸׷؝ٝذش؎ً٦آ OHJOYךغ٦آّٝ M l k M l dT
  28. 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ ꥺ㹱ח״׶7.װوءٝכ㠨׸׷կ㠨׸׋7.♳ד؟٦ؽأך竰竲ח䗳銲ז،فٔ؛٦ءָّٝ
 㹋遤ׁ׸גְ׋׵וֲׅ׷ַ 劄ת׃ְ朐䡾   植㖈ך朐䡾   !

  29. 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ ,VCFSOFUFTכؙٓأةך朐䡾׾竰竲涸ח劄ת׃ְ朐䡾ד֮׶竲ֽ״ֲה⹛⡲ׅ׷կ ظ٦سوءٝָ侧〴تؐٝ׃ג׮،فٔ؛٦ءّٝך㹋遤ח䗳銲זٔا٦أָ婍׏גְ׷
 ꣲ׶ծ绒㌀噟傈ח䗁傊ׅ׸לְְկ 劄ת׃ְ朐䡾   植㖈ך朐䡾  

    !
  30. 7.⚥䗰דכזֻչ؝ٝذش⚥䗰պך؎ٝؿٓ ,VCFSOFUFTכծ醱侧ך7. ظ٦س ׾אך㣐ֹזٔا٦أف٦ٕ ؙٓأة ה׃ג䪔ֲֿהד
 7.׾䬄韋⻉ׅ׷կ Ӝ ؙٓأةח㼎׃ג؝ٝذش׾رفٗ؎ׅ׷הծ׉ך؝ٝذشך㹋遤ח䗳銲זٔا٦أָ瑞ְגְ ׷ظ٦سחأ؛آُ٦ׁٕ׸׷

    Ӝ Ꟛ涪罏כծ暴㹀ך؝ٝذشָؙٓأةⰻךוךظ٦سד㹋遤ׁ׸גְ׷ַ׾孡חז׷䗳銲ָזְ
  31. ؙٓأةכظ٦سך꧊さ⡤ $16 3".(# ظ٦س $16 3".(# (16 $16 3".(# (16

    k p a f dT
  32. 7.ך䬄韋⻉ח⸇ִגծ*5؎ٝؿٓ׮䬄韋⻉ 7.ך䬄韋⻉ח⸇ִגծ؟٦ؽأך圓䧭׾չ،فٔ؛٦ءّٝ䭷ぢպח䬄韋⻉ׁ׸׋
 ,VCFSOFUFTؔـآؙؑزד䪔ֲկ Ӝ ٗ٦سغٓٝ؟װ宕竲أزٖ٦آזוך*5؎ٝؿٓ׾䬄韋⻉ Ӝ أذ٦زٖأַ׵أذ٦زؿٕתדׁתׂתז珏겲ך،فٔ؛٦ءّٝח㼎䘔ׅ׷
 㢳ֻךؔـآؙؑزָ欽䠐ׁ׸גְ׷ a b

    3 Lf y mdkY A03 3 3 r ) ! k
  33. מ׿ם׿זرفٗ؎ח罣ִ׵׸׷ꬊ䌢ח⮚׸׋堣腉 Ӝ מ׿ם׿זرفٗ؎׾〳腉חׅ׷չ㹑鎉涸鏣㹀պ رفٗ؎׃׋穠卓׾չ㹑鎉涸חպ鎸鶢׃׋鏣㹀ؿ؋؎ٕח״׷رفٗ؎ 然㹋זٗ٦ٕغحؙ׮㺁僒ח㹋植 Ӝ 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ 䌢ח⨳Ⰻז侧ך،فٔ؛٦ءָّٝ㹋遤ׁ׸גְ׷朐䡾׾⥂הֲהׅ׷ Ӝ 7.⟎䟝وءٝ⚥䗰דכזֻչ؝ٝذش⚥䗰ך؎ٝؿٓպ

    ⦐ղךوءٝך盖椚כ遤׻ז׻׆ծؙٓأة׾㣐ֹזوءٝה׃ג䪔ֲ ٗ٦سغٓٝ؟זוך*5؎ٝؿٓ׾䬄韋⻉׃ծ؟٦ؽأ圓䧭ך盖椚׾荈⹛⻉
  34. ➭ךفٓحزؿؓ٦يהך嫰鯰 Ӝ *OGSBTUSVDUVSFBTB4FSWJDF Ӝ 7. ر؍أؙ طحزٙ٦ؙ ,VCFSOFUFT Ӝ $POUBJOFSBTB4FSWJDF

    Ӝ ؝ٝذشך盖椚ծ㹋遤 Ӝ ֮׵ײ׷،فٔ؛٦ءّٝ $MPVE'PVOESZ Ӝ 1MBUGPSNBTB4FSWJDF Ӝ ؝٦سַ׵رفٗ؎ Ӝ )551ծ"1*ծ8FC 0QFO4UBDL
  35. Ӝ ؝ٝذشד荈歋䏝ה䬄韋⻉׾غٓ ٝأ״ֻ㹋植ׅ׷ Ӝ 䬄韋䏝ָ넝ֻծ欰欵䚍ָ넝ְ Ӝ ♧倯ד荈歋䏝ָ⡚ְ 暟椚؎ٝؿٓ 04 ؟٦غ⟎䟝⻉

    ىسٕؐؑ، ٓٝة؎ي ،فٔ؛٦ءّٝ 暟椚؎ٝؿٓ 04 ؟٦غ⟎䟝⻉ ىسٕؐؑ، ٓٝة؎ي ،فٔ؛٦ءّٝ 暟椚؎ٝؿٓ 04 ؟٦غ⟎䟝⻉ ىسٕؐؑ، ٓٝة؎ي ،فٔ؛٦ءّٝ ؝ٝذش⻉ *BB4 ,VCFSOFUFT 1BB4 Ӝ 荈歋䏝כ넝ְָ盖椚ׅ׷ ׮ךָ㢳ְ 荈歋䏝 欰欵䚍 h
  36. ,VCFSOFUFTך،٦ؗذؙثٍהؔـآؙؑز

  37. ,VCFSOFUFTך،٦ؗذؙثٍ Node Networking kubelet Container Runtime OS Master component etcd

    apiserver controller-manager scheduler kubectl REST
  38. ,VCFSOFUFTؔـآؙؑز ٙ٦ؙٗ٦سװ*5؎ٝؿٓ׾չ،فٔ؛٦ءّٝ䭷ぢպח䬄韋⻉׃׋׮ךדծꟚ涪罏כؔـآؙؑ ز׾وصؿؑأزהㄎל׸׷:".-ת׋כ+40/䕎䒭ךؿ؋؎ٕד鎸鶢ׅ׷կ Ӝ 1PE醱侧ך؝ٝذشה醱侧ךنُٔ٦ي Ӝ -BCFM㈓♧ךؚٕ٦ؾؚٝ堣腉 Ӝ 3FQMJDB4FU/⦐ך1PEָ㹋遤ׁ׸גְ׷朐䡾׾⥂א Ӝ

    %FQMPZNFOUٗ٦ؚٔٝ،حفر٦زծٗ٦ٕغحؙ Ӝ 4FSWJDF⟎䟝*1הه٦زծ؟٦ؽأךؒٝزٔه؎ٝز
  39. kubectl apply -f manifest.yaml

  40. 1PEهحس Ӝ 醱侧ך؝ٝذشה
 醱侧ךنُٔ٦ي Ӝ رفٗ؎ך剑㼭⽃⡘ Ӝ *1QFS1PE 1PE" نُٔ٦ي

    ؝ٝذش ؝ٝذش *1
  41. 1PEهحس Ӝ 醱侧ך؝ٝذشה
 醱侧ךنُٔ٦ي Ӝ رفٗ؎ך剑㼭⽃⡘ Ӝ *1QFS1PE apiVersion: v1

    kind: Pod metadata: name: nginx spec: containers: - name: nginx image: nginx:1.15.8 ports: - containerPort: 80
  42. 1PEهحس Ӝ 醱侧ך؝ٝذشה
 醱侧ךنُٔ٦ي Ӝ رفٗ؎ך剑㼭⽃⡘ Ӝ *1QFS1PE 1PE" نُٔ٦ي

    ؝ٝذش ؝ٝذش ظ٦س ظ٦س *1 1PE# *1 1PE$ *1 1PEחろת׸׷؝ٝذشכ
 䗳׆ずׄظ٦س♳ד㹋遤ׁ׸׷ ぐ1PEכؿٓحزזطحزٙ٦ؙך*1،سٖأ׾䭯׍ծظ٦س׾ת׋ְד鸐⥋דֹ׷
  43. 3FQMJDB4FUٖفٔؕإحز Ӝ /⦐ך1PEָ㹋遤ׁ׸גְ׷朐䡾׾⥂א 3FQMJDB4FU ٖفٔؕ侧
 إؙٖةBQQOHJOY ظ٦س" 1PEذٝفٖ٦ز ظ٦س# 1PE$

    BQQOHJOY BQQOHJOY 1PE# BQQOHJOY 1PE" BQQOHJOY 劄ת׃ְ侧植㖈ך侧
  44. 3FQMJDB4FUٖفٔؕإحز Ӝ /⦐ך1PEָ㹋遤ׁ׸גְ׷朐䡾׾⥂א apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx

    spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.8 ports: - containerPort: 80 1PEذٝفٖ٦ز ⡲䧭ׁ׸׷1PEך׮ההז׷鏣㹀 ٖفٔؕ侧
  45. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW
  46. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW
  47. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧
  48. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  49. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  50. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  51. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  52. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW 1PE## XFCW
  53. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW 1PE## XFCW
  54. %FQMPZNFOU Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ apiVersion: apps/v1 kind: Deployment metadata:

    name: nginx spec: replicas: 2 selector: matchLabels: app: nginx strategy: type: RollingUpdate rollingUpdate: maxSurge: 25% maxUnavailable: 25% template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.8 ports: - containerPort: 80 رفٗ؎䨌殛 ٖفٔؕ侧 1PEذٝفٖ٦ز
  55. 4FSWJDF؟٦ؽأ Ӝ ⟎䟝*1הه٦ز Ӝ ٓكٕإؙٖةח״׷1PEךؚٕ٦ؾؚٝ Ӝ ؟٦ؽأة؎ف $MVTUFS*1 *OUFSOBM 

    /PEF1PSU -PBE#BMBODFS &YUFSOBM   4FSWJDFNZTWD ة؎ف$MVTUFS*1
 إؙٖةBQQOHJOY ه٦زUDQ̔UDQ DMVTUFS*1 1PE" 1PE# 3FQMJDB4FU BQQOHJOY BQQOHJOY
  56. NZTWD 1PEē 4FSWJDFせח״׷؟٦ؽأر؍أؕغٔ 4FSWJDFNZTWD ة؎ف$MVTUFS*1
 إؙٖةBQQOHJOY ه٦زUDQ̔UDQ DMVTUFS*1 1PE" 1PE#

    3FQMJDB4FU BQQOHJOY BQQOHJOY
  57. 4FSWJDF؟٦ؽأ Ӝ ⟎䟝*1הه٦ز Ӝ ٓكٕإؙٖةח״׷1PEךؚٕ٦ؾؚٝ Ӝ ؟٦ؽأة؎ف $MVTUFS*1 *OUFSOBM 

    /PEF1PSU -PBE#BMBODFS &YUFSOBM   apiVersion: v1 kind: Service metadata: name: nginx spec: type: ClusterIP selector: app: nginx ports: - protocol: TCP port: 8080 targetPort: 80
  58. ׉ך➭ך⚺זؔـآؙؑز Ӝ /BNFTQBDFؙٓأة׾锷椚涸חⴓⶴׅ׷ Ӝ $POH.BQ،فٔ؛٦ءّٝה鏣㹀ךⴓꨄ Ӝ 4FDSFU،فٔ؛٦ءّٝהء٦ؙٖحزךⴓꨄ Ӝ 1FSTJTUFOU7PMVNF 1FSTJTUFOU7PMVNF$MBJN

    4UPSBHF$MBTT宕竲نُٔ٦ي Ӝ 4UBUFGVM4FUأذ٦زؿٕ،فٔ؛٦ءّٝ Ӝ +PCٙٝءّحزآّـ Ӝ $SPO+PCآّـך㹀劍㹋遤 Ӝ %BFNPO4FUⰋגךظ٦سד1PE׾㹋遤 Ӝ *OHSFTT)551頾蚚ⴓ侔ծغ٦ثٍٕمأزծ5-4穄畭 Ӝ )PMJ[POUBM1PE"VUP4DBMFS )1" ؔ٦زأ؛٦ٕ
  59. :BIPP+"1"/ך وط٦آس,VCFSOFUFT؟٦ؽأ׾佄ִ׷䪮遭

  60. َؿ٦ך,VCFSOFUFTBTB4FSWJDF َؿ٦吳䒭⠓爡ぢֽחئحزٓنָꟚ涪ׅ׷وط٦آس,VCFSOFUFT؟٦ؽأկ
 ؔٝفٖىأך0QFO4UBDL橆㞮ח,VCFSOFUFTؙٓأة׾⡲䧭ծ盖椚ׅ׷կ䎃剢갦ַ׵Ꟛ涪 ׾㨣׭ծَؿ٦ך,VCFSOFUFTךⵃ欽׾佄ִ׷կ Ӝ إٕؿ؟٦ؽأꟚ涪罏ְָאד׮㥨ֹזהֹח Ӝ وط٦آس孡ָ⸬ֹתׅ Ӝ أ؛٦ٓـٕ㼭ְׁ׮ךַ׵㣐ֹז׮ךתד

    Ӝ ءؚٕٝذشٝز杝׶⽑׭
  61. ז׈,VCFSOFUFTBTB4FSWJDFָ䗳銲זךַ 撕꧟ז,VCFSOFUFTךؔلٖ٦ءַّٝ׵麊欽罏׾鍑佝ׅ׷ Ӝ ؙٓأةך⡲䧭ծ⵴ꤐծ鏣㹀㢌刿 Ӝ ظ٦س 7. ך鷄⸇٥⵴ꤐ Ӝ ؙٓأةךئٗتؐٝة؎ي،حفؚٖ٦س

    Ӝ ꥺ㹱װ㉏겗ך֮׷ظ٦سך⥜䗁 Ӝ ر٦ةأز، FUDE ךغحؙ،حفծٔأز، ,VCFSOFUFTכ؝ٝذش⻉ׁ׸׋،فٔ؛٦ءّٝך麊欽ח㣐ֹזًٔحزָ֮׷♧倯דؙٓأةך 麊欽חכ㢳ֻךؔلٖ٦ءָّٝ䗳銲חז׷կ au a b M n L i b t T
  62. 撕꧟ז,VCFSOFUFTؙٓأةךؔلٖ٦ءّٝ Ӝ קה׿וכ寸ת׶ֹ׏׋⡲噟ך粸׶鵤׃ Ӝ :BIPP+"1"/כ㕂ⰻ㢩ח醱侧ךر٦ةإٝة׾䭯׍ծ侧♰〴鋉垷ך؟٦غך
 ؔلٖ٦ءָّٝ䗳銲חז׷ ➂꟦ךװ׷➬✲ׄׯזְկاؿزؐؑ،חװ׵ׇ׷կ AUTOMATE ALL THE

    THINGS
  63. ,VCFSOFUFTBTB4FSWJDFך銲⟝ Ӝ أ؛٦ٓـٕ 盖椚㼎韋ָ侧♰〴ד׮㉏겗זֻ⹛⡲ׅ׷ Ӝ ꬊず劍ٌرٕ وءٝ 7. װٗ٦سغٓٝ؟ך彊⪒זוⳢ椚ָ㸣✪ׅ׷תד儗꟦ַַָ׷ Ӝ

    㛙暕䚍 Ⳣ椚ָ㣟侁׃ג׮ⱄꟚדֹ׷ ءأذيך♧鿇ָ㠨׸ג׮ծⰋ⡤ָꥺ㹱׾饯ֿֿׅהכ鏩ׁ׸זְ
  64. ,VCFSOFUFT׾ⴓ侔ءأذيךؿٖ٦يٙ٦ؙה׃ג⢪ֲ ,VCFSOFUFTך㣐ֹז暴䗙ךאָ䭁䓸䚍 &YUFOTJCJMJUZ  Ӝ $VTUPN3FTPVSDF%FOJUJPOT $3%  ,VCFSOFUFTח⟣䠐ךٔا٦أة؎ف׾鷄⸇ׅ׷ Ӝ

    $VTUPN$POUSPMMFS ,VCFSOFUFTח⟣䠐ךؽآطأٗآحؙ׾鷄⸇ׅ׷
  65. $VTUPN3FTPVSDF%FOJUJPOT $3% "1* $POUSPMMFS *OTUBODF 3FQMJDB4FUT %FQMPZNFOUT  3FQMJDB4FU$POUSPMMFS $VTUPN$POUSPMMFS

     %FQMPZNFOU$POUSPMMFS 3FQMJDB4FU %FQMPZNFOU $VTUPN3FTPVSDF $3% 
  66. ,VCFSOFUFT$MVTUFS0QFSBUPSך$3%ה؝ٝزٗ٦ٓ "1* $POUSPMMFS *OTUBODF ,VCFSOFUFT$MVTUFST ,VCFSOFUFTFT ,VCFSOFUFT$MVTUFS
 $POUSPMMFS ,VCFSOFUFT
 $POUSPMMFS

    ,VCFSOFUFT$MVTUFS ,VCFSOFUFT .BDIJOF4FUT .BDIJOFT .BDIJOF4FU
 $POUSPMMFS .BDIJOF
 $POUSPMMFS .BDIJOF4FUT .BDIJOF    cL f O na M dT
  67. apiVersion: apiextensions/k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: kubernetesclusters.kubernetes.zlab.co.jp spec: scope: Namespaced

    group: kubernetes.zlab.co.jp version: v1alpha1 names: plural: kubernetesClusters singular: kubernetesCluster kind: KubernetesCluster shortNames: - kc additionalPrinterColumns: - name: ready type: boolean description: ”Phase of KubernetesCluster” JSONPath: .status.ready ŋŋŋ
  68. $ cat cluster01.yaml apiVersion: kubernetes.zlab.co.jp/v1alpha1 kind: KubernetesCluster metadata: name: cluster01

    spec: version: ”1.14.3” masterReplicas: 3 masterFlavor: large workerReplicas: 5 workerFlavor: large ŋŋŋ $ kubectl apply -f cluster.yaml Kubernetescluster “cluster01” created
  69. ,VCFSOFUFT$POUSPMMFS Ӝ ,VCFSOFUFTؔـآؙؑزך鷄⸇⵴ꤐ㢌刿׾湊鋔׃גծؔـآؙؑز׾乼⡲ׅ׷
 ؽآطأٗآحؙ׾㹋遤ׅ׷ 0CTFSWF "OBMZ[F "DU 6TFSBDUJPOT
 /FXPCKFDU SFDPOHVSF

    ,VCFSOFUFTFWFOUT
 $VSSFOUTUBUFPGDMVTUFS
  70. ,VCFSOFUFT0QFSBUPS Ӝ ,VCFSOFUFTךِ٦ؠח㢌׻׏ג醱꧟זأذ٦زؿٕ،فٔ؛٦ءّٝך؎ٝأةٝأ׾⡲䧭ծ أ؛٦ٕծ،حفؚٖ٦سծ鏣㹀㢌刿זו׾遤ֲ؝ٝزٗ٦ٓ Ӝ 㛇劤涸ז,VCFSOFUFTךٔا٦أה؝ٝزٗ٦ٓך嚊䙀׾׮הח׃גծ,VCFSOFUFTך䭁䓸堣腉׾ 崞欽׃،فٔ؛٦ءّٝ㔿剣ך麊欽شٖحآ׾اؿزؐؑ،ה׃גفؚٗٓيׅ׷ An Operator represents

    human operational knowledge in software to reliably manage an application. https://coreos.com/operators/
  71. ,VCFSOFUFT0QFSBUPSTJOUIF8JME Ӝ DPSFPTFUDEPQFSBUPS $SFBUFDPOHVSFNBOBHFFUDEDMVTUFSTBUPQ,VCFSOFUFT Ӝ DPSFPTQSPNFUIFVTPQFSBUPS $SFBUFDPOHVSFNBOBHF1SPNFUIFVTDMVTUFSTBUPQ,VCFSOFUFT Ӝ SPPLSPPL 4UPSBHF0SDIFTUSBUJPOGPS,VCFSOFUFT$FQI$PDLSPBDI%#

    Ӝ PSBDMFNZTRMPQFSBUPS $SFBUF PQFSBUFBOETDBMFTFMGIFBMJOH.Z42-DMVTUFSTJO,VCFSOFUFT Ӝ LVCFPXUGPQFSBUPS 5PPMTGPS.-5FOTPSPXPO,VCFSOFUFT https://github.com/operator-framework/awesome-operators
  72. 4JUF3FMJBCJMJUZ&OHJOFFSJOH
 اؿزؐؑ،׾剅ֻֿהד،فٔ؛٦ءّٝ׾麊欽ח荈⹛⻉׾׮׋׵ׅٗ٦ٕ ˑ׃׋ָ׏ג43&כծֿ׸תד麊欽ث٦يָ遤׏גֹ׋ֿה׾ اؿزؐؑ،ך㼔Ꟍ䚍׾䭯אؒٝآص،ָ遤ְծؒٝآص، ָ➂䩛ח״׷盖椚׾荈⹛⻉ׅ׷اؿزؐؑ،׾鏣鎘׃㹋鄲ׅ ׷腉⸂׾䭯׍ծ׉׸׾ְה׻זְהְֲֿהַ׵䧭׶甧׏ג ְתׅկ˒ https://www.oreilly.co.jp/books/9784873117911/

  73. ,VCFSOFUFT$MVTUFS0QFSBUPS 0CTFSWF "OBMZ[F "DU 6TFSBDUJPOT
 /FXDMVTUFS SFDPOHVSF ,VCFSOFUFTFWFOUT
 $VSSFOUTUBUFPGDMVTUFS 3

  74. clusterctl Kubectl 銮%$ 匌%$ 6TFS,VCFSOFUFT$MVTUFS ,VCFSOFUFTBTB4FSWJDF$MVTUFS

  75. ,VCFSOFUFTBTB4FSWJDFך⚺ז堣腉 Ӝ ؙٓأةךإٕؿؼ٦ؚٔٝ ꥺ㹱ך涪欰׃׋ظ٦س׾荈⹛涸חⴖ׶ꨄ׃ծ倜׃ְظ٦س׾鷄⸇ׅ׷ Ӝ ؙٓأةךأ؛٦ٕ،ؐز٥أ؛٦ٕ؎ٝ ظ٦س侧ך؝وٝسדך㢌刿 Ӝ ؙٓأةך荈⹛،حفؚٖ٦س ظ٦تؐٝة؎يծٗ٦ؚٔٝ،حفر٦ز

    Ӝ FUDEؙٓأةך荈⹛غحؙ،حف٥ٔأز، ر؍ؠأةٔؕغٔ
  76. ؙٓأةךإٕؿؼ٦ؚٔٝ Ӝ ꥺ㹱ך涪欰׃׋ظ٦س׾荈⹛涸ח⵴ꤐ׃ծ
 倜׋ח䗳銲זظ٦س׾ؙٓأةח鷄⸇ׅ׷կ ,VCFSOFUFT$MVTUFS0QFSBUPS ظ٦س$ ظ٦س# ظ٦س" ⵴ꤐ 鷄⸇

    !
  77. ״׶鑫׃ֻ濼׷חכ

  78. ,VCFSOFUFT׾״׶鑫׃ֻ濼׷חכ Ӝ Ⱅ䒭سًُؗٝزLVCFSOFUFTJP Ӝ ּׅ㨣׭׷חכ ٗ٦ٕؕחꟚ涪欽ؙٓأة׾圓眠.JOJLVCF %PDLFS%FTLUPQGPS.BDBOE8JOEPXT NJDSPLT ػـٔحؙؙٓؐس⥋걾ה㹋籐ך(PPHMF,VCFSOFUFT&OHJOF Ӝ

    傈劤؝ىُصذ؍,VCFSOFUFT4MBDLثٍٝطٕTMBDLLTJPKQVTFST KQFWFOUT Ӝ ى٦ز،حف ,VCFSOFUFT.FFUVQ5PLZPIUUQTLTKQDPOOQBTTDPN $MPVE/BUJWF.FFUVQ5PLZPIUUQTDMPVEOBUJWFDPOOQBTTDPN Ӝ ⹛歗 ,VCFSOFUFT.FFUVQ5PLZP:PV5VCFثٍٝطٕ $/$':PV5VCFثٍٝطٕ
  79. َؿ٦ך,VCFSOFUFTBTB4FSWJDF׾״׶鑫׃ֻ濼׷חכ +BQBO$POUBJOFS%BZTW  0QFO4UBDL%BZT5PLZP $MPVE/BUJWF%BZT5PLZP 

  80. תה׭

  81. ,VCFSOFUFT Ӝ ؝ٝذشؔ٦؛أزٖ٦ءّٝخ٦ٕ 醱侧ךوءٝظ٦سד圓䧭ׁ׸׷ؙٓأةח㼎׃ג
 ؝ٝذش،فٔ؛٦ءّٝךꂁ⪒ծ鏣㹀ծ盖椚׾遤ֲ Ӝ κυβερνήτηςؘٔءٍ铂ד乼菭㡦 Ӝ (PPHMFך爡ⰻءأذيַ׵؎ٝأػ؎،ׁ׸׋ “Kubernetes

    is open source—a contrast to Borg and Omega,
 which were developed as purely Google-internal systems. “ Borg, Omega, and Kubernetes
  82. acL 3 3A g A3 2 M dT ,VCFSOFUFT Ӝ

    䎃剢ח剑ⴱך؝ىحزָ֮׶ծ䎃剢ח׾ٔٔ٦أ 䎃剢儗挿ך剑倜غ٦آّٝכ Ӝ ⴱ劍כ(PPHMFךاؿزؐؑ،׌׏׋ָծ׉ך䖓
 $MPVE/BUJWF$PNQVUJOH'PVOEBUJPO$/$'ח隁床ׁ׸ծ
 植㖈כ$/$'ָمأزׅ׷ؔ٦فٝا٦أفٗآؙؑز Ӝ ؔٝفٖכ׮׍׹׿ծ⚺銲זؙٓؐسفٗغ؎تדوط٦آس؟٦ؽأָ䲿⣘ׁ׸גְ׷ (PPHMF,VCFSOFUFT&OHJOF "NB[PO&,4 "[VSF,VCFSOFUFT4FSJWJDFT Ӝ չ䎃⟃♳ח床׶؝ٝذش׾劤殢橆㞮ד麊欽׃גֹ׋(PPHMFך穗꿀ה
 㢳ֻך⟰噟ח״׷؝ىُصذ؍ך⮚׸׋،؎ر،ה䩛岀ָ穈׫鴥ת׸גְ׷պ
  83. َؿ٦ך,VCFSOFUFTBTB4FSWJDF َؿ٦吳䒭⠓爡ぢֽחئحزٓنָꟚ涪ׅ׷وط٦آس,VCFSOFUFT؟٦ؽأկ
 ؔٝفٖىأך0QFO4UBDL橆㞮ח,VCFSOFUFTؙٓأة׾⡲䧭ծ盖椚ׅ׷կ䎃剢갦ַ׵Ꟛ涪 ׾㨣׭ծَؿ٦ך,VCFSOFUFTךⵃ欽׾佄ִ׷կ Ӝ إٕؿ؟٦ؽأꟚ涪罏ְָאד׮㥨ֹזהֹח Ӝ وط٦آس孡ָ⸬ֹתׅ Ӝ أ؛٦ٓـٕ㼭ְׁ׮ךַ׵㣐ֹז׮ךתד

    Ӝ ءؚٕٝذشٝز杝׶⽑׭
  84. 2VFTUJPO Ӝ Kazuki Suda <LTVEB![MBCDPKQ !TVQFSCSPUIFST Ӝ 4MJEFIUUQTTQFBLFSEFDLDPNTVQFSCSPUIFST