$30 off During Our Annual Pro Sale. View Details »

Yahoo! JAPAN の マネージド Kubernetes サービスを支える技術 / The Technology Behind Managed Kubernetes Service of Yahoo! JAPAN

Yahoo! JAPAN の マネージド Kubernetes サービスを支える技術 / The Technology Behind Managed Kubernetes Service of Yahoo! JAPAN

ヤフー名古屋 Tech Meetup #2 - https://yahoo-nagoya.connpass.com/event/129398/

大規模環境でのコンテナの利用において、Kubernetes がデファクトスタンダードになりました。 Yahoo! JAPAN は、2017年10月頃から一部のサービスが本番環境で Kubernetes の利用を開始し、現在では多くのサービスが利用しています。 その結果、Yahoo! JAPAN では数百の Kubernetes クラスタが運用されています。 本セッションでは、Kubernetes の概要から、数百の Kubernetes クラスタの運用を可能にする Yahoo! JAPAN のプライベートマネージド Kubernetes サービスの紹介とそれを支える技術についてお話します。

Kazuki Suda

May 28, 2019
Tweet

More Decks by Kazuki Suda

Other Decks in Technology

Transcript

  1. َؿ٦せ〢㾊5FDI.FFUVQ   ,B[VLJ4VEBLTVEB![MBCDPKQ !TVQFSCSPUIFST :BIPP+"1"/ך وط٦آس,VCFSOFUFT؟٦ؽأ׾佄ִ׷䪮遭

  2. ,B[VLJ4VEB!TVQFSCSPUIFST Ӝ ئحزٓن吳䒭⠓爡اؿزؐؑ،ؒٝآص، Ӝ 䎃חَؿ٦吳䒭⠓爡ח倜⼼Ⰵ爡ծ痥➿랲䌒։,VCFSOFUFT։ Ӝ 䎃ئحزٓن吳䒭⠓爡ח⳿ぢ Ӝ ,VCFSOFUFT.FFUVQ5PLZP $MPVE/BUJWF%FFQ%JWFⰟず⚺⪵罏

    Ӝ $/$'"NCBTTBEPS Ӝ չ,VCFSOFUFT㹋騧ⰅꟌպծչ׫׿זך%PDLFS,VCFSOFUFTպⰟ衼剅 Ӝ չⰅꟌ1SPNFUIFVTպ湊鏬剅
  3. ئحزٓن吳䒭⠓爡;-BC$PSQPSBUJPO Ӝ 䎃ח鏣甧ׁ׸׋َؿ٦吳䒭⠓爡ך㶨⠓爡 Ӝ ؎ٝؿٓ㛇湍䪮遭ך锃叨٥灇瑔Ꟛ涪 Ӝ َؿ٦吳䒭⠓爡ぢֽךوط٦آس,VCFSOFUFT؟٦ؽأךꟚ涪 Ӝ IUUQT[MBCDPKQ

  4. ،آؑٝت  ,VCFSOFUFTהכזחַ  َؿ٦ה,VCFSOFUFT  ז׈,VCFSOFUFTזךַ  ,VCFSOFUFTך،٦ؗذؙثٍהؔـآؙؑز 

    :BIPP+"1"/ךوط٦آس,VCFSOFUFT؟٦ؽأ׾佄ִ׷䪮遭  ״׶鑫׃ֻ濼׷חכ  תה׭
  5. ,VCFSOFUFTהכזחַ

  6. ,VCFSOFUFT Ӝ ؝ٝذشؔ٦؛أزٖ٦ءّٝخ٦ٕ 醱侧ךوءٝظ٦سד圓䧭ׁ׸׷ؙٓأةח㼎׃ג
 ؝ٝذش،فٔ؛٦ءّٝךꂁ⪒ծ鏣㹀ծ盖椚׾遤ֲ Ӝ κυβερνήτηςؘٔءٍ铂ד乼菭㡦 Ӝ (PPHMFך爡ⰻءأذيַ׵؎ٝأػ؎،ׁ׸׋ b

    a xm M kLf “Kubernetes is open source—a contrast to Borg and Omega,
 which were developed as purely Google-internal systems. “ Borg, Omega, and Kubernetes
  7. acL 3 3A g A3 2 M dT ,VCFSOFUFT Ӝ

    䎃剢ח剑ⴱך؝ىحزָ֮׶ծ䎃剢ח׾ٔٔ٦أ 䎃剢儗挿ך剑倜غ٦آّٝכ Ӝ ⴱ劍כ(PPHMFךاؿزؐؑ،׌׏׋ָծ׉ך䖓
 $MPVE/BUJWF$PNQVUJOH'PVOEBUJPO$/$'ח隁床ׁ׸ծ
 植㖈כ$/$'ָمأزׅ׷ؔ٦فٝا٦أفٗآؙؑز Ӝ ؔٝفٖכ׮׍׹׿ծ⚺銲זؙٓؐسفٗغ؎تדوط٦آس؟٦ؽأָ䲿⣘ׁ׸גְ׷ (PPHMF,VCFSOFUFT&OHJOF "NB[PO&,4 "[VSF,VCFSOFUFT4FSJWJDFT Ӝ չ䎃⟃♳ח床׶؝ٝذش׾劤殢橆㞮ד麊欽׃גֹ׋(PPHMFך穗꿀ה
 㢳ֻך⟰噟ח״׷؝ىُصذ؍ך⮚׸׋،؎ر،ה䩛岀ָ穈׫鴥ת׸גְ׷պ
  8. https://twitter.com/brendandburns/status/585479466648018944 psp FP b KT u o Fu

  9. َؿ٦ה,VCFSOFUFT

  10. َؿ٦ה,VCFSOFUFT 䎃剢갦ַ׵♧鿇؟٦ؽأך劤殢橆㞮ה׃ג,VCFSOFUFTךⵃ欽׾Ꟛ㨣կ植㖈דכծ㢳ֻך ؟٦ؽأדⵃ欽׃גְ׷կ Ӝ ر٦ةإٝة Ӝ   ؟٦غ Ӝ

     0QFO4UBDLؙٓأة Ӝ  ,VCFSOFUFTؙٓأة ˟䎃剢植㖈 n
  11. َؿ٦ך,VCFSOFUFTBTB4FSWJDF َؿ٦吳䒭⠓爡ぢֽחئحزٓنָꟚ涪ׅ׷وط٦آس,VCFSOFUFT؟٦ؽأկ
 ؔٝفٖىأך0QFO4UBDL橆㞮ח,VCFSOFUFTؙٓأة׾⡲䧭ծ盖椚ׅ׷կ䎃剢갦ַ׵Ꟛ涪 ׾㨣׭ծَؿ٦ך,VCFSOFUFTךⵃ欽׾佄ִ׷կ Ӝ إٕؿ؟٦ؽأꟚ涪罏ְָאד׮㥨ֹזהֹח Ӝ وط٦آس孡ָ⸬ֹתׅ Ӝ أ؛٦ٓـٕ㼭ְׁ׮ךַ׵㣐ֹז׮ךתד

    Ӝ ءؚٕٝذشٝز杝׶⽑׭ m SdT
  12. ז׈,VCFSOFUFTזךַ cL f ka

  13. ,VCFSOFUFTכؔ٦فٝד֮׷ Ӝ ؔ٦فٝا٦أاؿزؐؑ، HJUIVCDPNLVCFSOFUFTLVCFSOFUFT "QBDIF-JDFOTF  Ӝ ؔ٦فٝرؠ؎ٝ HJUIVCDPNLVCFSOFUFTDPNNVOJUZ Ӝ

    ؔ٦فٝ؝ىُصذ؍ $MPVE/BUJWF$PNQVUJOH'PVOEBUJPO 4QFDJBM*OUFSFTU(SPVQT 4*(T  Ӝ 4MBDL 4UBDL0WFSPX Ⰻ⚅歲ד⟃♳ךى٦ز،حف ,VCFSOFUFT4MBDLٙ٦ؙأل٦أTMBDLLTJPKQVTFST bF A03 3 3 )33 A A A03 m S dT
  14. None
  15. ,VCFSOFUFTכه٦ةـٕד֮׷ Ӝ ٗ٦سغٓٝ؟ծأزٖ٦آזוך*5؎ٝؿٓך乼⡲׾䬄韋⻉ DMPVEDPOUSPMMFSNBOBHFSؙٓؐسך乼⡲ ($1ծ"84ծ"[VSFծ0QFO4UBDLזו Ӝ ؝ٝذشطحزٙ٦ؙך垥彊⻉ $/*$POUBJOFS/FUXPSL*OUFSGBDF $BMJDPծ'MBOOFMծ8FBWF/FUזו Ӝ

    ؝ٝذش乼⡲ך垥彊⻉ $3*$POUBJOFS3VOUJNF*OUFSGBDF %PDLFSDPOUBJOFSEծDSJPծSLU t f uu f Kk
  16. ,VCFSOFUFTכ䧭ꞿ׃גְ׷ ,VCFSOFUFT %PDLFS4XBSN "QBDIF.FTPT Ⰻ窟鎘 $POUSJCVUPST    

    $PNNJUT       麓⿠٠剢 $POUSJCVUPST    $PNNJUT      https://www.openhub.net/p/_compare?project_0=Kubernetes&project_1=docker+swarm&project_2=Apache+Mesos 䎃剢傈儗挿
  17. b a Y dT https://www.flickr.com/photos/143247548@N03/46985430585/in/album-72157707188120301/ KubeCon + CloudNativeCon Europe 2019

    Barcelona, Spain (May 20 - 23, 2019)
  18. ז׈,VCFSOFUFTזךַ Ӝ ,VCFSOFUFTכؔ٦فٝד֮׷ Ӝ ,VCFSOFUFTכه٦ةـٕד֮׷ Ӝ ,VCFSOFUFTכ䧭ꞿ׃גְ׷ t F uu

    F t f K F oM k
  19. ז׈,VCFSOFUFTזךַ

  20. ז׈,VCFSOFUFTזךַ מ׿ם׿זرفٗ؎ח罣ִ׵׸׷ꬊ䌢ח⮚׸׋堣腉׾䭯א Ӝ ֿ׸תדךꟚ涪כծ،فٔ؛٦ءّٝךرفٗ؎ח侧傈ծ侧鹈꟦ַַ׷ֿה׮ Ӝ 儗꟦ךַַ׷رفٗ؎כծ㉏겗ָ֮׏׋הֹך⥜姻׮鹼ֻז׷ Ӝ 鷞חְֲהמ׿ם׿זرفٗ؎כ㉏겗׾ּׅח⥜姻דֹծٔأؙ׾《׸׷ אמהאךرفٗ؎ָ鯪ֻז׶نُٔ٦يծٔأؙ ꟦麩ְַ׵鴼鸞ח䗁傊דֹ׷״ֲחז׶

    ״׶㢳ֻךٔأؙ׾ה׷ֿהָדֹ׷ S m L bgO PilkLM ea Y k
  21. מ׿ם׿זرفٗ؎ח罣ִ׵׸׷ꬊ䌢ח⮚׸׋堣腉 Ӝ מ׿ם׿זرفٗ؎׾〳腉חׅ׷չ㹑鎉涸鏣㹀պ Ӝ 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ Ӝ 7.⟎䟝وءٝ⚥䗰דכזֻչ؝ٝذش⚥䗰պך؎ٝؿٓ

  22. מ׿ם׿זرفٗ؎׾〳腉חׅ׷չ㹑鎉涸鏣㹀պ ֿ׸תדך"׾㹋遤׃גַ׵#׾㹋遤ׅ׷״ֲז♧鸬ך䩛竲ֹח״׷رفٗ؎כծぐ䩛竲ָֹ⟃⵸ ך朐䡾ח⣛㶷ׅ׷կ Ӝ ♧鿇ךمأزָ✲⵸ח㢌刿ׁ׸גְג׮رفٗ؎ח䧭⸆ׅ׷ Ӝ ،فٔ؛٦ءّٝח㉏겗ָ֮׏׋הֹח然㹋חٗ٦ٕغحؙדֹ׷

  23. ♧鸬ך䩛竲ֹח״׷رفٗ؎ B A C ⋆ ♥ 乼⡲ך갫䎷ָ㢌׻׏ג׮䗳׆$ך朐䡾תד׋ו׶滠ֽ׷ "תד姻然חٗ٦ٕغحؙׅ׷חכ "ַ׵#ך㢌刿 #ַ׵$ך㢌刿

  24. 㹑鎉涸鏣㹀ח״׷رفٗ؎ رفٗ؎׃׋穠卓ך،فٔ؛٦ءّٝךչ劄ת׃ְ朐䡾պ׾鎸鶢׃׋鏣㹀ؿ؋؎ٕ׾⿾僥ׅ׷կ Ӝ ✲⵸חו׿ז㢌刿ָ⸇ִ׵׸גְג׮ծئַٗ׵ך圓眠ד֮׏ג׮䗳׆鏣㹀ؿ؋؎ٕח鎸鶢ׁ׸ ׋չ劄ת׃ְ朐䡾պחז׷ Ӝ ٗ٦ٕغحؙכ⟃⵸ך䨱׃׋ְ儗挿דך鏣㹀ؿ؋؎ٕ׾⿾僥ׅ׷׌ֽחז׷ 3 f T

  25. 㹑鎉涸鏣㹀ח״׷رفٗ؎ A C C 劄ת׃ְ朐䡾ָ鎸鶢ׁ׸׋鏣㹀ؿ؋؎ٕ "ַ׵$חז׷״ֲח荈䖒涸ח⹛⡲ׅ׷

  26. 㹑鎉涸鏣㹀ח״׷ٗ٦ٕغحؙ C A A ٗ٦ٕغحؙכ⟃⵸ך朐䡾ך鏣㹀ؿ؋؎ٕ׾⿾僥ׅ׷׌ֽ $ַ׵"חז׷״ֲח荈䖒涸ח⹛⡲ׅ׷

  27. apiVersion: apps/v1 kind: Deployment metadata: name: nginx spec: replicas: 3

    selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.13.3 ports: - containerPort: 80 ,VCFSOFUFTחֶֽ׷㹑鎉涸鏣㹀,VCFSOFUFTوصؿؑأز ٖفٔؕ侧 㹋遤ׁ׸גְ׷ץֹ؝ٝذش侧 㹋遤ׁ׸׷؝ٝذش؎ً٦آ OHJOYךغ٦آّٝ M l k M l dT
  28. 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ ꥺ㹱ח״׶7.װوءٝכ㠨׸׷կ㠨׸׋7.♳ד؟٦ؽأך竰竲ח䗳銲ז،فٔ؛٦ءָّٝ
 㹋遤ׁ׸גְ׋׵וֲׅ׷ַ 劄ת׃ְ朐䡾   植㖈ך朐䡾   !

  29. 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ ,VCFSOFUFTכؙٓأةך朐䡾׾竰竲涸ח劄ת׃ְ朐䡾ד֮׶竲ֽ״ֲה⹛⡲ׅ׷կ ظ٦سوءٝָ侧〴تؐٝ׃ג׮،فٔ؛٦ءّٝך㹋遤ח䗳銲זٔا٦أָ婍׏גְ׷
 ꣲ׶ծ绒㌀噟傈ח䗁傊ׅ׸לְְկ 劄ת׃ְ朐䡾   植㖈ך朐䡾  

    !
  30. 7.⚥䗰דכזֻչ؝ٝذش⚥䗰պך؎ٝؿٓ ,VCFSOFUFTכծ醱侧ך7. ظ٦س ׾אך㣐ֹזٔا٦أف٦ٕ ؙٓأة ה׃ג䪔ֲֿהד
 7.׾䬄韋⻉ׅ׷կ Ӝ ؙٓأةח㼎׃ג؝ٝذش׾رفٗ؎ׅ׷הծ׉ך؝ٝذشך㹋遤ח䗳銲זٔا٦أָ瑞ְגְ ׷ظ٦سחأ؛آُ٦ׁٕ׸׷

    Ӝ Ꟛ涪罏כծ暴㹀ך؝ٝذشָؙٓأةⰻךוךظ٦سד㹋遤ׁ׸גְ׷ַ׾孡חז׷䗳銲ָזְ
  31. ؙٓأةכظ٦سך꧊さ⡤ $16 3".(# ظ٦س $16 3".(# (16 $16 3".(# (16

    k p a f dT
  32. 7.ך䬄韋⻉ח⸇ִגծ*5؎ٝؿٓ׮䬄韋⻉ 7.ך䬄韋⻉ח⸇ִגծ؟٦ؽأך圓䧭׾չ،فٔ؛٦ءّٝ䭷ぢպח䬄韋⻉ׁ׸׋
 ,VCFSOFUFTؔـآؙؑزד䪔ֲկ Ӝ ٗ٦سغٓٝ؟װ宕竲أزٖ٦آזוך*5؎ٝؿٓ׾䬄韋⻉ Ӝ أذ٦زٖأַ׵أذ٦زؿٕתדׁתׂתז珏겲ך،فٔ؛٦ءّٝח㼎䘔ׅ׷
 㢳ֻךؔـآؙؑزָ欽䠐ׁ׸גְ׷ a b

    3 Lf y mdkY A03 3 3 r ) ! k
  33. מ׿ם׿זرفٗ؎ח罣ִ׵׸׷ꬊ䌢ח⮚׸׋堣腉 Ӝ מ׿ם׿זرفٗ؎׾〳腉חׅ׷չ㹑鎉涸鏣㹀պ رفٗ؎׃׋穠卓׾չ㹑鎉涸חպ鎸鶢׃׋鏣㹀ؿ؋؎ٕח״׷رفٗ؎ 然㹋זٗ٦ٕغحؙ׮㺁僒ח㹋植 Ӝ 䓼⸂ז荈䊹㔐䗁堣腉չإٕؿؼ٦ؚٔٝպ 䌢ח⨳Ⰻז侧ך،فٔ؛٦ءָّٝ㹋遤ׁ׸גְ׷朐䡾׾⥂הֲהׅ׷ Ӝ 7.⟎䟝وءٝ⚥䗰דכזֻչ؝ٝذش⚥䗰ך؎ٝؿٓպ

    ⦐ղךوءٝך盖椚כ遤׻ז׻׆ծؙٓأة׾㣐ֹזوءٝה׃ג䪔ֲ ٗ٦سغٓٝ؟זוך*5؎ٝؿٓ׾䬄韋⻉׃ծ؟٦ؽأ圓䧭ך盖椚׾荈⹛⻉
  34. ➭ךفٓحزؿؓ٦يהך嫰鯰 Ӝ *OGSBTUSVDUVSFBTB4FSWJDF Ӝ 7. ر؍أؙ طحزٙ٦ؙ ,VCFSOFUFT Ӝ $POUBJOFSBTB4FSWJDF

    Ӝ ؝ٝذشך盖椚ծ㹋遤 Ӝ ֮׵ײ׷،فٔ؛٦ءّٝ $MPVE'PVOESZ Ӝ 1MBUGPSNBTB4FSWJDF Ӝ ؝٦سַ׵رفٗ؎ Ӝ )551ծ"1*ծ8FC 0QFO4UBDL
  35. Ӝ ؝ٝذشד荈歋䏝ה䬄韋⻉׾غٓ ٝأ״ֻ㹋植ׅ׷ Ӝ 䬄韋䏝ָ넝ֻծ欰欵䚍ָ넝ְ Ӝ ♧倯ד荈歋䏝ָ⡚ְ 暟椚؎ٝؿٓ 04 ؟٦غ⟎䟝⻉

    ىسٕؐؑ، ٓٝة؎ي ،فٔ؛٦ءّٝ 暟椚؎ٝؿٓ 04 ؟٦غ⟎䟝⻉ ىسٕؐؑ، ٓٝة؎ي ،فٔ؛٦ءّٝ 暟椚؎ٝؿٓ 04 ؟٦غ⟎䟝⻉ ىسٕؐؑ، ٓٝة؎ي ،فٔ؛٦ءّٝ ؝ٝذش⻉ *BB4 ,VCFSOFUFT 1BB4 Ӝ 荈歋䏝כ넝ְָ盖椚ׅ׷ ׮ךָ㢳ְ 荈歋䏝 欰欵䚍 h
  36. ,VCFSOFUFTך،٦ؗذؙثٍהؔـآؙؑز

  37. ,VCFSOFUFTך،٦ؗذؙثٍ Node Networking kubelet Container Runtime OS Master component etcd

    apiserver controller-manager scheduler kubectl REST
  38. ,VCFSOFUFTؔـآؙؑز ٙ٦ؙٗ٦سװ*5؎ٝؿٓ׾չ،فٔ؛٦ءّٝ䭷ぢպח䬄韋⻉׃׋׮ךדծꟚ涪罏כؔـآؙؑ ز׾وصؿؑأزהㄎל׸׷:".-ת׋כ+40/䕎䒭ךؿ؋؎ٕד鎸鶢ׅ׷կ Ӝ 1PE醱侧ך؝ٝذشה醱侧ךنُٔ٦ي Ӝ -BCFM㈓♧ךؚٕ٦ؾؚٝ堣腉 Ӝ 3FQMJDB4FU/⦐ך1PEָ㹋遤ׁ׸גְ׷朐䡾׾⥂א Ӝ

    %FQMPZNFOUٗ٦ؚٔٝ،حفر٦زծٗ٦ٕغحؙ Ӝ 4FSWJDF⟎䟝*1הه٦زծ؟٦ؽأךؒٝزٔه؎ٝز
  39. kubectl apply -f manifest.yaml

  40. 1PEهحس Ӝ 醱侧ך؝ٝذشה
 醱侧ךنُٔ٦ي Ӝ رفٗ؎ך剑㼭⽃⡘ Ӝ *1QFS1PE 1PE" نُٔ٦ي

    ؝ٝذش ؝ٝذش *1
  41. 1PEهحس Ӝ 醱侧ך؝ٝذشה
 醱侧ךنُٔ٦ي Ӝ رفٗ؎ך剑㼭⽃⡘ Ӝ *1QFS1PE apiVersion: v1

    kind: Pod metadata: name: nginx spec: containers: - name: nginx image: nginx:1.15.8 ports: - containerPort: 80
  42. 1PEهحس Ӝ 醱侧ך؝ٝذشה
 醱侧ךنُٔ٦ي Ӝ رفٗ؎ך剑㼭⽃⡘ Ӝ *1QFS1PE 1PE" نُٔ٦ي

    ؝ٝذش ؝ٝذش ظ٦س ظ٦س *1 1PE# *1 1PE$ *1 1PEחろת׸׷؝ٝذشכ
 䗳׆ずׄظ٦س♳ד㹋遤ׁ׸׷ ぐ1PEכؿٓحزזطحزٙ٦ؙך*1،سٖأ׾䭯׍ծظ٦س׾ת׋ְד鸐⥋דֹ׷
  43. 3FQMJDB4FUٖفٔؕإحز Ӝ /⦐ך1PEָ㹋遤ׁ׸גְ׷朐䡾׾⥂א 3FQMJDB4FU ٖفٔؕ侧
 إؙٖةBQQOHJOY ظ٦س" 1PEذٝفٖ٦ز ظ٦س# 1PE$

    BQQOHJOY BQQOHJOY 1PE# BQQOHJOY 1PE" BQQOHJOY 劄ת׃ְ侧植㖈ך侧
  44. 3FQMJDB4FUٖفٔؕإحز Ӝ /⦐ך1PEָ㹋遤ׁ׸גְ׷朐䡾׾⥂א apiVersion: apps/v1 kind: ReplicaSet metadata: name: nginx

    spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.8 ports: - containerPort: 80 1PEذٝفٖ٦ز ⡲䧭ׁ׸׷1PEך׮ההז׷鏣㹀 ٖفٔؕ侧
  45. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW
  46. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW
  47. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧
  48. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  49. 1PE"" XFCW %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW

    ٖفٔؕ侧 3FQMJDB4FU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  50. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  51. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW
  52. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE"# XFCW 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW 1PE## XFCW
  53. %FQMPZNFOUرفٗ؎ًٝز Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ %FQMPZNFOU" DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU"

    DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 3FQMJDB4FU# DPOUBJOFST JNBHFXFCW ٖفٔؕ侧 1PE#" XFCW 1PE## XFCW
  54. %FQMPZNFOU Ӝ ٗ٦ؚٔٝ،حفر٦ز Ӝ ٗ٦ٕغحؙ apiVersion: apps/v1 kind: Deployment metadata:

    name: nginx spec: replicas: 2 selector: matchLabels: app: nginx strategy: type: RollingUpdate rollingUpdate: maxSurge: 25% maxUnavailable: 25% template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15.8 ports: - containerPort: 80 رفٗ؎䨌殛 ٖفٔؕ侧 1PEذٝفٖ٦ز
  55. 4FSWJDF؟٦ؽأ Ӝ ⟎䟝*1הه٦ز Ӝ ٓكٕإؙٖةח״׷1PEךؚٕ٦ؾؚٝ Ӝ ؟٦ؽأة؎ف $MVTUFS*1 *OUFSOBM 

    /PEF1PSU -PBE#BMBODFS &YUFSOBM   4FSWJDFNZTWD ة؎ف$MVTUFS*1
 إؙٖةBQQOHJOY ه٦زUDQ̔UDQ DMVTUFS*1 1PE" 1PE# 3FQMJDB4FU BQQOHJOY BQQOHJOY
  56. NZTWD 1PEē 4FSWJDFせח״׷؟٦ؽأر؍أؕغٔ 4FSWJDFNZTWD ة؎ف$MVTUFS*1
 إؙٖةBQQOHJOY ه٦زUDQ̔UDQ DMVTUFS*1 1PE" 1PE#

    3FQMJDB4FU BQQOHJOY BQQOHJOY
  57. 4FSWJDF؟٦ؽأ Ӝ ⟎䟝*1הه٦ز Ӝ ٓكٕإؙٖةח״׷1PEךؚٕ٦ؾؚٝ Ӝ ؟٦ؽأة؎ف $MVTUFS*1 *OUFSOBM 

    /PEF1PSU -PBE#BMBODFS &YUFSOBM   apiVersion: v1 kind: Service metadata: name: nginx spec: type: ClusterIP selector: app: nginx ports: - protocol: TCP port: 8080 targetPort: 80
  58. ׉ך➭ך⚺זؔـآؙؑز Ӝ /BNFTQBDFؙٓأة׾锷椚涸חⴓⶴׅ׷ Ӝ $POH.BQ،فٔ؛٦ءّٝה鏣㹀ךⴓꨄ Ӝ 4FDSFU،فٔ؛٦ءّٝהء٦ؙٖحزךⴓꨄ Ӝ 1FSTJTUFOU7PMVNF 1FSTJTUFOU7PMVNF$MBJN

    4UPSBHF$MBTT宕竲نُٔ٦ي Ӝ 4UBUFGVM4FUأذ٦زؿٕ،فٔ؛٦ءّٝ Ӝ +PCٙٝءّحزآّـ Ӝ $SPO+PCآّـך㹀劍㹋遤 Ӝ %BFNPO4FUⰋגךظ٦سד1PE׾㹋遤 Ӝ *OHSFTT)551頾蚚ⴓ侔ծغ٦ثٍٕمأزծ5-4穄畭 Ӝ )PMJ[POUBM1PE"VUP4DBMFS )1" ؔ٦زأ؛٦ٕ
  59. :BIPP+"1"/ך وط٦آس,VCFSOFUFT؟٦ؽأ׾佄ִ׷䪮遭

  60. َؿ٦ך,VCFSOFUFTBTB4FSWJDF َؿ٦吳䒭⠓爡ぢֽחئحزٓنָꟚ涪ׅ׷وط٦آس,VCFSOFUFT؟٦ؽأկ
 ؔٝفٖىأך0QFO4UBDL橆㞮ח,VCFSOFUFTؙٓأة׾⡲䧭ծ盖椚ׅ׷կ䎃剢갦ַ׵Ꟛ涪 ׾㨣׭ծَؿ٦ך,VCFSOFUFTךⵃ欽׾佄ִ׷կ Ӝ إٕؿ؟٦ؽأꟚ涪罏ְָאד׮㥨ֹזהֹח Ӝ وط٦آس孡ָ⸬ֹתׅ Ӝ أ؛٦ٓـٕ㼭ְׁ׮ךַ׵㣐ֹז׮ךתד

    Ӝ ءؚٕٝذشٝز杝׶⽑׭
  61. ז׈,VCFSOFUFTBTB4FSWJDFָ䗳銲זךַ 撕꧟ז,VCFSOFUFTךؔلٖ٦ءַّٝ׵麊欽罏׾鍑佝ׅ׷ Ӝ ؙٓأةך⡲䧭ծ⵴ꤐծ鏣㹀㢌刿 Ӝ ظ٦س 7. ך鷄⸇٥⵴ꤐ Ӝ ؙٓأةךئٗتؐٝة؎ي،حفؚٖ٦س

    Ӝ ꥺ㹱װ㉏겗ך֮׷ظ٦سך⥜䗁 Ӝ ر٦ةأز، FUDE ךغحؙ،حفծٔأز، ,VCFSOFUFTכ؝ٝذش⻉ׁ׸׋،فٔ؛٦ءّٝך麊欽ח㣐ֹזًٔحزָ֮׷♧倯דؙٓأةך 麊欽חכ㢳ֻךؔلٖ٦ءָّٝ䗳銲חז׷կ au a b M n L i b t T
  62. 撕꧟ז,VCFSOFUFTؙٓأةךؔلٖ٦ءّٝ Ӝ קה׿וכ寸ת׶ֹ׏׋⡲噟ך粸׶鵤׃ Ӝ :BIPP+"1"/כ㕂ⰻ㢩ח醱侧ךر٦ةإٝة׾䭯׍ծ侧♰〴鋉垷ך؟٦غך
 ؔلٖ٦ءָّٝ䗳銲חז׷ ➂꟦ךװ׷➬✲ׄׯזְկاؿزؐؑ،חװ׵ׇ׷կ AUTOMATE ALL THE

    THINGS
  63. ,VCFSOFUFTBTB4FSWJDFך銲⟝ Ӝ أ؛٦ٓـٕ 盖椚㼎韋ָ侧♰〴ד׮㉏겗זֻ⹛⡲ׅ׷ Ӝ ꬊず劍ٌرٕ وءٝ 7. װٗ٦سغٓٝ؟ך彊⪒זוⳢ椚ָ㸣✪ׅ׷תד儗꟦ַַָ׷ Ӝ

    㛙暕䚍 Ⳣ椚ָ㣟侁׃ג׮ⱄꟚדֹ׷ ءأذيך♧鿇ָ㠨׸ג׮ծⰋ⡤ָꥺ㹱׾饯ֿֿׅהכ鏩ׁ׸זְ
  64. ,VCFSOFUFT׾ⴓ侔ءأذيךؿٖ٦يٙ٦ؙה׃ג⢪ֲ ,VCFSOFUFTך㣐ֹז暴䗙ךאָ䭁䓸䚍 &YUFOTJCJMJUZ  Ӝ $VTUPN3FTPVSDF%FOJUJPOT $3%  ,VCFSOFUFTח⟣䠐ךٔا٦أة؎ف׾鷄⸇ׅ׷ Ӝ

    $VTUPN$POUSPMMFS ,VCFSOFUFTח⟣䠐ךؽآطأٗآحؙ׾鷄⸇ׅ׷
  65. $VTUPN3FTPVSDF%FOJUJPOT $3% "1* $POUSPMMFS *OTUBODF 3FQMJDB4FUT %FQMPZNFOUT  3FQMJDB4FU$POUSPMMFS $VTUPN$POUSPMMFS

     %FQMPZNFOU$POUSPMMFS 3FQMJDB4FU %FQMPZNFOU $VTUPN3FTPVSDF $3% 
  66. ,VCFSOFUFT$MVTUFS0QFSBUPSך$3%ה؝ٝزٗ٦ٓ "1* $POUSPMMFS *OTUBODF ,VCFSOFUFT$MVTUFST ,VCFSOFUFTFT ,VCFSOFUFT$MVTUFS
 $POUSPMMFS ,VCFSOFUFT
 $POUSPMMFS

    ,VCFSOFUFT$MVTUFS ,VCFSOFUFT .BDIJOF4FUT .BDIJOFT .BDIJOF4FU
 $POUSPMMFS .BDIJOF
 $POUSPMMFS .BDIJOF4FUT .BDIJOF    cL f O na M dT
  67. apiVersion: apiextensions/k8s.io/v1beta1 kind: CustomResourceDefinition metadata: name: kubernetesclusters.kubernetes.zlab.co.jp spec: scope: Namespaced

    group: kubernetes.zlab.co.jp version: v1alpha1 names: plural: kubernetesClusters singular: kubernetesCluster kind: KubernetesCluster shortNames: - kc additionalPrinterColumns: - name: ready type: boolean description: ”Phase of KubernetesCluster” JSONPath: .status.ready ŋŋŋ
  68. $ cat cluster01.yaml apiVersion: kubernetes.zlab.co.jp/v1alpha1 kind: KubernetesCluster metadata: name: cluster01

    spec: version: ”1.14.3” masterReplicas: 3 masterFlavor: large workerReplicas: 5 workerFlavor: large ŋŋŋ $ kubectl apply -f cluster.yaml Kubernetescluster “cluster01” created
  69. ,VCFSOFUFT$POUSPMMFS Ӝ ,VCFSOFUFTؔـآؙؑزך鷄⸇⵴ꤐ㢌刿׾湊鋔׃גծؔـآؙؑز׾乼⡲ׅ׷
 ؽآطأٗآحؙ׾㹋遤ׅ׷ 0CTFSWF "OBMZ[F "DU 6TFSBDUJPOT
 /FXPCKFDU SFDPOHVSF

    ,VCFSOFUFTFWFOUT
 $VSSFOUTUBUFPGDMVTUFS
  70. ,VCFSOFUFT0QFSBUPS Ӝ ,VCFSOFUFTךِ٦ؠח㢌׻׏ג醱꧟זأذ٦زؿٕ،فٔ؛٦ءّٝך؎ٝأةٝأ׾⡲䧭ծ أ؛٦ٕծ،حفؚٖ٦سծ鏣㹀㢌刿זו׾遤ֲ؝ٝزٗ٦ٓ Ӝ 㛇劤涸ז,VCFSOFUFTךٔا٦أה؝ٝزٗ٦ٓך嚊䙀׾׮הח׃גծ,VCFSOFUFTך䭁䓸堣腉׾ 崞欽׃،فٔ؛٦ءّٝ㔿剣ך麊欽شٖحآ׾اؿزؐؑ،ה׃גفؚٗٓيׅ׷ An Operator represents

    human operational knowledge in software to reliably manage an application. https://coreos.com/operators/
  71. ,VCFSOFUFT0QFSBUPSTJOUIF8JME Ӝ DPSFPTFUDEPQFSBUPS $SFBUFDPOHVSFNBOBHFFUDEDMVTUFSTBUPQ,VCFSOFUFT Ӝ DPSFPTQSPNFUIFVTPQFSBUPS $SFBUFDPOHVSFNBOBHF1SPNFUIFVTDMVTUFSTBUPQ,VCFSOFUFT Ӝ SPPLSPPL 4UPSBHF0SDIFTUSBUJPOGPS,VCFSOFUFT$FQI$PDLSPBDI%#

    Ӝ PSBDMFNZTRMPQFSBUPS $SFBUF PQFSBUFBOETDBMFTFMGIFBMJOH.Z42-DMVTUFSTJO,VCFSOFUFT Ӝ LVCFPXUGPQFSBUPS 5PPMTGPS.-5FOTPSPXPO,VCFSOFUFT https://github.com/operator-framework/awesome-operators
  72. 4JUF3FMJBCJMJUZ&OHJOFFSJOH
 اؿزؐؑ،׾剅ֻֿהד،فٔ؛٦ءّٝ׾麊欽ח荈⹛⻉׾׮׋׵ׅٗ٦ٕ ˑ׃׋ָ׏ג43&כծֿ׸תד麊欽ث٦يָ遤׏גֹ׋ֿה׾ اؿزؐؑ،ך㼔Ꟍ䚍׾䭯אؒٝآص،ָ遤ְծؒٝآص، ָ➂䩛ח״׷盖椚׾荈⹛⻉ׅ׷اؿزؐؑ،׾鏣鎘׃㹋鄲ׅ ׷腉⸂׾䭯׍ծ׉׸׾ְה׻זְהְֲֿהַ׵䧭׶甧׏ג ְתׅկ˒ https://www.oreilly.co.jp/books/9784873117911/

  73. ,VCFSOFUFT$MVTUFS0QFSBUPS 0CTFSWF "OBMZ[F "DU 6TFSBDUJPOT
 /FXDMVTUFS SFDPOHVSF ,VCFSOFUFTFWFOUT
 $VSSFOUTUBUFPGDMVTUFS 3

  74. clusterctl Kubectl 銮%$ 匌%$ 6TFS,VCFSOFUFT$MVTUFS ,VCFSOFUFTBTB4FSWJDF$MVTUFS

  75. ,VCFSOFUFTBTB4FSWJDFך⚺ז堣腉 Ӝ ؙٓأةךإٕؿؼ٦ؚٔٝ ꥺ㹱ך涪欰׃׋ظ٦س׾荈⹛涸חⴖ׶ꨄ׃ծ倜׃ְظ٦س׾鷄⸇ׅ׷ Ӝ ؙٓأةךأ؛٦ٕ،ؐز٥أ؛٦ٕ؎ٝ ظ٦س侧ך؝وٝسדך㢌刿 Ӝ ؙٓأةך荈⹛،حفؚٖ٦س ظ٦تؐٝة؎يծٗ٦ؚٔٝ،حفر٦ز

    Ӝ FUDEؙٓأةך荈⹛غحؙ،حف٥ٔأز، ر؍ؠأةٔؕغٔ
  76. ؙٓأةךإٕؿؼ٦ؚٔٝ Ӝ ꥺ㹱ך涪欰׃׋ظ٦س׾荈⹛涸ח⵴ꤐ׃ծ
 倜׋ח䗳銲זظ٦س׾ؙٓأةח鷄⸇ׅ׷կ ,VCFSOFUFT$MVTUFS0QFSBUPS ظ٦س$ ظ٦س# ظ٦س" ⵴ꤐ 鷄⸇

    !
  77. ״׶鑫׃ֻ濼׷חכ

  78. ,VCFSOFUFT׾״׶鑫׃ֻ濼׷חכ Ӝ Ⱅ䒭سًُؗٝزLVCFSOFUFTJP Ӝ ּׅ㨣׭׷חכ ٗ٦ٕؕחꟚ涪欽ؙٓأة׾圓眠.JOJLVCF %PDLFS%FTLUPQGPS.BDBOE8JOEPXT NJDSPLT ػـٔحؙؙٓؐس⥋걾ה㹋籐ך(PPHMF,VCFSOFUFT&OHJOF Ӝ

    傈劤؝ىُصذ؍,VCFSOFUFT4MBDLثٍٝطٕTMBDLLTJPKQVTFST KQFWFOUT Ӝ ى٦ز،حف ,VCFSOFUFT.FFUVQ5PLZPIUUQTLTKQDPOOQBTTDPN $MPVE/BUJWF.FFUVQ5PLZPIUUQTDMPVEOBUJWFDPOOQBTTDPN Ӝ ⹛歗 ,VCFSOFUFT.FFUVQ5PLZP:PV5VCFثٍٝطٕ $/$':PV5VCFثٍٝطٕ
  79. َؿ٦ך,VCFSOFUFTBTB4FSWJDF׾״׶鑫׃ֻ濼׷חכ +BQBO$POUBJOFS%BZTW  0QFO4UBDL%BZT5PLZP $MPVE/BUJWF%BZT5PLZP 

  80. תה׭

  81. ,VCFSOFUFT Ӝ ؝ٝذشؔ٦؛أزٖ٦ءّٝخ٦ٕ 醱侧ךوءٝظ٦سד圓䧭ׁ׸׷ؙٓأةח㼎׃ג
 ؝ٝذش،فٔ؛٦ءّٝךꂁ⪒ծ鏣㹀ծ盖椚׾遤ֲ Ӝ κυβερνήτηςؘٔءٍ铂ד乼菭㡦 Ӝ (PPHMFך爡ⰻءأذيַ׵؎ٝأػ؎،ׁ׸׋ “Kubernetes

    is open source—a contrast to Borg and Omega,
 which were developed as purely Google-internal systems. “ Borg, Omega, and Kubernetes
  82. acL 3 3A g A3 2 M dT ,VCFSOFUFT Ӝ

    䎃剢ח剑ⴱך؝ىحزָ֮׶ծ䎃剢ח׾ٔٔ٦أ 䎃剢儗挿ך剑倜غ٦آّٝכ Ӝ ⴱ劍כ(PPHMFךاؿزؐؑ،׌׏׋ָծ׉ך䖓
 $MPVE/BUJWF$PNQVUJOH'PVOEBUJPO$/$'ח隁床ׁ׸ծ
 植㖈כ$/$'ָمأزׅ׷ؔ٦فٝا٦أفٗآؙؑز Ӝ ؔٝفٖכ׮׍׹׿ծ⚺銲זؙٓؐسفٗغ؎تדوط٦آس؟٦ؽأָ䲿⣘ׁ׸גְ׷ (PPHMF,VCFSOFUFT&OHJOF "NB[PO&,4 "[VSF,VCFSOFUFT4FSJWJDFT Ӝ չ䎃⟃♳ח床׶؝ٝذش׾劤殢橆㞮ד麊欽׃גֹ׋(PPHMFך穗꿀ה
 㢳ֻך⟰噟ח״׷؝ىُصذ؍ך⮚׸׋،؎ر،ה䩛岀ָ穈׫鴥ת׸גְ׷պ
  83. َؿ٦ך,VCFSOFUFTBTB4FSWJDF َؿ٦吳䒭⠓爡ぢֽחئحزٓنָꟚ涪ׅ׷وط٦آس,VCFSOFUFT؟٦ؽأկ
 ؔٝفٖىأך0QFO4UBDL橆㞮ח,VCFSOFUFTؙٓأة׾⡲䧭ծ盖椚ׅ׷կ䎃剢갦ַ׵Ꟛ涪 ׾㨣׭ծَؿ٦ך,VCFSOFUFTךⵃ欽׾佄ִ׷կ Ӝ إٕؿ؟٦ؽأꟚ涪罏ְָאד׮㥨ֹזהֹח Ӝ وط٦آس孡ָ⸬ֹתׅ Ӝ أ؛٦ٓـٕ㼭ְׁ׮ךַ׵㣐ֹז׮ךתד

    Ӝ ءؚٕٝذشٝز杝׶⽑׭
  84. 2VFTUJPO Ӝ Kazuki Suda <LTVEB![MBCDPKQ !TVQFSCSPUIFST Ӝ 4MJEFIUUQTTQFBLFSEFDLDPNTVQFSCSPUIFST