やるぜ！Kubernetesハンズオン！

2019/05/09 Fusic社内Kubernetesハンズオン 資料

Transcript

1. ΍Δͥʂ,VCFSOFUFTϋϯζΦϯʂ  ໟར
   ܒଠ
2. ຊ೔ͷ໨ඪ ‣ ,VCFSOFUFTͱ͓༑ୡʹͳΔ
    ΠϝʔδΛ௫Ή
    ৮ͬͯΈΔ ϩʔΧϧɾ&,4ɾͪΐͬͱ(,&
    ߏஙɾӡ༻͕Ͱ͖ΔΑ͏ͳؾ෼ʹͳΔ
   ‣ ޽ʹ͋;ΕΔ,VCFSOFUFTͷهࣄɾεϥΠυ͕
 ཧղͰ͖ΔΑ͏ʹͳΔ 2
3. 3 લఏ஌ࣝɾ४උ
4. %PDLFS
   
   %PDLFS
   $PNQPTF ‣ ͜͜Ͱ͸લఏ஌ࣝͱͯ͠ѻ͏ͷͰઆ໌͠·ͤΜ
    IUUQTHJUIVCDPNOPIFEPDLFSMFTTPOT
    IUUQTXXXTMJEFTIBSFOFU[FNCVUTVEPDLFSDPNQPTF HVJEFCPPL 4
5. %PDLFS
   GPS
   %FTLUPQ ‣ .BD8JOEPXT༻ͷ%PDLFS؀ڥ
    IUUQTXXXEPDLFSDPNQSPEVDUTEPDLFSEFTLUPQ 5
6. LVCFDUM ‣ ,VCFSOFUFTΫϥελΛૢ࡞͢Δ$-*
    IUUQTLVCFSOFUFTJPEPDTUBTLTUPPMTJOTUBMMLVCFDUM
   ‣ .BDͷ৔߹
    CSFX
   JOTUBMM
   LVCFSOFUFTDMJ 6
7. "84ΞΧ΢ϯτ
   
   *".Ϣʔβʔ
   
   "84
   $-* ‣ ݸਓͷ"84ΞΧ΢ϯτ
   ‣ *".Ϣʔβʔ
    "ENJOJTUSBUPS"DDFTTͷϙϦγʔΛΞλον
   w ݖݶߜΔͷ͕໘౗ͩͬͨ
    ΞΫηεΩʔΛ࡞੒
   ‣ "84
   $-*
    IUUQTHJUIVCDPNBXTBXTDMJ
    ࡞੒ͨ͠*".ϢʔβʔΞΫηεΩʔΛ࢖ͬͯQSPpMFΛઃఆ͓ͯ͘͠
    QSPpMFΛ؀ڥม਺
   "84@130'*-&
   ʹઃఆ 7
8. BXTJBNBVUIFOUJDBUPS ‣ *".ͷೝূ৘ใΛ࢖༻ͯ͠,VCFSOFUFTΫϥελʔ΁ͷ
 ೝূΛߦ͏πʔϧ
    IUUQTEPDTBXTBNB[PODPNKB@KQFLTMBUFTU VTFSHVJEFJOTUBMMBXTJBNBVUIFOUJDBUPSIUNM
   ‣ .BDͷ৔߹
    CSFX
   JOTUBMM
   BXTJBNBVUIFOUJDBUPS 8
9. FLTDUM ‣ &,4ͰΫϥελΛ࡞੒͢ΔͨΊͷ$-*
    IUUQTFLTDUMJP
   ‣ .BDͷ৔߹
    CSFX
   UBQ
   XFBWFXPSLTUBQ
    CSFX
   JOTUBMM
   XFBWFXPSLTUBQFLTDUM 9
10. ؀ڥ֬ೝ 10 $ docker -v Docker version 18.09.2, build 6247962 $ kubectl version --short --client Client Version: v1.13.4 $ aws --version aws-cli/1.16.125 Python/2.7.14 Darwin/18.2.0 botocore/1.12.115 $ aws-iam-authenticator help (লུ) $ eksctl version [ℹ] version.Info{BuiltAt:"", GitCommit:"", GitTag:"0.1.31"}
11. 11 ,VCFSOFUFTͱ͸
12. ,VCFSOFUFTͱ͸ ‣ ίϯςφΦʔέετϨʔγϣϯγεςϜ
     ͨ͘͞Μͷαʔόʔʹ
     ͨ͘͞ΜͷίϯςφΛஔ͍ͯ
     ࿈ܞͤ͞ΔΑ͏ͳΞϓϦέʔγϣϯΛ
     σϓϩΠɾ؅ཧɾεέʔϧͱ͔ͤ͞Δ΍ʔͭ
    ‣ ͷɺσϑΝΫτελϯμʔυ 12
13. ,VCFSOFUFTͱ͸ ‣ ೥ʹ(PPHMF͕044ͱͯ͠ެ։
     (PPHMFͷ௕͖ʹΘͨΔίϯςφӡ༻ͷ஌ݟ͕٧·͍ͬͯΔ
    ‣ ݱࡏ͸$/$'(Cloud Native Computing Foundation)͕؅ཧ
     Ϋϥ΢υωΠςΟϒͳ044ٕज़ͷਪਐΛߦ͏ஂମ
    ‣ %PDLFS΋ެࣜʹαϙʔτ
     %PDLFS
    4XBSN͋Δͷʹʂ 13
14. ,VCFSOFUFTͱ͸ ‣ ($1͚ͩͰͳ͘ΦϯϓϨ΍ଞͷΫϥ΢υͰ΋ಈ͘
    ‣ ֤Ϋϥ΢υͰϚωʔδυαʔϏε͕ग़͍ͯΔ
     ($1
    (,&
     "84
    &,4
    &$4͋Δͷʹʂ
     "[VSF
    ",4
     *#.
    $MPVE چ#MVFNJY 
    *,4
     "MJCBCB
    $POUBJOFS
    4FSWJDF
    GPS
    ,VCFSOFUFT
    ‣ ਺೥ܦͯ͹౰ͨΓલͷٕज़ʹͳ͍ͬͯΔ͔΋ 14
15. 15 ,VCFSOFUFTͬͯ
 Կ͕͏Ε͍͠ͷʁ
16. ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ 16 MySQL Nginx App Redis SERVER Nginx App SERVER
17. ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ 17 docker-compose01.yaml docker-compose02.yaml MySQL Nginx App Redis SERVER Nginx App SERVER Ͳͷαʔόʔʹ
 Ͳͷίϯςφ͕͋Δ͔
 ؅ཧ͠ͳͪ͘Ό ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ
18. App ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ 18 docker-compose01.yaml docker-compose02.yaml MySQL Nginx App Redis SERVER Nginx SERVER ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ ίϯςφ͕ࢮΜͩΒ
 Ͳ͏΍ͬͯؾͮ͘ʁ Ͳ͏΍ͬͯճ෮͢Δʁ Ͳͷαʔόʔʹ
 Ͳͷίϯςφ͕͋Δ͔
 ؅ཧ͠ͳͪ͘Ό
19. App ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ 19 docker-compose01.yaml docker-compose02.yaml MySQL Nginx App Redis SERVER Nginx SERVER ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ αʔόʔΛލ͍ͩ
 ίϯςφؒͷ௨৴͸
 Ͳ͏͠Α͏ʁ ίϯςφ͕ࢮΜͩΒ
 Ͳ͏΍ͬͯؾͮ͘ʁ Ͳ͏΍ͬͯճ෮͢Δʁ Ͳͷαʔόʔʹ
 Ͳͷίϯςφ͕͋Δ͔
 ؅ཧ͠ͳͪ͘Ό
20. App ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ 20 docker-compose01.yaml docker-compose02.yaml MySQL Nginx App Redis SERVER Nginx SERVER ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ αʔόʔΛލ͍ͩ
 ίϯςφؒͷ௨৴͸
 Ͳ͏͠Α͏ʁ v1.1.0 ίϯςφͷߋ৽ Ͳ͏͠Α͏ʁ
 ॱংɺґଘؔ܎etc… ίϯςφ͕ࢮΜͩΒ
 Ͳ͏΍ͬͯؾͮ͘ʁ Ͳ͏΍ͬͯճ෮͢Δʁ Ͳͷαʔόʔʹ
 Ͳͷίϯςφ͕͋Δ͔
 ؅ཧ͠ͳͪ͘Ό
21. App ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ 21 docker-compose01.yaml docker-compose02.yaml MySQL Nginx App Redis SERVER Nginx SERVER ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ αʔόʔΛލ͍ͩ
 ίϯςφؒͷ௨৴͸
 Ͳ͏͠Α͏ʁ Load Balancer ϩʔυόϥϯγϯά
 ͠ͳ͍ͱͶ ίϯςφ͕ࢮΜͩΒ
 Ͳ͏΍ͬͯؾͮ͘ʁ Ͳ͏΍ͬͯճ෮͢Δʁ v1.1.0 ίϯςφͷߋ৽ Ͳ͏͠Α͏ʁ
 ॱংɺґଘؔ܎etc… Ͳͷαʔόʔʹ
 Ͳͷίϯςφ͕͋Δ͔
 ؅ཧ͠ͳͪ͘Ό
22. App SERVER App ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ 22 docker-compose01.yaml docker-compose02.yaml MySQL Nginx App Redis SERVER Nginx SERVER ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ αʔόʔΛލ͍ͩ
 ίϯςφؒͷ௨৴͸
 Ͳ͏͠Α͏ʁ Load Balancer ϩʔυόϥϯγϯά
 ͠ͳ͍ͱͶ ίϯςφ͕ࢮΜͩΒ
 Ͳ͏΍ͬͯؾͮ͘ʁ Ͳ͏΍ͬͯճ෮͢Δʁ v1.1.0 ίϯςφͷߋ৽ Ͳ͏͠Α͏ʁ
 ॱংɺґଘؔ܎etc… Nginx εέʔϦϯά… Ͳͷαʔόʔʹ
 Ͳͷίϯςφ͕͋Δ͔
 ؅ཧ͠ͳͪ͘Ό
23. App SERVER App ίϯςφͰΞϓϦέʔγϣϯͷӡ༻ 23 docker-compose01.yaml docker-compose02.yaml MySQL Nginx App Redis SERVER Nginx SERVER ‣ ͨ͘͞Μͷαʔόʔʹͨ͘͞ΜͷίϯςφΛஔ͍ͯ࿈ܞ
     %PDLFS$PNQPTFͰͲ͏ʹ͔഑ஔͯ͠Έͨͱ͜Ζ αʔόʔΛލ͍ͩ
 ίϯςφؒͷ௨৴͸
 Ͳ͏͠Α͏ʁ Load Balancer ϩʔυόϥϯγϯά
 ͠ͳ͍ͱͶ ίϯςφ͕ࢮΜͩΒ
 Ͳ͏΍ͬͯؾͮ͘ʁ Ͳ͏΍ͬͯճ෮͢Δʁ v1.1.0 ίϯςφͷߋ৽ Ͳ͏͠Α͏ʁ
 ॱংɺґଘؔ܎etc… Nginx εέʔϦϯά… Ͳͷαʔόʔʹ
 Ͳͷίϯςφ͕͋Δ͔
 ؅ཧ͠ͳͪ͘Ό ϩάͷ؅ཧ…
24. ਓྨʹ͸ߴ౓͗͢Δ 24
25. Ͱ͖ΔΜͰ͢
    ͦ͏ɺ,VCFSOFUFTͳΒͶ 25
26. 26 ,VCFSOFUFTͷ
 ΠϝʔδΛ௫Ή
27. ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 27 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ
28. ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 28 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ Nginx App Redis MySQL Nginx App ͍͍ײ͡ʹ৔ॴΛ൑அͯ͠ ίϯςφΛ഑ஔ
29. ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 29 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ Nginx App Redis MySQL Nginx ͍͍ײ͡ʹ৔ॴΛ൑அͯ͠ ίϯςφΛ഑ஔ App App ίϯςφ͕ࢮΜͩΒ ࣗಈճ෮
30. ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 30 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ Nginx App Redis MySQL Nginx ͍͍ײ͡ʹ৔ॴΛ൑அͯ͠ ίϯςφΛ഑ஔ ίϯςφ͕ࢮΜͩΒ ࣗಈճ෮ αʔόʔؒͷ ωοτϫʔΫ΋ ͍͍ײ͡ʹ App App
31. ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 31 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ
    v1.1.0 Nginx App Redis MySQL Nginx ͍͍ײ͡ʹ৔ॴΛ൑அͯ͠ ίϯςφΛ഑ஔ ίϯςφ͕ࢮΜͩΒ ࣗಈճ෮ αʔόʔؒͷ ωοτϫʔΫ΋ ͍͍ײ͡ʹ App App App v1.1.0 App v1.1.0 ϩʔϦϯάΞοϓσʔτ Blue/GreenσϓϩΠϝϯτ
32. ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 32 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ
    v1.1.0 Nginx App Redis MySQL Nginx ͍͍ײ͡ʹ৔ॴΛ൑அͯ͠ ίϯςφΛ഑ஔ ίϯςφ͕ࢮΜͩΒ ࣗಈճ෮ αʔόʔؒͷ ωοτϫʔΫ΋ ͍͍ײ͡ʹ App App App v1.1.0 App v1.1.0 ϩʔϦϯάΞοϓσʔτ Blue/GreenσϓϩΠϝϯτ Load Balancer ϩʔυόϥϯγϯά΋ Ͱ͖Δ
33. App SERVER Nginx ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 33 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ
    v1.1.0 Nginx App Redis MySQL Nginx ͍͍ײ͡ʹ৔ॴΛ൑அͯ͠ ίϯςφΛ഑ஔ ίϯςφ͕ࢮΜͩΒ ࣗಈճ෮ αʔόʔؒͷ ωοτϫʔΫ΋ ͍͍ײ͡ʹ App App App v1.1.0 App v1.1.0 ϩʔϦϯάΞοϓσʔτ Blue/GreenσϓϩΠϝϯτ Load Balancer ϩʔυόϥϯγϯά΋ Ͱ͖Δ αʔόʔͷ ΦʔτεέʔϦϯάʂ
34. App SERVER Nginx ϕʔεͱͳΔΞΠσΞ ‣ ίϯςφͷ؅ཧΛ͍͍ײ͡ʹ΍ͬͯ͘ΕΔγεςϜ͕͋Ε͹͍͍͡ΌΜ 34 SERVER SERVER ͍͍ײ͡ͷ γεςϜ ஫จॻ
    v1.1.0 Nginx App Redis MySQL Nginx ͍͍ײ͡ʹ৔ॴΛ൑அͯ͠ ίϯςφΛ഑ஔ ίϯςφ͕ࢮΜͩΒ ࣗಈճ෮ αʔόʔؒͷ ωοτϫʔΫ΋ ͍͍ײ͡ʹ App App App v1.1.0 App v1.1.0 ϩʔϦϯάΞοϓσʔτ Blue/GreenσϓϩΠϝϯτ Load Balancer ϩʔυόϥϯγϯά΋ Ͱ͖Δ αʔόʔͷ ΦʔτεέʔϦϯάʂ ϩάऩू΋Χϯλϯ
35. Ұ୴͜͜·ͰΛ,VCFSOFUFT༻ޠʹม׵ 35 Kubernetes Cluster ϚχϑΣετ kubectl (CLI for k8s) = Control Plane Data Plane
36. ϚχϑΣετ ‣ γεςϜͷ͋Δ΂͖࢟Λॻ͍͍ͯΔZBNMϑΝΠϧ
     ʮએݴతઃఆʯͱݺ͹ΕΔ
    
    
    
    ʮ໋ྩతઃఆʯ
     એݴͨ͠ঢ়ଶΛҡ࣋͠Α͏ͱͯ͘͠ΕΔ
    ‣ ໋ྩతઃఆ
    
    ී௨ͷͦ͹԰͞Μ
     ͟Δͦ͹ͭ஫จ͢Δ
    
    ͦ͹Λग़ͨ͋͠ͱ͸஌Βͳ͍
     ͓͔ΘΓ͢Δͱ͖͸ผ్஫จ͕ඞཁ
    ‣ એݴతઃఆ
    
    ΘΜͦ͜͹
     ͓࿶ʹͦ͹͕͋Δঢ়ଶ͕͋Δ΂͖࢟
    
    ͦ͹Λৗʹ؂ࢹ͍ͯ͠Δ
     ͦ͹Λ৯΂ͨΒࣗಈͰ௥Ճͯ͘͠ΕΔ 36
37. ,VCFSOFUFTͷΠϝʔδ ‣ એݴతʹॻ͍ͨϚχϑΣετΛ
    ‣ LVCFDUMΛ࢖ͬͯNBTUFSʹ౉͢ͱ
    ‣ ֤OPEFʹίϯςφΛσϓϩΠͨ͠Γͯ͘͠Εͯ
    ‣ ͦͷޙ͸͍͍ײ͡ʹ؂ࢹɾҡ࣋Λͯ͘͠ΕΔ΍ʔͭ 37 Kubernetes Cluster ϚχϑΣετ kubectl (CLI for k8s) = Control Plane Data Plane
38. ͱΓ͋͑ͣ΍ͬͯΈΑ͏ 38
39. 39 ,VCFSOFUFTΛ
 ϩʔΧϧͰࢼͯ͠ΈΑ͏
40. ,VCFSOFUFT༗ޮԽ 40 1SFGFSFODFTը໘ 
    ,VCFSOFUFTλϒΛબ୒ 
    νΣοΫʂ 
    "QQMZ ॳճ͸༗ޮʹͳΔ·Ͱ
 ෼΄Ͳ͔͔Γ·͢
41. ,VCFSOFUFT༗ޮԽ 41 Ҏલ࢖ͬͨ͜ͱ͋Δਓ͸Ұ୴Ϧηοτ͓ͯ͘͠ͱ͍͍͔΋ 
    3FTFUλϒΛબ୒ Ϧηοτ
42. LVCFDUMͷDPOpHઃఆ 42 # kubectlͰૢ࡞͢ΔKubernetesΫϥελʔΛdocker-for-desktopʹ $ kubectl config user-context docker-for-desktop # ͪΌΜͱઃఆ͞Ε͍ͯΔ͔֬ೝ $ kubectl config current-context docker-for-desktop
43. ಈ࡞֬ೝͱԼ४උ 43 # ಈ࡞֬ೝɻͳΜ͔͍Ζ͍Ζಈ͍ͯΔ(આ໌͸͋ͱͰ) $ kubectl get pods --namespace=kube-system NAMESPACE NAME READY STATUS RESTARTS AGE kube-system etcd-docker-for-desktop 1/1 Running 0 1m kube-system kube-apiserver-docker-for-desktop 1/1 Running 0 1m kube-system kube-controller-manager-docker-for-desktop 1/1 Running 0 1m kube-system kube-dns-86f4d74b45-xb4qh 3/3 Running 0 2m kube-system kube-proxy-8r45p 1/1 Running 0 2m kube-system kube-scheduler-docker-for-desktop 1/1 Running 0 1m # docker-for-desktopͰingress͕࢖͑ΔΑ͏Լ४උɻ͋·Γؾʹ͠ͳͯ͘OK $ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/ mandatory.yaml $ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/ provider/cloud-generic.yaml
44. σϓϩΠ 44 # αϯϓϧΞϓϦέʔγϣϯͷϚχϑΣετϑΝΠϧΛऔͬͯ͘Δ $ git clone git@github.com:kubernetes/examples.git # PCͷෛ୲ΛԼ͛ΔͨΊͪΐͬͱௐ੔ $ vi examples/guestbook/frontend-deployment.yaml 10ߦ໨ replicas: 3 <- ͜ΕΛ1ʹมߋ $ vi examples/guestbook/redis-slave-deployment.yaml 11ߦ໨ replicas: 2 <- ͜ΕΛ1ʹมߋ # σϓϩΠ $ kubectl apply -f examples/guestbook/
45. ϒϥ΢β͔ΒΞΫηεͰ͖ΔΑ͏ʹ 45 $ cat << 'EOT' >./guestbook-ingress.yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: name: guestbook-ingress spec: rules: - http: paths: - path: / backend: serviceName: frontend servicePort: 80 EOT # σϓϩΠ $ kubectl apply -f guestbook-ingress.yaml # ADDRESS͕localhostʹͳΔ·Ͱ଴ػ $ kubectl get ingress NAME HOSTS ADDRESS PORTS AGE guestbook-ingress * localhost 80 24m
46. Ͱ͖ͨʂ 46 IUUQMPDBMIPTU
    1$ͷঢ়گʹΑͬͯ͸͏·͘ΞΫηεͰ͖ͳ͔ͬͨΓ͢Δ͔΋
    IUUQͩͱ͏·͍͘͘৔߹΋͋Δͱ͔
47. Կ͕ىͬͨ͜ 47
48. ͜Μͳͷ͕Ͱ͖ͯ·͢ 48 Kubernetes Cluster Redis-Master Guestbook-App gcr.io/google-samples/gb-frontend:v4 k8s.gcr.io/redis:e2e Redis-Slave gcr.io/google_samples/gb-redisslave:v1 localhost
49. ୯७ͳΞϓϦͰ͢Ͷ 49 Kubernetes Cluster Redis-Master Guestbook-App gcr.io/google-samples/gb-frontend:v4 k8s.gcr.io/redis:e2e Redis-Slave gcr.io/google_samples/gb-redisslave:v1 localhost
50. 
    
    
    
    
    1PE ‣ ,VCFSOFUFTͷ࠷খσϓϩΠ୯Ґ
    ‣ ͭҎ্ͷίϯςφͱετϨʔδϘϦϡʔϜͷू·Γ
    ‣ ಉҰ1PE಺ͷίϯςφ͸ಉҰ/PEFʹ഑ஔ͞ΕΔ
     ʮಉҰ/PEFͰಈ࡞͢Δඞཁ͕͋Δ͔ʁʯ͕1PEߏ੒ͷҰͭͷج४
    ‣ ͭͷ1PE಺ͷίϯςφ͸ಉ͡*1ΞυϨεͱϙʔτΛ࢖༻͢Δ
     1PE಺ͷίϯςφؒͷ௨৴͸ϓϩηεؒ௨৴ͱͯ͠ߦ͏ 50
51. 
    
    
    
    3FQMJDB4FU ‣ ಉ͡࢓༷ͷ1PE͕ࢦఆͨ͠਺͚ͩଘࡏ͢ΔΑ͏ੜ੒ɾ؅ཧ͢Δ
     1PE͕ࢮΜͩͱ͖΋ࢦఆͨ͠਺ʹͳΔΑ͏ࣗಈճ෮ͯ͘͠ΕΔ
    ‣ 1PEͱ3FQMJDB4FU͸ૄ݁߹
     -BCFMͱ͍͏ϝλσʔλΛ࢖ͬͯ౎౓ݕࡧ͍ͯ͠Δ
     खಈͰ1PEͷ-BCFMΛॻ͖׵͑Ε͹ɺ
 3FQMJDB4FU͔Β੾Γ཭ͯ͠σόοά͢Δͱ͍ͬͨ͜ͱ΋Մೳ 51
52. 
    
    
    
    %FQMPZNFOU ‣ ৽͍͠όʔδϣϯͷϦϦʔεΛ؅ཧ͢ΔͨΊͷ࢓૊Έ
     3FQMJDB4FUͷมߋΛ҆શʹ൓өͤ͞Δ
    
    ੈ୅؅ཧ͢Δ
     1PEͷεέʔϧɺίϯςφͷߋ৽ɺϩʔϧόοΫFUD
    ‣ ͭͷ%FQMPZNFOUઓུ
     3FDSFBUF
     3PMMJOH6QEBUF
    ‣ 3FQMJDB4FUͱ%FQMPZNFOU΋ૄ݁߹ 52
53. GSPOUFOEEFQMPZNFOUZBNM 53 apiVersion: apps/v1 # apply࣌ʹ࢖༻͢ΔAPIͷछผɻϦιʔε(kind)ʹΑܾͬͯ·Δ kind: Deployment # DeploymentͷϚχϑΣετ metadata: name: frontend # DeploymentϦιʔεͷ໊લɻʮmetadata.name + ϥϯμϜจࣈྻʯͷ໊લͰReplicaSet͕ੜ੒͞ΕΔ spec: selector: matchLabels: # ReplicaSet͕PodΛݕࡧ͢Δͱ͖ͷLabel app: guestbook tier: frontend replicas: 1 # ReplicaSet͕ੜ੒ɾ؅ཧ͢ΔPodͷ਺ template: # ---͔͜͜ΒPodͷఆٛ-------------------------------------------- metadata: labels: # PodͷLabelɻReplicaSet͕؅ཧԼͷPodΛݕࡧ͢Δͱ͖ʹ࢖͏ app: guestbook tier: frontend spec: containers: - name: php-redis # ίϯςφ໊ image: gcr.io/google-samples/gb-frontend:v4 # ίϯςφΠϝʔδ resources: # ࢖༻͢ΔCPU, Memoryͷࢦఆ requests: cpu: 100m memory: 100Mi env: # ؀ڥม਺ - name: GET_HOSTS_FROM value: dns ports: # EXPOSE͢Δϙʔτͷࢦఆ - containerPort: 80
54. SFEJTNBTUFSEFQMPZNFOUZBNM 54 apiVersion: apps/v1 kind: Deployment metadata: name: redis-master spec: selector: matchLabels: app: redis role: master tier: backend replicas: 1 template: metadata: labels: app: redis role: master tier: backend spec: containers: - name: master image: k8s.gcr.io/redis:e2e resources: requests: cpu: 100m memory: 100Mi ports: - containerPort: 6379
55. SFEJTTMBWFEFQMPZNFOUZBNM 55 apiVersion: apps/v1 kind: Deployment metadata: name: redis-slave spec: selector: matchLabels: app: redis role: slave tier: backend replicas: 1 template: metadata: labels: app: redis role: slave tier: backend spec: containers: - name: slave image: gcr.io/google_samples/gb-redisslave:v1 resources: requests: cpu: 100m memory: 100Mi env: - name: GET_HOSTS_FROM value: dns ports: - containerPort: 6379
56. 
    
    
    
    4FSWJDF ‣ 1PEͷू߹ ओʹ3FQMJDB4FU ʹର͢Δܦ࿏΍
 αʔϏεσΟεΧόϦΛఏڙ
     Ϋϥελ಺%/4Ͱɺ4FSWJDF໊/BNFTQBDF໊Ͱ໊લղܾՄೳʹ
     ಉ͡/BNFTQBDF಺ͳΒ4FSWJDF໊͚ͩͰ0,
    ‣ ͜͜Ͱ΋-BCFMʹΑͬͯର৅ͷ1PE͕ݕࡧ͞ΕΔ
     ର৅ͷ1PE͕ಈతʹೖΕସΘͬͨΓͯ͠΋ɺ
 -BCFM͍͍͑ͭͯ͞Ε͹Ұ؏໊ͨ͠લͰΞΫηεͰ͖Δ 56
57. GSPOUFOETFSWJDFZBNM 57 apiVersion: v1 kind: Service # ServiceͷϚχϑΣετ metadata: name: frontend # ServiceϦιʔεͷ໊લ labels: # Serviceʹ͚ͭΔLabel app: guestbook tier: frontend spec: type: NodePort # ServiceͷछผɻNodePort͸Ϋϥελ֎͔ΒΞΫηεͰ͖Δ΍ͭ ports: - port: 80 # ΞΫηεΛड͚෇͚Δϙʔτ selector: # ର৅ͷPodΛݕࡧ͢Δͱ͖ͷLabel app: guestbook tier: frontend
58. SFEJTNBTUFSTFSWJDFZBNM 58 apiVersion: v1 kind: Service metadata: name: redis-master labels: app: redis role: master tier: backend spec: # লུ͞Ε͍ͯΔ͚Ͳtype͸σϑΥϧτͷ"ClusterIP"ɻΫϥελ্ͷ಺෦IPΞυϨεʹServiceΛެ։ ports: - port: 6379 targetPort: 6379 selector: app: redis role: master tier: backend
59. SFEJTTMBWFTFSWJDFZBNM 59 apiVersion: v1 kind: Service metadata: name: redis-slave labels: app: redis role: slave tier: backend spec: ports: - port: 6379 selector: app: redis role: slave tier: backend
60. 
    
    
    
    *OHSFTT ‣ 4FSWJDFΛΫϥελ֎ʹެ։
    ‣ /PEF1PSUλΠϓͷ4FSWJDFͱҧ͍ɺ
 ύεϕʔεͰసૹઌͷ4FSWJDFΛ੾Γସ͑Δͱ͍ͬͨ͜ͱ΋Մೳ
     4FSWJDF /PEF1PSU
    
    -ϨϕϧͰͷ੍ޚ
     *OHSFTT
    
    -ϨϕϧͰͷ੍ޚ 60
61. HVFTUCPPLJOHSFTTZBNM 61 apiVersion: extensions/v1beta1 kind: Ingress # IngressͷϚχϑΣετ metadata: name: guestbook-ingress # IngressϦιʔεͷ໊લ spec: rules: # ϧʔςΟϯάͷϧʔϧͷ഑ྻ - http: paths: - path: / backend: # "frontend"Serviceͷ80൪ϙʔτʹΞΫηε serviceName: frontend servicePort: 80
62. ͜Θ͘ͳʔ͍ 62 Kubernetes Cluster Redis-Master Guestbook-App gcr.io/google-samples/gb-frontend:v4 k8s.gcr.io/redis:e2e Redis-Slave gcr.io/google_samples/gb-redisslave:v1 localhost
63. தΛ೷͍ͯΈΑ͏ 63
64. %FQMPZNFOUΛݟͯΈΔ 64 # Deploymentৄࡉ (ը໘ʹೖΒͳ͍ͷͰखݩͰݟͯʂ) $ kubectl describe deploy frontend 3FQMJDB4FUΛݕࡧ͢ΔͨΊͷ-BCFMηϨΫλ kubectl get rs -l app=guestbook,tier=frontend ͱ͍ͬͨײ͡Ͱର৅ͷ3FQMJDB4FUΛݕࡧͰ͖Δ # DeoloymentҰཡ ("-o wide"͸ৄࡉΛݟΔͨΊͷΦϓγϣϯ) $ kubectl get deploy -o wide NAME (ུ) SELECTOR frontend ... app=guestbook,tier=frontend redis-master ... app=redis,role=master,tier=backend redis-slave ... app=redis,role=slave,tier=backend
65. 3FQMJDB4FUΛݟͯΈΔ 65 # ReplicaSetҰཡ $ kubectl get rs -o wide NAME (ུ) SELECTOR frontend-5c548f4769 ... app=guestbook,pod-template-hash=1710490325,tier=frontend redis-master-55db5f7567 ... app=redis,pod-template-hash=1186193123,role=master,tier=backend redis-slave-584c66c5b5 ... app=redis,pod-template-hash=1407227161,role=slave,tier=backend # ReplicaSetৄࡉ (ը໘ʹೖΒͳ͍ͷͰखݩͰݟͯʂ) $ kubectl describe rs frontend-5c548f4769 %FQMPZNFOU໊
    
    ϥϯμϜจࣈྻ 3PMMJOH6QEBUFͳͲͰಉ͡UFNQMBUFͷ1PEΛ
 ؅ཧ͢Δ3FQMJDB4FU͕ෳ਺ࠞࡏͯ͠΋େৎ෉ͳΑ͏ʹ
 ݻ༗஋ͷ-BCFMΛࣗಈͰೖΕ͍ͯΔ
66. 1PEΛݟͯΈΔ 66 # PodҰཡ $ kubectl get pod -o wide NAME READY STATUS RESTARTS AGE IP NODE frontend-5c548f4769-xhpxz 1/1 Running 0 1d 10.1.1.65 docker-for-desktop redis-master-55db5f7567-2n4qp 1/1 Running 0 1d 10.1.1.67 docker-for-desktop redis-slave-584c66c5b5-z2fvj 1/1 Running 0 1d 10.1.1.66 docker-for-desktop # Podৄࡉ (ը໘ʹೖΒͳ͍ͷͰखݩͰݟͯʂ) # ઃఆ΍ىಈ೔࣌ɺঢ়ଶɺΠϕϯτ౳֬ೝͰ͖Δ $ kubectl describe pod frontend-5c548f4769-xhpxz # Podͷϩά (ը໘ʹೖΒͳ͍ͷͰखݩͰݟͯʂ) $ kubectl logs frontend-5c548f4769-xhpxz 3FQMJDB4FU໊
    
    ϥϯμϜจࣈྻ
67. 4FSWJDFΛݟͯΈΔ 67 # ServiceҰཡ $ kubectl get svc -o wide NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR frontend NodePort 10.102.204.76 <none> 80:30590/TCP 1d app=guestbook,tier=frontend kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 1d <none> redis-master ClusterIP 10.98.133.213 <none> 6379/TCP 1d app=redis,role=master,tier=backend redis-slave ClusterIP 10.107.141.173 <none> 6379/TCP 1d app=redis,role=slave,tier=backend # Serviceৄࡉ (ը໘ʹೖΒͳ͍ͷͰखݩͰݟͯʂ) $ kubectl describe svc frontend GSPOUFOE͕/PEF1PSUʹͳ͍ͬͯΔͷͰɺ
    IUUQMPDBMIPTU
    Ͱ΋ΞΫηεͰ͖Δʢͨͩ͠-੍ޚʣ
68. *OHSFTTΛݟͯΈΔ 68 # IngressҰཡ $ kubectl get ing NAME HOSTS ADDRESS PORTS AGE guestbook-ingress * localhost 80 1d # Ingressৄࡉ (ը໘ʹೖΒͳ͍ͷͰखݩͰݟͯʂ) $ kubectl describe ing guestbook-ingress
69. εέʔϧͤͯ͞ΈΑ͏ 69
70. GSPOUFOEͷ1PEΛͭʹ૿΍͢ 70 $ vi examples/guestbook/frontend-deployment.yaml 10ߦ໨ replicas: 1 <- ͜ΕΛ2ʹมߋ # σϓϩΠ $ kubectl apply -f examples/guestbook/frontend-deployment.yaml # ૿͑ͯΔʂ $ kubectl get pod NAME READY STATUS RESTARTS AGE frontend-5c548f4769-vltkv 1/1 Running 0 15s frontend-5c548f4769-xhpxz 1/1 Running 0 1d redis-master-55db5f7567-2n4qp 1/1 Running 0 1d redis-slave-584c66c5b5-z2fvj 1/1 Running 0 1d
71. ࣗಈճ෮ͤͯ͞ΈΑ͏ 71
72. ҙਤతʹ1PEΛ࡟আͯ͠ΈΔ 72 # ઌ΄Ͳ૿͑ͨ2ͭΊͷPodΛ࡟আͯ͠ΈΔ $ kubectl delete pod frontend-5c548f4769-vltkv # ৽͍͠pod͕Ͱ͖ͯΔʂ $ kubectl get pod NAME READY STATUS RESTARTS AGE frontend-5c548f4769-ns5q2 1/1 Running 0 8s frontend-5c548f4769-xhpxz 1/1 Running 0 1d redis-master-55db5f7567-2n4qp 1/1 Running 0 1d redis-slave-584c66c5b5-z2fvj 1/1 Running 0 1d
73. %FQMPZNFOUͷ
    σϓϩΠ؅ཧͬ΀ΓΛݟͯΈΑ͏ 73
74. 1PEͷઃఆΛม͑ͯΈΔ 74 # Podͷมߋ͕ى͜Βͳ͍ͱཤྺ͕ه࿥͞Εͳ͍(εέʔϧ͡Όμϝ)ͷͰɺࢼ͠ʹ࢖༻ϝϞϦΛม͑ͯΈΔ $ examples/guestbook/frontend-deployment.yaml 23ߦ໨ memory: 100Mi <- ͜ΕΛ120Miʹมߋ $ kubectl apply -f examples/guestbook/frontend-deployment.yaml # ঃʑʹ੾ΓସΘ͍ͬͯΔʂ $ kubectl get pod NAME READY STATUS RESTARTS AGE frontend-5c548f4769-ns5q2 1/1 Running 0 11m frontend-5c548f4769-xhpxz 1/1 Running 0 1d frontend-68dd74b969-ztcdw 0/1 ContainerCreating 0 5s redis-master-55db5f7567-2n4qp 1/1 Running 0 1d redis-slave-584c66c5b5-z2fvj 1/1 Running 0 1d $ kubectl get pod NAME READY STATUS RESTARTS AGE frontend-5c548f4769-xhpxz 1/1 Running 0 1d frontend-68dd74b969-6shhj 0/1 ContainerCreating 0 6s frontend-68dd74b969-ztcdw 1/1 Running 0 21s redis-master-55db5f7567-2n4qp 1/1 Running 0 1d redis-slave-584c66c5b5-z2fvj 1/1 Running 0 1d $ kubectl get pod NAME READY STATUS RESTARTS AGE frontend-68dd74b969-6shhj 1/1 Running 0 26s frontend-68dd74b969-ztcdw 1/1 Running 0 41s redis-master-55db5f7567-2n4qp 1/1 Running 0 1d redis-slave-584c66c5b5-z2fvj 1/1 Running 0 1d
75. ཤྺΛݟͯΈΔ 75 # REVISIONͷ਺஋͕େ͖͍΄͏͕৽͍͠ # CHANGE-CAUSE͸ϚχϑΣετϑΝΠϧʹ"Annotation"ͱݺ͹ΕΔ৘ใΛ෇͚Ճ͑Δͱग़ͯ͘Δ(ࠓճ͸ؾʹ͠ͳ͍) $ kubectl rollout history deployments frontend deployment.extensions/frontend REVISION CHANGE-CAUSE 1 <none> 2 <none> # REVISION=2ͷৄࡉΛݟͯΈΔ(ը໘ʹೖΒͳ͍ͷͰखݩͰݟͯʂ) $ kubectl rollout history deployments frontend --revision=2
76. ϩʔϧόοΫͯ͠ΈΔ 76 # 1ͭલʹ໭Δͱ͖͸"--to-revision"͸লུͰ͖Δ $ kubectl rollout undo deployments frontend --to-revision=1 # ΋͏Ұ౓ཤྺΛݟΔͱɺREVISION=1͕ফ͍͑ͯΔ # ϩʔϧόοΫͰ΋ϦϏδϣϯ͸ੵ·ΕΔ # ಉ಺༰ͷϦϏδϣϯ͸ཤྺ͔Βফ͑Δ $ kubectl rollout history deployments frontend deployment.extensions/frontend REVISION CHANGE-CAUSE 2 <none> 3 <none>
77. ͱ͜ΖͰ.BTUFSͬͯ
    Կ΍ͬͯΔͷʁ 77
78. ࠷ॳʹݟͨ͜ͷதʹ͋Δ 78 # ಈ࡞֬ೝɻͳΜ͔͍Ζ͍Ζಈ͍ͯΔ(આ໌͸͋ͱͰ) $ kubectl get pods --namespace=kube-system NAMESPACE NAME READY STATUS RESTARTS AGE kube-system etcd-docker-for-desktop 1/1 Running 0 1m kube-system kube-apiserver-docker-for-desktop 1/1 Running 0 1m kube-system kube-controller-manager-docker-for-desktop 1/1 Running 0 1m kube-system kube-dns-86f4d74b45-xb4qh 3/3 Running 0 2m kube-system kube-proxy-8r45p 1/1 Running 0 2m kube-system kube-scheduler-docker-for-desktop 1/1 Running 0 1m
79. .BTUFS΋1PEͷू·Γ ‣
    
    
    
    
    
    
    FUDE
     Ϋϥελ಺ͷ͞·͟·ͳσʔλΛอଘ͍ͯ͠ΔҰ؏ੑͷ͋ΔߴՄ༻ੑͷ,74
    ‣
    
    
    
    
    
    
    LVCFBQJTFSWFS
     Ϋϥελʹର͢Δશͯͷૢ࡞Λ࢘Δ"1*αʔόʔ
     ೝূ΍ೝՄͷॲཧͳͲ΋ߦ͏
    ‣
    
    
    
    
    
    
    LVCFTDIFEVMFS
     1PEͷ/PEF΁ͷׂΓ౰ͯΛߦ͏εέδϡʔϥʔ
     1PEΛ഑ஔ͢Δ/PEFͷબ୒΋ߦ͏
    ‣
    
    
    
    
    
    
    LVCFDPOUSPMMFSNBOBHFS
     ֤छ,VCFSOFUFTΦϒδΣΫτͷίϯτϩʔϥʔΛىಈ͠؅ཧ͢ΔϚωʔδϟʔ 79
80. /PEFʹ΋.BTUFSͱ࿈ܞ͢Δ΋ͷ͕͍Δ ‣
    
    
    
    
    
    
    LVCFMFU
     /PEFͷϝΠϯॲཧͰ͋Δ1PEͷىಈɾ؅ཧΛߦ͏ΤʔδΣϯτ
    ‣
    
    
    
    
    
    
    LVCFQSPYZ
     4FSWJDF͕࣋ͭԾ૝తͳ*1ΞυϨε $MVTUFS*1 ΁ͷΞΫηεΛ
 ϧʔςΟϯά͢Δ 80
81. ͭ·Γ͸͜Μͳײ͡ 81 Kubernetes Cluster kubectl (CLI for k8s) ,74 "1*αʔόʔ 1PEΛ
    ϊʔυ΁ׂ౰ͯ ֤छ
 ίϯτϩʔϥʔ ϧʔςΟϯά 1PE؅ཧ
 ΤʔδΣϯτ ͢΂ͯͷૢ࡞͸
 APIαʔόʔܦ༝
82. ϩʔΧϧͰ༡Ϳͷ͸͜͜·Ͱ✋ 82
83. HVFTUCPPLΞϓϦέʔγϣϯΛ࡟আ 83 # guestbookΞϓϦέʔγϣϯΛ࡟আ $ kubectl delete -f examples/guestbook/ # ͠͹Β͘͢Δͱ࡟আ͞ΕΔ $ kubectl get pod No resources found.
84. ,VCFSOFUFTແޮԽ 84 1SFGFSFODFTը໘ 
    ,VCFSOFUFTλϒΛબ୒ 
    νΣοΫΛ֎͢ 
    "QQMZ 
    3FTFUλϒΛબ୒ ҰԠϦηοτ
85. 85 &,4ͰΫϥελΛ࡞ͬͯΈΑ͏
86. &,4ͬͯͲΜͳαʔϏεʁ 86 Kubernetes Cluster ϚχϑΣετ kubectl (CLI for k8s) = Control Plane Data Plane ͜ΕͷϚωʔδυαʔϏε
87. &,4ͬͯͲΜͳαʔϏεʁ ‣ ෳ਺";ͰNBTUFSΛ৑௕ߏ੒࣮ͯ͠ߦ
    ‣ NBTUFSͷ؂ࢹɾࣗಈճ෮
    ‣ ࣗಈΞοϓάϨʔυɾύονద༻
    ‣ ଞͷ"84αʔϏεͱͷ౷߹
    ‣ %BUB1MBOFʢ&$ʣ͸ࣗલͰ༻ҙ͢Δඞཁ͕͋Δ
    ‣ ͬ͘͟Γඅ༻ײ
    
    ݄
    ݱࡏɾ౦ژϦʔδϣϯ
    
    &$අ༻͸ผ్ 87
88. &,4αʔϏεϩʔϧΛ࡞੒ 88 $ cat << 'EOT' >./eks_iam_role-trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "eks.amazonaws.com" }, "Action": "sts:AssumeRole" } ] } EOT $ aws iam create-role --role-name eksServiceRole --assume-role-policy-document file:// eks_iam_role-trust-policy.json $ aws iam attach-role-policy --role-name eksServiceRole --policy-arn "arn:aws:iam::aws:policy/ AmazonEKSClusterPolicy" $ aws iam attach-role-policy --role-name eksServiceRole --policy-arn "arn:aws:iam::aws:policy/ AmazonEKSServicePolicy" $ export EKS_SERVICE_ROLE=`aws iam get-role --role-name eksServiceRole2 --query "Role.Arn" -- output text`
89. υΩϡϝϯτʹԊͬͯߏஙͯ͠ΈΑ͏ ‣ https://docs.aws.amazon.com/ja_jp/eks/latest/userguide/getting- started.html
    ‣ ʜͱ͸ݴ͍ͭͭࠓճ͸खॱΛ֬ೝ͢Δ͚ͩ
     ຊདྷͲΜͳखॱ͕ඞཁͳͷ͔͸஌͓ͬͯ͘ͱ͍͍
     ΋ͬͱ͍͍΍Γํ͕͋ΔͷͰ࣮ࡍʹ΍Δͷ͸͕࣌ؒ΋͍ͬͨͳ͍
    ‣ ڵຯ͕͋Ε͹΍ͬͯΈͯ 89
90. $GOͰ71$·ΘΓΛ࡞੒ 90 $ aws cloudformation create-stack --stack-name eks-vpc --template-url https://amazon-eks.s3-us- west-2.amazonaws.com/cloudformation/2018-11-07/amazon-eks-vpc-sample.yaml $ export SECURITY_GROUPS=`aws cloudformation describe-stacks --stack-name eks-vpc --query "Stacks[0].Outputs[?OutputKey=='SecurityGroups'].OutputValue | [0]" --output text` $ export VPC_ID=`aws cloudformation describe-stacks --stack-name eks-vpc --query "Stacks[0].Outputs[?OutputKey=='VpcId'].OutputValue | [0]" --output text` $ export SUBNET_IDS=`aws cloudformation describe-stacks --stack-name eks-vpc --query "Stacks[0].Outputs[?OutputKey=='SubnetIds'].OutputValue | [0]" --output text`
91. $GOͰ71$·ΘΓΛ࡞੒ 91 AWS Cloud VPC 192.168.0.0/16 ap-northeast-1a ap-northeast-1c ap-northeast-1d Internet Gateway Subnet01 Subnet02 Subnet03 192.168.64.0/18 192.168.128.0/18 192.168.192.0/18 ap-northeast-1
92. &,4
    $MVTUFSΛ࡞੒ 92 $ aws eks create-cluster --name eks-handson --role-arn $EKS_SERVICE_ROLE --resources-vpc-config subnetIds=$SUBNET_IDS,securityGroupIds=$SECURITY_GROUPS # CREATING -> ACTIVEʹͳͬͨΒ׬ྃ(8෼͙Β͍) $ aws eks describe-cluster --name eks-handson --query cluster.status
93. &,4
    $MVTUFSΛ࡞੒ 93 AWS Cloud VPC 192.168.0.0/16 ap-northeast-1a ap-northeast-1c ap-northeast-1d Internet Gateway Subnet01 Subnet02 Subnet03 192.168.64.0/18 192.168.128.0/18 192.168.192.0/18 ap-northeast-1 EKS
94. LVCFDUMͷ&,4༻DPOpHϑΝΠϧΛ࡞੒ 94 $ aws eks update-kubeconfig --name eks-handson # ಈ࡞֬ೝ $ kubectl get all NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 1h
95. $GOͰ/PEFΛ࡞੒ 95 $ aws cloudformation create-stack --stack-name eks-handson-workernodes --template-url https:// amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2018-11-07/amazon-eks-nodegroup.yaml -- parameters ParameterKey=KeyName,ParameterValue=k_mohri ParameterKey=NodeImageId,ParameterValue=ami-07fdc9272ce5b0ce5 ParameterKey=NodeInstanceType,ParameterValue=t2.small ParameterKey=NodeAutoScalingGroupMinSize,ParameterValue=2 ParameterKey=NodeAutoScalingGroupMaxSize,ParameterValue=3 ParameterKey=NodeVolumeSize,ParameterValue=10 ParameterKey=ClusterName,ParameterValue=eks- handson ParameterKey=NodeGroupName,ParameterValue=eks-handson-node-group ParameterKey=ClusterControlPlaneSecurityGroup,ParameterValue=$SECURITY_GROUPS ParameterKey=VpcId,ParameterValue=$VPC_ID ParameterKey=Subnets,ParameterValue='$SUBNET_IDS' -- capabilities CAPABILITY_IAM /PEF*NBHF*E͸͜͜Λࢀর
    
    
    
    IUUQTEPDTBXTBNB[PODPNFLTMBUFTUVTFSHVJEFFLTPQUJNJ[FEBNJIUNM
    
    
    
    ೔ຊޠ൛͸ߋ৽͞Ε͍ͯͳ͍ͷͰ஫ҙ
96. $GOͰ/PEFΛ࡞੒ 96 AWS Cloud VPC 192.168.0.0/16 ap-northeast-1a ap-northeast-1c ap-northeast-1d Internet Gateway Subnet01 Subnet02 Subnet03 192.168.64.0/18 192.168.128.0/18 192.168.192.0/18 ap-northeast-1 EKS Auto Scaling Group
97. /PEFͱ&,4Λ݁߹͢Δ 97 # ΠϯελϯεϩʔϧΛऔಘ $ aws cloudformation describe-stacks --stack-name eks-handson-workernodes --query "Stacks[0].Outputs[?OutputKey=='NodeInstanceRole'].OutputValue | [0]" --output text # EKSͱNodeΛ݁߹͢ΔͨΊͷConfigMapΛ࡞੒ $ curl -O https://amazon-eks.s3-us-west-2.amazonaws.com/cloudformation/2018-11-07/aws-auth- cm.yaml $ vi aws-auth-cm.yaml apiVersion: v1 kind: ConfigMap metadata: name: aws-auth namespace: kube-system data: mapRoles: | - rolearn: <͜͜ʹΠϯελϯεϩʔϧΛೖΕΔ> username: system:node:{{EC2PrivateDNSName}} groups: - system:bootstrappers - system:nodes
98. /PEFͱ&,4Λ݁߹͢Δ 98 # ConfigMapΛద༻ $ kubectl apply -f aws-auth-cm.yaml # ͠͹Β͘͢ΔͱNode͕ೝࣝ͞ΕΔ $ kubectl get nodes NAME STATUS ROLES AGE VERSION ip-192-168-103-86.ap-northeast-1.compute.internal Ready <none> 24s v1.11.5 ip-192-168-183-21.ap-northeast-1.compute.internal Ready <none> 27s v1.11.5 ip-192-168-247-164.ap-northeast-1.compute.internal Ready <none> 22s v1.11.5
99. /PEFͱ&,4Λ݁߹͢Δ 99 AWS Cloud VPC 192.168.0.0/16 ap-northeast-1a ap-northeast-1c ap-northeast-1d Internet Gateway Subnet01 Subnet02 Subnet03 192.168.64.0/18 192.168.128.0/18 192.168.192.0/18 ap-northeast-1 EKS Auto Scaling Group
100. $MVTUFS͕Ͱ͖ͨʂ 100 Kubernetes Cluster Data Plane
101. αϯϓϧΞϓϦΛσϓϩΠ͢Δ 101 # Ұ౓มߋΛ΋ͱʹ໭͢ $ cd example $ cd git reset --hard $ cd ../ # frontend-serviceͷλΠϓΛLoadBalancerʹมߋ # Ϋϥ΢υͷϩʔυόϥϯαʔ(͜͜Ͱ͸ELB)ͱ࿈ܞ͢ΔλΠϓ $ vi examples/guestbook/frontend-service.yaml 9-13ߦ໨ # comment or delete the following line if you want to use a LoadBalancer type: NodePort <- ͜͜ΛίϝϯτΞ΢τ # if your cluster supports it, uncomment the following to automatically create # an external load-balanced IP for the frontend service. # type: LoadBalancer <- ͜͜ΛΞϯίϝϯτ # apply! $ kubectl apply -f examples/guestbook/
102. αϯϓϧΞϓϦΛσϓϩΠ͢Δ 102 # ͠͹Β͘͢Δͱߏங׬ྃ $ kubectl get all NAME READY STATUS RESTARTS AGE pod/frontend-56f7975f44-2vtbr 1/1 Running 0 8s pod/frontend-56f7975f44-j25zn 1/1 Running 0 8s pod/frontend-56f7975f44-mss7q 1/1 Running 0 8s pod/redis-master-6b464554c8-wrjrp 1/1 Running 0 8s pod/redis-slave-b58dc4644-ft2fd 1/1 Running 0 7s pod/redis-slave-b58dc4644-p59fk 1/1 Running 0 7s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/frontend LoadBalancer 10.100.61.11 xxxxxx.ap-northeast-1.elb.amazonaws.com 80:31673/TCP 8s service/kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 23m service/redis-master ClusterIP 10.100.137.217 <none> 6379/TCP 7s service/redis-slave ClusterIP 10.100.217.57 <none> 6379/TCP 7s NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.apps/frontend 3 3 3 3 8s deployment.apps/redis-master 1 1 1 1 8s deployment.apps/redis-slave 2 2 2 2 7s NAME DESIRED CURRENT READY AGE replicaset.apps/frontend-56f7975f44 3 3 3 8s replicaset.apps/redis-master-6b464554c8 1 1 1 8s replicaset.apps/redis-slave-b58dc4644 2 2 2 7s http://xxxxxx.ap-northeast-1.elb.amazonaws.com
     ʹΞΫηεʂ
103. ΋Ζ΋Ζ࡟আ 103 # Cluster্ʹߏஙͨ͠΋Ζ΋ΖΛ࡟আ $ kubectl delete -f examples/guestbook/ # ͠͹Β͘͢Δͱݩ௨Γ $ kubectl get all NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 1h # stack, EKS ClusterΛ࡟আ $ aws cloudformation delete-stack --stack-name eks-handson-workernodes $ aws eks delete-cluster --name eks-handson $ aws cloudformation delete-stack --stack-name eks-vpc
104. LVCFDUMͷίϯςΩετͷ੾Γସ͑ɾ࡟আ 104 # ίϯςΩετҰཡ $ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE * arn:aws:...:cluster/eks-handson arn:aws:...:cluster/eks-handson arn:aws:...:cluster/eks-handson docker-for-desktop docker-for-desktop-cluster docker-for-desktop # ίϯςΩετΛdocker-for-desktopʹ໭͢ $ kubectl config use-context docker-for-desktop # EKSͷίϯςΩετΛ࡟আ $ kubectl config delete-context arn:aws:...:cluster/eks-handson
105. ΊΜͲ͗͘͢͞ͳ͍ʁ 105
106. ΋͏ͪΐͬͱ͏·͍΍Γํ ‣ FLTDUM ඇެࣜɾσϑΝΫτελϯμʔυ
      https://eksctl.io/
      ίϚϯυҰͭͰΫϥελߏங
     ‣ ΫΠοΫελʔτ ެࣜɾ࠷ۙग़ͨ
      https://aws.amazon.com/jp/quickstart/architecture/amazon-eks/
      $GOΛ࢖ͬͯΫϥελߏங
      ϕετϓϥΫςΟεʹै͍ͬͯΔͷͰ
 ݁ߏ߽՚ͳߏ੒ 106
107. 107 FLTDUMΛ࢖ͬͯΈΑ͏
108. FLTDUMͱ͸ʁ ‣ ίϚϯυҰͭͰ&,4ͷ$MVTUFS͕Ͱ͖ͪΌ͏πʔϧ
     ‣ $GOͷςϯϓϨʔτΛࣗಈੜ੒ͯ͠ߏஙͯ͠Δ
     ‣ OPEFͷΦʔτεέʔϦϯάͳͲɺศརͳػೳ΋
     ‣ ͱ͸͍͑·ͩൃల్্
     ‣ ͝ཡͷͱ͓ΓHPͰͰ͖ͯ·͢
     108
109. ΍ͬͯΈΑ͏ʂ ‣ ͜Ε͚ͩʂ
     ‣ ͦͷଞΦϓγϣϯ͸ϔϧϓΛࢀর 109 $ eksctl create cluster \ --name eksctl-handson \ --region ap-northeast-1 \ --nodes 3 \ --nodes-min 3 \ --nodes-max 3 \ --node-type t2.medium \ --ssh-public-key <ΩʔϖΞ໊> $ eksctl create cluster -h ˞ͨͩ͠ߏங׬ྃ·Ͱ෼͙Β͍͔͔Γ·͢ʜ
     ˞BQOPSUIFBTUC͕બ୒Ͱ͖Δݹ͍"84ΞΧ΢ϯτ͸ɺ";ࢦఆ΋ඞཁ
110. ͖ͬ͞ͷͱ΄΅ಉ͡ͷ͕Ͱ͖Δ 110 AWS Cloud VPC ap-northeast-1a ap-northeast-1c ap-northeast-1d Internet Gateway PublicSubnet01 PublicSubnet02 PublicSubnet03 ap-northeast-1 EKS Auto Scaling Group NAT Gateway ΦϓγϣϯʹΑͬͯ PrivateSubnetʹNodeΛ ഑ஔՄೳͳͷͰɺ ͦͷͱ͖༻ʁ PrivateSubnet01 PrivateSubnet02 PrivateSubnet03
111. ͱΓ͋͑ͣσϓϩΠ͠Α͏ 111
112. αϯϓϧΞϓϦΛσϓϩΠ͢Δ 112 # Ұ౓มߋΛ΋ͱʹ໭͢ $ cd examples $ git reset --hard $ cd ../ # frontend-serviceͷλΠϓΛLoadBalancerʹมߋ # Ϋϥ΢υͷϩʔυόϥϯαʔ(͜͜Ͱ͸ELB)ͱ࿈ܞ͢ΔλΠϓ $ vi examples/guestbook/frontend-service.yaml 9-13ߦ໨ # comment or delete the following line if you want to use a LoadBalancer type: NodePort <- ͜͜ΛίϝϯτΞ΢τ # if your cluster supports it, uncomment the following to automatically create # an external load-balanced IP for the frontend service. # type: LoadBalancer <- ͜͜ΛΞϯίϝϯτ # apply! $ kubectl apply -f examples/guestbook/
113. αϯϓϧΞϓϦΛσϓϩΠ͢Δ 113 # ͠͹Β͘͢Δͱߏங׬ྃ $ kubectl get all NAME READY STATUS RESTARTS AGE pod/frontend-56f7975f44-2vtbr 1/1 Running 0 8s pod/frontend-56f7975f44-j25zn 1/1 Running 0 8s pod/frontend-56f7975f44-mss7q 1/1 Running 0 8s pod/redis-master-6b464554c8-wrjrp 1/1 Running 0 8s pod/redis-slave-b58dc4644-ft2fd 1/1 Running 0 7s pod/redis-slave-b58dc4644-p59fk 1/1 Running 0 7s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/frontend LoadBalancer 10.100.61.11 xxxxxx.ap-northeast-1.elb.amazonaws.com 80:31673/TCP 8s service/kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 23m service/redis-master ClusterIP 10.100.137.217 <none> 6379/TCP 7s service/redis-slave ClusterIP 10.100.217.57 <none> 6379/TCP 7s NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.apps/frontend 3 3 3 3 8s deployment.apps/redis-master 1 1 1 1 8s deployment.apps/redis-slave 2 2 2 2 7s NAME DESIRED CURRENT READY AGE replicaset.apps/frontend-56f7975f44 3 3 3 8s replicaset.apps/redis-master-6b464554c8 1 1 1 8s replicaset.apps/redis-slave-b58dc4644 2 2 2 7s http://xxxxxx.ap-northeast-1.elb.amazonaws.com
     ʹΞΫηεʂ
114. μογϡϘʔυΛ
     Πϯετʔϧͯ͠ΈΑ͏ 114
115. μογϡϘʔυͷΠϯετʔϧ 115 # μογϡϘʔυ༻ͷPodΛapply $ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/ recommended/kubernetes-dashboard.yaml $ μογϡϘʔυʹϩάΠϯ͢ΔͨΊͷτʔΫϯΛऔಘ $ aws-iam-authenticator token -i eksctl-handson | jq -r '.status.token' # ϓϩΩγܦ༝ͰμογϡϘʔυʹΞΫηε $ kubectl proxy --port=8000 --address='0.0.0.0' --disable-filter=true http://localhost:8000/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/ ͜͜ʹτʔΫϯΛೖྗͯ͠
 αΠϯΠϯ
116. ͜Μͳײ͡ 116
117. ϩάΛऩूͯ͠ΈΑ͏ 117
118. ,VCFSOFUFTͷϩάΛ$MPVE8BUDI-PHT΁ ‣ %PDLFSίϯςφͰ͸ɺඪ४ग़ྗ͕ϩάͱͯ͠ѻΘΕΔ
     ‣ ϩά͸σϑΥϧτͰKTPOܕࣜͷϑΝΠϧʹͳΔ
     ‣ ֤/PEFʹqVFOUEͷ1PEΛஔ͖ɺ/PEF಺ͷ1PE͕ग़ྗͨ͠ϩάϑΝΠϧΛऩूͯ͠
 $MPVE8BUDIʹૹΔ 118 CloudWatchLogs
119. ,VCFSOFUFTͷϩάΛ$MPVE8BUDI-PHT΁ ‣ %PDLFSίϯςφͰ͸ɺඪ४ग़ྗ͕ϩάͱͯ͠ѻΘΕΔ
     ‣ ϩά͸σϑΥϧτͰKTPOܕࣜͷϑΝΠϧʹͳΔ
     ‣ ֤/PEFʹqVFOUEͷ1PEΛஔ͖ɺ/PEF಺ͷ1PE͕ग़ྗͨ͠ϩάϑΝΠϧΛऩूͯ͠
 $MPVE8BUDIʹૹΔ 119 CloudWatchLogs Ͳ͏΍ͬͯʜʁ
120. 
     
     
     
     %BFNPO4FU ‣ Ϋϥελ಺ͷ֤/PEFʹରͯ͠ඞͣͭ഑ஔ͞ΕΔ1PEΛ؅ཧ
      ֤/PEFͷΤʔδΣϯτͳͲͷ༻్ʹ 120 CloudWatchLogs
121. *".
     3PMFͷ͓ੈ࿩ 121 # NodeͷEC2ʹඥ෇͍͍ͯΔIAM Roleͷ໊લΛऔಘ $ INSTANCE_PROFILE_NAME=$(aws iam list-instance-profiles | jq -r '.InstanceProfiles[].InstanceProfileName' | grep nodegroup) $ ROLE_NAME=$(aws iam get-instance-profile --instance-profile-name $INSTANCE_PROFILE_NAME | jq -r '.InstanceProfile.Roles[] | .RoleName') # IAM Roleʹϩάऩू༻ͷΠϯϥΠϯϙϦγʔΛ௥Ճ $ cat << "EoF" > ./k8s-logs-policy.json { "Version": "2012-10-17", "Statement": [ { "Action": [ "logs:DescribeLogGroups", "logs:DescribeLogStreams", "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents" ], "Resource": "*", "Effect": "Allow" } ] } EoF $ aws iam put-role-policy --role-name $ROLE_NAME --policy-name Logs-Policy-For-Worker --policy-document file://k8s- logs-policy.json
122. qVFOUEΛσϓϩΠ 122 # ϚχϑΣετϑΝΠϧΛऔಘ $ wget https://eksworkshop.com/logging/deploy.files/fluentd.yml # Ϋϥελ໊Λมߋ $ vi fluentd.yml 197ߦ໨ value: us-east-1 <- ap-northeast-1 ʹมߋ 199ߦ໨ value: eksworkshop-eksctl <- eksctl-handson ʹมߋ # σϓϩΠ $ kubectl apply -f fluentd.yml
123. $MPVE8BUDIͷϩάʹ্͕ͬͯ͘Δ 123 ͋ͱ͸&MBTUJDTFBSDI4FSWJDFʹྲྀ͢ͳΓԿͳΓͱ
124. qVFOUEZNM 124 (ུ) --- apiVersion: extensions/v1beta1 kind: DaemonSet # DaemonSetͷϚχϑΣετ metadata: name: fluentd-cloudwatch namespace: kube-system labels: k8s-app: fluentd-cloudwatch spec: template: # ---͔͜͜ΒPodͷఆٛ-------------------------------------------- metadata: labels: k8s-app: fluentd-cloudwatch spec: serviceAccountName: fluentd terminationGracePeriodSeconds: 30 # Because the image's entrypoint requires to write on /fluentd/etc but we mount configmap there which is read-only, # this initContainers workaround or other is needed. # See https://github.com/fluent/fluentd-kubernetes-daemonset/issues/90 initContainers: - name: copy-fluentd-config image: busybox command: ['sh', '-c', 'cp /config-volume/..data/* /fluentd/etc'] volumeMounts: - name: config-volume mountPath: /config-volume - name: fluentdconf mountPath: /fluentd/etc ʢུʣ
125. )FMNΛ࢖ͬͯΈΑ͏⛑ 125
126. 
     
     
     
     )FMN ‣ ,VCFSOFUFT༻ͷύοέʔδ؅ཧπʔϧ
      ύοέʔδ͸$IBSUͱݺ͹ΕɺϚχϑΣετϑΝΠϧͷςϯϓϨʔτؚ͕·ΕΔ
      5JMMFSͱݺ͹ΕΔαʔόʔΞϓϦέʔγϣϯ ͜Ε΋1PE Λհͯ͠Ϋϥελ಺ʹύοέʔδΛΠϯετʔϧ
     ‣ ͪͳΈʹIFMN͸דͰ͸ͳ͘ધͷ଩ɺDIBSU͸ւਤɺUJMMFS͸଩ฑͷҙຯ 126 helm (CLI) Tiller Repository Chart A Chart B Chart C Manifest Files Custom Values
127. 3PMF#BTFE
     "DDFTT
     $POUSPM
     3#"$ ‣ ,VCFSOFUFTͷݖݶ੍ޚͷ࢓૊Έ
      ,VCFSOFUFTͷϦιʔε΁ͷΞΫηεΛϩʔϧʹΑ੍ͬͯޚ
      ϢʔβʔͱϩʔϧΛ#JOEJOHʹΑͬͯඥ෇͚Δ͜ͱʹΑͬͯػೳ͢Δ
     ‣ Ϣʔβʔछผ
      ೝূϢʔβʔɾάϧʔϓ
     
     Ϋϥελ֎͔Β,VCFSOFUFT
     "1*Λૢ࡞͢ΔͨΊͷϢʔβʔ
      4FSWJDF"DDPVOU
     
     1PE͕,VCFSOFUFT
     "1*Λૢ࡞͢ΔͨΊͷϢʔβʔ
     ‣ ϩʔϧछผ
      3PMF
     
     ࢦఆͷOBNFTQBDF಺ͰͷΈ༗ޮ
      $MVTUFS3PMF
     
     ΫϥελશମͰ༗ޮ
     ‣ )FMNʹ΋3#"$Λ༗ޮʹͰ͖Δ$IBSU͕
 ଟ͘؅ཧ͞Ε͍ͯΔ 127 ServiceAccount 認証ユーザーグループ 認証ユーザー RoleBinding ClusterRoleBinding Role ClusterRole
128. )FMNͷΠϯετʔϧ ‣ https://helm.sh/docs/using_helm/#installing-helm
     ‣ .BDͷ৔߹
      CSFX
     JOTUBMM
     LVCFSOFUFTIFMN 128
129. )FMNͷηοτΞοϓ 129 # Tiller༻ͷαʔϏεΞΧ΢ϯτͷϚχϑΣετϑΝΠϧΛ࡞੒ # "cluster-admin"͸σϑΥϧτͰଘࡏ͢ΔClusterRole $ cat <<EoF > tiller_rbac.yaml --- apiVersion: v1 kind: ServiceAccount metadata: name: tiller namespace: kube-system --- apiVersion: rbac.authorization.k8s.io/v1beta1 kind: ClusterRoleBinding metadata: name: tiller roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: tiller namespace: kube-system EoF # Tiller༻ͷαʔϏεΞΧ΢ϯτΛ࡞੒ $ kubectl apply -f tiller_rbac.yaml # TillerͷαʔϏεΞΧ΢ϯτΛࢦఆͯ͠ΫϥελʹHelmΛಋೖ # ͜ΕͰTillerͷPod͕kube-systemωʔϜεϖʔεʹσϓϩΠ͞ΕΔ $ helm init --service-account tiller
130. )FMNͰ+FOLJOTΛΠϯετʔϧ 130 # CustomValueϑΝΠϧΛ࡞੒ # ύϥϝʔλʔৄࡉ͸ "helm inspect values stable/jenkins" Ͱ֬ೝ $ cat <<EoF > jenkins.yaml rbac: create: true master: service_port: 8080 persistence: size: 1Gi EoF # JenkinsΛΠϯετʔϧ $ helm install -f jenkins.yaml --name jenkins stable/jenkins # ͠͹Β͘͢ΔͱσϓϩΠ׬ྃ(2-3෼͙Β͍) $ kubectl get all NAME READY STATUS RESTARTS AGE pod/jenkins-f65b9477-89s69 1/1 Running 0 33m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/jenkins LoadBalancer 10.100.22.208 xxxxxx.ap-northeast-1.elb.amazonaws.com 8081:30196/TCP 33m service/jenkins-agent ClusterIP 10.100.58.26 <none> 50000/TCP 33m service/kubernetes ClusterIP 10.100.0.1 <none> 443/TCP 2h NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.apps/jenkins 1 1 1 1 33m NAME DESIRED CURRENT READY AGE replicaset.apps/jenkins-f65b9477 1 1 1 33m
131. Πϯετʔϧͨ͠+FOLJOTʹϩάΠϯ 131 # ύεϫʔυऔಘ $ printf $(kubectl get secret --namespace default jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo XXXXXXXX # ϩάΠϯURLऔಘ $ export SERVICE_IP=$(kubectl get svc --namespace default jenkins --template "{{ range (index .status.loadBalancer.ingress 0) }} {{ . }}{{ end }}") $ echo http://$SERVICE_IP:8080/login http://xxxxxx.ap-northeast-1.elb.amazonaws.com:8080/login VTFSOBNF
     
     BENJO
     QBTTXPSE
     
     ίϚϯυͰऔಘͨ͠΍ͭ
132. +FOLJOTΛΞϯΠϯετʔϧ 132 # JenkinsΛΞϯΠϯετʔϧ # --purge͸Φϓγϣϯɻ෇͚ͳ͍৔߹ɺϦϏδϣϯͷه࿥͕࢒ΓɺϩʔϧόοΫ͕Ͱ͖Δ $ helm delete --purge jenkins
133. ؂ࢹͰ͖ΔΑ͏ʹͯ͠ΈΑ͏2 133
134. 
     
     
     
     1SPNFUIFVT
     
     
     
     
     
     (SBGBOB ‣ 1SPNFUIFVT
      044ͷϦιʔε؂ࢹπʔϧ
      ಋೖ͕Χϯλϯɺ͍͍ײ͡ʹ௨஌͘ΕΔɺߴੑೳͳͲͰਓؾ͕ߴ͍
      ͨͩɺσʔλͷՄࢹԽ͕ຊۀͰ͸ͳ͍ͷͰྗෆ଍
     ‣ (SBGBOB
      044ͷϩάɾσʔλՄࢹԽπʔϧ
      1SPNFUIFVT͕ऩूͨ͠σʔλΛ͔ͬ͜Α͘දࣔͰ͖Δ 134
135. )FMNͰ1SPNFUIFVTΛΠϯετʔϧ 135 # CustomValueϑΝΠϧΛ࡞੒ # ύϥϝʔλʔৄࡉ͸ "helm inspect values stable/prometheus" Ͱ֬ೝ $ cat <<EoF > prometheus.yaml alertmanager: persistentVolume: size: 1Gi storageClass: "gp2" server: persistentVolume: size: 1Gi storageClass: "gp2" retention: "12h" pushgateway: enabled: false EoF # PrometheusΛΠϯετʔϧ $ kubectl create namespace prometheus $ helm install -f prometheus.yaml --name prometheus --namespace prometheus stable/prometheus # σϓϩΠ͕׬ྃͨ͠ΒΞΫηεͯ͠ΈΔʢ্ཱ͕ͪΔ·Ͱ਺෼͔͔Δʣ # URLͷऔಘίϚϯυ͕Α͘มΘΔͷͰɺhelm installͰग़͖ͯͨϩάͷํΛࢀߟʹͨ͠΄͏͕͍͍͔΋ $ export POD_NAME=$(kubectl get pods --namespace prometheus -l "app=prometheus,component=server" -o jsonpath="{.items[0].metadata.name}") $ kubectl --namespace prometheus port-forward $POD_NAME 9090
136. 1SPNFUIFVTʹΞΫηε 136 http://localhost:9090/targets
137. )FMNͰ(SBGBOBΛΠϯετʔϧ 137 # CustomValueϑΝΠϧΛ࡞੒ # ύϥϝʔλʔৄࡉ͸ "helm inspect values stable/grafana" Ͱ֬ೝ $ cat <<EoF > grafana.yaml persistence: storageClassName: gp2 adminPassword: password datasources: datasources.yaml: apiVersion: 1 datasources: - name: Prometheus type: prometheus url: "http://prometheus-server.prometheus.svc.cluster.local" access: proxy isDefault: true service: type: LoadBalancer EoF # GrafanaΛΠϯετʔϧ $ kubectl create namespace grafana $ helm install -f grafana.yaml --name grafana --namespace grafana stable/grafana # σϓϩΠ͕׬ྃͨ͠ΒΞΫηεͯ͠ΈΔʢ্ཱ͕ͪΔ·Ͱ਺෼͔͔Δʣ # URLͷऔಘίϚϯυ͕Α͘มΘΔͷͰɺhelm installͰग़͖ͯͨϩάͷํΛࢀߟʹͨ͠΄͏͕͍͍͔΋ $ export ELB=$(kubectl get svc -n grafana grafana -o jsonpath='{.status.loadBalancer.ingress[0].hostname}') $ echo "http://$ELB" http://xxxxxx.ap-northeast-1.elb.amazonaws.com
138. (SBGBOBʹΞΫηε 138 VTFSOBNF
     
     BENJO
     QBTTXPSE
     
     QBTTXPSE
     HSBGBOBZBNMʹॻ͍ͨ
139. μογϡϘʔυͷ*%Λೖྗ 139 Λೖྗ ެ։͞Ε͍ͯΔςϯϓϨʔτͷ൪߸ ΫϦοΫ
140. Φϓγϣϯઃఆ 140 ΫϦοΫ 1SPNFUIFVTΛબ୒
141. ͦΕΒ͍͠ͷ͕ग़͖ͯͨʂ 141
142. ͳΒ͜Μͳײ͡ 142
143. ͓ย෇͚ 143
144. ͓ย෇͚ 144 # PrometheusͱGrafanaΛ࡟আ # EBSΛ࡞͍ͬͯΔͷͰɺ͍͖ͳΓeksctlͰΫϥελΛ࡟আ͢ΔͱEBS͕࢒ͬͯ͠·͏ $ helm delete --purge prometheus $ helm delete --purge grafana # fluentdΛ࡟আ $ kubectl delete -f fluentd.yml # μογϡϘʔυΛ࡟আ $ kubectl delete -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/ kubernetes-dashboard.yaml # guestbookΞϓϦΛ࡟আ $ kubectl delete -f examples/guestbook/ # ϩάऩू༻ʹIAM Roleʹ͚ͬͭͨ͘PolicyΛ֎͢ # $ROLE_NAME͸121ϖʔδͰઃఆ͞Ε͍ͯΔ # ֎͓͔ͯ͠ͳ͍ͱeksctlͰΫϥελΛ࡟আ͢Δͱ͖ʹcloudformationʹౖΒΕΔ $ aws iam delete-role-policy --role-name $ROLE_NAME --policy-name Logs-Policy-For-Worker # eksctlͰΫϥελΛ࡟আ # ్த·Ͱ͔͠ݟͯ͘Εͳ͍ͷͰɺAWSίϯιʔϧͰຊ౰ʹ࡟আ͞Ε͔ͨ֬ೝͨ͠΄͏͕͍͍ $ eksctl delete cluster --name eksctl-handson
145. 145 ͓·͚
     (,&ͰΫϥελΛ࡞ͬͯΈΑ͏
146. (,&ͰΫϥελΛ࡞Δ ‣ ΍ͬͺΓຊՈ΋ݟ͓͔ͯͳ͍ͱɺͱ͍͏͜ͱͰͪΐͬͱ͚ͩ
     ‣ $-*΍ϓϩδΣΫτͷઃఆ͸ऴΘ͍ͬͯΔલఏ 146
147. (,&ͰΫϥελΛ࡞Δ 147 # ίϚϯυ1ͭͰNode΋ؚΊͯΫϥελΛ࡞ͬͯ͘ΕΔ # 3෼൒͙Β͍ͰͰ͖Δʂʂ $ gcloud container clusters create gke-handson --cluster-version=1.12.7-gke.10 --machine-type=n1-standard-1 --num-nodes=3 # kubectlͷconfig΋มΘͬͯΔ
 $ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE docker-for-desktop docker-for-desktop-cluster docker-for-desktop * xxxxxx_asia-northeast1-a_gke-handson xxxxxx_asia-northeast1-a_gke-handson xxxxxx_asia-northeast1-a_gke-handson # σϑΥϧτͰfluentd΍prometheus͕ೖͬͯΔ!? $ kubectl get pods --namespace=kube-system NAME READY STATUS RESTARTS AGE event-exporter-v0.2.3-f9c896d75-52nkr 2/2 Running 0 2m5s fluentd-gcp-scaler-69d79984cb-zm56b 1/1 Running 0 113s fluentd-gcp-v3.2.0-5mncb 2/2 Running 0 63s fluentd-gcp-v3.2.0-9sdg7 2/2 Running 0 74s fluentd-gcp-v3.2.0-t59sr 2/2 Running 0 54s heapster-v1.6.0-beta.1-6fc8df6cb8-54qrk 3/3 Running 0 85s kube-dns-autoscaler-76fcd5f658-22l8j 1/1 Running 0 104s kube-dns-b46cc9485-5kspm 4/4 Running 0 92s kube-dns-b46cc9485-j8fmn 4/4 Running 0 2m5s kube-proxy-gke-gke-handson-default-pool-d757b1ec-9ld7 1/1 Running 0 108s kube-proxy-gke-gke-handson-default-pool-d757b1ec-lcl8 1/1 Running 0 110s kube-proxy-gke-gke-handson-default-pool-d757b1ec-z2m6 1/1 Running 0 110s l7-default-backend-6f8697844f-s8lgv 1/1 Running 0 2m6s metrics-server-v0.3.1-5b4d6d8d98-tn8cw 2/2 Running 0 87s prometheus-to-sd-4v26j 1/1 Running 0 111s prometheus-to-sd-k4jj7 1/1 Running 0 110s prometheus-to-sd-mhs8h 1/1 Running 0 110s
148. /PEFͷΫϥελ͕Ͱ͖ͨ 148
149. ΞϓϦέʔγϣϯΛσϓϩΠ 149 # ΋ͪΖΜࠓ·Ͱ࢖͖ͬͯͨϚχϑΣετϑΝΠϧ͕࢖͑Δ $ kubectl apply -f examples/guestbook/
150. ͓ย෇͚ 150 # ࡟আ΋3෼൒͙Β͍ͰऴΘΔʂʂ $ gcloud container clusters delete gke-handson # kubectlͷconfig΋ফ͑ͯΔ
 $ kubectl config get-contexts CURRENT NAME CLUSTER AUTHINFO NAMESPACE docker-for-desktop docker-for-desktop-cluster docker-for-desktop # ೦ͷͨΊ࢒͍ͬͯΔϦιʔε͕ແ͍͔ίϯιʔϧͰ֬ೝ͓ͯ͘͠ͱ٢
151. &,4ͱ(,& ‣ ॊೈੑ
      &,4͸ϢʔβʔʹΑΔΧελϚΠζͷ෯͕େ͖͍
     w Ͱ΋΍ͬͺΓ໘౗ͳͷͰFLTDUMͱ͔͕஀ੜ͍ͯͨ͠Γ͢Δʜ
      (,&͸͍͍ײ͡ʹͯ͘͠ΕΔ
     ‣ ্ཱ͕ͪΓͷ଎͞
      &,4͸ FLTDUMར༻Ͱ ෼͙Β͍
      (,&͸෼൒͙Β͍
     ‣ අ༻
      &,4͸NBTUFSʹඅ༻͕͔͔Δ ౦ژϦʔδϣϯͰ݄͙Β͍
      (,&͸NBTUFSʹඅ༻͕͔͔Βͳ͍ʂ
     ‣ ͲͬͪΛબ΅͏ʁ
      ७ਮʹ,VCFSOFUFTΛ࢖͍͍͚ͨͩͳΒѹ౗తʹ(,&
      "84ͷ΋Ζ΋ΖͷαʔϏεͱ߹Θͤͯ࢖͍͍ͨͳΒ&,4 151
152. 152 ͦͷଞ΋Ζ΋Ζ
153. ొ৔͠ͳ͔ͬͨϦιʔεͨͪ 153
154. 
     
     
     
     +PC ‣ ୯ൃͷॲཧΛ؅ཧ
      ࢦఆͨ͠਺͚ͩ1PEΛ࡞੒ͯ͠ॲཧΛ࣮ߦ 154 apiVersion: batch/v1 kind: Job # JobͷϚχϑΣετ metadata: name: example_job labels: app: example spec: parallelism: 3 # ಉ࣌ʹ࣮ߦ͢ΔPodͷ਺ template: # ---͔͜͜ΒPodͷఆٛ-------------------------------------------- metadata: labels: app: example spec: (ུ)
155. 
     
     
     
     $SPO+PC ‣ ఆظ࣮ߦ͢ΔॲཧΛ؅ཧ
      εέδϡʔϧʹԊͬͯ1PEΛ࡞੒ͯ͠ॲཧΛ࣮ߦ 155 apiVersion: batch/v1beta1 kind: CronJob # CronJobͷϚχϑΣετ metadata: name: example_job labels: app: example spec: schedule: "*/1 * * * *" # ىಈεέδϡʔϧΛcronͱಉ͡ܕࣜͰఆٛ jobTemplate:
 spec: template: # ---͔͜͜ΒPodͷఆٛ-------------------------------------------- metadata: labels: app: example spec: (ུ)
156. 
     
     
     
     $POpH.BQ ‣ ΞϓϦέʔγϣϯͷઃఆ৘ใΛఆٛͯ͠1PEʹఏڙ
      ؀ڥม਺ͱͯ͠ఏڙ
      7PMVNFͱͯ͠ఏڙ 156
157. 
     
     
     
     $POpH.BQ 157 apiVersion: v1 kind: ConfigMap metadata: name: cm-example data: # key-valueܕࣜͰઃఆ৘ใΛॻ͍͍ͯ͘ EXAMPLE: this_is_example example.txt: | this is example # Podͷఆٛத (ུ) env: # ؀ڥม਺ͱͯ͠ఏڙ - name: EXAMPLE valueFrom: configMapKeyRef: name: cm-example key: EXAMPLE (ུ) containers: # Volumeͱͯ͠ఏڙɻ͜ΕͰ /config/example.txt ͕ѻ͑ΔΑ͏ʹͳΔ - image: alpine (ུ) volumeMounts: ## ίϯςφ಺ͰͷVolumeͷϚ΢ϯτઃఆ - name: cm-volume mountPath: /config volumes: ## Volumeͷఆٛ - name: cm-volume configMap: name: cm-example (ུ)
158. 
     
     
     
     4FDSFU ‣ ΞϓϦέʔγϣϯͷػີ৘ใΛఆٛͯ͠1PEʹఏڙ
      ؀ڥม਺ͱͯ͠ఏڙ
      7PMVNFͱͯ͠ఏڙ
     ‣ $POpH.BQͱͷҧ͍
      จࣈྻΛ#BTFΤϯίʔυͨ͠ঢ়ଶͰѻ͏ όΠφϦσʔλΛѻ͑ΔΑ͏ʹ
     w ΋ͪΖΜ҉߸Խ໨త͡Όͳ͍ͷͰɺͦͷ··(JUIVCʹ্͛ͨΓͨ͠Βμϝ
      ઃఆʹΑͬͯ FUDE্ʹ҉߸Խͨ͠ঢ়ଶͰอଘ͞ΕΔ
      /PEFͰ͸1PEͷUNQGT ཁ͢ΔʹϝϞϦ ্ʹอଘ͞ΕΔ
      /PEF͸ׂΓ౰ͯΒΕͨ1PE͕ࢀর͢Δ4FDSFUҎ֎͸ΞΫηεͰ͖ͳ͍
      ͍͔ͭ͘5ZQF͕͋Δ
     w 0QBRVF
     
     $POpH.BQͱಉ͡ߏ଄Խ͞Εͯͳ͍,FZ7BMVFܗࣜ
     w LVCFSOFUFTJPUMT
     
     5-4ͷൿີݤͱެ։ݤΛ֨ೲ
     w LVCFSOFUFTJPTFSWJDFBDDPVOUUPLFO
     
     ,VCFSOFUFTͷαʔϏεΞΧ΢ϯτͷΫϨσϯγϟϧ 158
159. 
     
     
     
     4FDSFU 159 apiVersion: v1 kind: Secret metadata: name: secret-example stringData: password: xxxxxxxxxxxx # Base64Τϯίʔυ͞Εͨจࣈྻ credential.txt: | xxxxxxxxxxxxx xxxxxxxxxxxxx xxxxxxxxxxxxx # Podͷఆٛத (ུ) env: # ؀ڥม਺ͱͯ͠ఏڙ - name: PASSWORD valueFrom: secretKeyRef: name: secret-example key: password (ུ) containers: # Volumeͱͯ͠ఏڙɻ͜ΕͰ /secrets/credential.txt ͕ѻ͑ΔΑ͏ʹͳΔ - image: alpine (ུ) volumeMounts: ## ίϯςφ಺ͰͷVolumeͷϚ΢ϯτઃఆ - name: secret-volume mountPath: /secrets volumes: ## Volumeͷఆٛ - name: secret-volume secret: secretName: secret-data (ུ)
160. 
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     
     ετϨʔδؔ࿈΋Ζ΋Ζ ‣ 1FSTJTUFOU7PMVNF
     
      ετϨʔδͷ࣮ମ
      "84ͳΒ/PEFͷ&$͕࣋ͭ&#4
     ‣ 1FSTJTUFOU7PMVNF$MBJN
     
      ετϨʔδΛ࿦ཧతʹந৅Խͨ͠Ϧιʔε
      1FSTJTUFOU7PMVNFʹରͯ͠ඞཁͳ༰ྔΛಈతʹ֬อ
     ‣ 4UPSBHF$MBTT
      1FSTJTUFOU7PMVNF͕֬อ͢ΔετϨʔδͷछྨΛఆٛ
      "84ͳΒ
     JP HQTDTU
     ‣ 4UBUFGVM4FU
      ܧଓతʹσʔλΛӬଓԽ͢ΔεςʔτϑϧͳΞϓϦέʔγϣϯͷ؅ཧʹ޲͍ͨϦιʔε
      ؅ཧԼͷ1PEʹ͸࿈൪ͷࣝผࢠ͕෇༩͞Εɺ࠶࡞੒͞Εͯ΋ಉࣝ͡ผࢠͰ͋Ε͹ಉ͡ετϨʔδΛࢀর͢Δ
     ‣ Ͱ΋ɺΫϥ΢υͰ,VCFSOFUFT࢖͏ͳΒجຊతʹϚωʔδυͷ%#αʔϏεΛ࢖͏ 160
161. ϚχϑΣετϑΝΠϧͷ
     ϑΥʔϚοτΛௐ΂Δ 161
162. ϚχϑΣετϑΝΠϧͷϑΥʔϚοτ 162 # "kubectl explain [Ϧιʔε] --recursive" ͰશମͷϑΥʔϚοτΛදࣔ $ kubectl explain service --recursive KIND: Service VERSION: v1 DESCRIPTION: Service is a named abstraction of software service (for example, mysql) consisting of local port (for example 3306) that the proxy listens on, and the selector that determines which pods will answer requests sent through the proxy. FIELDS: apiVersion <string> kind <string> metadata <Object> annotations <map[string]string> (ུ) # ֊૚Λࢦఆ͢Δͱͦͷ߲໨ͷઆ໌͕֬ೝͰ͖Δ $ kubectl explain service.spec.type KIND: Service VERSION: v1 FIELD: type <string> DESCRIPTION: type determines how the Service is exposed. Defaults to ClusterIP. Valid options are ExternalName, ClusterIP, NodePort, and LoadBalancer. (ུ)
163. ӡ༻࣌ͷجຊߏ੒Πϝʔδ 163
164. ӡ༻࣌ͷجຊߏ੒Πϝʔδ 164 GitHub CircleCI DockerImage Deploy ECR DockerHub CloudWatchLogs Prometheus Grafana Log Metrics Kubernetes Cluster CI/CD ؂ࢹ ϩά؅ཧ
165. 165 ·ͱΊ
166. ຊ೔ͷ໨ඪ ࠶ܝ ‣ ,VCFSOFUFTͱ͓༑ୡʹͳΔ
      ΠϝʔδΛ௫Ή
      ৮ͬͯΈΔ ϩʔΧϧɾ&,4ɾͪΐͬͱ(,&
      ߏஙɾӡ༻͕Ͱ͖ΔΑ͏ͳؾ෼ʹͳΔ
     ‣ ޽ʹ͋;ΕΔ,VCFSOFUFTͷهࣄɾεϥΠυ͕
 ཧղͰ͖ΔΑ͏ʹͳΔ 166
167. ୡ੒Ͱ͖ͨͰ͠ΐ͏͔ 167
168. ͱ͸͍͑ʜ ‣ ,VCFSOFUFT͸େن໛ͰෳࡶͳγεςϜͰਅՁΛൃش
     ‣ ୯७ͳ΋ͷͳΒ'BSHBUFͱ͔&$4ͱ͔ͷ΄͏͕ɻɻ
      ৔߹ʹΑͬͯ͸&$Λ࢖ͬͨϨΨγʔͳߏ੒Ͱ΋ࣄ଍ΓΔ 168
169. ͱ͸͍͑ʜ ‣ ,VCFSOFUFT͸େن໛ͰෳࡶͳγεςϜͰਅՁΛൃش
     ‣ ୯७ͳ΋ͷͳΒ'BSHBUFͱ͔&$4ͱ͔ͷ΄͏͕ɻɻ
      ৔߹ʹΑͬͯ͸&$Λ࢖ͬͨϨΨγʔͳߏ੒Ͱ΋ࣄ଍ΓΔ
     ‣ Ͱ΋ɺ਺೥ޙʹ͸౰ͨΓલͷٕज़ʹͳ͍ͬͯΔՄೳੑ͕ߴ͍
      ʜͱࢥ͍ͬͯΔ
      ΋ͬͱΧϯλϯʹ࢖͑ΔΑ͏ʹͳ͍ͬͯΔ͔΋ʜ
      &,4ͷ'BSHBUFରԠ΋͋Δͱ͔ͳ͍ͱ͔ʜ
     ‣ ݱ࣌఺Ͱ΋ࣗ෼ͷબ୒ࢶ͕૿͑Δͷ͸͍͍͜ͱͩΑͶʂ 169
170. ·ͱΊ ‣ ͱΓ͋͑ͣ໨ඪ͕ୡ੒Ͱ͖͍ͯͨͳΒ޾͍Ͱ͢
     ‣ ,VCFSOFUFTপͷೖΓޱʹཱͬͨ͹͔Γ
      ࠓճ঺հ͖͠Εͳ͔ͬͨ༻ޠɺΤίγεςϜͳͲ·ͩ๲େʹʜ
     ‣ Զͨͪͷઓ͍͸͜Ε͔Βͩʂ 170
171. 171 ࢀߟࢿྉ
172. ࢀߟࢿྉ ‣ ,VCFSOFUFTެࣜαΠτ
     - https://kubernetes.io/ ‣ ,VCFSOFUFTΞΠίϯ
     - https://github.com/kubernetes/community/tree/master/icons ‣ %PDLFS,VCFSOFUFT
     ࣮ફίϯςφ։ൃೖ໳
     - https://www.amazon.co.jp/dp/4297100339/ref=cm_sw_em_r_mt_dp_U_vNpSCbV87QB5G ‣ ೖ໳
     ,VCFSOFUFT
     - https://www.amazon.co.jp/dp/4873118409/ref=cm_sw_em_r_mt_dp_U_AMpSCb8MKY1MN ‣ "NB[PO
     &,4
     ͷ࢖༻։࢝
     - https://docs.aws.amazon.com/ja_jp/eks/latest/userguide/getting-started.html ‣ "NB[PO
     &,4
     8PSLTIPQ
     - https://eksworkshop.com/jenkinsworld/ ‣ ,VCFSOFUFT
     PO
     "84&,4ϕετϓϥΫςΟε
     - https://speakerdeck.com/mumoshu/eksbesutopurakuteisu2019-dot-2-number-jawsdays ‣ ,VCFSOFUFT͸ۜͷ஄ؙͰ͸ͳ͍
     "84ʹ͓͚ΔίϯςφΦʔέετϨʔγϣϯͷݱࡏ஍ͱՄೳੑ
     - https://logmi.jp/tech/articles/305690 172
173. ࢀߟࢿྉ ‣ ෼͘Β͍ͰΘ͔Δɺ,VCFSOFUFTͱ&,4ͷԿ͕ศརͳͷ͔
     - https://qiita.com/masachaco/items/3e50a1ac65cdd661a734 ‣ ,VCFSOFUFT
     ߏ੒ίϯϙʔωϯτҰཡ
     - https://qiita.com/tkusumi/items/c2a92cd52bfdb9edd613 ‣ 5PLVT
     #MPH
     ,VCFSOFUFTಓ৔
     - https://cstoku.dev/tags/kubernetes-dojo/ 173