Wantedly Visit on Kubernetes

089fe44e41bb1fa2d9421f919a99173c?s=47 Shimpei Otsubo
September 26, 2018
Programming
3
700

Wantedly Visit on Kubernetes

089fe44e41bb1fa2d9421f919a99173c?s=128

Shimpei Otsubo

September 26, 2018
Tweet

Want more?

089fe44e41bb1fa2d9421f919a99173c?s=48 potsbo
1
210
089fe44e41bb1fa2d9421f919a99173c?s=48 potsbo
1
380
089fe44e41bb1fa2d9421f919a99173c?s=48 potsbo
6
1.7k
3ab1249be442027903e1180025340b3f?s=48 reverentgeek
12
620
A9a491b0fcbe0fbce3d64063a37add99?s=48 rmw
3
320
766b9746b59e6f09e280cd33cf4ed419?s=48 skipperchong
0
210
61857dafbd287b3027c4dcea9008ad3c?s=48 carmenhchung
8
400
06f8b41980eb4c577fa40c41d5030c19?s=48 keathley
7
240
C0390e8b11079f8761c0cd3274ed61cb?s=48 productmarketing
3
320
C35e01544a0dd94a2cf1619ee8a42ebb?s=48 clairelew
4
620
6bb82b13cda86d3b821fa44100335ddf?s=48 thoeni
1
180
B3ace07412a5178ea778e7eec161fc0a?s=48 jcasabona
4
260

Transcript

  1. 1.

    ©2018 Wantedly, Inc. Wantedly Visit on Kubernetes Wantedly ͷ͢΂ͯͷαʔϏεΛ Kubernetes

    ΛҠߦͤͨ͞ཧ༝ͱۤ࿑ SRE Lounge #5 26.Sep.2018 - Shimpei Otsubo @potsbo
  2. 2.

    ©2018 Wantedly, Inc. ͸ͨΒ͘Λ໘ന͘ ༗໊اۀ΍ϕϯνϟʔɺελʔτΞοϓاۀɺؾʹ ͳΔΦϑΟεʹ࿩Λฉ͖ʹ͍͜͏ɻ
 ΩϥͬͱޫΔγΰτ΍ϓϩδΣΫτɺձࣾΛݟ͚ͭ ͯΤϯτϦʔ͠·͠ΐ͏ɻ஌Γ߹͍ͷͭͳ͕Γ͕෼ ͔ͬͯ҆৺Ͱ͢ɻ
 ϓϩϑΟʔϧΛॆ࣮ͤ͞Δ͜ͱͰɺ࠾༻୲౰ऀ͔Β

    ௚઀ΦϑΟε๚໰ͷট଴͕དྷΔ͔΋͠Ε·ͤΜɻ
  3. 3.

    ©2018 Wantedly, Inc. ड͚औ໊ͬͨࢗશͯΛɺճͷࡱӨͰ ಉ࣌ʹಡΈࠐΈɻ
 ಡΈࠐΜ໊ͩࢗ͸ਓ޻஌ೳ͕ଈ࣌σʔ λԽ͠ɺ࿈བྷாʹ௥Ճ͞Ε·͢ɻ ݡ໊ࢗ͘؅ཧ

  4. 4.

    ©2018 Wantedly, Inc. 4IJNQFJ0UTVCP!QPUTCP Infrastructure Team, Wantedly, Inc. 2018 New

    Grads Kubernetes Dev Tools CI / CD AuthN / Z Productivity Dvorak
  5. 5.

    ©2018 Wantedly, Inc.

  6. 6.

    ©2018 Wantedly, Inc. ڞ௨σʔλج൫ ΠϯϑϥνʔϜ͕࠷ۙ΍͍ͬͯΔ͜ͱ αʔϏεϝογϡ෼ࢄτϨʔγϯά ΤϯδχΞͷੜ࢈ੑ 43&͚ͩͰ͸ͳ͍

  7. 7.

    ©2018 Wantedly, Inc. ڞ௨σʔλج൫ ΠϯϑϥνʔϜ͕࠷ۙ΍͍ͬͯΔ͜ͱ αʔϏεϝογϡ෼ࢄτϨʔγϯά ΤϯδχΞͷੜ࢈ੑ 43&͚ͩͰ͸ͳ͍ ࣗ෼ͷڵຯ

  8. 8.

    ©2018 Wantedly, Inc. ڧ͍ਓ ೖࣾि໨ ࣗ෼ ʮࣾ಺ͷੜ࢈ੑ͸๻্͕͛·͢ʂʂʯ ʮ͓લੜ࢈ੑͬͯԿ͔Θ͔ͬͯΜͷ͔ʁʯ

  9. 9.

    ©2018 Wantedly, Inc. ྔ ୯Ґ࣌ؒ͋ͨΓʹҰਓͷΤϯδχΞ͕࡞ΕΔՁ஋ͷ ࣗ෼ͷճ౴

  10. 10.

    ©2018 Wantedly, Inc. ࣭ ୯Ґ࣌ؒ͋ͨΓʹҰਓͷΤϯδχΞ͕࡞ΕΔՁ஋ͷ ྔ ͱ ڧ͍ਓͷճ౴

  11. 11.

    ©2018 Wantedly, Inc. ड͚औ໊ͬͨࢗશͯΛɺճͷࡱӨͰ ಉ࣌ʹಡΈࠐΈɻ
 ಡΈࠐΜ໊ͩࢗ͸ਓ޻஌ೳ͕ଈ࣌σʔ λԽ͠ɺ࿈བྷாʹ௥Ճ͞Ε·͢ɻ ݡ໊ࢗ͘؅ཧ ,VCFSOFUFT͕ͳ͚Ε͹8BOUFEMZ1FPQMF͸ͳ͔ͬͨ ͦ΋ͦ΋࡞ΕΔͱ͍͏ൃ૝ʹͳΒͳ͔ͬͨ

  12. 12.

    ©2018 Wantedly, Inc.

  13. 13.

    ©2018 Wantedly, Inc. ͸ͨΒ͘Λ໘ന͘ ༗໊اۀ΍ϕϯνϟʔɺελʔτΞοϓاۀɺؾʹ ͳΔΦϑΟεʹ࿩Λฉ͖ʹ͍͜͏ɻ
 ΩϥͬͱޫΔγΰτ΍ϓϩδΣΫτɺձࣾΛݟ͚ͭ ͯΤϯτϦʔ͠·͠ΐ͏ɻ஌Γ߹͍ͷͭͳ͕Γ͕෼ ͔ͬͯ҆৺Ͱ͢ɻ
 ϓϩϑΟʔϧΛॆ࣮ͤ͞Δ͜ͱͰɺ࠾༻୲౰ऀ͔Β

    ௚઀ΦϑΟε๚໰ͷট଴͕དྷΔ͔΋͠Ε·ͤΜɻ 8BOUFEMZ7JTJU΋,VCFSOFUFTʹࡌͤΔͱ։ൃͷ্࣭͕͕Δ͔΋ʁ ͦ΋ͦ΋࡞ΕΔͱ͍͏ൃ૝ʹͳΒͳ͔ͬͨ΋ͷ͕࡞ΕΔ͔΋ʁ
  14. 14.

    ©2018 Wantedly, Inc. ׂͱϞϊϦγοΫ ϚΠΫϩαʔϏε શ෦ Ұ෦ ೥໨ ೥໨ ͜Ε·Ͱ

  15. 15.

    ©2018 Wantedly, Inc. ׂͱϞϊϦγοΫ ϚΠΫϩαʔϏε શ෦ ೥໨ ೥໨ શ෦ ͜Ε͔Β

  16. 16.

    ©2018 Wantedly, Inc. ͢΂ͯͷαʔϏε͕ ݸͷେখ༷ʑͳ microservice

  17. 17.

    ©2018 Wantedly, Inc. ݸͷେখ༷ʑͳ microservice ΄΅ ௒γϯϓϧͳ΍͕ͭ)FSPLVʹ͍ͨΓ Ҡߦ͍ͨ͠΍ͭ͸શ෦Ҡߦ͖ͬͨ͠ མͪͯࠔΔ΍ͭ ͢΂ͯͷαʔϏε͕

  18. 18.

    ©2018 Wantedly, Inc. on KubernetesԽ࠷ޙͷࡆ

  19. 19.

    ©2018 Wantedly, Inc. Կނ೉͔ͬͨ͠ʁ ͦΕͰ΋΍ͬͨཧ༝͸ʁ ҠߦલޙͷϑϩʔͦΕͧΕ ੾ସ࡞ۀ ·ͱΊ Ҡߦ࡞ۀ

  20. 20.

    ©2018 Wantedly, Inc. Կނ೉͔ͬͨ͠ʁ

  21. 21.

    ©2018 Wantedly, Inc. ن໛͕େ͖͗ͯ͢૝ఆ֎ ՝୊ Өڹൣғ͕޿͗͢Δ

  22. 22.

    ©2018 Wantedly, Inc. Created github.com/wantedly/wantedly Commits Docker Image Repo ՝୊

    Sep 6, 2011 147,687 517.76MB 4GB ن໛͕େ͖͗ͯ͢૝ఆ֎ %PDLFSCVJMEʹ਺෼Ҏ্͔͔Δ͜ͱΛ૝ఆͯ͠ͳ͍ ࣾ಺ͷ,VCFSOFUFTͷӡ༻ϑϩʔʹWJTJU͕ͦͷ··͸৐Εͳ͍ සൟʹߋ৽͞ΕΔXFCαʔόʔ͸લఏʹ͍ͯ͠ͳ͍
  23. 23.

    ©2018 Wantedly, Inc. Ք͙͓ۚ github.com/wantedly/wantedly ґଘ͢ΔαʔϏε ՝୊ ͨ͘͞Μ Өڹൣғ͕޿͗͢Δ 6TFSσʔλ͸7JTJUͷ΋ͷΛ1FPQMFͰ΋ྲྀ༻͍ͯ͠Δ

    7JTJU͕མͪΔͱ1FPQMF΋མͪΔ ͪΐͬͱ஗͘ͳΔ͚ͩͰച্ʹӨڹ ͨ͘͞Μ ࠷ॳظ͔Β͋ΔαʔϏεͳͷͰׂͱ੹຿͕େ͖͍
  24. 24.

    ©2018 Wantedly, Inc. Ϟνϕʔγϣϯ ΠϯϑϥͷҰຊԽ 7JTJUΠϯϑϥͷෛ࠴Խ

  25. 25.

    ©2018 Wantedly, Inc. ҟͳΔ̎ͭͷΠϯϑϥΛ؅ཧ͢Δͷ͕ίετ "VUPTDBMF͍ͨ͠ *TUJP͍Ε͍ͨ 3#"$͍ͨ͠ ৽͍͜͠ͱΛ͍ͨ࣌͠ Πϯγσϯτ͕ىͬͨ࣌͜ 8PSLFSΛࢭΊ͍ͨʂ

    8FCΛεέʔϧ͍ͤͨ͞ʂ Ϟνϕ ΠϯϑϥͷҰຊԽ ՕॴͰಉ͜͡ͱΛҧ͏ํ๏Ͱʜ ঢ়گʹ߹Θͤͯਖ਼͍͠ํ๏Λબ୒ʜ
  26. 26.

    ©2018 Wantedly, Inc. ݟΕΔਓ͕গͳ͘ͳ͖ͬͯͨ ͍ΖΜͳ΋ͷΛॳظ͔Β࢖͍ͬͯΔ ͪΐͬͱ৽͍͜͠ͱΛ΍Δ͚ͩͰҰۤ࿑ Ϟνϕ 7JTJUΠϯϑϥͷෛ࠴Խ ౰࣌͸ඪ४Ͱ͸Ͱ͖ͳ͔ͬͨ͜ͱΛϋοΫ͍ͯͨ͠ ࠓͰ͸ࣗ෼ͰؤுΒͳ͍͍ͯ͘͜ͱ΋ଟʑ͋Δ

    มԽʹڧ͘ͳ͍ʜ %PDLFS͸Ҏલ͔ΒLT͸೥Ҏ্લ͔Β
  27. 27.

    ©2018 Wantedly, Inc. Ϟνϕʔγϣϯ ΠϯϑϥͷҰຊԽ 7JTJUΠϯϑϥͷෛ࠴Խ ن໛͕େ͖͗ͯ͢૝ఆ֎ ՝୊ Өڹൣғ͕޿͗͢Δ

  28. 28.

    ©2018 Wantedly, Inc. ଔઌഐͷνϟϨϯδʂ ΠϯϑϥνʔϜ৽ଔ͕ຖ೥νϟϨϯδ ଔઌഐͷνϟϨϯδʂ ࣌ظঘૣ ࣌ظঘૣ ଔͷԶͷνϟϨϯδʂ ࠓͳΒߦ͚Δʂ

  29. 29.

    ©2018 Wantedly, Inc. Kubernetes Ҡߦ͸Կ౓΋௅ઓ͖ͯͨ͠ େ͖ͳোน͕গͳ͘Ϟνϕ΋͋Δࠓ͔͠ͳ͍ʂ ͍·ͷLTʹ͸YY͕ͳ͍ͷͰࠓ͸அ೦ʜ ͦΕ͕࣏ͬͨࠒʹ͸ผͷ໰୊͕ʜ

  30. 30.

    ©2018 Wantedly, Inc. ҠߦલޙͷEFQMPZϑϩʔ

  31. 31.

    ©2018 Wantedly, Inc. ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ શ෦Ϛωʔδυ set image check pull deploy

    push push hook build build
  32. 32.

    ©2018 Wantedly, Inc. tag ͸ commit hash ͱ latest ͷΈ

    GitHub ͷ commit status Ͱ image ͕ ready ͔Θ͔Δ ΄΅೚ҙͷ commit ༻ͷ image ͕ଘࡏ push push hook build build ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ શ෦Ϛωʔδυ
  33. 33.

    ©2018 Wantedly, Inc. `kube prod deploy master` Ͱ deploy Ͱ͖Δ

    ಺෦Ͱ͸ kubectl set image ؀ڥ͝ͱͷࠩҟ͸؀ڥม਺ͷΈͰ؅ཧ set image check pull deploy ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ શ෦Ϛωʔδυ
  34. 34.

    ©2018 Wantedly, Inc. શ෦AWS build hook push hook intermediate ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ

    ssh deploy hook restart build push pull prod pull ೉͍͠…
  35. 35.

    ©2018 Wantedly, Inc. શ෦AWS ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ ೉͍͠… cache ͳ͠Ͱਅ໘໨ʹ docker build

    ͢Δͱ1࣌ؒ FROM wantedly/wantedly:intermediate tag ͸ prod / qa / intermediate ࡞੒౰࣌͸ແݶʹ image Λஔ͚Δ registry ͕ͳ͔ͬͨ Docker run Ͱಈ͔͔ͨͬͨ͠ͷͰ .env ΋ image ͷͳ͔ build hook push hook intermediate
  36. 36.

    ©2018 Wantedly, Inc. શ෦AWS ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ ೉͍͠… Deploy ࣌ʹ build ઐ༻αʔόʔͰ

    build Deploy ͸ pull and restart Blue / green ssh deploy hook restart build push pull prod pull
  37. 37.

    ©2018 Wantedly, Inc. CVJME 7JTJU #VJME%FQMPZϑϩʔࠩ෼ UBH ؀ڥม਺ 1FPQMF ઐ༻Πϯελϯε

    5SBWJT ؀ڥʹରԠ $PNNJUIBTIʹରԠ JNBHFͷத LTͷTFDSFUT EFQMPZ CMVFHSFFO SPMMJOH શ෦"84 ΄΅Ϛωʔδυ Πϯϑϥ 1FPQMFଆʹऩଋ͍ͤͨ͞
  38. 38.

    ©2018 Wantedly, Inc. Ҡߦ࡞ۀ

  39. 39.

    ©2018 Wantedly, Inc. NBTUFSQVTI͔ΒEFQMPZ׬ྃ·Ͱ෼ ཁ݅ 5SBWJTͰCVJME͍ͨ͠

  40. 40.

    ©2018 Wantedly, Inc. Ͱ͖ΔݶΓͷ͜ͱΛ build ҎલʹऴΘΒͤΔΑ͏ʹ ൿಗ৘ใΛҰ੾ೖΕͳ͍EPDLFSJNBHFʹมߋ $*ͰCVJME͢ΔͨΊʹ *OUFSNFEJBUFΛ࡮৽ͯ͠શ؀ڥ౷ҰͷJNBHF ΄ͱΜͲͷ͕࣌ؒ

    asset sync -> gem ʹ patch ͦͷଞॾʑ…
  41. 41.

    ©2018 Wantedly, Inc. 1VTI͔Β10෼Ͱbuild ׬ྃ ෼ͰऴΘΒ͍͕ͤͨҰ୴͸े෼Ͱ͋Δͱ൑அ Ͱ΍ͬͯ΋

  42. 42.

    ©2018 Wantedly, Inc. ෼ͰऴΘΒ͍͕ͤͨҰ୴͸े෼Ͱ͋Δͱ൑அ ΋ͬͱؤுΔํ๏ $JSDMF$*ʹҠߦ͢Δ 'SPOUFOEΛผϨϙδτϦʹ͢Δ Ϋϥελ಺ͰCVJME͢Δ -BZFSDBDIF͕࢖͑ͯૣ͍͸ͣ Ͱ΍ͬͯ΋

    1VTI͔Β10෼Ͱbuild ׬ྃ
  43. 43.

    ©2018 Wantedly, Inc. %FQMPZ3PMMCBDL଎౓Λམͱͨ͘͠ͳ͍

  44. 44.

    ©2018 Wantedly, Inc. 3PMMCBDLͷ଎౓͕མͪΔͷ͸ఘΊΔ #MVFHSFFOͱಉ͡଎౓͸CMVFHSFFO͡Όͳ͍ͱແཧ Ͳ͏ͤSPMMCBDL͢Δ͔ͷ൑அࣗମʹ෼࢖͍ͬͯΔͷͰ͘͢͝େࣄͰ͸ͳ͍ʁ

  45. 45.

    ©2018 Wantedly, Inc. %FQMPZ3PMMCBDLΛԿ౓΋ߦͬͯ଎౓ܭଌ #MVFHSFFOΛ΍ΊΔͨΊʹ ΞϓϦέʔγϣϯνʔϜͱΑ͘σΟεΧογϣϯΛͨ͠ ݁ہίϛϡχέʔγϣϯ͕େࣄ Կ͕Ͱ͖ͯԿ͕Ͱ͖ͳ͍ͷ͔Λ໌֬ʹ

  46. 46.

    ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂ

  47. 47.

    ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂͷલʹ

  48. 48.

    ©2018 Wantedly, Inc. ։ൃݕূ؀ڥ͚ͩҠߦͨ͠ Ҡߦ࡞ۀࢼݧ ৽͍͠ΦϖϨʔγϣϯͷ໰୊఺Λચ͍ग़͢

  49. 49.

    ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂ ͜ΜͲͦ͜

  50. 50.

    ©2018 Wantedly, Inc. 1SPEΛϑϦʔζͯ྆͠ํʹEFQMPZ چ"-# ৽"-# JOUFSOBM DNS

  51. 51.

    ©2018 Wantedly, Inc. چ"-# ৽"-# JOUFSOBM DNS Ұ෦τϥϑΟοΫΛOHJOY͔Βྲྀ͢ ༧Ί༻ҙ͓͍ͯͨ͠ύλʔϯʹҰॠͰมߋͰ͖ΔΑ͏ʹ͓͍ͯͨ͠ BCPSU͕༰қ

  52. 52.

    ©2018 Wantedly, Inc. શτϥϑΟοΫΛLTʹྲྀ͢ چ"-# ৽"-# JOUFSOBM DNS

  53. 53.

    ©2018 Wantedly, Inc. Ұ୴BCPSUͨ͠ چ"-# ৽"-# JOUFSOBM DNS ͜ͷ࣌গ͠ࣄނΛى͕ͨ͜͠ɺҰॠͰBCPSUͰ͖ΔQBUIΛ༻ҙ͓͍ͯͨ͠ͷ͸ྑ͔ͬͨ

  54. 54.

    ©2018 Wantedly, Inc. ࢓੾Γ௚͠ چ"-# ৽"-# JOUFSOBM DNS

  55. 55.

    ©2018 Wantedly, Inc. %/4੾Γସ͑ چ"-# ৽"-# JOUFSOBM DNS

  56. 56.

    ©2018 Wantedly, Inc. چ"-# ৽"-# JOUFSOBM DNS ׬ྃʂʂʂ

  57. 57.

    ©2018 Wantedly, Inc. DNS ׬ྃʂʂʂ ৽"-#

  58. 58.

    ©2018 Wantedly, Inc. %/4੾Γସ͑ ৽ALB͕ड͚ͨϦΫΤετͷׂ߹ DNS੾ସ

  59. 59.

    ©2018 Wantedly, Inc. %/4੾Γସ͑ ৽ALB͕ड͚ͨϦΫΤετͷׂ߹ DNS੾ସ

  60. 60.

    ©2018 Wantedly, Inc. ίϚϯυ࣮ߦ͕ૣ͘ͳͬͨ %FQMPZ͕ૣ͘ͳͬͨ ݁Ռ DBෛՙ͕ݮͬͨ Autoscale͕Ͱ͖ΔΑ͏ʹͳͬͨ ϫʔετ෼ Ұൠతʹ͸෼ఔ౓

    CMVFHSFFOͰͳ͘ͳͬͨͨΊ
  61. 61.

    ©2018 Wantedly, Inc. ϑϩʔ͕͋ΔͱҠߦ͠΍͍͢ ֶΜͩ͜ͱ ϑΥʔϧόοΫઌ͸৻ॏʹ Ϛωʔδυ͸ָ ࢼݧΛͯ͠΋ࣄނ͸ى͜Δ ఘΊΔ͜ͱ΋େࣄ 1PTUNPSUFNେࣄ

    ࣄނΛىͨ͜͠ຊਓ΋ཱͪ௚ΕΔ
  62. 62.

    ©2018 Wantedly, Inc. We’re Hiring .JDSPTFSWJDFT.POEBZ HJUIVCDPNXBOUFEMZNJDSPTFSWJDFT@NPOEBZ