Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Wantedly Visit on Kubernetes

Wantedly Visit on Kubernetes

089fe44e41bb1fa2d9421f919a99173c?s=128

Shimpei Otsubo

September 26, 2018
Tweet

Transcript

  1. ©2018 Wantedly, Inc. Wantedly Visit on Kubernetes Wantedly ͷ͢΂ͯͷαʔϏεΛ Kubernetes

    ΛҠߦͤͨ͞ཧ༝ͱۤ࿑ SRE Lounge #5 26.Sep.2018 - Shimpei Otsubo @potsbo
  2. ©2018 Wantedly, Inc. ͸ͨΒ͘Λ໘ന͘ ༗໊اۀ΍ϕϯνϟʔɺελʔτΞοϓاۀɺؾʹ ͳΔΦϑΟεʹ࿩Λฉ͖ʹ͍͜͏ɻ
 ΩϥͬͱޫΔγΰτ΍ϓϩδΣΫτɺձࣾΛݟ͚ͭ ͯΤϯτϦʔ͠·͠ΐ͏ɻ஌Γ߹͍ͷͭͳ͕Γ͕෼ ͔ͬͯ҆৺Ͱ͢ɻ
 ϓϩϑΟʔϧΛॆ࣮ͤ͞Δ͜ͱͰɺ࠾༻୲౰ऀ͔Β

    ௚઀ΦϑΟε๚໰ͷট଴͕དྷΔ͔΋͠Ε·ͤΜɻ
  3. ©2018 Wantedly, Inc. ड͚औ໊ͬͨࢗશͯΛɺճͷࡱӨͰ ಉ࣌ʹಡΈࠐΈɻ
 ಡΈࠐΜ໊ͩࢗ͸ਓ޻஌ೳ͕ଈ࣌σʔ λԽ͠ɺ࿈བྷாʹ௥Ճ͞Ε·͢ɻ ݡ໊ࢗ͘؅ཧ

  4. ©2018 Wantedly, Inc. 4IJNQFJ0UTVCP!QPUTCP Infrastructure Team, Wantedly, Inc. 2018 New

    Grads Kubernetes Dev Tools CI / CD AuthN / Z Productivity Dvorak
  5. ©2018 Wantedly, Inc.

  6. ©2018 Wantedly, Inc. ڞ௨σʔλج൫ ΠϯϑϥνʔϜ͕࠷ۙ΍͍ͬͯΔ͜ͱ αʔϏεϝογϡ෼ࢄτϨʔγϯά ΤϯδχΞͷੜ࢈ੑ 43&͚ͩͰ͸ͳ͍

  7. ©2018 Wantedly, Inc. ڞ௨σʔλج൫ ΠϯϑϥνʔϜ͕࠷ۙ΍͍ͬͯΔ͜ͱ αʔϏεϝογϡ෼ࢄτϨʔγϯά ΤϯδχΞͷੜ࢈ੑ 43&͚ͩͰ͸ͳ͍ ࣗ෼ͷڵຯ

  8. ©2018 Wantedly, Inc. ڧ͍ਓ ೖࣾि໨ ࣗ෼ ʮࣾ಺ͷੜ࢈ੑ͸๻্͕͛·͢ʂʂʯ ʮ͓લੜ࢈ੑͬͯԿ͔Θ͔ͬͯΜͷ͔ʁʯ

  9. ©2018 Wantedly, Inc. ྔ ୯Ґ࣌ؒ͋ͨΓʹҰਓͷΤϯδχΞ͕࡞ΕΔՁ஋ͷ ࣗ෼ͷճ౴

  10. ©2018 Wantedly, Inc. ࣭ ୯Ґ࣌ؒ͋ͨΓʹҰਓͷΤϯδχΞ͕࡞ΕΔՁ஋ͷ ྔ ͱ ڧ͍ਓͷճ౴

  11. ©2018 Wantedly, Inc. ड͚औ໊ͬͨࢗશͯΛɺճͷࡱӨͰ ಉ࣌ʹಡΈࠐΈɻ
 ಡΈࠐΜ໊ͩࢗ͸ਓ޻஌ೳ͕ଈ࣌σʔ λԽ͠ɺ࿈བྷாʹ௥Ճ͞Ε·͢ɻ ݡ໊ࢗ͘؅ཧ ,VCFSOFUFT͕ͳ͚Ε͹8BOUFEMZ1FPQMF͸ͳ͔ͬͨ ͦ΋ͦ΋࡞ΕΔͱ͍͏ൃ૝ʹͳΒͳ͔ͬͨ

  12. ©2018 Wantedly, Inc.

  13. ©2018 Wantedly, Inc. ͸ͨΒ͘Λ໘ന͘ ༗໊اۀ΍ϕϯνϟʔɺελʔτΞοϓاۀɺؾʹ ͳΔΦϑΟεʹ࿩Λฉ͖ʹ͍͜͏ɻ
 ΩϥͬͱޫΔγΰτ΍ϓϩδΣΫτɺձࣾΛݟ͚ͭ ͯΤϯτϦʔ͠·͠ΐ͏ɻ஌Γ߹͍ͷͭͳ͕Γ͕෼ ͔ͬͯ҆৺Ͱ͢ɻ
 ϓϩϑΟʔϧΛॆ࣮ͤ͞Δ͜ͱͰɺ࠾༻୲౰ऀ͔Β

    ௚઀ΦϑΟε๚໰ͷট଴͕དྷΔ͔΋͠Ε·ͤΜɻ 8BOUFEMZ7JTJU΋,VCFSOFUFTʹࡌͤΔͱ։ൃͷ্࣭͕͕Δ͔΋ʁ ͦ΋ͦ΋࡞ΕΔͱ͍͏ൃ૝ʹͳΒͳ͔ͬͨ΋ͷ͕࡞ΕΔ͔΋ʁ
  14. ©2018 Wantedly, Inc. ׂͱϞϊϦγοΫ ϚΠΫϩαʔϏε શ෦ Ұ෦ ೥໨ ೥໨ ͜Ε·Ͱ

  15. ©2018 Wantedly, Inc. ׂͱϞϊϦγοΫ ϚΠΫϩαʔϏε શ෦ ೥໨ ೥໨ શ෦ ͜Ε͔Β

  16. ©2018 Wantedly, Inc. ͢΂ͯͷαʔϏε͕ ݸͷେখ༷ʑͳ microservice

  17. ©2018 Wantedly, Inc. ݸͷେখ༷ʑͳ microservice ΄΅ ௒γϯϓϧͳ΍͕ͭ)FSPLVʹ͍ͨΓ Ҡߦ͍ͨ͠΍ͭ͸શ෦Ҡߦ͖ͬͨ͠ མͪͯࠔΔ΍ͭ ͢΂ͯͷαʔϏε͕

  18. ©2018 Wantedly, Inc. on KubernetesԽ࠷ޙͷࡆ

  19. ©2018 Wantedly, Inc. Կނ೉͔ͬͨ͠ʁ ͦΕͰ΋΍ͬͨཧ༝͸ʁ ҠߦલޙͷϑϩʔͦΕͧΕ ੾ସ࡞ۀ ·ͱΊ Ҡߦ࡞ۀ

  20. ©2018 Wantedly, Inc. Կނ೉͔ͬͨ͠ʁ

  21. ©2018 Wantedly, Inc. ن໛͕େ͖͗ͯ͢૝ఆ֎ ՝୊ Өڹൣғ͕޿͗͢Δ

  22. ©2018 Wantedly, Inc. Created github.com/wantedly/wantedly Commits Docker Image Repo ՝୊

    Sep 6, 2011 147,687 517.76MB 4GB ن໛͕େ͖͗ͯ͢૝ఆ֎ %PDLFSCVJMEʹ਺෼Ҏ্͔͔Δ͜ͱΛ૝ఆͯ͠ͳ͍ ࣾ಺ͷ,VCFSOFUFTͷӡ༻ϑϩʔʹWJTJU͕ͦͷ··͸৐Εͳ͍ සൟʹߋ৽͞ΕΔXFCαʔόʔ͸લఏʹ͍ͯ͠ͳ͍
  23. ©2018 Wantedly, Inc. Ք͙͓ۚ github.com/wantedly/wantedly ґଘ͢ΔαʔϏε ՝୊ ͨ͘͞Μ Өڹൣғ͕޿͗͢Δ 6TFSσʔλ͸7JTJUͷ΋ͷΛ1FPQMFͰ΋ྲྀ༻͍ͯ͠Δ

    7JTJU͕མͪΔͱ1FPQMF΋མͪΔ ͪΐͬͱ஗͘ͳΔ͚ͩͰച্ʹӨڹ ͨ͘͞Μ ࠷ॳظ͔Β͋ΔαʔϏεͳͷͰׂͱ੹຿͕େ͖͍
  24. ©2018 Wantedly, Inc. Ϟνϕʔγϣϯ ΠϯϑϥͷҰຊԽ 7JTJUΠϯϑϥͷෛ࠴Խ

  25. ©2018 Wantedly, Inc. ҟͳΔ̎ͭͷΠϯϑϥΛ؅ཧ͢Δͷ͕ίετ "VUPTDBMF͍ͨ͠ *TUJP͍Ε͍ͨ 3#"$͍ͨ͠ ৽͍͜͠ͱΛ͍ͨ࣌͠ Πϯγσϯτ͕ىͬͨ࣌͜ 8PSLFSΛࢭΊ͍ͨʂ

    8FCΛεέʔϧ͍ͤͨ͞ʂ Ϟνϕ ΠϯϑϥͷҰຊԽ ՕॴͰಉ͜͡ͱΛҧ͏ํ๏Ͱʜ ঢ়گʹ߹Θͤͯਖ਼͍͠ํ๏Λબ୒ʜ
  26. ©2018 Wantedly, Inc. ݟΕΔਓ͕গͳ͘ͳ͖ͬͯͨ ͍ΖΜͳ΋ͷΛॳظ͔Β࢖͍ͬͯΔ ͪΐͬͱ৽͍͜͠ͱΛ΍Δ͚ͩͰҰۤ࿑ Ϟνϕ 7JTJUΠϯϑϥͷෛ࠴Խ ౰࣌͸ඪ४Ͱ͸Ͱ͖ͳ͔ͬͨ͜ͱΛϋοΫ͍ͯͨ͠ ࠓͰ͸ࣗ෼ͰؤுΒͳ͍͍ͯ͘͜ͱ΋ଟʑ͋Δ

    มԽʹڧ͘ͳ͍ʜ %PDLFS͸Ҏલ͔ΒLT͸೥Ҏ্લ͔Β
  27. ©2018 Wantedly, Inc. Ϟνϕʔγϣϯ ΠϯϑϥͷҰຊԽ 7JTJUΠϯϑϥͷෛ࠴Խ ن໛͕େ͖͗ͯ͢૝ఆ֎ ՝୊ Өڹൣғ͕޿͗͢Δ

  28. ©2018 Wantedly, Inc. ଔઌഐͷνϟϨϯδʂ ΠϯϑϥνʔϜ৽ଔ͕ຖ೥νϟϨϯδ ଔઌഐͷνϟϨϯδʂ ࣌ظঘૣ ࣌ظঘૣ ଔͷԶͷνϟϨϯδʂ ࠓͳΒߦ͚Δʂ

  29. ©2018 Wantedly, Inc. Kubernetes Ҡߦ͸Կ౓΋௅ઓ͖ͯͨ͠ େ͖ͳোน͕গͳ͘Ϟνϕ΋͋Δࠓ͔͠ͳ͍ʂ ͍·ͷLTʹ͸YY͕ͳ͍ͷͰࠓ͸அ೦ʜ ͦΕ͕࣏ͬͨࠒʹ͸ผͷ໰୊͕ʜ

  30. ©2018 Wantedly, Inc. ҠߦલޙͷEFQMPZϑϩʔ

  31. ©2018 Wantedly, Inc. ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ શ෦Ϛωʔδυ set image check pull deploy

    push push hook build build
  32. ©2018 Wantedly, Inc. tag ͸ commit hash ͱ latest ͷΈ

    GitHub ͷ commit status Ͱ image ͕ ready ͔Θ͔Δ ΄΅೚ҙͷ commit ༻ͷ image ͕ଘࡏ push push hook build build ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ શ෦Ϛωʔδυ
  33. ©2018 Wantedly, Inc. `kube prod deploy master` Ͱ deploy Ͱ͖Δ

    ಺෦Ͱ͸ kubectl set image ؀ڥ͝ͱͷࠩҟ͸؀ڥม਺ͷΈͰ؅ཧ set image check pull deploy ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ શ෦Ϛωʔδυ
  34. ©2018 Wantedly, Inc. શ෦AWS build hook push hook intermediate ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ

    ssh deploy hook restart build push pull prod pull ೉͍͠…
  35. ©2018 Wantedly, Inc. શ෦AWS ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ ೉͍͠… cache ͳ͠Ͱਅ໘໨ʹ docker build

    ͢Δͱ1࣌ؒ FROM wantedly/wantedly:intermediate tag ͸ prod / qa / intermediate ࡞੒౰࣌͸ແݶʹ image Λஔ͚Δ registry ͕ͳ͔ͬͨ Docker run Ͱಈ͔͔ͨͬͨ͠ͷͰ .env ΋ image ͷͳ͔ build hook push hook intermediate
  36. ©2018 Wantedly, Inc. શ෦AWS ࣾ಺ͷ,VCFSOFUFTͷϑϩʔ ೉͍͠… Deploy ࣌ʹ build ઐ༻αʔόʔͰ

    build Deploy ͸ pull and restart Blue / green ssh deploy hook restart build push pull prod pull
  37. ©2018 Wantedly, Inc. CVJME 7JTJU #VJME%FQMPZϑϩʔࠩ෼ UBH ؀ڥม਺ 1FPQMF ઐ༻Πϯελϯε

    5SBWJT ؀ڥʹରԠ $PNNJUIBTIʹରԠ JNBHFͷத LTͷTFDSFUT EFQMPZ CMVFHSFFO SPMMJOH શ෦"84 ΄΅Ϛωʔδυ Πϯϑϥ 1FPQMFଆʹऩଋ͍ͤͨ͞
  38. ©2018 Wantedly, Inc. Ҡߦ࡞ۀ

  39. ©2018 Wantedly, Inc. NBTUFSQVTI͔ΒEFQMPZ׬ྃ·Ͱ෼ ཁ݅ 5SBWJTͰCVJME͍ͨ͠

  40. ©2018 Wantedly, Inc. Ͱ͖ΔݶΓͷ͜ͱΛ build ҎલʹऴΘΒͤΔΑ͏ʹ ൿಗ৘ใΛҰ੾ೖΕͳ͍EPDLFSJNBHFʹมߋ $*ͰCVJME͢ΔͨΊʹ *OUFSNFEJBUFΛ࡮৽ͯ͠શ؀ڥ౷ҰͷJNBHF ΄ͱΜͲͷ͕࣌ؒ

    asset sync -> gem ʹ patch ͦͷଞॾʑ…
  41. ©2018 Wantedly, Inc. 1VTI͔Β10෼Ͱbuild ׬ྃ ෼ͰऴΘΒ͍͕ͤͨҰ୴͸े෼Ͱ͋Δͱ൑அ Ͱ΍ͬͯ΋

  42. ©2018 Wantedly, Inc. ෼ͰऴΘΒ͍͕ͤͨҰ୴͸े෼Ͱ͋Δͱ൑அ ΋ͬͱؤுΔํ๏ $JSDMF$*ʹҠߦ͢Δ 'SPOUFOEΛผϨϙδτϦʹ͢Δ Ϋϥελ಺ͰCVJME͢Δ -BZFSDBDIF͕࢖͑ͯૣ͍͸ͣ Ͱ΍ͬͯ΋

    1VTI͔Β10෼Ͱbuild ׬ྃ
  43. ©2018 Wantedly, Inc. %FQMPZ3PMMCBDL଎౓Λམͱͨ͘͠ͳ͍

  44. ©2018 Wantedly, Inc. 3PMMCBDLͷ଎౓͕མͪΔͷ͸ఘΊΔ #MVFHSFFOͱಉ͡଎౓͸CMVFHSFFO͡Όͳ͍ͱແཧ Ͳ͏ͤSPMMCBDL͢Δ͔ͷ൑அࣗମʹ෼࢖͍ͬͯΔͷͰ͘͢͝େࣄͰ͸ͳ͍ʁ

  45. ©2018 Wantedly, Inc. %FQMPZ3PMMCBDLΛԿ౓΋ߦͬͯ଎౓ܭଌ #MVFHSFFOΛ΍ΊΔͨΊʹ ΞϓϦέʔγϣϯνʔϜͱΑ͘σΟεΧογϣϯΛͨ͠ ݁ہίϛϡχέʔγϣϯ͕େࣄ Կ͕Ͱ͖ͯԿ͕Ͱ͖ͳ͍ͷ͔Λ໌֬ʹ

  46. ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂ

  47. ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂͷલʹ

  48. ©2018 Wantedly, Inc. ։ൃݕূ؀ڥ͚ͩҠߦͨ͠ Ҡߦ࡞ۀࢼݧ ৽͍͠ΦϖϨʔγϣϯͷ໰୊఺Λચ͍ग़͢

  49. ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂ ͜ΜͲͦ͜

  50. ©2018 Wantedly, Inc. 1SPEΛϑϦʔζͯ྆͠ํʹEFQMPZ چ"-# ৽"-# JOUFSOBM DNS

  51. ©2018 Wantedly, Inc. چ"-# ৽"-# JOUFSOBM DNS Ұ෦τϥϑΟοΫΛOHJOY͔Βྲྀ͢ ༧Ί༻ҙ͓͍ͯͨ͠ύλʔϯʹҰॠͰมߋͰ͖ΔΑ͏ʹ͓͍ͯͨ͠ BCPSU͕༰қ

  52. ©2018 Wantedly, Inc. શτϥϑΟοΫΛLTʹྲྀ͢ چ"-# ৽"-# JOUFSOBM DNS

  53. ©2018 Wantedly, Inc. Ұ୴BCPSUͨ͠ چ"-# ৽"-# JOUFSOBM DNS ͜ͷ࣌গ͠ࣄނΛى͕ͨ͜͠ɺҰॠͰBCPSUͰ͖ΔQBUIΛ༻ҙ͓͍ͯͨ͠ͷ͸ྑ͔ͬͨ

  54. ©2018 Wantedly, Inc. ࢓੾Γ௚͠ چ"-# ৽"-# JOUFSOBM DNS

  55. ©2018 Wantedly, Inc. %/4੾Γସ͑ چ"-# ৽"-# JOUFSOBM DNS

  56. ©2018 Wantedly, Inc. چ"-# ৽"-# JOUFSOBM DNS ׬ྃʂʂʂ

  57. ©2018 Wantedly, Inc. DNS ׬ྃʂʂʂ ৽"-#

  58. ©2018 Wantedly, Inc. %/4੾Γସ͑ ৽ALB͕ड͚ͨϦΫΤετͷׂ߹ DNS੾ସ

  59. ©2018 Wantedly, Inc. %/4੾Γସ͑ ৽ALB͕ड͚ͨϦΫΤετͷׂ߹ DNS੾ସ

  60. ©2018 Wantedly, Inc. ίϚϯυ࣮ߦ͕ૣ͘ͳͬͨ %FQMPZ͕ૣ͘ͳͬͨ ݁Ռ DBෛՙ͕ݮͬͨ Autoscale͕Ͱ͖ΔΑ͏ʹͳͬͨ ϫʔετ෼ Ұൠతʹ͸෼ఔ౓

    CMVFHSFFOͰͳ͘ͳͬͨͨΊ
  61. ©2018 Wantedly, Inc. ϑϩʔ͕͋ΔͱҠߦ͠΍͍͢ ֶΜͩ͜ͱ ϑΥʔϧόοΫઌ͸৻ॏʹ Ϛωʔδυ͸ָ ࢼݧΛͯ͠΋ࣄނ͸ى͜Δ ఘΊΔ͜ͱ΋େࣄ 1PTUNPSUFNେࣄ

    ࣄނΛىͨ͜͠ຊਓ΋ཱͪ௚ΕΔ
  62. ©2018 Wantedly, Inc. We’re Hiring .JDSPTFSWJDFT.POEBZ HJUIVCDPNXBOUFEMZNJDSPTFSWJDFT@NPOEBZ