Wantedly Visit on Kubernetes

Transcript

1. ©2018 Wantedly, Inc. Wantedly Visit on Kubernetes Wantedly ͷ͢΂ͯͷαʔϏεΛ Kubernetes

   ΛҠߦͤͨ͞ཧ༝ͱۤ࿑ SRE Lounge #5 26.Sep.2018 - Shimpei Otsubo @potsbo

2. ©2018 Wantedly, Inc. ͸ͨΒ͘Λ໘ന͘ ༗໊اۀ΍ϕϯνϟʔɺελʔτΞοϓاۀɺؾʹ ͳΔΦϑΟεʹ࿩Λฉ͖ʹ͍͜͏ɻ
   
 ΩϥͬͱޫΔγΰτ΍ϓϩδΣΫτɺձࣾΛݟ͚ͭ ͯΤϯτϦʔ͠·͠ΐ͏ɻ஌Γ߹͍ͷͭͳ͕Γ͕෼ ͔ͬͯ҆৺Ͱ͢ɻ
   
 ϓϩϑΟʔϧΛॆ࣮ͤ͞Δ͜ͱͰɺ࠾༻୲౰ऀ͔Β

   ௚઀ΦϑΟε๚໰ͷট଴͕དྷΔ͔΋͠Ε·ͤΜɻ

3. ©2018 Wantedly, Inc. ड͚औ໊ͬͨࢗશͯΛɺ
   ճͷࡱӨͰ ಉ࣌ʹಡΈࠐΈɻ
   
 ಡΈࠐΜ໊ͩࢗ͸ਓ޻஌ೳ͕ଈ࣌σʔ λԽ͠ɺ
   ࿈བྷாʹ௥Ճ͞Ε·͢ɻ ݡ໊ࢗ͘؅ཧ

4. ©2018 Wantedly, Inc. 4IJNQFJ
   0UTVCP
   !QPUTCP Infrastructure Team, Wantedly, Inc. 2018 New

   Grads Kubernetes Dev Tools CI / CD AuthN / Z Productivity Dvorak

5. ©2018 Wantedly, Inc.

6. ©2018 Wantedly, Inc. ڞ௨σʔλج൫ ΠϯϑϥνʔϜ͕࠷ۙ΍͍ͬͯΔ͜ͱ αʔϏεϝογϡ
   
   ෼ࢄτϨʔγϯά ΤϯδχΞͷੜ࢈ੑ 43&͚ͩͰ͸ͳ͍

7. ©2018 Wantedly, Inc. ڞ௨σʔλج൫ ΠϯϑϥνʔϜ͕࠷ۙ΍͍ͬͯΔ͜ͱ αʔϏεϝογϡ
   
   ෼ࢄτϨʔγϯά ΤϯδχΞͷੜ࢈ੑ 43&͚ͩͰ͸ͳ͍ ࣗ෼ͷڵຯ

8. ©2018 Wantedly, Inc. ڧ͍ਓ ೖࣾि໨ ࣗ෼ ʮࣾ಺ͷੜ࢈ੑ͸๻্͕͛·͢ʂʂʯ ʮ͓લੜ࢈ੑͬͯԿ͔Θ͔ͬͯΜͷ͔ʁʯ

9. ©2018 Wantedly, Inc. ྔ ୯Ґ࣌ؒ͋ͨΓʹҰਓͷΤϯδχΞ͕࡞ΕΔՁ஋ͷ ࣗ෼ͷճ౴

10. ©2018 Wantedly, Inc. ࣭ ୯Ґ࣌ؒ͋ͨΓʹҰਓͷΤϯδχΞ͕࡞ΕΔՁ஋ͷ ྔ ͱ ڧ͍ਓͷճ౴

11. ©2018 Wantedly, Inc. ड͚औ໊ͬͨࢗશͯΛɺ
    ճͷࡱӨͰ ಉ࣌ʹಡΈࠐΈɻ
    
 ಡΈࠐΜ໊ͩࢗ͸ਓ޻஌ೳ͕ଈ࣌σʔ λԽ͠ɺ
    ࿈བྷாʹ௥Ճ͞Ε·͢ɻ ݡ໊ࢗ͘؅ཧ ,VCFSOFUFT
    ͕ͳ͚Ε͹
    8BOUFEMZ
    1FPQMF
    ͸ͳ͔ͬͨ ͦ΋ͦ΋࡞ΕΔͱ͍͏ൃ૝ʹͳΒͳ͔ͬͨ

12. ©2018 Wantedly, Inc.

13. ©2018 Wantedly, Inc. ͸ͨΒ͘Λ໘ന͘ ༗໊اۀ΍ϕϯνϟʔɺελʔτΞοϓاۀɺؾʹ ͳΔΦϑΟεʹ࿩Λฉ͖ʹ͍͜͏ɻ
    
 ΩϥͬͱޫΔγΰτ΍ϓϩδΣΫτɺձࣾΛݟ͚ͭ ͯΤϯτϦʔ͠·͠ΐ͏ɻ஌Γ߹͍ͷͭͳ͕Γ͕෼ ͔ͬͯ҆৺Ͱ͢ɻ
    
 ϓϩϑΟʔϧΛॆ࣮ͤ͞Δ͜ͱͰɺ࠾༻୲౰ऀ͔Β

    ௚઀ΦϑΟε๚໰ͷট଴͕དྷΔ͔΋͠Ε·ͤΜɻ 8BOUFEMZ
    7JTJU
    ΋
    ,VCFSOFUFT
    ʹࡌͤΔͱ։ൃͷ্࣭͕͕Δ͔΋ʁ ͦ΋ͦ΋࡞ΕΔͱ͍͏ൃ૝ʹͳΒͳ͔ͬͨ΋ͷ͕࡞ΕΔ͔΋ʁ

14. ©2018 Wantedly, Inc. ׂͱϞϊϦγοΫ ϚΠΫϩαʔϏε શ෦ Ұ෦ ೥໨ ೥໨ ͜Ε·Ͱ

15. ©2018 Wantedly, Inc. ׂͱϞϊϦγοΫ ϚΠΫϩαʔϏε શ෦ ೥໨ ೥໨ શ෦ ͜Ε͔Β

16. ©2018 Wantedly, Inc. ͢΂ͯͷαʔϏε͕ ݸͷେখ༷ʑͳ microservice

17. ©2018 Wantedly, Inc. ݸͷେখ༷ʑͳ microservice ΄΅ ௒γϯϓϧͳ΍͕ͭ
    )FSPLV
    ʹ͍ͨΓ
    Ҡߦ͍ͨ͠΍ͭ͸શ෦Ҡߦ͖ͬͨ͠ མͪͯࠔΔ΍ͭ ͢΂ͯͷαʔϏε͕

18. ©2018 Wantedly, Inc. on Kubernetes
    Խ࠷ޙͷࡆ

19. ©2018 Wantedly, Inc. Կނ೉͔ͬͨ͠ʁ ͦΕͰ΋΍ͬͨཧ༝͸ʁ ҠߦલޙͷϑϩʔͦΕͧΕ ੾ସ࡞ۀ ·ͱΊ Ҡߦ࡞ۀ

20. ©2018 Wantedly, Inc. Կނ೉͔ͬͨ͠ʁ

21. ©2018 Wantedly, Inc. ن໛͕େ͖͗ͯ͢૝ఆ֎ ՝୊ Өڹൣғ͕޿͗͢Δ

22. ©2018 Wantedly, Inc. Created github.com/wantedly/wantedly Commits Docker Image Repo ՝୊

    Sep 6, 2011 147,687 517.76MB 4GB ن໛͕େ͖͗ͯ͢૝ఆ֎ %PDLFS
    CVJME
    ʹ਺෼Ҏ্͔͔Δ͜ͱΛ૝ఆͯ͠ͳ͍ ࣾ಺ͷ
    ,VCFSOFUFT
    ͷӡ༻ϑϩʔʹ
    WJTJU
    ͕ͦͷ··͸৐Εͳ͍ සൟʹߋ৽͞ΕΔXFCαʔόʔ͸લఏʹ͍ͯ͠ͳ͍

23. ©2018 Wantedly, Inc. Ք͙͓ۚ github.com/wantedly/wantedly ґଘ͢ΔαʔϏε ՝୊ ͨ͘͞Μ Өڹൣғ͕޿͗͢Δ 6TFS
    σʔλ͸
    7JTJU
    ͷ΋ͷΛ
    1FPQMF
    Ͱ΋ྲྀ༻͍ͯ͠Δ

    7JTJU
    ͕མͪΔͱ
    1FPQMF
    ΋མͪΔ ͪΐͬͱ஗͘ͳΔ͚ͩͰച্ʹӨڹ ͨ͘͞Μ ࠷ॳظ͔Β͋ΔαʔϏεͳͷͰׂͱ੹຿͕େ͖͍

24. ©2018 Wantedly, Inc. Ϟνϕʔγϣϯ ΠϯϑϥͷҰຊԽ 7JTJU
    Πϯϑϥͷෛ࠴Խ

25. ©2018 Wantedly, Inc. ҟͳΔ̎ͭͷΠϯϑϥΛ؅ཧ͢Δͷ͕ίετ "VUPTDBMF
    ͍ͨ͠ *TUJP
    ͍Ε͍ͨ 3#"$͍ͨ͠ ৽͍͜͠ͱΛ͍ͨ࣌͠ Πϯγσϯτ͕ىͬͨ࣌͜ 8PSLFS
    ΛࢭΊ͍ͨʂ

    8FC
    Λεέʔϧ͍ͤͨ͞ʂ Ϟνϕ ΠϯϑϥͷҰຊԽ ՕॴͰಉ͜͡ͱΛҧ͏ํ๏Ͱʜ ঢ়گʹ߹Θͤͯਖ਼͍͠ํ๏Λબ୒ʜ

26. ©2018 Wantedly, Inc. ݟΕΔਓ͕গͳ͘ͳ͖ͬͯͨ ͍ΖΜͳ΋ͷΛॳظ͔Β࢖͍ͬͯΔ ͪΐͬͱ৽͍͜͠ͱΛ΍Δ͚ͩͰҰۤ࿑ Ϟνϕ 7JTJUΠϯϑϥͷෛ࠴Խ ౰࣌͸ඪ४Ͱ͸Ͱ͖ͳ͔ͬͨ͜ͱΛϋοΫ͍ͯͨ͠ ࠓͰ͸ࣗ෼ͰؤுΒͳ͍͍ͯ͘͜ͱ΋ଟʑ͋Δ

    มԽʹڧ͘ͳ͍ʜ %PDLFS͸Ҏલ͔Β
    
    LT
    ͸೥Ҏ্લ͔Β

27. ©2018 Wantedly, Inc. Ϟνϕʔγϣϯ ΠϯϑϥͷҰຊԽ 7JTJU
    Πϯϑϥͷෛ࠴Խ ن໛͕େ͖͗ͯ͢૝ఆ֎ ՝୊ Өڹൣғ͕޿͗͢Δ

28. ©2018 Wantedly, Inc. ଔઌഐͷνϟϨϯδʂ ΠϯϑϥνʔϜ৽ଔ͕ຖ೥νϟϨϯδ ଔઌഐͷνϟϨϯδʂ ࣌ظঘૣ ࣌ظঘૣ ଔͷԶͷνϟϨϯδʂ ࠓͳΒߦ͚Δʂ

29. ©2018 Wantedly, Inc. Kubernetes Ҡߦ͸Կ౓΋௅ઓ͖ͯͨ͠ େ͖ͳোน͕গͳ͘Ϟνϕ΋͋Δࠓ͔͠ͳ͍ʂ ͍·ͷ
    LT
    ʹ͸
    YY
    ͕ͳ͍ͷͰࠓ͸அ೦ʜ ͦΕ͕࣏ͬͨࠒʹ͸ผͷ໰୊͕ʜ

30. ©2018 Wantedly, Inc. Ҡߦલޙͷ
    EFQMPZ
    ϑϩʔ

31. ©2018 Wantedly, Inc. ࣾ಺ͷ
    ,VCFSOFUFT
    ͷ
    ϑϩʔ શ෦Ϛωʔδυ set image check pull deploy

    push push hook build build

32. ©2018 Wantedly, Inc. tag ͸ commit hash ͱ latest ͷΈ

    GitHub ͷ commit status Ͱ image ͕ ready ͔Θ͔Δ ΄΅೚ҙͷ commit ༻ͷ image ͕ଘࡏ push push hook build build ࣾ಺ͷ
    ,VCFSOFUFT
    ͷ
    ϑϩʔ શ෦Ϛωʔδυ

33. ©2018 Wantedly, Inc. `kube prod deploy master` Ͱ deploy Ͱ͖Δ

    ಺෦Ͱ͸ kubectl set image ؀ڥ͝ͱͷࠩҟ͸؀ڥม਺ͷΈͰ؅ཧ set image check pull deploy ࣾ಺ͷ
    ,VCFSOFUFT
    ͷ
    ϑϩʔ શ෦Ϛωʔδυ

34. ©2018 Wantedly, Inc. શ෦AWS build hook push hook intermediate ࣾ಺ͷ
    ,VCFSOFUFT
    ͷ
    ϑϩʔ

    ssh deploy hook restart build push pull prod pull ೉͍͠…

35. ©2018 Wantedly, Inc. શ෦AWS ࣾ಺ͷ
    ,VCFSOFUFT
    ͷ
    ϑϩʔ ೉͍͠… cache ͳ͠Ͱਅ໘໨ʹ docker build

    ͢Δͱ1࣌ؒ FROM wantedly/wantedly:intermediate tag ͸ prod / qa / intermediate ࡞੒౰࣌͸ແݶʹ image Λஔ͚Δ registry ͕ͳ͔ͬͨ Docker run Ͱಈ͔͔ͨͬͨ͠ͷͰ .env ΋ image ͷͳ͔ build hook push hook intermediate

36. ©2018 Wantedly, Inc. શ෦AWS ࣾ಺ͷ
    ,VCFSOFUFT
    ͷ
    ϑϩʔ ೉͍͠… Deploy ࣌ʹ build ઐ༻αʔόʔͰ

    build Deploy ͸ pull and restart Blue / green ssh deploy hook restart build push pull prod pull

37. ©2018 Wantedly, Inc. CVJME 7JTJU #VJME
    
    %FQMPZ
    ϑϩʔࠩ෼ UBH ؀ڥม਺ 1FPQMF ઐ༻Πϯελϯε

    5SBWJT ؀ڥʹରԠ $PNNJU
    IBTI
    ʹରԠ JNBHF
    ͷத LT
    ͷ
    TFDSFUT EFQMPZ CMVF
    
    HSFFO SPMMJOH શ෦"84 ΄΅Ϛωʔδυ Πϯϑϥ 1FPQMF
    ଆʹऩଋ͍ͤͨ͞

38. ©2018 Wantedly, Inc. Ҡߦ࡞ۀ

39. ©2018 Wantedly, Inc. NBTUFS
    QVTI
    ͔Β
    EFQMPZ
    ׬ྃ·Ͱ
    ෼ ཁ݅ 5SBWJT
    Ͱ
    CVJME
    ͍ͨ͠

40. ©2018 Wantedly, Inc. Ͱ͖ΔݶΓͷ͜ͱΛ build ҎલʹऴΘΒͤΔΑ͏ʹ ൿಗ৘ใΛҰ੾ೖΕͳ͍
    EPDLFS
    JNBHF
    ʹมߋ $*
    Ͱ
    CVJME
    ͢ΔͨΊʹ *OUFSNFEJBUF
    Λ࡮৽ͯ͠શ؀ڥ౷ҰͷJNBHF ΄ͱΜͲͷ͕࣌ؒ

    asset sync -> gem ʹ patch ͦͷଞॾʑ…

41. ©2018 Wantedly, Inc. 1VTI
    ͔Β10෼Ͱ
    build ׬ྃ ෼ͰऴΘΒ͍͕ͤͨҰ୴͸े෼Ͱ͋Δͱ൑அ Ͱ΍ͬͯ΋

42. ©2018 Wantedly, Inc. ෼ͰऴΘΒ͍͕ͤͨҰ୴͸े෼Ͱ͋Δͱ൑அ ΋ͬͱؤுΔํ๏ $JSDMF$*
    ʹҠߦ͢Δ 'SPOUFOEΛผϨϙδτϦʹ͢Δ Ϋϥελ಺Ͱ
    CVJME
    ͢Δ -BZFS
    DBDIF
    ͕࢖͑ͯૣ͍͸ͣ Ͱ΍ͬͯ΋

    1VTI
    ͔Β10෼Ͱ
    build ׬ྃ

43. ©2018 Wantedly, Inc. %FQMPZ
    
    3PMMCBDL
    ଎౓Λམͱͨ͘͠ͳ͍

44. ©2018 Wantedly, Inc. 3PMMCBDL
    ͷ଎౓͕མͪΔͷ͸ఘΊΔ #MVF
    
    HSFFO
    ͱಉ͡଎౓͸
    CMVF
    HSFFO
    ͡Όͳ͍ͱແཧ Ͳ͏ͤ
    SPMMCBDL
    ͢Δ͔ͷ൑அࣗମʹ෼࢖͍ͬͯΔͷͰ͘͢͝େࣄͰ͸ͳ͍ʁ

45. ©2018 Wantedly, Inc. %FQMPZ
    
    3PMMCBDL
    ΛԿ౓΋ߦͬͯ଎౓ܭଌ #MVF
    HSFFO
    Λ΍ΊΔͨΊʹ ΞϓϦέʔγϣϯνʔϜͱΑ͘σΟεΧογϣϯΛͨ͠ ݁ہίϛϡχέʔγϣϯ͕େࣄ Կ͕Ͱ͖ͯԿ͕Ͱ͖ͳ͍ͷ͔Λ໌֬ʹ

46. ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂ

47. ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂͷલʹ

48. ©2018 Wantedly, Inc. ։ൃݕূ؀ڥ͚ͩҠߦͨ͠ Ҡߦ࡞ۀࢼݧ ৽͍͠ΦϖϨʔγϣϯͷ໰୊఺Λચ͍ग़͢

49. ©2018 Wantedly, Inc. ͋͞Ҡߦͩʂ ͜ΜͲͦ͜

50. ©2018 Wantedly, Inc. 1SPE
    ΛϑϦʔζͯ྆͠ํʹ
    EFQMPZ چ"-# ৽"-# JOUFSOBM DNS

51. ©2018 Wantedly, Inc. چ"-# ৽"-# JOUFSOBM DNS Ұ෦τϥϑΟοΫΛ
    OHJOY
    ͔Βྲྀ͢ ༧Ί༻ҙ͓͍ͯͨ͠ύλʔϯʹҰॠͰมߋͰ͖ΔΑ͏ʹ͓͍ͯͨ͠ BCPSU
    ͕༰қ

52. ©2018 Wantedly, Inc. શτϥϑΟοΫΛ
    LT
    ʹྲྀ͢ چ"-# ৽"-# JOUFSOBM DNS

53. ©2018 Wantedly, Inc. Ұ୴
    BCPSU
    ͨ͠ چ"-# ৽"-# JOUFSOBM DNS ͜ͷ࣌গ͠ࣄނΛى͕ͨ͜͠ɺҰॠͰ
    BCPSU
    Ͱ͖Δ
    QBUI
    Λ༻ҙ͓͍ͯͨ͠ͷ͸ྑ͔ͬͨ

54. ©2018 Wantedly, Inc. ࢓੾Γ௚͠ چ"-# ৽"-# JOUFSOBM DNS

55. ©2018 Wantedly, Inc. %/4੾Γସ͑ چ"-# ৽"-# JOUFSOBM DNS

56. ©2018 Wantedly, Inc. چ"-# ৽"-# JOUFSOBM DNS ׬ྃʂʂʂ

57. ©2018 Wantedly, Inc. DNS ׬ྃʂʂʂ ৽"-#

58. ©2018 Wantedly, Inc. %/4੾Γସ͑ ৽ALB͕ड͚ͨϦΫΤετͷׂ߹ DNS੾ସ

59. ©2018 Wantedly, Inc. %/4੾Γସ͑ ৽ALB͕ड͚ͨϦΫΤετͷׂ߹ DNS੾ସ

60. ©2018 Wantedly, Inc. ίϚϯυ࣮ߦ͕ૣ͘ͳͬͨ %FQMPZ
    ͕ૣ͘ͳͬͨ ݁Ռ DBෛՙ͕ݮͬͨ Autoscale͕Ͱ͖ΔΑ͏ʹͳͬͨ ϫʔετ෼ Ұൠతʹ͸෼ఔ౓

    CMVFHSFFO
    Ͱͳ͘ͳͬͨͨΊ

61. ©2018 Wantedly, Inc. ϑϩʔ͕͋ΔͱҠߦ͠΍͍͢ ֶΜͩ͜ͱ ϑΥʔϧόοΫઌ͸৻ॏʹ Ϛωʔδυ͸ָ ࢼݧΛͯ͠΋ࣄނ͸ى͜Δ ఘΊΔ͜ͱ΋େࣄ 1PTUNPSUFN
    େࣄ

    ࣄނΛىͨ͜͠ຊਓ΋ཱͪ௚ΕΔ

62. ©2018 Wantedly, Inc. We’re Hiring .JDSPTFSWJDFT
    .POEBZ HJUIVCDPNXBOUFEMZNJDSPTFSWJDFT@NPOEBZ