k8s - Kubernetes 8 Factors

by Shimpei Otsubo

Slide 1

Slide 1 text

Slide 2

Slide 2 text

Slide 3

Slide 3 text

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Slide 10

Slide 10 text

Slide 11

Slide 11 text

Slide 12

Slide 12 text

Slide 13

Slide 13 text

©2019 Wantedly, Inc. എܠ Wantedly ʹ͓͚Δ Kubernetes App Engineer ͱͷૄ݁߹Λ໨ࢦ͍ͯ͠Δ ΋ͪΖΜݱ࣌఺Ͱ͸ίϛϡχέʔγϣϯ͸ॏཁ Ͱ΋ࣾ಺ Heroku ʹͳΓ͍ͨͷͰ஌Βͳͯ͘΋ྑ͍ͱ͜Ζ͸໨ࢦ͍ͨ͠ ͲΜͳίϯςφΛಈ͔͍ͯ͠Δͷ͔ΠϯϑϥνʔϜ͕஌Βͳͯ͘ྑ͍ঢ়ଶ αʔϏε༻Ϧιʔε͸είʔϓ֎ ΋ͪΖΜΠϯϑϥνʔϜͰਪ঑/ېࢭͳͲͷضৼΓ͸͢Δ Ͱ΋Ϋϥελ؅ཧʹ͓͍ͯ͸ࣗ༝ͱͯ͠࢒͢෦෼ αʔϏε༻ʹϦιʔε؅ཧ͸ΞϓϦέʔγϣϯΤϯδχΞʹ೚ͤΔ

Slide 14

Slide 14 text

Slide 15

Slide 15 text

Slide 16

Slide 16 text

©2019 Wantedly, Inc. എܠ 2018೥ʹͳΓ etcd ͷ version Λ্͛Δඞཁੑ͕ग़͖ͯͨ 2015೥͔Β ͏ͪΠϯϑϥ͸6ਓ Wantedly Ͱىͬͨ͜͜ͱ k8s Λॳظ͔Βݕূ/ӡ༻͖ͯͨ͠ v1.1ͩͬͨ ݱࡏ͸ 40ਓͷΤϯδχΞͰ߹ܭ 300 ͷ namespace Λ؅ཧ͍ͯ͠Δ ͜ͷͨΊॳظͷ component ͕࢒ͬͯ͠·͍ͬͯΔ in-place ͷ upgrade ͸ϦεΫ͕େ͖͍ ࠶࡞੒ͩʂ ʮ৽͍͠ͷ࡞͔ͬͨΒͦΕͧΕҠߦ͠ͱ͍ͯʯ͸ඇݱ࣮త

Slide 17

Slide 17 text

Slide 18

Slide 18 text

Slide 19

Slide 19 text

©2019 Wantedly, Inc. https://12factor.net/ja/ https://content.pivotal.io/ebooks/beyond-the-12-factor-app THE TWELVE FACTORS CloudNative ࣌୅ͷվྑ൛ Application αΠυͰ͸ʮϙʔλϏϦςΟΛอͭݪଇ͕஌ΒΕ͍ͯΔʯ ͔͠͠Πϯϑϥ͕࣋ͭ΂͖ੑ࣭͸Α͘·ͱΊΒΕͯ͸͍ͳ͍ ࡞ͬͯ͠·͓͏ʂ

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Slide 22

Slide 22 text

©2019 Wantedly, Inc. ίʔυϕʔε ґଘએݴ ίʔυϕʔε಺֎ʹอ؅ͯ͠஫ೖ͢Δ ୯Ұํ޲Ϗϧυ ؀ڥҰக ඇ໊લ؅ཧ όοΫΞοϓ ઃఆ͸͢΂ͯίʔυԽ͢Δ ґଘ͢ΔίϯϙʔωϯτͷWFSTJPOએݴ͢Δ ઃఆ෼཭ ίʔυϕʔε͔ΒΫϥελ΁ͷҰํ޲ͷΈͷ൓ө ࠩҟ͸ઃఆʹཹΊΔ ϥϕϧΤΠϦΞεΛ࢖͏ ϦιʔεఆٛΛ෮ݩՄೳͳঢ়ଶͰอ؅͢Δ 12-factor app ͱڞ௨ New!! ॏෳܽଛ؅ཧ ॏෳͱܽଛͷ͍ͣΕ͔ͷΈΛकΔ

Slide 23

Slide 23 text

Slide 24

Slide 24 text

©2019 Wantedly, Inc. શ͘ಉ͡ΫϥελΛ࡞ΔͨΊʹ͸ґଘΛ໌֬ʹએݴ͢Δ͜ͱ͕ॏཁ ར༻͢ΔΦʔέετϨʔγϣϯπʔϧͰՄೳͳݶΓWFSTJPOΛݻఆ͢Δ - etcdMembers: - instanceGroup: master-ap-northeast-1c name: c - instanceGroup: master-ap-northeast-1b name: b - instanceGroup: master-ap-northeast-1c-2 name: c2 name: main version: 3.0.17 ґଘએݴ ґଘ͢ΔίϯϙʔωϯτͷWFSTJPOએݴ͢Δ 5IF5XFMWF'BDUPS"QQͰ͸ʮ**ґଘؔ܎ʯʹ૬౰ ؅ཧ͢Δ΋ͷ ؅ཧ͠ͳ͍΋ͷ αʔϏε༻ͷ … ࠓΫϥελΛࣦͬͯ΋ಉ͡ΫϥελΛ͙͢ʹͭ͘ΕΔ͔ʁ कΓ͍ͨ͜ͱ

Slide 25

Slide 25 text

©2019 Wantedly, Inc. Code Config Deploy + = THE TWELVE FACTOR APP Code Config Cluster + = Kubernetes 8 factors ઃఆ෼཭ ίʔυϕʔε಺֎ʹอ؅ͯ͠஫ೖ͢Δ 5IF5XFMWF'BDUPS"QQͰ͸ʮ***ઃఆʯʹ૬౰ ؀ڥࠩҟΛ໌ࣔతʹએݴ͢Δ ྫ͑͹ઃఆϑΝΠϧͷHFOFSBUPSΛ࡞Δ wantedly Ͱ͸࣮͸ఘΊ͍ͯΔ $POGJH͕ߴʑ਺ඦߦ 3FWJFXͰಉҰมߋͰ͋Δ͜ͱΛอূ ಉҰεϖοΫͷݕূ؀ڥΛ؆୯ʹݐͯΒΕΔ͔ʁ कΓ͍ͨ͜ͱ কདྷతʹਐΈ͍ͨಓ

Slide 26

Slide 26 text

©2019 Wantedly, Inc. શͯͷมߋʹSFWJFXΛཁٻ͢Δ Ϋϥελͱઃఆʹࠩ෼͕ͳ͍͔ʁ Ϋϥελͷঢ়ଶ͔ΒίʔυϕʔεʹࣗಈͰ13Λ࡞੒͢ΔΞϓϩʔν΋͋Δ ٞ࿦͕෼͔ΕΔͱ͜Ζ कΓ͍ͨ͜ͱ ࣗಈEFQMPZͩͱߋʹྑ͍ Code Cluster ୯Ұํ޲Ϗϧυ 5IF5XFMWF'BDUPS"QQͰ͸ʮ7ϏϧυɺϦϦʔεɺ࣮ߦʯʹ૬౰ ίʔυϕʔε͔ΒΫϥελ΁ͷҰํ޲ͷΈͷ൓ө $MVTUFSΛJNNVUBCMFʹ͢Δ ίʔυϕʔεͷมߋͳ͠ʹΫϥελΛม͑ͳ͍ XBOUFEMZͰ͸·ͩख࡞ۀ github.com/fluxcd/flux

Slide 27

Slide 27 text

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

©2019 Wantedly, Inc. 8BOUFEMZͰ͸ԼͷΑ͏ͳΫϥελ໊ʹ͍ͯͨͨ͠Ίେมͩͬͨ ඇ໊લ؅ཧ ϥϕϧΤΠϦΞεΛ࢖͏ prod.example.com api.prod.example.com dev.example.com api.dev.example.com stage.example.com api.stage.example.com ࠓޙ΋มΘΒͳͦ͞͏ͳงғؾ͕͋Δ Τϥʔݩ͕QSPEFYBNQMFDPNͷ৔߹͸QBHFSEVUZ ໊લʹґଘͨ͠ϩδοΫΛ࡞ͬͯ͠·ͬͨ Τϥʔݩ͕TUBHFFYBNQMFDPNͷ৔߹͸໷ؒ͸ແࢹ ΦʔέετϨʔγϣϯπʔϧ͕%/4Ϩίʔυ·Ͱ࡞੒ͯ͠͠·͏ Ҡߦ࣌ʹਖ਼͍͠BQJαʔόʔ͕ߦํෆ໌ʹͳΔ ڞ௨ͷূ໌ॻΛ༻͍ͳ͍ͱҠߦ࣌ʹπʔϧ͕յΕͯ͠·͏ Ϋϥελ໊ BQJαʔόʔͷ%/4Ϩίʔυ

Slide 31

Slide 31 text

©2019 Wantedly, Inc. ґଘͰ͖ͳͦ͞͏ͳ໊લϨίʔυʹมߋ ඇ໊લ؅ཧ ϥϕϧΤΠϦΞεΛ࢖͏ prod201907.example.com prod201907.api.example.com dev201907.example.com dev201907.api.example.com stage201907.example.com stage201907.api.example.com ௚઀ͷࢀরΛආ͚Δ ϩδοΫͷ۠ผ͸ϥϕϧΛ༻͍Δ ໊લʹҙຯΛ࣋ͨͤͳ͍ FOWQSPEVDUJPOTUBHFEFW *.api.example.com ͷূ໌ॻΛڞ௨ͯ͠ར༻Ͱ͖Δ ґଘͰ͖ͳͦ͞͏ͳΞυϨεΛ࠾༻ FOWBQJFYBNQMFDPN͔Β$/".&ΛுΔ͜ͱͰґଘπʔϧͷมߋΛෆཁʹͰ͖Δ Ϋϥελ໊ BQJαʔόʔͷ%/4Ϩίʔυ

Slide 32

Slide 32 text

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

©2019 Wantedly, Inc. ґଘͰ͖ͳͦ͞͏ͳ໊લϨίʔυʹมߋ ඇ໊લ؅ཧ ϥϕϧΤΠϦΞεΛ࢖͏ prod201907.example.com prod201907.api.example.com dev201907.example.com dev201907.api.example.com stage201907.example.com stage201907.api.example.com ௚઀ͷࢀরΛආ͚Δ ΫογϣϯͱͳΔϨΠϠʔΛඞͣڬΉ Ωʔ͸௚઀ࢀর͠ͳ͍͜ͱ (BUFXBZͱ͍͏ղܾࡦ΋͋Δ *.api.example.com ͷূ໌ॻΛڞ௨ͯ͠ར༻Ͱ͖Δ ґଘͰ͖ͳͦ͞͏ͳΞυϨεΛ࠾༻ {env}.api.example.com ͔Β CNAME ΛுΔ͜ͱͰґଘπʔϧͷมߋΛෆཁʹͰ͖Δ Ϋϥελ໊ BQJαʔόʔͷ%/4Ϩίʔυ ͜ͷߟ͑͸LT௨ͯ͠ڞ௨Ͱ͋Δͱߟ͍͑ͯΔ

Slide 36

Slide 36 text

©2019 Wantedly, Inc. ໊લΛ௚઀ࢀর͠ͳ͍ߟ͑ͷଞͷదԠྫ ඇ໊લ؅ཧͷ༨ஊ ΫϥελͷҠߦʹࡍͯ͠શϩʔυόϥϯαʔ͕࠶࡞੒͞ΕΔ ϩʔυόϥϯαʔͷ࡞੒ΛτϦΨʔʹ$%/Λ࡞Δ࢓૊Έ͕͋ͬͨ Ҏ߱ڬΉΑ͏ʹͨ͠ $/".&ͷϫϯΫογϣϯ͕ݩ͔Β͋Ε͹Ҡߦ͸ָͩͬͨ ࣮ࡍʹ͸ख࡞ۀͰ΍͍ͬͯΔϑϩʔ͕ͩͬͨࣗಈͰ΋ಉ͡ όοΫΤϯυͷͭͳ͗௚͕͠େมʹͳͬͨ CDN Deployment Load Balancer

Slide 37

Slide 37 text

©2019 Wantedly, Inc. ໊લΛ௚઀ࢀর͠ͳ͍ߟ͑ͷଞͷదԠྫ ඇ໊લ؅ཧͷ༨ஊ ΫϥελͷҠߦʹࡍͯ͠શϩʔυόϥϯαʔ͕࠶࡞੒͞ΕΔ ϩʔυόϥϯαʔͷ࡞੒ΛτϦΨʔʹ$%/Λ࡞Δ࢓૊Έ͕͋ͬͨ Ҏ߱ڬΉΑ͏ʹͨ͠ $/".&ͷϫϯΫογϣϯ͕ݩ͔Β͋Ε͹Ҡߦ͸ָͩͬͨ ࣮ࡍʹ͸ख࡞ۀͰ΍͍ͬͯΔϑϩʔ͕ͩͬͨࣗಈͰ΋ಉ͡ όοΫΤϯυͷղܾ͕େมʹͳͬͨ CDN CNAME Load Balancer Deployment

Slide 38

Slide 38 text

©2019 Wantedly, Inc. όοΫΞοϓ ϦιʔεఆٛΛ෮ݩՄೳͳঢ়ଶͰอ؅͢Δ ͦΕͧΕʹҠߦͯ͠΋Β͏ϑϩʔΛऔΕͳ͍͜ͱ͕͋Δ ํ๏͸໰Θͳ͍ Ұׅͯ͠Ϛωδϝϯτͯ͋͛͠Δ͜ͱΛ೦಄ʹஔ͘ ΞϓϦέʔγϣϯΤϯδχΞͷෛ୲ͳ͠ʹҠߦ … … … ίί 8BOUFEMZͰ͸IFQUJPWFMFSPΛ࠾༻ EFQMPZͷͨͼʹZBNMΛߋ৽BQQMZ͢Δϑϩʔ΋औΕΔ 8BOUFEMZͰ͸ࢿ࢈తʹZBNMͱຊ൪ঢ়ଶʹࠩ෼͕ग़Δ͜ͱΛڐ༰ THE TWELVE FACTOR APP Ͱ͸ѻΘͳ͍ϨΠϠ ҰൠͷΞϓϦέʔγϣϯͰ͸%#ͷόοΫΞοϓʹ૬౰͢Δ Ϋϥελ؅ཧͰ͸ආ͚ΒΕͳ͍෦෼

Slide 39

Slide 39 text

©2019 Wantedly, Inc. Ϋϥελ಺ͷεςʔτϑϧͳ΋ͷ͕ଘࡏ͢ΔͱҠߦ͕ඇৗʹେม SFBESFQMJDBΛ৽نΫϥελͰཱͯͯঢ֨ εςʔτϑϧͱ͸ʁ +PC΍$SPO+PC΋͋Δҙຯεςʔτϑϧ 4UBUFGVMTFU͸΋ͪΖΜεςʔτϑϧ ࣮ߦ͔ͨ͠ʁͷTUBUF͕ඇৗʹॏཁͰ͋ΔͨΊ ॏෳܽଛ؅ཧ ॏෳͱܽଛͷ͍ͣΕ͔ͷΈΛकΔ 8BOUFEMZͰ͸εςʔτϑϧηοτͷຊ൪ར༻Λجຊతʹېࢭ͍ͯ͠Δ ࠷ѱσʔλ͕ඈͿ͜ͱΛڐ༰Ͱ͖Δ৔߹͸PL ॏෳͱܽଛͷ͍ͣΕ͔ͷΈΛकΔ Ҡߦ࣌ʹEFMFUFͱDSFBUFΛͲͷॱͰߦ͏͔ʁ ͍ͣΕͷॱͰ΋QPEͷॏෳܽଛͷ͍ͣΕ͔͸ൃੜ͠͏Δ

Slide 40

Slide 40 text

©2019 Wantedly, Inc. ॏෳܽଛ؅ཧ ॏෳͱܽଛͷ͍ͣΕ͔ͷΈΛकΔ at-most-one at-least-one ࠷େ1͔ͭ͠ଘࡏͯ͠͸ߦ͚ͳ͍ resource ࠷௿1ͭҎ্͸ଘࡏ͠ͳ͍ͱ͍͚ͳ͍ resource Muti-Cluster Strategy Λఆٛ ͜ͷ strategy ͝ͱʹҠߦ࣌ͷѻ͍Λม͑ΔΑ͏ʹͨ͠ ෳ਺Ϋϥελ͕ଘࡏ͢Δͱ͖ʹ Value ྫ Deployment(worker), CronJob, StatefulSet Deployment (web), ConfigMap, Service at-most-one delete and create at-least-one create and delete ΑΓཧ૝ͷੈք શͯͷ resource ͰॏෳΛڐ༰͢Δ શͯͷ Job Ͱႈ౳ੑΛ୲อ͢Δ Wantedly Ͱ͸ະ࣮ݱ Kubernetes Ͱ State Λ࣋ͬͨ resource Λ্ख͘ѻ͏ख๏΋͋Δ ͨͩʮ୭͕ͲΜͳίϯςφΛཱͯͯ΋͍͍ʯΛอͭʹ͸͋Δఔ౓ͷׂ੾΋ඞཁ

Slide 41

Slide 41 text

©2019 Wantedly, Inc. ॏෳഉআ ΫϥελͷҠߦ࣌ʹ ॏෳͱܽଛͷ͍ͣΕ͔ͷΈΛकΔ SFBESFQMJDBΛ৽نΫϥελͰཱͯͯঢ֨ εςʔτϑϧͱ͸ʁ +PC΍$SPO+PC΋͋Δҙຯεςʔτϑϧ 4UBUFGVMTFU͸΋ͪΖΜεςʔτϑϧ ෳ਺ΫϥελͰಉ࣌ʹଘࡏͯ͠Α͍͔ʁ͕Ұͭͷ൑அج४ ࣮ߦ͔ͨ͠ʁͷTUBUF͕ඇৗʹॏཁͰ͋ΔͨΊ BUNPTUPOF BUMFBTUPOF ෳ਺Ϋϥελ͕ଘࡏͯ͠΋͔ͭ͠ଘࡏͯ͠͸ߦ͚ͳ͍SFTPVSDF ෳ਺Ϋϥελ͕ଘࡏͯ͠΋ͭҎ্͸ଘࡏ͠ͳ͍ͱ͍͚ͳ͍SFTPVSDF .VUJ$MVTUFS4USBUFHZΛఆٛ ͜ͷTUSBUFHZ͝ͱʹҠߦ࣌ͷѻ͍Λม͑ΔΑ͏ʹͨ͠

Slide 42

Slide 42 text

Slide 43

Slide 43 text

©2019 Wantedly, Inc. heptio/ark(౰࣌) Ͱ backup ࣄલ ಉҰઃఆͰΫϥελΛ্ཱͪ͛Δ at-least-one resource ͷΈΛ ৽Ϋϥελʹ restore at-most-one resource ΛݱΫϥελ͔Β delete at-most-one resource Λ৽Ϋϥελʹ restore DNS Λ੾Γସ͑ ౰೔ ؂ࢹΛશͯϥϕϧ؅ཧʹม͑Δ ֤ॴʹΤΠϦΞεΛࠩ͠ࠐΉ ༨ஊ ࣮ࡍͷҠߦϑϩʔ Ops πʔϧ༻ͷূ໌ॻࠩ͠ସ͑ 1ϲ݄ఔ౓ 1೔

Slide 44

Slide 44 text

Slide 45

Slide 45 text