Adventure around Kubernetes at Chatwork

by Ryo Sakamoto

Slide 1

Slide 1 text

,VCFSOFUFTΛΊ͙Δ๯ݥ +"846(ίϯςφࢧ෦ $IBUXPSL43&νʔϜࡔຊ

Slide 2

Slide 2 text

Slide 3

Slide 3 text

▸ ϏδωενϟοταʔϏε ▸ άϧʔϓνϟοτɺλεΫ؅ཧɺ  ϑΝΠϧڞ༗ɺϏσΦɾԻ੠௨࿩ ▸ ಋೖاۀ 246,000 ࣾ  (2019೥12݄ݱࡏ)

Slide 4

Slide 4 text

Slide 5

Slide 5 text

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Slide 8

Slide 8 text

© Chatwork &,4ͷಋೖ 8 ▸ EKSར༻લ͸kube-awsͱ͍͏ͷΛར༻ͯ͠, ࣗલͰϗεςΟϯά ▸ https://github.com/kubernetes-incubator/kube-aws ▸ EKSͷొ৔ʹΑΓkube-awsͷϢʔβݮ… ▸ ࡉ͔͍ػೳʹόά͕ଟ͘, దٓPRରԠͭͭ͠΋ࣗલύονӡ༻ ▸ EKSࣗମ͸ίϯτϩʔϥʔ෦෼͚ͩͰ, ࢖͍ʹ͍͘ͳ͊…ͱϓϨϏϡʔͰ ࢥ͍͕ͬͯͨeksctlʹΑΓͦ͏͍͏ͱ͜Ζ͕·Δͬͱָʹ

Slide 9

Slide 9 text

Slide 10

Slide 10 text

ΦϑΟγϟϧͳͷͰɺEKS৽͍͠ػೳ͸େମAWSͷൃදͱಉ࣌ʹରԠ͞Ε͍ͯΔ

Slide 11

Slide 11 text

© Chatwork FLTDUMͱLVCFBXTͷൺֱ 11 ▸ kube-aws ▸ Ϋϥελͷߏ੒͸updateՄೳ ▸ ϊʔυάϧʔϓͷઃఆมߋΛ͍ͨ͠৔߹͸ɺچϊʔυάϧʔϓ͔Β ϩʔϦϯάΞοϓσʔτ(CloudFormationͷupdateͳͷͰ...) ▸ EKSΛར༻͢Δ΋ͷͰ͸ͳ͘, controller, etcdؚΊͯ࡞੒ ▸ ੜKubernetesͳͷͰɺΧελϚΠζੑ͸͔ͳΓߴ͍ ▸ KubernetesͷઃఆϑΝΠϧ͕௕େͳyamlʹͳΔ(Productionͷ686ߦ)

Slide 12

Slide 12 text

© Chatwork FLTDUMͱLVCFBXTͷൺֱ 12 ▸ ௕ظతʹݟΔͱ, eksctlͷ΄͏͕ϝϦοτ͸େ͖͍ ▸ Ϣʔβೝূ͕iamͰͰ͖ͨΓ, podʹiam roleΛ෇͚ΒΕͨΓ, eksͦͷ΋ͷ ͷϝϦοτ΋େ͖͍ ▸ iam role for pod͸ͱͯ΋خ͍͠ ▸ kube2iam, kiam, kube-aws-iam-controller͢΂ͯӡ༻͖ͯͨ͠ܦݧ ▸ ͜Ε͸͜ΕͰखܰ͞΋͋Δ͚ΕͲ

Slide 13

Slide 13 text

ΫϥελΛ࡞Δ͚ͩͳΒ͜ΕͰOK  ͔͠͠ɾɾ

Slide 14

Slide 14 text

Slide 15

Slide 15 text

͜ΕΛ৐Γӽ͑Δӡ༻ɾ࢓૊Έ͕ඞཁ

Slide 16

Slide 16 text

Slide 17

Slide 17 text

Slide 18

Slide 18 text

© Chatwork FLTDUM͚ͩͰ͸ͳͥͩΊͳͷ͔ 18 ▸ eksctl͚ͩͰ΋े෼ͳ৔߹΋ଟ͍ ▸ ͱΓ͋͑ͣࢼݧ؀ڥΛىಈͤ͞Δʹ͸ίϚϯυ͚ͩͰे෼ ▸ ͨͩ͠ࡉ͔͍ઃఆΛ͠Α͏ͱ͢Δͱ, ϑΝΠϧͰͷઃఆͷ΄͏ָ͕ ▸ sampleҎ֎ͷυΩϡϝϯτ͸ͳ͍ͷͰ, ஫ҙ ▸ https://github.com/weaveworks/eksctl/blob/master/pkg/apis/eksctl.io/ v1alpha5/types.go ͔Β୳͔͢͠ͳ͍ͱ͖΋͋Δ ▸ ઃఆΛ౉ͤΔVariantͰϥοϓͯ͠, eksctl༻ઃఆϑΝΠϧͷ஋Λஔ׵

Slide 19

Slide 19 text

# Ϋϥελ໊ # αϒωοτ # cluster-autoscalerͷiam roleͭ͘Δ # eksຊମ(ϊʔυάϧʔϓআ͘)ͷeksctlͷઃఆϑΝΠϧ

Slide 20

Slide 20 text

© Chatwork FLTDUM͚ͩͰ͸ͳͥͩΊͳͷ͔ 20 ▸ Ϋϥελ࡞Δ͚ͩͳΒ, ୯ମར༻Ͱ΋͍͚ͳ͘͸ͳ͍ ▸ FluxͩΕ͕ೖΕΔͷ໰୊ɺ΁ͷରԠ ▸ Flux ... GitOps tool ޙड़͠·͢ ▸ GitOpsͰઃఆͷ൓ө΍ΞϓϦέʔγϣϯͷσϓϩΠΛ͢Δ ▸ Ͱ͸, GitOpsΛ࣮ݱ͢ΔͨΊͷπʔϧ͸Ұମ୭͕ೖΕΔͷ͔...! ▸ GitOps ͸Kubernetes಺෦ʹ࣮ݱ͢ΔͨΊͷΞϓϦέʔγϣϯ͕͍Δ

Slide 21

Slide 21 text

© Chatwork 21 ▸ eksctlίϚϯυͰFluxΛ͍ΕΔ͜ͱ͸Մೳ ▸ υΩϡϝϯτϕʔεͰ΋ରԠͰ͖Δ͕ɺԼهͷϑϩʔΛ·ͱΊ͍ͨ ▸ eksctlͷઃఆϑΝΠϧ࡞੒ ▸ eksctl create cluster ▸ eksctl enable repo ... (Fluxͷinstall) ▸ ͜ΕΒΛvariantͰλεΫ੍ޚ(Infrastructure as codeͷҰ؀) FLTDUM͚ͩͰ͸ͳͥͩΊͳͷ͔

Slide 22

Slide 22 text

Slide 23

Slide 23 text

Slide 24

Slide 24 text

͜͜·Ͱ͸Kubernetesࣗମͷߏங ͔͜͜Β͸Kubernetes಺෦ͷߏங

Slide 25

Slide 25 text

© Chatwork 25 ▸ https://toris.io/2019/12/what-i-think-about-when-i-think-about-kubernetes- and-ecs/ ▸ (ͳΜ͔ଜ্य़थͷλΠτϧͬΆ͍...) ▸ EKS͸ϚωʔδυͰ͕͢ɺϚωʔδϝϯτίϯιʔϧͰ࡞੒͚ͨͩ͠Ͱ͸ ΄΅Կ΋Ͱ͖·ͤΜ ▸ ϊʔυάϧʔϓ(࣮ࡍʹίϯςφ͕ಈ͘؀ڥ)Λ࡞ͬͯ΋ɺ·ͩϩά΋ૹΕ ͣɺϝτϦΫε΋ݟΕͣɺͰ͢ ▸ ͳͷͰɺEKSͰ΋؅ཧܥͷΞϓϦέʔγϣϯ(ex. Datadog)ͷӡ༻͕ඞਢ &$4ͱͷҧ͍

Slide 26

Slide 26 text

© Chatwork IFMN IFMNpMF 26 ▸ helm ▸ The package manager for Kubernetes ▸ KubernetesʹΞϓϦέʔγϣϯΛೖΕΔࡍʹɺඞཁͳ΋ͷΛύοέʔδ Խͯ͠ɺೖΕͯ͘ΕΔπʔϧ ▸ ChatworkͰ͸΄ͱΜͲΞϓϦέʔγϣϯΛHelmΛར༻ͯ͠σϓϩΠ ▸ Chartࣗମ͸؀ڥʹґଘ͠ͳ͍ʂ ▸ σϓϩΠͷ࠶ݱੑ(ࣗલKubernetes(kube-aws) -> EKSͰ΋ͦͷ··)

Slide 27

Slide 27 text

© Chatwork IFMN IFMNpMF 27 ▸ helmﬁle ▸ helm chartΛͲ͏΍ͬͯద༻͢Δͷ͔Λએݴతʹॻ͚Δ ▸ helmͰσϓϩΠ͢ΔࡍͷΞϓϦέʔγϣϯͷઃఆ஋΍ɺhelmίϚϯυ ͷΦϓγϣϯͳͲΛҰׅͰهࡌ͠ɺhelmͰσϓϩΠͰ͖Δπʔϧ ▸ https://github.com/helm/charts/tree/master/stable ▸ ͍ΖΜͳΞϓϦέʔγϣϯͷhelm chart ▸ https://developers.freee.co.jp/entry/2019/12/03/122657

Slide 28

Slide 28 text

© Chatwork ͳͥIFMNpMF͕ඞཁͳͷ͔ 28 ▸ ΞϓϦέʔγϣϯ͝ͱʹ࢖͍͍ͨΦϓγϣϯ(΍લॲཧ)͕มΘΔ ▸ ྫ͑͹)ΞϓϦέʔγϣϯಉ࢜Ͱґଘؔ܎ͷ͋Δ΋ͷ, ͳͲ ▸ helmͷઃఆϑΝΠϧ͕؀ڥ͝ͱ(test, stg, prod)ͰมΘΔ ▸ ྫ͑͹)AWSͷΞΧ΢ϯτ͝ͱʹҧ͏, ϩάͷు͖ग़͠ઌ͕ҧ͏ ▸ go template ΋࢖͑ΔͷͰ, helmͷઃఆ஋Λಈతʹੜ੒͢Δ͜ͱ΋Մೳ ▸ ChatworkͰ͸ɺ͞ΒʹVariantͰhelmﬁleΛแΈࠐΈ

Slide 29

Slide 29 text

Slide 30

Slide 30 text

Slide 31

Slide 31 text

Slide 32

Slide 32 text

͜͜·Ͱ͸Kubernetesӡ༻ ͔͜͜Β͸CI/CD

Slide 33

Slide 33 text

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Slide 36

Slide 36 text

Slide 37

Slide 37 text

Slide 38

Slide 38 text

© Chatwork $IBUXPSLͷ(JU0QTߏ૝ 38 ▸ Flux + Argo CDͷϋΠϒϦουߏ੒ ▸ Flux ▸ ѻ͑Δͷ͸ݪଇmanifestͷΈͰ, 1ﬂux - 1repo - 1branch ▸ Argo CDࣗ਎ͷΠϯετʔϧ΍namespace, aws-auth(eksಠࣗ)ͳͲ manifestద༻Ͱ͋·Γมߋ͕ͳ͍΋ͷ ▸ Argo CD ▸ GUI͕͋ͬͨΓ, helmʹରԠ͍ͯͨ͠Γɺͱػೳ͕ଟ͘࢖͍΍͍͢