Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Adventure around Kubernetes at Chatwork
Search
Ryo Sakamoto
January 23, 2020
Technology
6
7.7k
Adventure around Kubernetes at Chatwork
Ryo Sakamoto
January 23, 2020
Tweet
Share
More Decks by Ryo Sakamoto
See All by Ryo Sakamoto
いろいろなAWSアカウントのArgo CDを統合した話
cwsakamoto
1
880
ArgoCDとGitHub Self Hosted Runnerを使って リリース時間を1/4にした話
cwsakamoto
0
2.1k
チャットワークにおけるKubernetesOnAWS.pdf
cwsakamoto
0
86
チャットワークにおけるKubernetesOnAWS.pdf
cwsakamoto
0
77
Kubernetes on AWS at Chatwork
cwsakamoto
0
1.7k
Other Decks in Technology
See All in Technology
JAWS DAYS 2025 アーキテクチャ道場 事前説明会 / JAWS DAYS 2025 briefing document
naospon
0
2.5k
Autonomous Database Serverless 技術詳細 / adb-s_technical_detail_jp
oracle4engineer
PRO
17
45k
ABWG2024採択者が語るエンジニアとしての自分自身の見つけ方〜発信して、つながって、世界を広げていく〜
maimyyym
1
190
OSS構成管理ツールCMDBuildを使ったAWSリソース管理の自動化
satorufunai
0
650
自分だけの仮想クラスタを高速かつ効率的に作る kubefork
donkomura
0
110
実は強い 非ViTな画像認識モデル
tattaka
3
1.3k
いまからでも遅くない!コンテナでWebアプリを動かしてみよう!コンテナハンズオン編
nomu
0
170
JavaにおけるNull非許容性
skrb
2
2.7k
Introduction to OpenSearch Project - Search Engineering Tech Talk 2025 Winter
tkykenmt
2
120
Охота на косуль у древних
ashapiro
0
110
DevinでAI AWSエンジニア製造計画 序章 〜CDKを添えて〜/devin-load-to-aws-engineer
tomoki10
0
180
手を動かしてレベルアップしよう!
maruto
0
230
Featured
See All Featured
Large-scale JavaScript Application Architecture
addyosmani
511
110k
Fireside Chat
paigeccino
34
3.2k
The Pragmatic Product Professional
lauravandoore
32
6.4k
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
356
29k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
248
1.3M
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Git: the NoSQL Database
bkeepers
PRO
427
65k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
100
18k
Being A Developer After 40
akosma
89
590k
Docker and Python
trallard
44
3.3k
Keith and Marios Guide to Fast Websites
keithpitt
411
22k
Transcript
,VCFSOFUFTΛΊ͙Δݥ +"846(ίϯςφࢧ෦ $IBUXPSL43&νʔϜࡔຊ
© Chatwork ࣗݾհ 2 ▸ ࡔຊ ྒ (͔͞ͱɹΓΐ͏) ▸ Chatwork
SREΤϯδχΞ ▸ ϥϯχϯάΤϯδχΞ ▸ ϑϧϚϥιϯ͕Α͏͘3࣌ؒ17͙Β͍(20193݄) ▸ 2019ͷߦڑ2700km͙Β͍ ▸ NY- LA4000kmͳͷͰ, ೦ͳ͕ΒΞϝϦΧԣஅͰ͖ͣ
▸ ϏδωενϟοταʔϏε ▸ άϧʔϓνϟοτɺλεΫཧɺ ϑΝΠϧڞ༗ɺϏσΦɾԻ௨ ▸ ಋೖاۀ 246,000 ࣾ (201912݄ݱࡏ)
© Chatwork ΞδΣϯμ 4 ▸ ChatworkͷKubernetesͷྺ࢙ΛৼΓฦΓͳ͕Βݱࡏͷӡ༻ʹ͍ͭͯ ▸ ྺ࢙ ▸ ӡ༻πʔϧ
▸ ChatworkͷKubernetes CI/CD
© Chatwork ҙ 5 Fargateͷ͋Γ·ͤΜ
© Chatwork લఏ$IBUXPSLͷ,VCFSOFUFTར༻ܗଶ 6 ▸ Ϛϧνςφϯτ ▸ ΫϥελࣗମSRE͕࡞Δ ▸ ཧܥΞϓϦέʔγϣϯ(Datadog,
Fluentd ͳͲ) -> SRE ▸ αʔϏεΞϓϦέʔγϣϯ -> DevνʔϜ
© Chatwork $IBUXPSLͷ,VCFSOFUFTͷྺ࢙ 7 ▸ ಋೖ2016 ▸ ࠷ॳͷProductionͷόʔδϣϯ1.5 ▸ ϝοηʔδϯάγεςϜͷ৽ϓϩδΣΫτ
▸ AWS Summit Tokyo 2017 Ͱൃද ▸ ͦͷޙԿ͔ΫϥελҠߦ(όʔδϣϯΞοϓ)Λͭͭ͠ɺݱࡏʹࢸΔ ▸ ࣗલKubernetes, EKSͷࠞ߹ঢ়ଶ
© Chatwork &,4ͷಋೖ 8 ▸ EKSར༻લkube-awsͱ͍͏ͷΛར༻ͯ͠, ࣗલͰϗεςΟϯά ▸ https://github.com/kubernetes-incubator/kube-aws ▸
EKSͷొʹΑΓkube-awsͷϢʔβݮ… ▸ ࡉ͔͍ػೳʹόά͕ଟ͘, దٓPRରԠͭͭࣗ͠લύονӡ༻ ▸ EKSࣗମίϯτϩʔϥʔ෦͚ͩͰ, ͍ʹ͍͘ͳ͊…ͱϓϨϏϡʔͰ ࢥ͍͕ͬͯͨeksctlʹΑΓͦ͏͍͏ͱ͜Ζ͕·Δͬͱָʹ
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 9 ▸ eksctl ▸ ։ൃ͕ૣ͍ ▸ جຊతʹΫϥελͷߏupdateͰ͖ͳ͍
▸ ϊʔυάϧʔϓͷઃఆมߋΛ͍ͨ͠߹ɺ৽ن࡞ɾچϊʔυά ϧʔϓআ ▸ ϊʔυάϧʔϓͷߏΛࢼ͍ͯ͠Δͱ͖ͪΐͬͱखؒ
ΦϑΟγϟϧͳͷͰɺEKS৽͍͠ػೳେମAWSͷൃදͱಉ࣌ʹରԠ͞Ε͍ͯΔ
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 11 ▸ kube-aws ▸ ΫϥελͷߏupdateՄೳ ▸ ϊʔυάϧʔϓͷઃఆมߋΛ͍ͨ͠߹ɺچϊʔυάϧʔϓ͔Β
ϩʔϦϯάΞοϓσʔτ(CloudFormationͷupdateͳͷͰ...) ▸ EKSΛར༻͢ΔͷͰͳ͘, controller, etcdؚΊͯ࡞ ▸ ੜKubernetesͳͷͰɺΧελϚΠζੑ͔ͳΓߴ͍ ▸ KubernetesͷઃఆϑΝΠϧ͕େͳyamlʹͳΔ(Productionͷ686ߦ)
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 12 ▸ ظతʹݟΔͱ, eksctlͷ΄͏͕ϝϦοτେ͖͍ ▸ Ϣʔβೝূ͕iamͰͰ͖ͨΓ, podʹiam
roleΛ͚ΒΕͨΓ, eksͦͷͷ ͷϝϦοτେ͖͍ ▸ iam role for podͱͯخ͍͠ ▸ kube2iam, kiam, kube-aws-iam-controllerͯ͢ӡ༻͖ͯͨ͠ܦݧ ▸ ͜Ε͜ΕͰखܰ͋͞Δ͚ΕͲ
ΫϥελΛ࡞Δ͚ͩͳΒ͜ΕͰOK ͔͠͠ɾɾ
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 14 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜ΕΛΓӽ͑Δӡ༻ɾΈ͕ඞཁ
© Chatwork ,VCFSOFUFTͷӡ༻Λࢧ͑Δπʔϧͨͪ 16 ▸ eksctl (kube-aws) ▸ Ϋϥελࣗମ͜ΕͰ࡞ ▸
Variant ▸ https://github.com/mumoshu/variant ▸ helm, helmfile ▸ https://github.com/roboll/helmfile
© Chatwork 7BSJBOU 17 ▸ task runner ▸ eksctlͳͲͷίϚϯυΛϥοϓ͢Δ ▸
wrapper shellͰ͍͍͕ɺͪ͜ΒyamlͰॻ͚ͯɺύϥϝʔλͳͲͷࢦ ఆΓ͍͢ ▸ dockerϕʔεͷtask runner͕ఆٛͰ͖ɺڥґଘΛݮΒͤΔ ▸ ڥͱίϚϯυΛηοτʹͰ͖Δ
© Chatwork FLTDUM͚ͩͰͳͥͩΊͳͷ͔ 18 ▸ eksctl͚ͩͰेͳ߹ଟ͍ ▸ ͱΓ͋͑ͣࢼݧڥΛىಈͤ͞ΔʹίϚϯυ͚ͩͰे ▸ ͨͩ͠ࡉ͔͍ઃఆΛ͠Α͏ͱ͢Δͱ,
ϑΝΠϧͰͷઃఆͷ΄͏ָ͕ ▸ sampleҎ֎ͷυΩϡϝϯτͳ͍ͷͰ, ҙ ▸ https://github.com/weaveworks/eksctl/blob/master/pkg/apis/eksctl.io/ v1alpha5/types.go ͔Β୳͔͢͠ͳ͍ͱ͖͋Δ ▸ ઃఆΛͤΔVariantͰϥοϓͯ͠, eksctl༻ઃఆϑΝΠϧͷΛஔ
# Ϋϥελ໊ # αϒωοτ # cluster-autoscalerͷiam roleͭ͘Δ # eksຊମ(ϊʔυάϧʔϓআ͘)ͷeksctlͷઃఆϑΝΠϧ
© Chatwork FLTDUM͚ͩͰͳͥͩΊͳͷ͔ 20 ▸ Ϋϥελ࡞Δ͚ͩͳΒ, ୯ମར༻Ͱ͍͚ͳ͘ͳ͍ ▸ FluxͩΕ͕ೖΕΔͷɺͷରԠ ▸
Flux ... GitOps tool ޙड़͠·͢ ▸ GitOpsͰઃఆͷөΞϓϦέʔγϣϯͷσϓϩΠΛ͢Δ ▸ Ͱ, GitOpsΛ࣮ݱ͢ΔͨΊͷπʔϧҰମ୭͕ೖΕΔͷ͔...! ▸ GitOps Kubernetes෦ʹ࣮ݱ͢ΔͨΊͷΞϓϦέʔγϣϯ͕͍Δ
© Chatwork 21 ▸ eksctlίϚϯυͰFluxΛ͍ΕΔ͜ͱՄೳ ▸ υΩϡϝϯτϕʔεͰରԠͰ͖Δ͕ɺԼهͷϑϩʔΛ·ͱΊ͍ͨ ▸ eksctlͷઃఆϑΝΠϧ࡞ ▸
eksctl create cluster ▸ eksctl enable repo ... (Fluxͷinstall) ▸ ͜ΕΒΛvariantͰλεΫ੍ޚ(Infrastructure as codeͷҰ) FLTDUM͚ͩͰͳͥͩΊͳͷ͔
© Chatwork 22 FLTDUMΛ7BSJBOUͰXSBQͨ݁͠Ռ ͜Ε͚ͩͰEKSڥOK https://en.wikipedia.org/wiki/Achlis EKS࡞ίϚϯυΛachlisͱ໋໊
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 23 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜͜·ͰKubernetesࣗମͷߏங ͔͜͜ΒKubernetes෦ͷߏங
© Chatwork 25 ▸ https://toris.io/2019/12/what-i-think-about-when-i-think-about-kubernetes- and-ecs/ ▸ (ͳΜ͔ଜ্य़थͷλΠτϧͬΆ͍...) ▸ EKSϚωʔδυͰ͕͢ɺϚωʔδϝϯτίϯιʔϧͰ࡞͚ͨͩ͠Ͱ
΄΅ԿͰ͖·ͤΜ ▸ ϊʔυάϧʔϓ(࣮ࡍʹίϯςφ͕ಈ͘ڥ)Λ࡞ͬͯɺ·ͩϩάૹΕ ͣɺϝτϦΫεݟΕͣɺͰ͢ ▸ ͳͷͰɺEKSͰཧܥͷΞϓϦέʔγϣϯ(ex. Datadog)ͷӡ༻͕ඞਢ &$4ͱͷҧ͍
© Chatwork IFMN IFMNpMF 26 ▸ helm ▸ The package
manager for Kubernetes ▸ KubernetesʹΞϓϦέʔγϣϯΛೖΕΔࡍʹɺඞཁͳͷΛύοέʔδ Խͯ͠ɺೖΕͯ͘ΕΔπʔϧ ▸ ChatworkͰ΄ͱΜͲΞϓϦέʔγϣϯΛHelmΛར༻ͯ͠σϓϩΠ ▸ Chartࣗମڥʹґଘ͠ͳ͍ʂ ▸ σϓϩΠͷ࠶ݱੑ(ࣗલKubernetes(kube-aws) -> EKSͰͦͷ··)
© Chatwork IFMN IFMNpMF 27 ▸ helmfile ▸ helm chartΛͲ͏ͬͯద༻͢Δͷ͔Λએݴతʹॻ͚Δ
▸ helmͰσϓϩΠ͢ΔࡍͷΞϓϦέʔγϣϯͷઃఆɺhelmίϚϯυ ͷΦϓγϣϯͳͲΛҰׅͰهࡌ͠ɺhelmͰσϓϩΠͰ͖Δπʔϧ ▸ https://github.com/helm/charts/tree/master/stable ▸ ͍ΖΜͳΞϓϦέʔγϣϯͷhelm chart ▸ https://developers.freee.co.jp/entry/2019/12/03/122657
© Chatwork ͳͥIFMNpMF͕ඞཁͳͷ͔ 28 ▸ ΞϓϦέʔγϣϯ͝ͱʹ͍͍ͨΦϓγϣϯ(લॲཧ)͕มΘΔ ▸ ྫ͑)ΞϓϦέʔγϣϯಉ࢜Ͱґଘؔͷ͋Δͷ, ͳͲ ▸
helmͷઃఆϑΝΠϧ͕ڥ͝ͱ(test, stg, prod)ͰมΘΔ ▸ ྫ͑)AWSͷΞΧϯτ͝ͱʹҧ͏, ϩάͷు͖ग़͠ઌ͕ҧ͏ ▸ go template ͑ΔͷͰ, helmͷઃఆΛಈతʹੜ͢Δ͜ͱՄೳ ▸ ChatworkͰɺ͞ΒʹVariantͰhelmfileΛแΈࠐΈ
© Chatwork 29 IFMNpMF IFMN Λ7BSJBOUͰXSBQͨ݁͠Ռ ͜Ε͚ͩͰKubernetes෦OK
© Chatwork 30 ͍Ζ͍Ζ7BSJBOUͰXSBQͨ݁͠Ռ
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 31 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜͜·ͰKubernetesӡ༻ ͔͜͜ΒCI/CD
© Chatwork ,VCFSOFUFTͷӡ༻ͰҰ൪େมͳ͜ͱ 33 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
© Chatwork ,VCFSOFUFTڥͷ$*$% 34 ▸ KubernetesͰͷCI/CD ▸ αʔϏεͷΞϓϦέʔγϣϯ ▸ Concourse
CIΛར༻ͯ͠CI/CD ▸ ͯ͢ΛyamlܗࣜͷpipelineͰఆٛͰ͖Δπʔϧ
© Chatwork ,VCFSOFUFTڥͷ$*$% 35 ▸ ࠔͬͨ ▸ CIͳΓϩʔΧϧ͔ΒσϓϩΠ͢Δ߹, ςϯϓϨʔτԽ͍ͯ͠Δͱͯ͠ ɺΫϥελ͕Ͱ͖Δͨͼʹ͖ઌΛม͑Δඞཁ͕͋Δ
▸ ͦͦαʔϏεΞϓϦͷσϓϩΠΛߦ͏Concourse CIࣗΛͲ͜Ͱӡ ༻͢Δͷ͔
© Chatwork (JU0QTʂ 36 https://www.weave.works/technologies/gitops/
© Chatwork (JU0QTʂ!$IBUXPSL 37 Argoͷ෦·ͩରԠͰ͖͍ͯͳ͍
© Chatwork $IBUXPSLͷ(JU0QTߏ 38 ▸ Flux + Argo CDͷϋΠϒϦουߏ ▸
Flux ▸ ѻ͑ΔͷݪଇmanifestͷΈͰ, 1flux - 1repo - 1branch ▸ Argo CDࣗͷΠϯετʔϧnamespace, aws-auth(eksಠࣗ)ͳͲ manifestద༻Ͱ͋·Γมߋ͕ͳ͍ͷ ▸ Argo CD ▸ GUI͕͋ͬͨΓ, helmʹରԠ͍ͯͨ͠Γɺͱػೳ͕ଟ͍͍͘͢
© Chatwork $IBUXPSLͷ(JU0QTߏ 39 ▸ αʔϏεͷΞϓϦέʔγϣϯ͚ͩͰͳ͘ɺཧܥͷΞϓϦͷΞϓϦέʔ γϣϯGitOpsͰద༻͢Δํࣜʹมߋ༧ఆ ▸ Kubernetes͕Ͱ͖ΔͱɺඞཁͳΞϓϦέʔγϣϯͯ͢উखʹೖ͍ͬͯ Δঢ়ଶ
▸ αʔϏεͷΞϓϦέʔγϣϯʹؔͯͪ͠ΖΜΓସ͑ͳͲ͕ඞཁ
© Chatwork (JU0QTΛΊ͙Δڥ 40
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 41 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
© Chatwork ·ͱΊ 42 ▸ Chatworkʹ͓͚ΔKubernetesͷӡ༻(ߏங) ▸ eksctl, variant, helm,
helmfile ▸ GitOpsߏ ▸ EKSɺֹʂ