Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Adventure around Kubernetes at Chatwork
Search
Ryo Sakamoto
January 23, 2020
Technology
5
7.9k
Adventure around Kubernetes at Chatwork
Ryo Sakamoto
January 23, 2020
Tweet
Share
More Decks by Ryo Sakamoto
See All by Ryo Sakamoto
いろいろなAWSアカウントのArgo CDを統合した話
cwsakamoto
1
1k
ArgoCDとGitHub Self Hosted Runnerを使って リリース時間を1/4にした話
cwsakamoto
0
2.3k
チャットワークにおけるKubernetesOnAWS.pdf
cwsakamoto
0
94
チャットワークにおけるKubernetesOnAWS.pdf
cwsakamoto
0
88
Kubernetes on AWS at Chatwork
cwsakamoto
0
1.8k
Other Decks in Technology
See All in Technology
JTCにおける内製×スクラム開発への挑戦〜内製化率95%達成の舞台裏/JTC's challenge of in-house development with Scrum
aeonpeople
0
260
AIエージェントで90秒の広告動画を制作!台本・音声・映像・編集をつなぐAWS最新アーキテクチャの実践
nasuvitz
3
350
普通のチームがスクラムを会得するたった一つの冴えたやり方 / the best way to scrum
okamototakuyasr2
0
110
Android Audio: Beyond Winning On It
atsushieno
0
2.9k
共有と分離 - Compose Multiplatform "本番導入" の設計指針
error96num
2
1.2k
新規プロダクトでプロトタイプから正式リリースまでNext.jsで開発したリアル
kawanoriku0
1
210
20250910_障害注入から効率的復旧へ_カオスエンジニアリング_生成AIで考えるAWS障害対応.pdf
sh_fk2
3
280
Unlocking the Power of AI Agents with LINE Bot MCP Server
linedevth
0
120
品質視点から考える組織デザイン/Organizational Design from Quality
mii3king
0
210
Platform開発が先行する Platform Engineeringの違和感
kintotechdev
4
590
2つのフロントエンドと状態管理
mixi_engineers
PRO
3
130
テストを軸にした生き残り術
kworkdev
PRO
0
220
Featured
See All Featured
Designing Experiences People Love
moore
142
24k
Building Applications with DynamoDB
mza
96
6.6k
How GitHub (no longer) Works
holman
315
140k
Building Adaptive Systems
keathley
43
2.7k
Making Projects Easy
brettharned
117
6.4k
Building a Scalable Design System with Sketch
lauravandoore
462
33k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
27k
Mobile First: as difficult as doing things right
swwweet
224
9.9k
Site-Speed That Sticks
csswizardry
10
820
Understanding Cognitive Biases in Performance Measurement
bluesmoon
29
1.9k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
RailsConf & Balkan Ruby 2019: The Past, Present, and Future of Rails at GitHub
eileencodes
139
34k
Transcript
,VCFSOFUFTΛΊ͙Δݥ +"846(ίϯςφࢧ෦ $IBUXPSL43&νʔϜࡔຊ
© Chatwork ࣗݾհ 2 ▸ ࡔຊ ྒ (͔͞ͱɹΓΐ͏) ▸ Chatwork
SREΤϯδχΞ ▸ ϥϯχϯάΤϯδχΞ ▸ ϑϧϚϥιϯ͕Α͏͘3࣌ؒ17͙Β͍(20193݄) ▸ 2019ͷߦڑ2700km͙Β͍ ▸ NY- LA4000kmͳͷͰ, ೦ͳ͕ΒΞϝϦΧԣஅͰ͖ͣ
▸ ϏδωενϟοταʔϏε ▸ άϧʔϓνϟοτɺλεΫཧɺ ϑΝΠϧڞ༗ɺϏσΦɾԻ௨ ▸ ಋೖاۀ 246,000 ࣾ (201912݄ݱࡏ)
© Chatwork ΞδΣϯμ 4 ▸ ChatworkͷKubernetesͷྺ࢙ΛৼΓฦΓͳ͕Βݱࡏͷӡ༻ʹ͍ͭͯ ▸ ྺ࢙ ▸ ӡ༻πʔϧ
▸ ChatworkͷKubernetes CI/CD
© Chatwork ҙ 5 Fargateͷ͋Γ·ͤΜ
© Chatwork લఏ$IBUXPSLͷ,VCFSOFUFTར༻ܗଶ 6 ▸ Ϛϧνςφϯτ ▸ ΫϥελࣗମSRE͕࡞Δ ▸ ཧܥΞϓϦέʔγϣϯ(Datadog,
Fluentd ͳͲ) -> SRE ▸ αʔϏεΞϓϦέʔγϣϯ -> DevνʔϜ
© Chatwork $IBUXPSLͷ,VCFSOFUFTͷྺ࢙ 7 ▸ ಋೖ2016 ▸ ࠷ॳͷProductionͷόʔδϣϯ1.5 ▸ ϝοηʔδϯάγεςϜͷ৽ϓϩδΣΫτ
▸ AWS Summit Tokyo 2017 Ͱൃද ▸ ͦͷޙԿ͔ΫϥελҠߦ(όʔδϣϯΞοϓ)Λͭͭ͠ɺݱࡏʹࢸΔ ▸ ࣗલKubernetes, EKSͷࠞ߹ঢ়ଶ
© Chatwork &,4ͷಋೖ 8 ▸ EKSར༻લkube-awsͱ͍͏ͷΛར༻ͯ͠, ࣗલͰϗεςΟϯά ▸ https://github.com/kubernetes-incubator/kube-aws ▸
EKSͷొʹΑΓkube-awsͷϢʔβݮ… ▸ ࡉ͔͍ػೳʹόά͕ଟ͘, దٓPRରԠͭͭࣗ͠લύονӡ༻ ▸ EKSࣗମίϯτϩʔϥʔ෦͚ͩͰ, ͍ʹ͍͘ͳ͊…ͱϓϨϏϡʔͰ ࢥ͍͕ͬͯͨeksctlʹΑΓͦ͏͍͏ͱ͜Ζ͕·Δͬͱָʹ
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 9 ▸ eksctl ▸ ։ൃ͕ૣ͍ ▸ جຊతʹΫϥελͷߏupdateͰ͖ͳ͍
▸ ϊʔυάϧʔϓͷઃఆมߋΛ͍ͨ͠߹ɺ৽ن࡞ɾچϊʔυά ϧʔϓআ ▸ ϊʔυάϧʔϓͷߏΛࢼ͍ͯ͠Δͱ͖ͪΐͬͱखؒ
ΦϑΟγϟϧͳͷͰɺEKS৽͍͠ػೳେମAWSͷൃදͱಉ࣌ʹରԠ͞Ε͍ͯΔ
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 11 ▸ kube-aws ▸ ΫϥελͷߏupdateՄೳ ▸ ϊʔυάϧʔϓͷઃఆมߋΛ͍ͨ͠߹ɺچϊʔυάϧʔϓ͔Β
ϩʔϦϯάΞοϓσʔτ(CloudFormationͷupdateͳͷͰ...) ▸ EKSΛར༻͢ΔͷͰͳ͘, controller, etcdؚΊͯ࡞ ▸ ੜKubernetesͳͷͰɺΧελϚΠζੑ͔ͳΓߴ͍ ▸ KubernetesͷઃఆϑΝΠϧ͕େͳyamlʹͳΔ(Productionͷ686ߦ)
© Chatwork FLTDUMͱLVCFBXTͷൺֱ 12 ▸ ظతʹݟΔͱ, eksctlͷ΄͏͕ϝϦοτେ͖͍ ▸ Ϣʔβೝূ͕iamͰͰ͖ͨΓ, podʹiam
roleΛ͚ΒΕͨΓ, eksͦͷͷ ͷϝϦοτେ͖͍ ▸ iam role for podͱͯخ͍͠ ▸ kube2iam, kiam, kube-aws-iam-controllerͯ͢ӡ༻͖ͯͨ͠ܦݧ ▸ ͜Ε͜ΕͰखܰ͋͞Δ͚ΕͲ
ΫϥελΛ࡞Δ͚ͩͳΒ͜ΕͰOK ͔͠͠ɾɾ
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 14 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜ΕΛΓӽ͑Δӡ༻ɾΈ͕ඞཁ
© Chatwork ,VCFSOFUFTͷӡ༻Λࢧ͑Δπʔϧͨͪ 16 ▸ eksctl (kube-aws) ▸ Ϋϥελࣗମ͜ΕͰ࡞ ▸
Variant ▸ https://github.com/mumoshu/variant ▸ helm, helmfile ▸ https://github.com/roboll/helmfile
© Chatwork 7BSJBOU 17 ▸ task runner ▸ eksctlͳͲͷίϚϯυΛϥοϓ͢Δ ▸
wrapper shellͰ͍͍͕ɺͪ͜ΒyamlͰॻ͚ͯɺύϥϝʔλͳͲͷࢦ ఆΓ͍͢ ▸ dockerϕʔεͷtask runner͕ఆٛͰ͖ɺڥґଘΛݮΒͤΔ ▸ ڥͱίϚϯυΛηοτʹͰ͖Δ
© Chatwork FLTDUM͚ͩͰͳͥͩΊͳͷ͔ 18 ▸ eksctl͚ͩͰेͳ߹ଟ͍ ▸ ͱΓ͋͑ͣࢼݧڥΛىಈͤ͞ΔʹίϚϯυ͚ͩͰे ▸ ͨͩ͠ࡉ͔͍ઃఆΛ͠Α͏ͱ͢Δͱ,
ϑΝΠϧͰͷઃఆͷ΄͏ָ͕ ▸ sampleҎ֎ͷυΩϡϝϯτͳ͍ͷͰ, ҙ ▸ https://github.com/weaveworks/eksctl/blob/master/pkg/apis/eksctl.io/ v1alpha5/types.go ͔Β୳͔͢͠ͳ͍ͱ͖͋Δ ▸ ઃఆΛͤΔVariantͰϥοϓͯ͠, eksctl༻ઃఆϑΝΠϧͷΛஔ
# Ϋϥελ໊ # αϒωοτ # cluster-autoscalerͷiam roleͭ͘Δ # eksຊମ(ϊʔυάϧʔϓআ͘)ͷeksctlͷઃఆϑΝΠϧ
© Chatwork FLTDUM͚ͩͰͳͥͩΊͳͷ͔ 20 ▸ Ϋϥελ࡞Δ͚ͩͳΒ, ୯ମར༻Ͱ͍͚ͳ͘ͳ͍ ▸ FluxͩΕ͕ೖΕΔͷɺͷରԠ ▸
Flux ... GitOps tool ޙड़͠·͢ ▸ GitOpsͰઃఆͷөΞϓϦέʔγϣϯͷσϓϩΠΛ͢Δ ▸ Ͱ, GitOpsΛ࣮ݱ͢ΔͨΊͷπʔϧҰମ୭͕ೖΕΔͷ͔...! ▸ GitOps Kubernetes෦ʹ࣮ݱ͢ΔͨΊͷΞϓϦέʔγϣϯ͕͍Δ
© Chatwork 21 ▸ eksctlίϚϯυͰFluxΛ͍ΕΔ͜ͱՄೳ ▸ υΩϡϝϯτϕʔεͰରԠͰ͖Δ͕ɺԼهͷϑϩʔΛ·ͱΊ͍ͨ ▸ eksctlͷઃఆϑΝΠϧ࡞ ▸
eksctl create cluster ▸ eksctl enable repo ... (Fluxͷinstall) ▸ ͜ΕΒΛvariantͰλεΫ੍ޚ(Infrastructure as codeͷҰ) FLTDUM͚ͩͰͳͥͩΊͳͷ͔
© Chatwork 22 FLTDUMΛ7BSJBOUͰXSBQͨ݁͠Ռ ͜Ε͚ͩͰEKSڥOK https://en.wikipedia.org/wiki/Achlis EKS࡞ίϚϯυΛachlisͱ໋໊
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 23 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜͜·ͰKubernetesࣗମͷߏங ͔͜͜ΒKubernetes෦ͷߏங
© Chatwork 25 ▸ https://toris.io/2019/12/what-i-think-about-when-i-think-about-kubernetes- and-ecs/ ▸ (ͳΜ͔ଜ্य़थͷλΠτϧͬΆ͍...) ▸ EKSϚωʔδυͰ͕͢ɺϚωʔδϝϯτίϯιʔϧͰ࡞͚ͨͩ͠Ͱ
΄΅ԿͰ͖·ͤΜ ▸ ϊʔυάϧʔϓ(࣮ࡍʹίϯςφ͕ಈ͘ڥ)Λ࡞ͬͯɺ·ͩϩάૹΕ ͣɺϝτϦΫεݟΕͣɺͰ͢ ▸ ͳͷͰɺEKSͰཧܥͷΞϓϦέʔγϣϯ(ex. Datadog)ͷӡ༻͕ඞਢ &$4ͱͷҧ͍
© Chatwork IFMN IFMNpMF 26 ▸ helm ▸ The package
manager for Kubernetes ▸ KubernetesʹΞϓϦέʔγϣϯΛೖΕΔࡍʹɺඞཁͳͷΛύοέʔδ Խͯ͠ɺೖΕͯ͘ΕΔπʔϧ ▸ ChatworkͰ΄ͱΜͲΞϓϦέʔγϣϯΛHelmΛར༻ͯ͠σϓϩΠ ▸ Chartࣗମڥʹґଘ͠ͳ͍ʂ ▸ σϓϩΠͷ࠶ݱੑ(ࣗલKubernetes(kube-aws) -> EKSͰͦͷ··)
© Chatwork IFMN IFMNpMF 27 ▸ helmfile ▸ helm chartΛͲ͏ͬͯద༻͢Δͷ͔Λએݴతʹॻ͚Δ
▸ helmͰσϓϩΠ͢ΔࡍͷΞϓϦέʔγϣϯͷઃఆɺhelmίϚϯυ ͷΦϓγϣϯͳͲΛҰׅͰهࡌ͠ɺhelmͰσϓϩΠͰ͖Δπʔϧ ▸ https://github.com/helm/charts/tree/master/stable ▸ ͍ΖΜͳΞϓϦέʔγϣϯͷhelm chart ▸ https://developers.freee.co.jp/entry/2019/12/03/122657
© Chatwork ͳͥIFMNpMF͕ඞཁͳͷ͔ 28 ▸ ΞϓϦέʔγϣϯ͝ͱʹ͍͍ͨΦϓγϣϯ(લॲཧ)͕มΘΔ ▸ ྫ͑)ΞϓϦέʔγϣϯಉ࢜Ͱґଘؔͷ͋Δͷ, ͳͲ ▸
helmͷઃఆϑΝΠϧ͕ڥ͝ͱ(test, stg, prod)ͰมΘΔ ▸ ྫ͑)AWSͷΞΧϯτ͝ͱʹҧ͏, ϩάͷు͖ग़͠ઌ͕ҧ͏ ▸ go template ͑ΔͷͰ, helmͷઃఆΛಈతʹੜ͢Δ͜ͱՄೳ ▸ ChatworkͰɺ͞ΒʹVariantͰhelmfileΛแΈࠐΈ
© Chatwork 29 IFMNpMF IFMN Λ7BSJBOUͰXSBQͨ݁͠Ռ ͜Ε͚ͩͰKubernetes෦OK
© Chatwork 30 ͍Ζ͍Ζ7BSJBOUͰXSBQͨ݁͠Ռ
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 31 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
͜͜·ͰKubernetesӡ༻ ͔͜͜ΒCI/CD
© Chatwork ,VCFSOFUFTͷӡ༻ͰҰ൪େมͳ͜ͱ 33 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
© Chatwork ,VCFSOFUFTڥͷ$*$% 34 ▸ KubernetesͰͷCI/CD ▸ αʔϏεͷΞϓϦέʔγϣϯ ▸ Concourse
CIΛར༻ͯ͠CI/CD ▸ ͯ͢ΛyamlܗࣜͷpipelineͰఆٛͰ͖Δπʔϧ
© Chatwork ,VCFSOFUFTڥͷ$*$% 35 ▸ ࠔͬͨ ▸ CIͳΓϩʔΧϧ͔ΒσϓϩΠ͢Δ߹, ςϯϓϨʔτԽ͍ͯ͠Δͱͯ͠ ɺΫϥελ͕Ͱ͖Δͨͼʹ͖ઌΛม͑Δඞཁ͕͋Δ
▸ ͦͦαʔϏεΞϓϦͷσϓϩΠΛߦ͏Concourse CIࣗΛͲ͜Ͱӡ ༻͢Δͷ͔
© Chatwork (JU0QTʂ 36 https://www.weave.works/technologies/gitops/
© Chatwork (JU0QTʂ!$IBUXPSL 37 Argoͷ෦·ͩରԠͰ͖͍ͯͳ͍
© Chatwork $IBUXPSLͷ(JU0QTߏ 38 ▸ Flux + Argo CDͷϋΠϒϦουߏ ▸
Flux ▸ ѻ͑ΔͷݪଇmanifestͷΈͰ, 1flux - 1repo - 1branch ▸ Argo CDࣗͷΠϯετʔϧnamespace, aws-auth(eksಠࣗ)ͳͲ manifestద༻Ͱ͋·Γมߋ͕ͳ͍ͷ ▸ Argo CD ▸ GUI͕͋ͬͨΓ, helmʹରԠ͍ͯͨ͠Γɺͱػೳ͕ଟ͍͍͘͢
© Chatwork $IBUXPSLͷ(JU0QTߏ 39 ▸ αʔϏεͷΞϓϦέʔγϣϯ͚ͩͰͳ͘ɺཧܥͷΞϓϦͷΞϓϦέʔ γϣϯGitOpsͰద༻͢Δํࣜʹมߋ༧ఆ ▸ Kubernetes͕Ͱ͖ΔͱɺඞཁͳΞϓϦέʔγϣϯͯ͢উखʹೖ͍ͬͯ Δঢ়ଶ
▸ αʔϏεͷΞϓϦέʔγϣϯʹؔͯͪ͠ΖΜΓସ͑ͳͲ͕ඞཁ
© Chatwork (JU0QTΛΊ͙Δڥ 40
© Chatwork ,VCFSOFUFTͷӡ༻Ͱେมͳ͜ͱ 41 ▸ Kubernetes version up 3ϲ݄ʹ1ճ
▸ ͯ͢ʹै͠ͳ͍ͱͯ͠, ʹ1ճ͙Β͍version up ▸ test, stg, productionڥͷ࡞Γม͑ɺσϓϩΠ͕සൃʂ ▸ EKSʹϩʔϦϯάΞοϓάϨʔυ͋Δ͚ΕͲ.... ▸ खಈ(υΩϡϝϯτରԠ)Ͱӡ༻ෛՙ͕ߴ͗͢Δ
© Chatwork ·ͱΊ 42 ▸ Chatworkʹ͓͚ΔKubernetesͷӡ༻(ߏங) ▸ eksctl, variant, helm,
helmfile ▸ GitOpsߏ ▸ EKSɺֹʂ