Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
自分だけの仮想クラスタを高速かつ効率的に作る kubefork
Search
Kazuki Obata
February 25, 2025
Technology
0
110
自分だけの仮想クラスタを高速かつ効率的に作る kubefork
Wantedly Tech Night 〜ウォンテッドリーで内製している開発用ツールの紹介〜
https://wantedly.connpass.com/event/341010/
Kazuki Obata
February 25, 2025
Tweet
Share
More Decks by Kazuki Obata
See All by Kazuki Obata
散らばったトレースを繋げる技術
donkomura
1
600
ウォンテッドリーのインフラチームに加わってみて
donkomura
0
120
AWS CLI で気軽にコスト改善やってみた
donkomura
1
140
入門 KRR
donkomura
0
230
Other Decks in Technology
See All in Technology
LINEギフトにおけるバックエンド開発
lycorptech_jp
PRO
0
400
開発者体験を定量的に把握する手法と活用事例
ham0215
0
110
LINE NEWSにおけるバックエンド開発
lycorptech_jp
PRO
0
340
"TEAM"を導入したら最高のエンジニア"Team"を実現できた / Deploying "TEAM" and Building the Best Engineering "Team"
yuj1osm
1
230
Exadata Database Service on Cloud@Customer セキュリティ、ネットワーク、および管理について
oracle4engineer
PRO
2
1.6k
Pwned Labsのすゝめ
ken5scal
2
540
Oracle Database Technology Night #87-1 : Exadata Database Service on Exascale Infrastructure(ExaDB-XS)サービス詳細
oracle4engineer
PRO
1
210
4th place solution Eedi - Mining Misconceptions in Mathematics
rist
0
150
AWSアカウントのセキュリティ自動化、どこまで進める? 最適な設計と実践ポイント
yuobayashi
7
1.1k
AI自体のOps 〜LLMアプリの運用、AWSサービスとOSSの使い分け〜
minorun365
PRO
9
810
手を動かしてレベルアップしよう!
maruto
0
250
[OpsJAWS Meetup33 AIOps] Amazon Bedrockガードレールで守る安全なAI運用
akiratameto
1
130
Featured
See All Featured
Side Projects
sachag
452
42k
Done Done
chrislema
182
16k
It's Worth the Effort
3n
184
28k
How to Ace a Technical Interview
jacobian
276
23k
YesSQL, Process and Tooling at Scale
rocio
172
14k
Fashionably flexible responsive web design (full day workshop)
malarkey
406
66k
Art, The Web, and Tiny UX
lynnandtonic
298
20k
What's in a price? How to price your products and services
michaelherold
244
12k
[Rails World 2023 - Day 1 Closing Keynote] - The Magic of Rails
eileencodes
33
2.1k
Six Lessons from altMBA
skipperchong
27
3.6k
個人開発の失敗を避けるイケてる考え方 / tips for indie hackers
panda_program
100
18k
The Myth of the Modular Monolith - Day 2 Keynote - Rails World 2024
eileencodes
21
2.5k
Transcript
© 2025 Wantedly, Inc. ͚ࣗͩͷԾΫϥελΛߴ͔ͭ ޮతʹ࡞Δ kubefork Wantedly Tech Night
#8 2025-02-25 Kazuki Obata
© 2025 Wantedly, Inc. ࣗݾհ ڊി थ (Obata Kazuki) @donkomura_
Wantedly, Inc. (2024/09 ~ ) ΠϯϑϥΤϯδχΞ #k8s #ϘϧμϦϯά🧗
© 2025 Wantedly, Inc. ࣍ • Ϟνϕʔγϣϯ • ՝ͱΓ͍ͨ͜ͱ •
Ͳ͏࣮ͬͯݱ͢Δ͔ ◦ Wantedly ։ൃͷ߹ ◦ ΞΠσΟΞͱ • ·ͱΊ
© 2025 Wantedly, Inc. Ϟνϕʔγϣϯɿ։ൃதͷϓϨϏϡʔ
© 2025 Wantedly, Inc. Ϟνϕʔγϣϯɿ։ൃதͷϓϨϏϡʔ • ؾܰʹϓϨϏϡʔͰ͖Δͱศར ◦ ຊ൪ͱಉ͡Α͏ͳڥͰಈ࡞֬ೝ͠ͳ͕Β։ൃͰ͖Δ ◦
ϨϏϡʔ͍͢͠ Πϝʔδɿhttps://github.com/marketplace/actions/deploy-pr-preview ։ൃதʹ ࣅͨΑ͏ͳ͜ ͱ͕͍ͨ͠
© 2025 Wantedly, Inc. ཧɿݱঢ়ͷ։ൃϓϩηεͷτϨʔυΦϑ ͍͍ͱ͜औΓΛ͍ͨ͠ ݸਓͷ։ൃαʔόʔʢओʹϩʔΧϧʣ - ✅ڥ͕ଞͷӨڹʹΑͬͯԚ͞Εͳ͍ -
✅Docker ͳͲͰຊ൪ʹ͍ۙڥ࣮ݱͰ͖Δ - ❌Πϯϑϥ෦ͯ͢Λ࠶ݱͰ͖ΔΘ͚Ͱ ͳ͍ - ❌ґଘͨ͠αʔϏε͕͋Δͱ։ൃ͕͍͠ ڞ༗ͷ։ൃαʔόʔʢओʹϦϞʔτʣ - ✅ຊ൪ʹ͍ۙڥͰ։ൃͰ͖Δ - ❌ଞͷਓͷ࡞ۀ͔ΒӨڹΛड͚Δɾ༩͑Δ - ❌҆ఆͨ͠ڥΛอͪʹ͍͘ - յΕΔ͜ͱ͕͋Δ - ❌ͷಛఆ͕͘͠ͳΔ - σϓϩΠ͕িಥ͢Δ - ϩά͕ࠞࡏ͢Δ
© 2025 Wantedly, Inc. Ͳ͏࣮ݱ͢Δ͔ - Wantedly ։ൃͷ߹
© 2025 Wantedly, Inc. Ͳ͏࣮ݱ͢ΔɿWantedly ։ൃͷ߹ લఏ • ϚΠΫϩαʔϏεʢΞʔΩςΫνϟʣ •
Πϯϑϥ Kubernetes ◦ 1Ϋϥελ60ݸͷϚΠΫϩαʔϏεΛӡ༻ • ։ൃऀ50ਓ
© 2025 Wantedly, Inc. ࣮ݱͷͨΊͷΞΠσΟΞɿΫϥελʔؙ͝ͱίϐʔ͢Δ • શһʹಉ͡։ൃڥΛఏڙ͢Δͷ͕ཧ • ͯ͢ͷϦιʔεΛຊ൪͔Βίϐʔ͢Δͱ࣮ݱͰ͖Δ ◦
1αʔϏε1αʔόʔͱͯ͠60 x 50 = 3000 αʔόʔ࡞͢Δ͜ͱʹͳΔ 🤔 ◦ Ϧιʔε͕५ʹ͋ΔΘ͚Ͱͳ͍
© 2025 Wantedly, Inc. ۪ʹ࣮ݱ͢Δͷͦ͠͏
© 2025 Wantedly, Inc. ͢Δ
© 2025 Wantedly, Inc. มߋ͍ͨ͠෦͚ͩઐ༻ ΓͷαʔϏεڞ༻
© 2025 Wantedly, Inc. • deployment ͱ service ͚ͩมߋ͢Δ
• ϧʔςΟϯά ◦ ԾΫϥελͷΞΫηεใΛ HTTP Header, gRPC Metadata Ͱൖ
© 2025 Wantedly, Inc. deployment ͱ service ͚ͩมߋ͢Δ
© 2025 Wantedly, Inc. ʮมߋ͍ͨ͠෦͚ͩʯઐ༻ • ΞϓϦέʔγϣϯ։ൃऀʹΑΔมߋ ◦ Stateless ͳϦιʔε͚ͩΛίϐʔ͢Εे
◦ Kubernetes ͩͱ Deployment • Deployment ͷίϐʔʹىҼ͢Δมߋ ◦ ίϐʔͨ͠ Deployment ʹ͚ͯϦΫΤετΛྲྀ͍ͨ͠ ◦ Kubernetes Ͱ Service ΛͬͯτϥϑΟοΫΛసૹ͢Δ
© 2025 Wantedly, Inc. ʮมߋ͍ͨ͠෦͚ͩʯઐ༻ • deployment ͱ service ͷΈίϐʔ͢Δ
◦ มߋ͍ͨ͠ϚΠΫϩαʔϏεͷ Deployment ͱ Service Λίϐʔ ◦ มߋ͠ͳ͍ϚΠΫϩαʔϏεڞ༻ͷͷΛ͏Α͏ʹϧʔςΟϯά
© 2025 Wantedly, Inc. ϧʔςΟϯά
© 2025 Wantedly, Inc. • มߋ෦ʢService, DeploymentʣͷΈΛίϐʔ ◦ ԾΫϥελʹΞΫηε͢Δ߹ ▪
ଞͷϚΠΫϩαʔϏε͔ΒͷϦΫΤετΛ͜ΕΒʹྲྀ͍ͨ͠ ▪ ϓϨϏϡʔͰ͖Δঢ়ଶʹ͍ͨ͠ Ͳ͏ͬͯίϐʔͨ͠ Service, Deployment ʹ ϦΫΤετΛྲྀ͔͢ ͜͜·Ͱ A B B
© 2025 Wantedly, Inc. Istio ͷϦιʔεΛͬͨϧʔςΟϯά • VirtualService ◦ ϚΠΫϩαʔϏεͷαʔϏεؒ௨৴ͷϧʔςΟϯάΛઃఆ͢Δ
resource ◦ header ͷΛجʹϦΫΤετͷϧʔςΟϯάઌΛมߋ͢Δ apiVersion: networking.istio.io/v1beta1 kind: VirtualService spec: hosts: - wantedly-x http: - match: - headers: x-fork-identifer: exact: donkomura route: - destination: host: kube-fork-donkomura x-fork-identifier ͱ͍͏ header Λݟͯ ࣗઐ༻ͷ host ʹ ϧʔςΟϯά͢Δ ίϐʔࡁΈͷ host
© 2025 Wantedly, Inc. ࣮Ͱ͖ͨ ӡ༻Ͱ͖Δʁ
© 2025 Wantedly, Inc. fork ʹ͓͚Δ VirtualService ӡ༻ͷ՝ • 1ͭͷ
VirtualService ʹԾΫϥελ͝ͱͷઃఆ͕ඞཁ • ྫ: 2ਓ͕ಉ͡αʔϏεΛมߋ͍ͨ͠ͱ͖ ◦ Ұͭͷ Virtual Service ʹ݅Λ·ͱΊΔ ◦ x-fork-identifer: cluster-A ʹϚον͢ΔͳΒ ▪ service-A ϧʔςΟϯά ◦ x-fork-identifer: cluster-B ʹϚον͢ΔͳΒ ▪ service-B ϧʔςΟϯ ◦ ͦΕҎ֎ͳΒ service-X ϧʔςΟϯά
© 2025 Wantedly, Inc. fork ʹ͓͚Δ VirtualService ӡ༻ͷ՝ • 1ͭͷ
VirtualService ʹԾΫϥελ͝ͱͷઃఆ͕ඞཁ • ྫ: 2ਓ͕ಉ͡αʔϏεΛมߋ͍ͨ͠ͱ͖ ◦ Ұͭͷ Virtual Service ʹ݅Λ·ͱΊΔ ◦ x-fork-identifer: cluster-A ʹϚον͢ΔͳΒ ▪ service-A ϧʔςΟϯά ◦ x-fork-identifer: cluster-B ʹϚον͢ΔͳΒ ▪ service-B ϧʔςΟϯ ◦ ͦΕҎ֎ͳΒ service-X ϧʔςΟϯά ධՁॱΛ੍ޚͰ͖ͳ͍ ίϯϑϦΫτ͢Δ
© 2025 Wantedly, Inc. VSConfig • VSConfig ◦ ԾΫϥελͷϧʔςΟϯάઃఆΛهड़͢ΔͨΊͷಠࣗϦιʔε ◦
࣮ΧελϜίϯτϩʔϥʔ ▪ ઃఆΛूͯ͠ VirtualService Λੜ͢Δ ◦ VirtualService ͷઃఆΛෳਓ͕৮Βͳͯ͘ྑ͍ ▪ ίϯϑϦΫτ͕ى͖ͳ͍ ◦ յΕͯࣗಈͰݩʹΔ VirtualService ͷҟͳΔมߋΛू͢Δ apiVersion: vsconfig.k8s.wantedly.com/v1beta1 kind: VSConfig spec: headerName: x-fork-identifer headerValue: donkomura host: # ϧʔςΟϯάରͷ Service wantedly-x service: # ϦΫΤετΛྲྀ͢ઌͷ Service kube-fork-donkomura
© 2025 Wantedly, Inc. ։ൃதʹԾΫϥελʹϦΫΤετΛඈ͍ͨ͠ • ͜͜·Ͱ ◦ ಛఆͷ header
ͷ͍ͭͨϦΫΤετԾΫϥελʹϧʔςΟϯά͞ΕΔ ◦ ϧʔςΟϯά VirtualService ͰߦΘΕ͍ͯΔ ◦ VirtualService ͷݸʑͷϧʔςΟϯάઃఆ VSConfig ʹهड़͞Ε͍ͯΔ ͜ΕͰे͔ʁ
© 2025 Wantedly, Inc. ϚΠΫϩαʔϏεʹ͓͚ΔϦΫΤετॲཧͷྲྀΕ A B C E ௨ৗͷϦΫΤετ
req B E A B C E req ؒతʹϦΫΤετΛड͚ͯ ڍಈ͕มΘΔ ίϐʔͨ͠ Deployment ԾΫϥελͷ ϦΫΤετ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req ԾΫϥελͷ ϦΫΤετ A B C E ௨ৗͷϦΫΤετ req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req A B C E req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ ԾΫϥελͷ ϦΫΤετ ௨ৗͷϦΫΤετ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req A B C E req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ ԾΫϥελͷ ϦΫΤετ ௨ৗͷϦΫΤετ ʮԾΫϥελͷΞΫηεͰ͋Δʯ ͱ͍͏ใʢίϯςΩετʣΛ ޙଓͷϚΠΫϩαʔϏε ൖͤ͞Δඞཁ͕͋Δ
© 2025 Wantedly, Inc. ԾΫϥελͷϦΫΤετͰ͋Δ͜ͱΛൖ͢Δ • ͜͜·Ͱʢ࠶ܝʣ ◦ ಛఆͷ header
ͷ͍ͭͨϦΫΤετԾΫϥελʹϧʔςΟϯά͞ΕΔ ◦ ϧʔςΟϯά VirtualService ͰߦΘΕ͍ͯΔ ◦ VirtualService ͷཧ VSConfig ͕ߦ͍ͬͯΔ ԾΫϥελͷ ID Λ࣋ͭ header Λ͚ͭͯ ϚΠΫϩαʔϏεͰ͏ɾൖͤ͞Δ → ϦΫΤετ͕ίϐʔͨ͠ Deployment → ԾతͳΫϥελʹΞΫηε͢Δମݧ
© 2025 Wantedly, Inc. ୭͕ header Λ͚͍ͭͯΔʁ Ͳ͏ͬͯ header Λൖ͍ͯ͠Δʁ
© 2025 Wantedly, Inc. ୭͕ header Λ͚͍ͭͯΔʁ Ͳ͏ͬͯ header Λൖ͍ͯ͠Δʁ
ຊͪ͜Β͚ͩઆ໌͠·͢
© 2025 Wantedly, Inc. ʮԾΫϥελʹΞΫηε͍ͯ͠Δʯ͜ͱΛൖ͢Δ
© 2025 Wantedly, Inc. • ಛఆͷ ID ͷ͍ͨϦΫΤετ (e.g. header)
Λड͚Δ ◦ ड͚ͨϚΠΫϩαʔϏεผͷϚΠΫϩαʔϏεϦΫΤετ ◦ ͦͷࡍʹಉ͡ ID Λ͚ͭͯϦΫΤετΛൃߦ͢Δ ԾΫϥελͷIDΛൖ͢Δ A B C E req id: hoge E req id: hoge
© 2025 Wantedly, Inc. • ಛఆͷ ID ͷ͍ͨϦΫΤετ (e.g. header)
Λड͚Δ ◦ ड͚ͨϚΠΫϩαʔϏεผͷϚΠΫϩαʔϏεϦΫΤετ ◦ ͦͷࡍʹಉ͡ ID Λ͚ͭͯϦΫΤετΛൃߦ͢Δ ԾΫϥελͷIDΛൖ͢Δ req id: hoge req id: hoge req id: hoge header Λൖ͞ ͓ͤͯ͘ A B C E req id: hoge E
© 2025 Wantedly, Inc. Context propagation • ʮԾΫϥελAʹདྷͨϦΫΤετͰ͋Δʯͱ͍͏ίϯςΩετΛ ϚΠΫϩαʔϏεʹൖ (propagate)
͢Δ ◦ ྫ ▪ X-Fork-Identifer: fork-a ͷΑ͏ͳ header ͖Ͱ request ͷॲཧத ▪ ผͷ microservice ʹ௨৴Λߦ͏߹ඞͣ X-Fork-Identifer: fork-a Λ͚Δ • servicex ͱ͍͏ϚΠΫϩαʔϏεڞ௨ϥΠϒϥϦͰ࣮ݱ ◦ wrap ͨ͠ΫϥΠΞϯτΛ͏ ◦ ଞͷϦΫΤετ࣌ʹಉ͡ϔομɾϝλσʔλΛ͏ ◦ ͏গ͠Γ͍ͨ > Wantedly Engineering Handbook
© 2025 Wantedly, Inc. ·ͱΊ • kubefork ԾΫϥελΛ࡞͢ΔΈ ◦ ։ൃऀ͕ࣗઐ༻ͷΫϥελΛ͍࣋ͬͯΔ͔ͷΑ͏ͳମݧΛఏڙ͢Δ
• ΠϯϑϥͱΞϓϦέʔγϣϯͷٕज़ΛΈ߹Θͤͯ Ͱ͖͍ͯΔ ◦ Kubernetes (VirtualService, VSConfig etc.), Istio ◦ servicex