Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
自分だけの仮想クラスタを高速かつ効率的に作る kubefork
Search
Kazuki Obata
February 25, 2025
Technology
0
100
自分だけの仮想クラスタを高速かつ効率的に作る kubefork
Wantedly Tech Night 〜ウォンテッドリーで内製している開発用ツールの紹介〜
https://wantedly.connpass.com/event/341010/
Kazuki Obata
February 25, 2025
Tweet
Share
More Decks by Kazuki Obata
See All by Kazuki Obata
散らばったトレースを繋げる技術
donkomura
1
590
ウォンテッドリーのインフラチームに加わってみて
donkomura
0
120
AWS CLI で気軽にコスト改善やってみた
donkomura
1
140
入門 KRR
donkomura
0
230
Other Decks in Technology
See All in Technology
IAMポリシーのAllow/Denyについて、改めて理解する
smt7174
2
210
Охота на косуль у древних
ashapiro
0
110
Oracle Database Technology Night #87-1 : Exadata Database Service on Exascale Infrastructure(ExaDB-XS)サービス詳細
oracle4engineer
PRO
1
180
わたしがEMとして入社した「最初の100日」の過ごし方 / EMConfJp2025
daiksy
14
5.1k
Amazon Q Developerの無料利用枠を使い倒してHello worldを表示させよう!
nrinetcom
PRO
2
120
"TEAM"を導入したら最高のエンジニア"Team"を実現できた / Deploying "TEAM" and Building the Best Engineering "Team"
yuj1osm
1
190
EDRの検知の仕組みと検知回避について
chayakonanaika
12
4.9k
開発組織を進化させる!AWSで実践するチームトポロジー
iwamot
2
380
1行のコードから社会課題の解決へ: EMの探究、事業・技術・組織を紡ぐ実践知 / EM Conf 2025
9ma3r
11
3.9k
クラウド食堂とは?
hiyanger
0
120
遷移の高速化 ヤフートップの試行錯誤
narirou
6
1.2k
急成長する企業で作った、エンジニアが輝ける制度/ 20250227 Rinto Ikenoue
shift_evolve
0
130
Featured
See All Featured
Building Applications with DynamoDB
mza
93
6.2k
Git: the NoSQL Database
bkeepers
PRO
427
65k
Rebuilding a faster, lazier Slack
samanthasiow
80
8.9k
The Art of Delivering Value - GDevCon NA Keynote
reverentgeek
10
1.3k
Designing Dashboards & Data Visualisations in Web Apps
destraynor
231
53k
How to Create Impact in a Changing Tech Landscape [PerfNow 2023]
tammyeverts
49
2.3k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
666
120k
Code Review Best Practice
trishagee
67
18k
JavaScript: Past, Present, and Future - NDC Porto 2020
reverentgeek
47
5.2k
Agile that works and the tools we love
rasmusluckow
328
21k
Build The Right Thing And Hit Your Dates
maggiecrowley
34
2.5k
Why Our Code Smells
bkeepers
PRO
336
57k
Transcript
© 2025 Wantedly, Inc. ͚ࣗͩͷԾΫϥελΛߴ͔ͭ ޮతʹ࡞Δ kubefork Wantedly Tech Night
#8 2025-02-25 Kazuki Obata
© 2025 Wantedly, Inc. ࣗݾհ ڊി थ (Obata Kazuki) @donkomura_
Wantedly, Inc. (2024/09 ~ ) ΠϯϑϥΤϯδχΞ #k8s #ϘϧμϦϯά🧗
© 2025 Wantedly, Inc. ࣍ • Ϟνϕʔγϣϯ • ՝ͱΓ͍ͨ͜ͱ •
Ͳ͏࣮ͬͯݱ͢Δ͔ ◦ Wantedly ։ൃͷ߹ ◦ ΞΠσΟΞͱ • ·ͱΊ
© 2025 Wantedly, Inc. Ϟνϕʔγϣϯɿ։ൃதͷϓϨϏϡʔ
© 2025 Wantedly, Inc. Ϟνϕʔγϣϯɿ։ൃதͷϓϨϏϡʔ • ؾܰʹϓϨϏϡʔͰ͖Δͱศར ◦ ຊ൪ͱಉ͡Α͏ͳڥͰಈ࡞֬ೝ͠ͳ͕Β։ൃͰ͖Δ ◦
ϨϏϡʔ͍͢͠ Πϝʔδɿhttps://github.com/marketplace/actions/deploy-pr-preview ։ൃதʹ ࣅͨΑ͏ͳ͜ ͱ͕͍ͨ͠
© 2025 Wantedly, Inc. ཧɿݱঢ়ͷ։ൃϓϩηεͷτϨʔυΦϑ ͍͍ͱ͜औΓΛ͍ͨ͠ ݸਓͷ։ൃαʔόʔʢओʹϩʔΧϧʣ - ✅ڥ͕ଞͷӨڹʹΑͬͯԚ͞Εͳ͍ -
✅Docker ͳͲͰຊ൪ʹ͍ۙڥ࣮ݱͰ͖Δ - ❌Πϯϑϥ෦ͯ͢Λ࠶ݱͰ͖ΔΘ͚Ͱ ͳ͍ - ❌ґଘͨ͠αʔϏε͕͋Δͱ։ൃ͕͍͠ ڞ༗ͷ։ൃαʔόʔʢओʹϦϞʔτʣ - ✅ຊ൪ʹ͍ۙڥͰ։ൃͰ͖Δ - ❌ଞͷਓͷ࡞ۀ͔ΒӨڹΛड͚Δɾ༩͑Δ - ❌҆ఆͨ͠ڥΛอͪʹ͍͘ - յΕΔ͜ͱ͕͋Δ - ❌ͷಛఆ͕͘͠ͳΔ - σϓϩΠ͕িಥ͢Δ - ϩά͕ࠞࡏ͢Δ
© 2025 Wantedly, Inc. Ͳ͏࣮ݱ͢Δ͔ - Wantedly ։ൃͷ߹
© 2025 Wantedly, Inc. Ͳ͏࣮ݱ͢ΔɿWantedly ։ൃͷ߹ લఏ • ϚΠΫϩαʔϏεʢΞʔΩςΫνϟʣ •
Πϯϑϥ Kubernetes ◦ 1Ϋϥελ60ݸͷϚΠΫϩαʔϏεΛӡ༻ • ։ൃऀ50ਓ
© 2025 Wantedly, Inc. ࣮ݱͷͨΊͷΞΠσΟΞɿΫϥελʔؙ͝ͱίϐʔ͢Δ • શһʹಉ͡։ൃڥΛఏڙ͢Δͷ͕ཧ • ͯ͢ͷϦιʔεΛຊ൪͔Βίϐʔ͢Δͱ࣮ݱͰ͖Δ ◦
1αʔϏε1αʔόʔͱͯ͠60 x 50 = 3000 αʔόʔ࡞͢Δ͜ͱʹͳΔ 🤔 ◦ Ϧιʔε͕५ʹ͋ΔΘ͚Ͱͳ͍
© 2025 Wantedly, Inc. ۪ʹ࣮ݱ͢Δͷͦ͠͏
© 2025 Wantedly, Inc. ͢Δ
© 2025 Wantedly, Inc. มߋ͍ͨ͠෦͚ͩઐ༻ ΓͷαʔϏεڞ༻
© 2025 Wantedly, Inc. • deployment ͱ service ͚ͩมߋ͢Δ
• ϧʔςΟϯά ◦ ԾΫϥελͷΞΫηεใΛ HTTP Header, gRPC Metadata Ͱൖ
© 2025 Wantedly, Inc. deployment ͱ service ͚ͩมߋ͢Δ
© 2025 Wantedly, Inc. ʮมߋ͍ͨ͠෦͚ͩʯઐ༻ • ΞϓϦέʔγϣϯ։ൃऀʹΑΔมߋ ◦ Stateless ͳϦιʔε͚ͩΛίϐʔ͢Εे
◦ Kubernetes ͩͱ Deployment • Deployment ͷίϐʔʹىҼ͢Δมߋ ◦ ίϐʔͨ͠ Deployment ʹ͚ͯϦΫΤετΛྲྀ͍ͨ͠ ◦ Kubernetes Ͱ Service ΛͬͯτϥϑΟοΫΛసૹ͢Δ
© 2025 Wantedly, Inc. ʮมߋ͍ͨ͠෦͚ͩʯઐ༻ • deployment ͱ service ͷΈίϐʔ͢Δ
◦ มߋ͍ͨ͠ϚΠΫϩαʔϏεͷ Deployment ͱ Service Λίϐʔ ◦ มߋ͠ͳ͍ϚΠΫϩαʔϏεڞ༻ͷͷΛ͏Α͏ʹϧʔςΟϯά
© 2025 Wantedly, Inc. ϧʔςΟϯά
© 2025 Wantedly, Inc. • มߋ෦ʢService, DeploymentʣͷΈΛίϐʔ ◦ ԾΫϥελʹΞΫηε͢Δ߹ ▪
ଞͷϚΠΫϩαʔϏε͔ΒͷϦΫΤετΛ͜ΕΒʹྲྀ͍ͨ͠ ▪ ϓϨϏϡʔͰ͖Δঢ়ଶʹ͍ͨ͠ Ͳ͏ͬͯίϐʔͨ͠ Service, Deployment ʹ ϦΫΤετΛྲྀ͔͢ ͜͜·Ͱ A B B
© 2025 Wantedly, Inc. Istio ͷϦιʔεΛͬͨϧʔςΟϯά • VirtualService ◦ ϚΠΫϩαʔϏεͷαʔϏεؒ௨৴ͷϧʔςΟϯάΛઃఆ͢Δ
resource ◦ header ͷΛجʹϦΫΤετͷϧʔςΟϯάઌΛมߋ͢Δ apiVersion: networking.istio.io/v1beta1 kind: VirtualService spec: hosts: - wantedly-x http: - match: - headers: x-fork-identifer: exact: donkomura route: - destination: host: kube-fork-donkomura x-fork-identifier ͱ͍͏ header Λݟͯ ࣗઐ༻ͷ host ʹ ϧʔςΟϯά͢Δ ίϐʔࡁΈͷ host
© 2025 Wantedly, Inc. ࣮Ͱ͖ͨ ӡ༻Ͱ͖Δʁ
© 2025 Wantedly, Inc. fork ʹ͓͚Δ VirtualService ӡ༻ͷ՝ • 1ͭͷ
VirtualService ʹԾΫϥελ͝ͱͷઃఆ͕ඞཁ • ྫ: 2ਓ͕ಉ͡αʔϏεΛมߋ͍ͨ͠ͱ͖ ◦ Ұͭͷ Virtual Service ʹ݅Λ·ͱΊΔ ◦ x-fork-identifer: cluster-A ʹϚον͢ΔͳΒ ▪ service-A ϧʔςΟϯά ◦ x-fork-identifer: cluster-B ʹϚον͢ΔͳΒ ▪ service-B ϧʔςΟϯ ◦ ͦΕҎ֎ͳΒ service-X ϧʔςΟϯά
© 2025 Wantedly, Inc. fork ʹ͓͚Δ VirtualService ӡ༻ͷ՝ • 1ͭͷ
VirtualService ʹԾΫϥελ͝ͱͷઃఆ͕ඞཁ • ྫ: 2ਓ͕ಉ͡αʔϏεΛมߋ͍ͨ͠ͱ͖ ◦ Ұͭͷ Virtual Service ʹ݅Λ·ͱΊΔ ◦ x-fork-identifer: cluster-A ʹϚον͢ΔͳΒ ▪ service-A ϧʔςΟϯά ◦ x-fork-identifer: cluster-B ʹϚον͢ΔͳΒ ▪ service-B ϧʔςΟϯ ◦ ͦΕҎ֎ͳΒ service-X ϧʔςΟϯά ධՁॱΛ੍ޚͰ͖ͳ͍ ίϯϑϦΫτ͢Δ
© 2025 Wantedly, Inc. VSConfig • VSConfig ◦ ԾΫϥελͷϧʔςΟϯάઃఆΛهड़͢ΔͨΊͷಠࣗϦιʔε ◦
࣮ΧελϜίϯτϩʔϥʔ ▪ ઃఆΛूͯ͠ VirtualService Λੜ͢Δ ◦ VirtualService ͷઃఆΛෳਓ͕৮Βͳͯ͘ྑ͍ ▪ ίϯϑϦΫτ͕ى͖ͳ͍ ◦ յΕͯࣗಈͰݩʹΔ VirtualService ͷҟͳΔมߋΛू͢Δ apiVersion: vsconfig.k8s.wantedly.com/v1beta1 kind: VSConfig spec: headerName: x-fork-identifer headerValue: donkomura host: # ϧʔςΟϯάରͷ Service wantedly-x service: # ϦΫΤετΛྲྀ͢ઌͷ Service kube-fork-donkomura
© 2025 Wantedly, Inc. ։ൃதʹԾΫϥελʹϦΫΤετΛඈ͍ͨ͠ • ͜͜·Ͱ ◦ ಛఆͷ header
ͷ͍ͭͨϦΫΤετԾΫϥελʹϧʔςΟϯά͞ΕΔ ◦ ϧʔςΟϯά VirtualService ͰߦΘΕ͍ͯΔ ◦ VirtualService ͷݸʑͷϧʔςΟϯάઃఆ VSConfig ʹهड़͞Ε͍ͯΔ ͜ΕͰे͔ʁ
© 2025 Wantedly, Inc. ϚΠΫϩαʔϏεʹ͓͚ΔϦΫΤετॲཧͷྲྀΕ A B C E ௨ৗͷϦΫΤετ
req B E A B C E req ؒతʹϦΫΤετΛड͚ͯ ڍಈ͕มΘΔ ίϐʔͨ͠ Deployment ԾΫϥελͷ ϦΫΤετ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req ԾΫϥελͷ ϦΫΤετ A B C E ௨ৗͷϦΫΤετ req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req A B C E req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ ԾΫϥελͷ ϦΫΤετ ௨ৗͷϦΫΤετ
© 2025 Wantedly, Inc. ʮԾΫϥελͷϦΫΤετʯͱ͍͏ίϯςΩετ B E A B C
E req A B C E req ϦΫΤετΛ ίϐʔͨ͠ deploymen ͚͍ͨ ԾΫϥελͷ ϦΫΤετ ௨ৗͷϦΫΤετ ʮԾΫϥελͷΞΫηεͰ͋Δʯ ͱ͍͏ใʢίϯςΩετʣΛ ޙଓͷϚΠΫϩαʔϏε ൖͤ͞Δඞཁ͕͋Δ
© 2025 Wantedly, Inc. ԾΫϥελͷϦΫΤετͰ͋Δ͜ͱΛൖ͢Δ • ͜͜·Ͱʢ࠶ܝʣ ◦ ಛఆͷ header
ͷ͍ͭͨϦΫΤετԾΫϥελʹϧʔςΟϯά͞ΕΔ ◦ ϧʔςΟϯά VirtualService ͰߦΘΕ͍ͯΔ ◦ VirtualService ͷཧ VSConfig ͕ߦ͍ͬͯΔ ԾΫϥελͷ ID Λ࣋ͭ header Λ͚ͭͯ ϚΠΫϩαʔϏεͰ͏ɾൖͤ͞Δ → ϦΫΤετ͕ίϐʔͨ͠ Deployment → ԾతͳΫϥελʹΞΫηε͢Δମݧ
© 2025 Wantedly, Inc. ୭͕ header Λ͚͍ͭͯΔʁ Ͳ͏ͬͯ header Λൖ͍ͯ͠Δʁ
© 2025 Wantedly, Inc. ୭͕ header Λ͚͍ͭͯΔʁ Ͳ͏ͬͯ header Λൖ͍ͯ͠Δʁ
ຊͪ͜Β͚ͩઆ໌͠·͢
© 2025 Wantedly, Inc. ʮԾΫϥελʹΞΫηε͍ͯ͠Δʯ͜ͱΛൖ͢Δ
© 2025 Wantedly, Inc. • ಛఆͷ ID ͷ͍ͨϦΫΤετ (e.g. header)
Λड͚Δ ◦ ड͚ͨϚΠΫϩαʔϏεผͷϚΠΫϩαʔϏεϦΫΤετ ◦ ͦͷࡍʹಉ͡ ID Λ͚ͭͯϦΫΤετΛൃߦ͢Δ ԾΫϥελͷIDΛൖ͢Δ A B C E req id: hoge E req id: hoge
© 2025 Wantedly, Inc. • ಛఆͷ ID ͷ͍ͨϦΫΤετ (e.g. header)
Λड͚Δ ◦ ड͚ͨϚΠΫϩαʔϏεผͷϚΠΫϩαʔϏεϦΫΤετ ◦ ͦͷࡍʹಉ͡ ID Λ͚ͭͯϦΫΤετΛൃߦ͢Δ ԾΫϥελͷIDΛൖ͢Δ req id: hoge req id: hoge req id: hoge header Λൖ͞ ͓ͤͯ͘ A B C E req id: hoge E
© 2025 Wantedly, Inc. Context propagation • ʮԾΫϥελAʹདྷͨϦΫΤετͰ͋Δʯͱ͍͏ίϯςΩετΛ ϚΠΫϩαʔϏεʹൖ (propagate)
͢Δ ◦ ྫ ▪ X-Fork-Identifer: fork-a ͷΑ͏ͳ header ͖Ͱ request ͷॲཧத ▪ ผͷ microservice ʹ௨৴Λߦ͏߹ඞͣ X-Fork-Identifer: fork-a Λ͚Δ • servicex ͱ͍͏ϚΠΫϩαʔϏεڞ௨ϥΠϒϥϦͰ࣮ݱ ◦ wrap ͨ͠ΫϥΠΞϯτΛ͏ ◦ ଞͷϦΫΤετ࣌ʹಉ͡ϔομɾϝλσʔλΛ͏ ◦ ͏গ͠Γ͍ͨ > Wantedly Engineering Handbook
© 2025 Wantedly, Inc. ·ͱΊ • kubefork ԾΫϥελΛ࡞͢ΔΈ ◦ ։ൃऀ͕ࣗઐ༻ͷΫϥελΛ͍࣋ͬͯΔ͔ͷΑ͏ͳମݧΛఏڙ͢Δ
• ΠϯϑϥͱΞϓϦέʔγϣϯͷٕज़ΛΈ߹Θͤͯ Ͱ͖͍ͯΔ ◦ Kubernetes (VirtualService, VSConfig etc.), Istio ◦ servicex