Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Testing with microservices in merpay
Search
kazegusuri
April 14, 2018
Programming
10
10k
Testing with microservices in merpay
Go Conference 2018 Spring
kazegusuri
April 14, 2018
Tweet
Share
More Decks by kazegusuri
See All by kazegusuri
go-sqlite3を使ってCloud Spannerエミュレーターを作ってみた / Cloud Spanner emulator with go-sqlite3
kazegusuri
5
6.6k
handy-spanner GCPUG
kazegusuri
4
1.9k
Open SKT: メルペイ開発の裏側 / builderscon tokyo 2019 Open SKT
kazegusuri
22
27k
Keep watching and extending features of gRPC
kazegusuri
3
2.5k
Real World Mercari API Architecture
kazegusuri
1
6.2k
gRPC and REST with gRPC in practice
kazegusuri
19
7.9k
Fluentdで始めるPrometheus / Prometheus Tokyo Meetup #1
kazegusuri
1
1.9k
GRPCの実践と現状での利点欠点 / Go Conference 2016 Spring
kazegusuri
44
32k
OutputとBufferedOutputの間の何か
kazegusuri
2
3.3k
Other Decks in Programming
See All in Programming
GUI操作LLMの最新動向: UI-TARSと関連論文紹介
kfujikawa
0
560
大規模FlutterプロジェクトのCI実行時間を約8割削減した話
teamlab
PRO
0
450
画像コンペでのベースラインモデルの育て方
tattaka
3
1.3k
オホーツクでコミュニティを立ち上げた理由―地方出身プログラマの挑戦 / TechRAMEN 2025 Conference
lemonade_37
1
440
Vibe coding コードレビュー
kinopeee
0
420
No Install CMS戦略 〜 5年先を見据えたフロントエンド開発を考える / no_install_cms
rdlabo
0
460
PHPUnitの限界をPlaywrightで補完するテストアプローチ
yuzneri
0
380
kiroでゲームを作ってみた
iriikeita
0
140
バイブスあるコーディングで ~PHP~ 便利ツールをつくるプラクティス
uzulla
1
320
実践 Dev Containers × Claude Code
touyu
1
140
抽象化という思考のツール - 理解と活用 - / Abstraction-as-a-Tool-for-Thinking
shin1x1
1
940
あなたとJIT, 今すぐアセンブ ル
sisshiki1969
0
410
Featured
See All Featured
Improving Core Web Vitals using Speculation Rules API
sergeychernyshev
18
1k
The Power of CSS Pseudo Elements
geoffreycrofte
77
5.9k
Optimizing for Happiness
mojombo
379
70k
Designing Experiences People Love
moore
142
24k
Site-Speed That Sticks
csswizardry
10
760
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
It's Worth the Effort
3n
185
28k
Making Projects Easy
brettharned
117
6.3k
Building an army of robots
kneath
306
45k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
31
2.5k
Connecting the Dots Between Site Speed, User Experience & Your Business [WebExpo 2025]
tammyeverts
8
430
Transcript
Go Conference 2018 Spring Masahiro Sano (@kazegusuri) Testing with microservices
in merpay
Agenda • ࣗݾհ • ϝϧϖΠͷܾࡁγεςϜ • ςετϑϨʔϜϫʔΫ • ςετͷछྨ εϥΠυ
Speaker Deck ʹ͋Γ·͢ʂ https://speakerdeck.com/kazegusuri
ࣗݾհ • ࠤ ਖ਼ߒ (@kazegusuri) • Mercari, Inc → Souzoh,
Inc → merpay, Inc • Principal Software Engineer • Backend Engineer • GoͰܾࡁγεςϜશൠΛ։ൃத • ͖ͳ͜ͱ • gRPC
@kazegusuri Go Conference 2016 Spring GRPCͷ࣮ફͱ ݱঢ়Ͱͷརɾܽ
ϝϧΧϦʹ͓͚ΔgRPC • 2015ࠒʹϝϧΧϦͰॳΊͯgRPCΛಋೖ • ͜ΕܾࡁγεςϜͷͨΊͷϚΠΫϩαʔϏεͷҰͭ • ͦͯ͠ϝϧϖΠ
None
None
ϝϧϖΠͷٕज़ελοΫ • Go + gRPC • Microservices Architecture • Kubernetes
(GKE) + (EKS?) • ͦͷଞ • CircleCI, Spinnaker • Stackdriver Logging, Profiler • Datadog (Monitoring, Tracing) • Pagerduty, Sentry
ܾࡁͷٕज़తͳ͠͞ͱָ͠͞ • ܾࡁ͍͠ (ͱݴΘΕΔ) • ܾࡁϓϩόΠμͱͷҰ؏ੑ • ΫϦςΟΧϧ • 0.1%ͷΤοδέʔεΛߟྀͨ͠γεςϜ
• ΤϯδχΞϦϯάͰղܾ
ϚΠΫϩαʔϏεͱҰ؏ੑ • Microservices Architecture is super difficult!! • Monolithic ArchitectureͰτϥϯβΫγϣϯͰ݁
• Microservices ArchitectureͰඞવతʹτϥϯβΫγϣϯ • ݁Ռ߹ੑ • Idempotency • Retry, retry, retry • Async, Message Queue
ϝϧϖΠͷܾࡁγεςϜ • Microservices Architecture • ػೳຖʹϚΠΫϩαʔϏεΛߏங • த֩ͱͳΔܾࡁγεςϜ͕֤ϚΠΫϩαʔϏεͷঢ়ଶΛཧ • ঢ়ଶભҠϞσϧ
• ঢ়ଶ͕֬ఆ͢Δ·ͰແݶʹϦτϥΠ • ͲΜͳλΠϛϯάͰམͪͯ࠶։Մೳ
ϝϧϖΠͷܾࡁγεςϜ Payment Point Sales Microservices Payment Provider 1 2 3
4 Init 1 2 3 4 Fin
Payment Point Sales Microservices Payment Provider 1 2 3 4
Init 1 2 3 4 Fin Keep retrying with delay ϝϧϖΠͷܾࡁγεςϜ
ਓྨ͕ਖ਼࣮͘͢͠Δͷ͍͠
Ͳ͏ͬͯςετ͢Δ͔ • ڞ௨ͷςετϑϨʔϜϫʔΫΛ࡞ • ֤ঢ়ଶຖʹ͋Δ͖ঢ়ଶ(σʔλͷଘࡏɺ)Λఆٛ • ঢ়ଶભҠͤ͞ͳ͕Βঢ়ଶΛ֬ೝ • ࠷ऴతʹΰʔϧ(डཧঢ়ଶ)ʹͳΔ •
͜ͷςετΛਖ਼ৗܥ͚ͩͰͳ༷͘ʑͳҟৗঢ়ଶΛద༻ͯ͠ਖ਼͘͠ಈ࡞͢ Δ͔֬ೝ͢Δ
ςετϑϨʔϜϫʔΫͷαϯϓϧίʔυ
Failure Injection • ֎෦௨৴ʹରͯ͠ҙͷλΠϛϯάͰΤϥʔΛൃੜͤ͞Δ • MySQL • ֎෦αʔϏεͷHTTPϦΫΤετ • ෦αʔϏεͷgRPCϦΫΤετ
• ༷ʑͳΤϥʔύλʔϯͰঢ়ଶભҠ͕ਖ਼͘͠ߦΘΕΔ͔֬ೝ
Failure Injection (MySQL) • ΤϥʔλΠϛϯά • Query • Execͷྃޙ •
Commitͷྃޙ • શͯͷMySQL queryʹରͯ͠hook͢Δ • github.com/shogo82148/go-sql-proxy • ༷ʑͳhook͕ࠐΊͯศར!
Failure Injection (MySQL)
Failure Injection (HTTP) • ΤϥʔλΠϛϯά • HTTPϦΫΤετͷྃޙ • ಛఆͷαʔϏεʹର͢Δ௨৴શͯhook͢Δ •
net/http.RoundTripper ʹΑΔ࣮
Failure Injection (gRPC) • ΤϥʔλΠϛϯά • gRPCϦΫΤετͷྃޙ • ಛఆͷαʔϏεʹର͢Δ௨৴શͯhook͢Δ •
Client InterceptorʹΑΔ࣮ • ·ͩಋೖͰ͖͍ͯͳ͍
Random or Deterministic? • Ͳ͏͍͏݅ͰΤϥʔΛൃੜͤ͞Δ͔ • ΤϥʔΛϥϯμϜͰൃੜͤ͞Δͱςετ͕ෆ҆ఆʹͳΔ • Τϥʔ֬Λ͋͛Δͱޭ͠ͳ͘ͳΓςετ͕࣌ؒ͘ͳΔ •
Τϥʔͷൃੜܾఆతʹߦ͍͍ͨ • stacktrace(runtime.Callers) Ͱܾఆ • Ұൃੜͨ͠ύεͰ2ͱൃੜͤ͞ͳ͍
αʔϏεͷϦΫΤετͷςετ • ςετͰ֎෦ɾ෦αʔϏεΛ͏ͷ͍͠ • ϩʔΧϧͰͷߏங(docker)ɺσʔλͷॳظԽɺςετؒͷґଘ • Mock୯७ͳUnitςετʹྑ͍ • ϦΫΤετʹର͢ΔϨεϙϯε͕ݻఆ͞Εͨঢ়ଶͰͷϦάϨογϣϯ ςετ
• ༧ଌෆೳͳঢ়ଶͰͷςετ
Fake Implementation • αʔϏεͷ୯७Խ͞Ε࣮ͨ • σʔλϕʔεͳͲʹґଘͤͣʹߴʹಈ࡞ • ϥΠϒϥϦଆͰఏڙ͍ͯ͠Δͷ͋Δ • Google
Cloud Pub/Sub • ༻్ʹΑ༷ͬͯʑͳϨϕϧͰ࡞͢Δ • αʔϏεɺpackage, interface
Google Cloud Pub/Sub ͷ Fake
Conformance Test for Fake • FakeͷৼΔ͍͕ຊདྷͷ࣮ͱဃ͍ͯ͠ͳ͍͔ • ظ͢ΔৼΔ͍Λςετέʔεʹ͢Δ • ຊདྷͷ࣮ͱFakeͷ྆ํʹର࣮ͯ͠ߦ
None
ςετͷछྨ • Unit Test / Integration Test • E2E Test
• Chaos Test • QA
Unit and Integration Test • packageͰߦ͏ςετ • ݫີͳ୯ମςετʹ͍ͯ͠ͳ͍ • ԼҐϨΠϠ·ͰؚΊͨςετ
• ґଘinterfaceશͯΠϯδΣΫγϣϯ͢Δ • ґଘαʔϏεͷଓFakeΛར༻ • σʔλϕʔεͷڞ༗͕
σʔλϕʔεͷςετؒͷಠཱੑ • ֤ςετͰDB·Ͱଓ͢ΔͱpackageؒͷςετͰ่յ͢Δ͜ͱ͕ଟʑ • DB͕ڞ༗͞Ε͍ͯΔ͜ͱ͕ • ͳΒDBΛςετຖʹཱͯΕྑ͍ʂ • TestMainͰpackageຖʹಠཱͨ͠databaseΛ࡞(instanceͰͳ͍) •
1ϑΝΠϧʹ·ͱΊͨschemaΛ༻ҙ • multiStatements=trueʹ͢ΔͱExecͰෳΫΤϦΛ·Δ͝ͱ͛ΒΕΔ • ࡞ʹ͔͔Δ࣌ؒ1ඵ΄Ͳ
E2E Test • ࣮ࡍʹαʔόΛىಈͯ͠ϦΫΤετΛ͛ͯৼΔ͍Λςετ • ՄೳͳݶΓґଘαʔϏεىಈ͢Δ • docker-composeͰڥߏங • ςετؒͷಠཱੑςετηοτຖʹಠཱͨ͠σʔλΛ࡞Δ
• ͜͜·ͩਏ͍ͱ͜Ζ͕ଟ͍ • E2EςετͰෳࡶͳςετΛ࣮ࢪ • ಉ࣌ϦΫΤετڝ߹͢ΔΑ͏ͳϦΫΤετ
Chaos Test (Trying) • github.com/Shopify/toxiproxy • framework for simulating network
conditions • Ԇͤͨ͞ΓଳҬ੍ݶͨ͠Γམͱͨ͠ΓͰ͖Δ • gRPCͩͱ͍Ͳ͜Ζ͕͍͠ • ࠓͷͱ͜ΖԆΛൃੜͤͯ͞ݱ࣮తͳϦΫΤετͷϨΠςϯγͷ ༧ଌʹར༻
QA (Quality Assurance) • PostmanΛͬͨAPIςετ • JSONͰϦΫΤετ͢ΔͨΊʹgrpc-gatewayΛར༻ • PostmanͷςετέʔεGitHubͰཧͯ͠CI •
QAͷਓ͕ࣗߦ͍ͬͯΔ!
·ͱΊ • ϚΠΫϩαʔϏε͍͠ • ςετϑϨʔϜϫʔΫͰςετ߲ͷύλʔϯԽ • Failure Injection TestͰΤοδέʔεͷόάݟ͚ͭ͘͢ •
MockΑΓFake
We’re Hiring! https://www.merpay.com/jp/careers/index.html