Long journey of Continuous Delivery at Mercari

Japan Community Day at KubeCon + CloudNativeCon Japan 2025
https://community.cncf.io/events/details/cncf-cloud-native-community-japan-presents-japan-community-day-at-kubecon-cloudnativecon-japan-2025/

This presentation details Mercari's evolving approach to Continuous Delivery. It covers Mercari's adoption of microservices on Kubernetes and outlines our journey through distinct phases.
This presentation provides insights into the implementation, challenges, and evolution of Continuous Delivery within a large-scale microservices environment.

Transcript

1. 1 Long journey of Continuous Delivery at Mercari @hisaharu (Hisaharu Ishii)
2. 2 @hisaharu @hisaharu • Ex-NTT, Ex-Google 
 • Joined Mercari in 2023 
 • Platform CI/CD team 
 • Leading a Continuous Delivery Platform migration program 
 • #シェル芸 
 @hisaharu (Hisaharu Ishii) 

3. 3 Microservices at Mercari Agenda Startup Phase (2018~) Expansion Phase (2021~) Migration journey (2023~) 02 03 04 01
4. 4 01 Microservices at Mercari 02.Startup 03.Expansion 04.Migration 01.Microservices
5. 5 Microservices Infrastructure on Kubernetes as of 2024 https://speakerdeck.com/tcnksm/platform-engineering-at-mercari-platform-engineering-kaigi-2024?slide=10 02.Startup 03.Expansion 04.Migration 01.Microservices
6. 6 Software Development Life Cycle https://speakerdeck.com/tcnksm/platform-engineering-at-mercari-platform-engineering-kaigi-2024?slide=9 02.Startup 03.Expansion 04.Migration 01.Microservices
7. 7 source code repos for microservices 1. commit a change CI resources 2. test and build Kubernetes clusters Customers What are CI and CD for Kubernetes apps? CD Developers container image registry 3. push container image 02.Startup 03.Expansion 04.Migration 01.Microservices
8. 8 source code repos for microservices 1. commit a change CI resources 2. test and build Kubernetes clusters Customers What are CI and CD for Kubernetes apps? CD How to manage 
 delivery configurations? 
 Developers container image registry 3. push container image 02.Startup 03.Expansion 04.Migration 01.Microservices
9. 9 02 Startup Phase (2018~) 03.Expansion 04.Migration 01.Microservices 02.Startup
10. 10 source code repos for microservices 1. commit a change CI container image registry resources 2. test and build 3. push container image Kubernetes clusters Spinnaker pipeline Customers CD with Spinnaker manifest monorepo write manifest configure pipeline PubSub 4. trigger pipeline 5. deploy manifest Developers 03.Expansion 04.Migration 01.Microservices 02.Startup
11. 11 03.Expansion 04.Migration 01.Microservices 02.Startup
12. 12 Pull-based and Push-based GitOps メルカリのマイクロサービスにおける Kubernetes manifest の運用 https://k8sjp.connpass.com/event/124114/ 03.Expansion 04.Migration 01.Microservices 02.Startup
13. 13 Monorepo for Kubernetes manifests メルカリのマイクロサービスにおける Kubernetes manifest の運用 https://k8sjp.connpass.com/event/124114/ 03.Expansion 04.Migration 01.Microservices 02.Startup
14. 14 GitOps with Spinnaker メルカリのマイクロサービスにおける Kubernetes manifest の運用 https://k8sjp.connpass.com/event/124114/ 03.Expansion 04.Migration 01.Microservices 02.Startup
15. 15 source code repos for microservices 1. commit a change CI container image registry resources 2. test and build 3. push container image Kubernetes clusters Spinnaker pipeline Customers CD with Spinnaker manifest monorepo write manifest configure pipeline PubSub 4. trigger pipeline 5. deploy manifest Developers 03.Expansion 04.Migration 01.Microservices 02.Startup
16. 16 03 Expansion Phase (2021~) 02.Startup 04.Migration 01.Microservices 03.Expansion
17. 17 1. Reliability / performance in large scale 2. Security concern 3. WebUI-based pipeline config management 4. Momentum of OSS community Challenges with Spinnaker 02.Startup 04.Migration 01.Microservices 03.Expansion
18. 18 02.Startup 04.Migration 01.Microservices 03.Expansion
19. 19 source code repos for microservices 1. commit a change CI resources 2. test and build 4. update image tag manifest monorepo Kubernetes clusters Customers CD with Warp Speed CD PubSub write manifest Developers delivery configs configure delivery Tekton ArgoCD 5.trigger pipeline 6. run pre-process 7. trigger sync 8. sync manifest container image registry 3. push container image Flux 02.Startup 04.Migration 01.Microservices 03.Expansion
20. 20 1. Reliability / performance in large scale 2. Security concern 3. WebUI-based pipeline config management 4. Momentum of OSS community Design concepts of Warp Speed CD 1. Pull-based, reconciliation loop 2. K8s native tenant isolation 3. GitOps for pipeline config management 4. Active OSS community 02.Startup 04.Migration 01.Microservices 03.Expansion
21. 21 Spinnaker Momentum of OSS community ArgoCD 02.Startup 04.Migration 01.Microservices 03.Expansion
22. 22 source code repos for microservices 1. commit a change CI resources 2. test and build 4. update image tag manifest monorepo Kubernetes clusters Customers CD with Warp Speed CD PubSub write manifest Developers delivery configs configure delivery Tekton ArgoCD 5.trigger pipeline 6. run pre-process 7. trigger sync 8. sync manifest container image registry 3. push container image Flux 02.Startup 04.Migration 01.Microservices 03.Expansion
23. 23 Platform Engineering = Migration Engineering 02.Startup 04.Migration 01.Microservices 03.Expansion
24. 24 04 Migration journey (2023~) 02.Startup 03.Expansion 01.Microservices 04.Migration
25. 25 • Total number of applications: 456 • Total number of pipelines: 2854 ◦ Templated pipelines: 678 ◦ Manually created pipelines: 2176 Spinnaker usages (as of 2022) 02.Startup 03.Expansion 01.Microservices 04.Migration
26. 26 • Make it default for new microservices • Focus on heavy users • Automation • Build momentum by migrating simple use cases Migration strategy for Warp Speed CD 02.Startup 03.Expansion 01.Microservices 04.Migration
27. 27 Make momentum by migrating simple use cases 02.Startup 03.Expansion 01.Microservices 04.Migration
28. 28 Migration helper tool manifest monorepo GitHub Actions workflow Secret Manager 1. checkout 2. load keys conversion logics 3. execute 4. Open a PR Spinnaker 3-1. fetch pipeline configs Team metadata server 3-2. fetch Slack channel and members 3-3. generate Warp Speed CD configs 02.Startup 03.Expansion 01.Microservices 04.Migration
29. 29 Conclusion
30. 30 • Adopted microservices on Kubernetes • Our initial CD approach was GitOps with Spinnaker • Developed “Warp Speed CD” to overcome challenges • We are still on a long migration journey Long journey of Continuous Delivery at Mercari
31. 31 Thank you!
32. 32 Challenges with Warp Speed CD • Huge manifest monorepo ◦ Flux image reflection ◦ PR throughput • Extensibility ◦ Tekton workflow ◦ Testability • BCP/DR ◦ Region-wide outage
33. 33 Material credits All product names, logos, and brands are property of their respective owners. Platform Engineering at Mercari (Platform Engineering Kaigi 2024), taichi nakashima https://speakerdeck.com/tcnksm/platform-engineering-at-mercari-platform-engineering-kaigi-2024 メルカリのマイクロサービスにおける Kubernetes manifest の運用, Masaki Ishiyama, https://k8sjp.connpass.com/event/124114/, https://www.youtube.com/watch?v=5NL1tEIcU-o GitHub, https://github.com/logos GitHub Actions, https://github.com/actions Kubernetes® and are registered trademarks of The Linux Foundation in the United States and other countries, and is used pursuant to a license from The Linux Foundation. Spinnaker™, 　　, Tekton™,　　, ArgoCD™,　　, Flux™,　　, are trademarks of The Linux Foundation. Kubernetes Icon Set, CC-BY-4.0, https://creativecommons.org/licenses/by/4.0/deed.en Slack is a trademark and service mark of Slack Technologies, Inc., registered in the U.S. and in other countries. Google Cloud Platform™ service is a trademark of Google LLC 　 X, https://about.x.com/en/who-we-are/brand-toolkit