MTC2018 - Microservices Platform at Mercari

Speaker: 中島 大一(Taichi Nakashima)

Mercariではサービスと組織の拡大に合わせてMonolithアーキテクチャからMicroservicesアーキテクチャへの移行を進めています．Microservices Platform TeamはこのMicroservicesのためのPlatformの構築を行っています．開発者のDeveloper Productivityを最大限に高め各チームがオーナーシップを持ち独立して高速にサービス開発サイクルを回せるようなPlatformを目指しています．本セッションではMicroservices Platform Teamの取り組みを紹介します。

As Mercari continues to grow as a service and organization, we decided it was time to make the move from a monolith architecture to microservices. The Microservices Platform Team is the team responsible for building the platform for this. The goal is to create a platform that allows developers to maximize their productivity for a rapid service development cycle where teams are independent and have ownership over their projects. In this session, Nakashima will introduce the initiatives the Microservices Platform Team are taking to achieve this goal.

Transcript

1. Microservices Platform at Mercari Taichi Nakashima Tech Lead (Microservices platform)
2. Tech Lead Microservices platform Taichi Nakashima
3. Microservices https://www.flickr.com/photos/ronduong/37605344086
4. 1000 Engineers Org. https://www.flickr.com/photos/owyzzz/3626718695
5. Speed https://www.flickr.com/photos/szymipl/6093250294
6. https://itrevolution.com/book/accelerate/ ACCELERATE The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
7. https://itrevolution.com/book/accelerate/
8. https://itrevolution.com/book/accelerate/
9. https://itrevolution.com/book/accelerate/ Organization Design
10. Develop Monolith Test Deploy Operate
11. Develop Monolith Test Deploy Operate Backend Team QA team SRE team
12. Test Deploy Operate Backend Team QA team SRE team Develop Service B Develop Service A Develop Service C
13. Test Deploy Operate Backend Team SRE team Develop Service B Develop Service A Develop Service C QA team
14. Test Deploy Operate Backend Team SRE team Develop Service B Develop Service A Develop Service C QA team
15. Test Deploy Operate Backend Team SRE team Develop Service B Develop Service A Develop Service C QA team
16. Develop Service B Test Deploy Operate Service B Team Develop Service A Test Deploy Operate Service A Team Develop Service C Test Deploy Operate Service C Team
17. https://medium.com/netflix-techblog/full-cycle-developers-at-netflix-a08c31f83249
18. Develop Service B Test Deploy Operate Service Team B Develop Service A Test Deploy Operate Service Team A Develop Service C Test Deploy Operate Service Team C
19. Develop Service B Test Deploy Operate Service Team B Develop Service A Test Deploy Operate Service Team A Develop Service C Test Deploy Operate Service Team C Developers need to act as SWE, SET and SRE
20. Develop Service B Test Deploy Operate Service Team B Develop Service A Test Deploy Operate Service Team A Develop Service C Test Deploy Operate Service Team C Too much expected
22. 1 year
23. 73 Developing 19 Production
24. What We Achieved
25. Pave the “Base” Road https://www.flickr.com/photos/martin_borjesson/7725354722
26. https://www.flickr.com/photos/martin_borjesson/7725354722 Architecture DevOps Culture
27. API Gateway Communication Template Access manage
28. MySQL Solr Monolith API
29. API gateway MySQL Solr Kubernetes Monolith API
30. API gateway Authority Service A MySQL Solr Kubernetes Monolith API
31. API gateway Authority Service A MySQL Solr Kubernetes Monolith API Service B
32. API gateway Authority Service A Spanner MySQL Solr Kubernetes Monolith API CloudSQL Service B
34. API gateway Authority Service A Spanner MySQL Solr Kubernetes Monolith API CloudSQL Service B
35. API Gateway Communication Template Access manage
36. API gateway Authority Service A CloudSQL Spanner MySQL Solr Kubernetes Service B Monolith API
39. Service Team PR generate platform-proto CircleCI
40. API Gateway Communication Template Access manage
43. Monitoring Error reporting Health checking Dockerfile CI setting
44. API Gateway Communication Template Access manage
45. API gateway Authority Service A CloudSQL Spanner MySQL Solr Kubernetes Service B Monolith API SRE team
46. API gateway Authority Service A CloudSQL Spanner MySQL Solr Kubernetes Service B Monolith API Service Team B Service Team A
47. GCP Project: Kubernetes Platform Team Kubernetes IAM
48. GCP Project: Kubernetes Platform Team Kubernetes IAM Service A Team
49. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes IAM RBAC Service A Team
50. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes IAM RBAC Container A Container A Container A Service A Team
51. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes IAM RBAC Container A Container A Container A GCP Project: Service A IAM Service A Team
52. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes IAM RBAC Container A Container A Container A Spanner GCP Project: Service A IAM Service A Team
53. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes IAM RBAC Container A Container A Container A Spanner GCP Project: Service A IAM Service B Team Service A Team
54. GCP Project: Kubernetes Platform Team Spanner Namespace: Service A Kubernetes GCP Project: Service A Namespace: Service B GCP Project: Service B RBAC IAM RBAC IAM IAM Container A Container A Container A Service B Team Service A Team
55. API Gateway Communication Template Access manage
56. https://www.flickr.com/photos/martin_borjesson/7725354722 Architecture DevOps Culture
57. https://www.flickr.com/photos/marc_sud13/8453080595 Provisioning Deploying
58. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes GCP Project: Service A RBAC IAM IAM Service A Team Spanner
59. Infrastructure as Code https://www.flickr.com/photos/trilanes/4714198782/
62. Starter-kit as terraform module
63. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes GCP Project: Service A Service A Team RBAC IAM IAM
65. GCP Project: Kubernetes Platform Team Namespace: Service A Kubernetes GCP Project: Service A Service A Team RBAC IAM IAM Spanner
66. PR Plan & Apply Platform Team Review microservices-terraform CircleCI Service A Team
67. Service B Team Service A Team Service C Team PR PR PR Plan & Apply Platform Team Review microservices-terraform CircleCI
68. PR Plan & Apply PR PR Platform Team Review microservices-terraform CircleCI Service B Team Service A Team Service C Team
69. PR Plan & Apply PR PR Platform Team Review microservices-terraform CircleCI Service B Team Service A Team Service C Team CODEOWNERS
70. Plan & Apply PR Platform Team Review microservices-terraform CircleCI Service B Team Service A Team Service C Team PR & Review PR & Review
73. https://www.flickr.com/photos/marc_sud13/8453080595 Provisioning Deploying
75. GCP Project: Kubernetes Namespace: Service A Kubernetes Namespace: Service B Container A Container A Container A Container A Container A Container B Service A Team Service B Team
77. 60+ pipelines
79. GUI ...
80. Infrastructure as Code https://www.flickr.com/photos/trilanes/4714198782/
81. Kubernetes v2 provider https://www.flickr.com/photos/trilanes/4714198782/
83. GCP Project: Kubernetes Namespace: Service A Kubernetes Namespace: Service B Container A Container A Container A Container A Container A Container B Service Team A Service Team B PR PR Sync Platform Team Review microservices-kubernetes
84. GCP Project: Kubernetes Namespace: Service A Kubernetes Namespace: Service B Container A Container A Container A Container A Container A Container B Service Team A Service Team B PR PR Sync Platform Team Review microservices-kubernetes Developers can learn k8s YAML, too
85. What’s Next?
86. 73 Developing 19 Production
87. 73 Developing 19 Production Reliable Service Operation
88. SLI/SLO Driven Operation https://www.flickr.com/photos/samarthshyamanur/8347584639
89. Chaos Testing https://www.flickr.com/photos/foxgrrl/9671201784/
90. Service Mesh https://www.flickr.com/photos/mjsonline/1971492464