Platform Engineering at Mercari

by taichi nakashima

Slide 1

Slide 1 text

Platform Engineering at Mercari Platform Engineering Meetup #2

Slide 2

Slide 2 text

Taichi Nakashima @deeeet Director of Platform Engineering

Slide 3

Slide 3 text

Agenda  ● Overview of Mercari Developer Platform ● Lessons learned

Slide 4

Slide 4 text

Overview of Mercari Developer Platform

Slide 5

Slide 5 text

Mission of Platform Engineering at Mercari  Providing core infrastructure, toolings and workﬂows for Mercari Group developers so that they can deliver new value and experience to Mercari customers quickly and reliably.

Slide 6

Slide 6 text

Slide 7

Slide 7 text

Platform ？

Slide 8

Slide 8 text

Slide 9

Slide 9 text

Build Test Deploy Operate Infrastructure Network Data Interface Documentation On-support Production readiness check Service catalog Template project Common language package/library Protobuf management CI platform QA environment & automation Cloud administration Kubernetes administration Terraform & Kubernetes monorepo CDN administration Cloud network Service mesh Data pipeline and lake Data governance (lineages) CD platform Experimentation platform Supply Chain Security protection Delivery conﬁguration management Observability platform Production access managements

Slide 10

Slide 10 text

Build Test Deploy Operate Platform DX CI/CD Infrastructure Network Platform Infra Network Interface Data Data Operations Experimentation

Slide 11

Slide 11 text

Lessons Learned

Slide 12

Slide 12 text

Lessons Learned  ● Business as First Priority ● Platform as Product ● Start from “Collaboration” ● Keep Appropriate Distance from Stream-aligned Team ● Platform Engineering = Migration Engineering

Slide 13

Slide 13 text

Slide 14

Slide 14 text

Business as First Priority 

Slide 15

Slide 15 text

Slide 16

Slide 16 text

Platform as Product

Slide 17

Slide 17 text

Platform as Product  "Platform as a Product" is a mindset where the internal developer platform is managed continuously like “Product”, by considering developers as “customer,” focusing on meeting its needs. Unlike a Project, which has a finite life cycle with a clear start and end based on achieving specific goals, this approach sees the platform as an ongoing concern that requires continuous refinement and improvement.

Slide 18

Slide 18 text

https://miro.com/app/board/uXjVPhewWSg=/

Slide 19

Slide 19 text

Discovery Exploring the issues developers are facing Evaluating the solution works or not. Deciding which issues to address in which order. Prioritization Validation

Slide 20

Slide 20 text

Slide 21

Slide 21 text

Start from “Collaboration”

Slide 22

Slide 22 text

https://teamtopologies.com/book

Slide 23

Slide 23 text

X-as-a-Service / Self-service Stream-Aligned Teams Platform Team

Slide 24

Slide 24 text

Collaboration X-as-a-Service / Self-service

Slide 25

Slide 25 text

Start from “Collaboration”  ● Easy Discovery & Prioritization: Platform team can actually facing the issue as Stream-aligned team and understand it clearly ● Easy Validation: Platform Team can get feedback directly from the team and iterate quickly

Slide 26

Slide 26 text

Slide 27

Slide 27 text

Keep Appropriate Distance from Stream-aligned Teams

Slide 28

Slide 28 text

Keep Appropriate Distance from Stream-aligned Teams

Slide 29

Slide 29 text

Collaboration X-as-a-Service / Self-service

Slide 30

Slide 30 text

Force of “Collaboration” … When “Collaboration”, there is a natural force to be the same team forever e.g., Platform Team is taking all operation or On-calling 👉 Platform becomes speciﬁc to the team. It can not be scaled Collaboration Getting too close

Slide 31

Slide 31 text

Against the force of “Collaboration”  ● Have a deadline of the “Collaboration” ● Keep future “X-as-a-Service” in mind ● Have a strict line on what NOT to do

Slide 32

Slide 32 text

Getting too further away Force of X-as-a-Service … When “X-as-a-Service”, there is a natural force to getting further and further away from the stream aligned team. Platform Team only cares about its internal … Steam-aligned Team does not know what Platform Team is doing … 👉 Platform Team can not properly discover the issue to be solved or validate the solution … X-as-a-Service

Slide 33

Slide 33 text

Against the force of “X-as-a-Service”  ● Keep the visibility of Platform activity ● Have direct feedback channels with Stream-aligned team ● Use “Collaboration” for exploring and validation

Slide 34

Slide 34 text

Slide 35

Slide 35 text

Platform Engineering = Migration Engineering

Slide 36

Slide 36 text

Migration Project at Mercari  ● Replacing legacy infrastructure/system with new platform ○ e.g., Moving monolith system from On-prem environment ○ e.g., Moving SaaS based CI platform to self-hosted ● Enabling new features ○ e.g., Migrating to GKE workload Identity ● Applying best practices to the service running on the platform ○ e.g., Improving resource utilization (FinOps)

Slide 37

Slide 37 text

Time Number of internal tools With Migration Without Migration Reducing the maintenance costs and technical debt Increasing the leverage of tooling improvements to everyone Keeping the system complexity lows Lots of similar tool is built but not maintained…

Slide 38

Slide 38 text

“X-as-a-Service” Makes Migration Hard  ● X-as-a-Service/Self-service means delegating responsibility of workﬂows or conﬁguration management to Stream-aligned Teams ○ = Platform Teams can not modify them ● Some can be only by Platform Team but the most of migration requires support from Stream-aligned Teams …

Slide 39

Slide 39 text

Tips: Successful Migration  ● Design with Migration in mind ● Start from “Collaboration” ● Reduce the cost of migration as much as possible ● Communicate, communicate, communicate

Slide 40

Slide 40 text

Conclusion  ● Business as First Priority ● Platform as Product ● Start from “Collaboration” ● Keep Appropriate Distance from Stream-aligned Team ● Platform Engineering = Migration Engineering