Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Platform Engineering at Mercari

taichi nakashima
May 15, 2023
4.6k

Platform Engineering at Mercari

Talk at Platform Engineering Meetup #2 https://platformengineering.connpass.com/event/280339/

taichi nakashima

May 15, 2023
Tweet

Transcript

  1. Mission of Platform Engineering at Mercari
 Providing core infrastructure, toolings

    and workflows for Mercari Group developers so that they can deliver new value and experience to Mercari customers quickly and reliably.
  2. Mission of Platform Engineering at Mercari
 Providing core infrastructure, toolings

    and workflows for Mercari Group developers so that they can deliver new value and experience to Mercari customers quickly and reliably.
  3. Mission of Platform Engineering at Mercari
 Providing core infrastructure, toolings

    and workflows for Mercari Group developers so that they can deliver new value and experience to Mercari customers quickly and reliably.
  4. Build Test Deploy Operate Infrastructure Network Data Interface Documentation On-support

    Production readiness check Service catalog Template project Common language package/library Protobuf management CI platform QA environment & automation Cloud administration Kubernetes administration Terraform & Kubernetes monorepo CDN administration Cloud network Service mesh Data pipeline and lake Data governance (lineages) CD platform Experimentation platform Supply Chain Security protection Delivery configuration management Observability platform Production access managements
  5. Build Test Deploy Operate Platform DX CI/CD Infrastructure Network Platform

    Infra Network Interface Data Data Operations Experimentation
  6. Lessons Learned
 • Business as First Priority • Platform as

    Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
  7. Lessons Learned
 • Business as First Priority • Platform as

    Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
  8. Lessons Learned
 • Business as First Priority • Platform as

    Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
  9. Platform as Product
 "Platform as a Product" is a mindset

    where the internal developer platform is managed continuously like “Product”, by considering developers as “customer,” focusing on meeting its needs. Unlike a Project, which has a finite life cycle with a clear start and end based on achieving specific goals, this approach sees the platform as an ongoing concern that requires continuous refinement and improvement.
  10. Discovery Exploring the issues developers are facing Evaluating the solution

    works or not. Deciding which issues to address in which order. Prioritization Validation
  11. Lessons Learned
 • Business as First Priority • Platform as

    Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
  12. Start from “Collaboration”
 • Easy Discovery & Prioritization: Platform team

    can actually facing the issue as Stream-aligned team and understand it clearly • Easy Validation: Platform Team can get feedback directly from the team and iterate quickly
  13. Lessons Learned
 • Business as First Priority • Platform as

    Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
  14. Force of “Collaboration” … When “Collaboration”, there is a natural

    force to be the same team forever e.g., Platform Team is taking all operation or On-calling 👉 Platform becomes specific to the team. It can not be scaled Collaboration Getting too close
  15. Against the force of “Collaboration”
 • Have a deadline of

    the “Collaboration” • Keep future “X-as-a-Service” in mind • Have a strict line on what NOT to do
  16. Getting too further away Force of X-as-a-Service … When “X-as-a-Service”,

    there is a natural force to getting further and further away from the stream aligned team. Platform Team only cares about its internal … Steam-aligned Team does not know what Platform Team is doing … 👉 Platform Team can not properly discover the issue to be solved or validate the solution … X-as-a-Service
  17. Against the force of “X-as-a-Service”
 • Keep the visibility of

    Platform activity • Have direct feedback channels with Stream-aligned team • Use “Collaboration” for exploring and validation
  18. Lessons Learned
 • Business as First Priority • Platform as

    Product • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering
  19. Migration Project at Mercari
 • Replacing legacy infrastructure/system with new

    platform ◦ e.g., Moving monolith system from On-prem environment ◦ e.g., Moving SaaS based CI platform to self-hosted • Enabling new features ◦ e.g., Migrating to GKE workload Identity • Applying best practices to the service running on the platform ◦ e.g., Improving resource utilization (FinOps)
  20. Time Number of internal tools With Migration Without Migration Reducing

    the maintenance costs and technical debt Increasing the leverage of tooling improvements to everyone Keeping the system complexity lows Lots of similar tool is built but not maintained…
  21. “X-as-a-Service” Makes Migration Hard
 • X-as-a-Service/Self-service means delegating responsibility of

    workflows or configuration management to Stream-aligned Teams ◦ = Platform Teams can not modify them • Some can be only by Platform Team but the most of migration requires support from Stream-aligned Teams …
  22. Tips: Successful Migration
 • Design with Migration in mind •

    Start from “Collaboration” • Reduce the cost of migration as much as possible • Communicate, communicate, communicate
  23. Conclusion
 • Business as First Priority • Platform as Product

    • Start from “Collaboration” • Keep Appropriate Distance from Stream-aligned Team • Platform Engineering = Migration Engineering