Micoservices Platform in Mercari

https://mercari.connpass.com/event/94466/

Transcript

1. Backend Engineer Drink Meetup #2
   Aug 15, 2018 / @b4b4r07
   Microservices Platform in Mercari
   

   View Slide

2. @b4b4r07 / babarot
   Blog / tellme.tokyo
   SRE in Microservices Platform
   Team at Mercari, Inc.
   

   View Slide

3. Why Mercari
   Choose Microservices
   

   View Slide

4. 4
   ● To scale the service more than ever
   ○ * (Code base is getting bigger and complex day by day...)
   ○ Make it easy to try new feature / technology
   ○ Make it easy to catch up the service logic / domain
   ○ Cut down on-boarding cost
   ○ Speed up the deployment cycle
   ■ Reduce coordination with other teams
   Why Mercari choose Microservices
   

   View Slide

5. 5
   ● Fault Tolerance / Resilience
   ○ As the service grows, some features become important
   components that should not be down (e.g. Timeline)
   ○ Do not spread system failure to the whole
   ○ Cut down operation cost
   Why Mercari choose Microservices
   

   View Slide

6. What are Microservices?
   

   View Slide

7. 7
   ● Simpler to deploy / Easy to understand
   ● Reusability across business
   ● Faster defect isolation
   ● Minimize the risk of code changes
   What are Microservices?
   

   View Slide

8. 8
   ● Autonomous
   ○ (Units that can be deployed independently)
   ○ Isolate perspective:
   ■ Data consistency and transaction boundaries
   ■ Team, Technology, Cohesion, Security, ...
   What are Microservices?
   

   View Slide

9. 9
   ● Autonomous
   ○ (Units that can be deployed independently)
   ○ Isolate perspective:
   ■ Data consistency and transaction boundaries
   ■ Team, Technology, Cohesion, Security, ...
   What are Microservices?
   

   View Slide

10. 10
    Development in Monolith
    Develop
    Monolith
    QA Deploy Operate
    QA
    

    View Slide

11. 11
    Development in Monolith
    Develop
    Monolith
    QA Deploy Operate
    QA
    Backend Team A
    Backend Team B
    Backend Team C
    

    View Slide

12. 12
    Development in Monolith
    Develop
    Monolith
    QA Deploy Operate
    QA
    Backend Team A
    Backend Team B
    Backend Team C QA team
    

    View Slide

13. 13
    Development in Monolith
    Develop
    Monolith
    QA Deploy Operate
    QA team SRE team
    QA
    Backend Team A
    Backend Team B
    Backend Team C
    

    View Slide

14. 14
    Development in Microservices
    Develop
    Service B
    QA Deploy Operate
    QA team SRE team
    Service Team A
    Service Team B
    Service Team C
    Develop
    Service A
    Develop
    Service C
    

    View Slide

15. 15
    Development in Microservices
    Develop
    Service B
    QA Deploy Operate
    Service Team A
    Service Team B
    Service Team C
    Develop
    Service A
    Develop
    Service C
    QA Deploy Operate
    QA Deploy Operate
    

    View Slide

16. Why Microservices Platform
    

    View Slide

17. 17
    ● Two reasons why Platform is required
    ○ Team side
    ○ System side
    Why Microservices Platform
    

    View Slide

18. 18
    Why Platform is required: Team side
    

    View Slide

19. 19
    Develop
    Monolith
    QA Deploy Operate
    QA
    Backend Team A
    Backend Team B
    Backend Team C
    Develop
    Service B
    QA Deploy Operate
    Service Team A
    Service Team B
    Service Team C
    Develop
    Service A
    Develop
    Service C
    QA Deploy Operate
    QA Deploy Operate
    

    View Slide

20. 20
    Develop
    Monolith
    QA Deploy Operate
    QA
    Backend Team A
    Backend Team B
    Backend Team C
    Develop
    Service B
    QA Deploy Operate
    Service Team A
    Service Team B
    Service Team C
    Develop
    Service A
    Develop
    Service C
    QA Deploy Operate
    QA Deploy Operate
    New team responsibility
    

    View Slide

21. 21
    Why Platform is required: System Side
    

    View Slide

22. 22
    Monolithic App
    Data Access
    Layer
    Business Logic
    UI
    Database
    

    View Slide

23. 23
    Monolithic App
    Data Access
    Layer
    Business Logic
    UI
    Database
    Microservices Apps
    Units that can be deployed independently
    

    View Slide

24. 24
    Monolithic App
    Data Access
    Layer
    Business Logic
    UI
    Database
    Microservices Apps
    Units that can be deployed independently
    

    View Slide

25. 25
    Monolithic App
    Data Access
    Layer
    Business Logic
    UI
    Database
    Microservices Apps
    Units that can be deployed independently
    

    View Slide

26. 26
    Why Platform is required: System side
    ● Bootstrap & Provision cost
    ○ In case of Monolith,
    ■ the infra can be used for every features (reusability)
    ○ Unlike Monolith,
    ■ need to build the infra for each micro service
    

    View Slide

27. Microservices Platform
    

    View Slide

28. 28
    Microservices Platform
    ● Centralized GKE Cluster
    ○ 1 cluster per 1 region
    ● Cloud resources for each microservice
    ○ GCP Projects (Cloud SQL, Spanner, etc.)
    ○ PagerDuty
    ○ Datadog Dashboard
    

    View Slide

29. View Slide

30. 30
    GCP project for GKE
    Centralized cluster
    Namespace: Service A
    Namespace: Service B
    IAM: SRE IAM: Team A
    IAM: Team B
    Service A
    Service B
    RBAC: Team A
    RBAC: Team B
    

    View Slide

31. 31
    GCP project for GKE
    Centralized cluster
    Namespace: Service A
    Namespace: Service B
    IAM: SRE IAM: Team A
    Service A
    Service B
    RBAC: Team A
    RBAC: Team B
    Bootstrap
    Bootstrap
    IAM: Team B
    

    View Slide

32. ● Microservice Starter Kit
    ○ It became easy for the developers to bootstrap own
    infrastructure by themselves
    ○ Just by running 1 command locally,
    all resources needed for their microservice are created
    

    View Slide

33. Technical Stacks
    

    View Slide

34. View Slide

35. View Slide

36. View Slide

37. Work In Progress
    

    View Slide

38. Work In Progress
    

    View Slide

39. View Slide