MTC2018 - Mercari API: from Monolithic to Microservices

Slide 1

Slide 1 text

Mercari API: from Monolithic to Microservices Tonghui Li Tech Lead (Backend)

Slide 2

Slide 2 text

Tech Lead Backend Tonghui Li

Slide 3

Slide 3 text

Mercari API: from Monolithic to Microservices

Slide 4

Slide 4 text

Background

Slide 5

Slide 5 text

Mercari API

Slide 6

Slide 6 text

Mercari API iOS Android web

Slide 7

Slide 7 text

Mercari API iOS Android web

Slide 8

Slide 8 text

Mercari API iOS Android web admin tools

Slide 9

Slide 9 text

Mercari API iOS Android web admin tools

Slide 10

Slide 10 text

● 500 KLOC ● 100+ contributors ● 100 PR / week Mercari API iOS Android web admin tools

Slide 11

Slide 11 text

● 500 KLOC ● 100+ contributors ● 100 PR / week ● And growing! Mercari API iOS Android web admin tools

Slide 12

Slide 12 text

How to migrate to microservices

Slide 13

Slide 13 text

How to migrate to microservices ● Implement new features in microservices ● Break down existing API into multiple services

Slide 14

Slide 14 text

How to implement new features

Slide 15

Slide 15 text

client Mercari API

Slide 16

Slide 16 text

client Service X Service Y Mercari API

Slide 17

Slide 17 text

client Service X Service Y Mercari API

Slide 18

Slide 18 text

API Gateway Service X Service Y Mercari API client

Slide 19

Slide 19 text

API Gateway Service X Service Y Mercari API client SSL termination DDos protection Authn and Authz Traffic routing Logging and tracing

Slide 20

Slide 20 text

Technical Stack

Slide 21

Slide 21 text

API Gateway Service X Service Y Mercari API client Sakura

Slide 22

Slide 22 text

API Gateway Service X Service Y Mercari API client Sakura GCP

Slide 23

Slide 23 text

API Gateway Service X Service Y Mercari API client Sakura kubernetes GCP

Slide 24

Slide 24 text

API Gateway Service X Service Y Mercari API client Sakura kubernetes GCP

Slide 25

Slide 25 text

API Gateway Service X Service Y Mercari API client Sakura kubernetes GCP

Slide 26

Slide 26 text

API Gateway Service X Service Y Mercari API client Sakura kubernetes GCP

Slide 27

Slide 27 text

How to break down existing API

Slide 28

Slide 28 text

API Gateway Service X Service Y Mercari API client

Slide 29

Slide 29 text

API Gateway Service X Service Y Mercari API client

Slide 30

Slide 30 text

Start from UX features

Slide 31

Slide 31 text

Mercari Login Buy Sell Search Shipping ...

Slide 32

Slide 32 text

Analyze fundamental components

Slide 33

Slide 33 text

Mercari Login Buy Sell Search Shipping ... User Item Photo User Item Address

Slide 34

Slide 34 text

Split them into microservices

Slide 35

Slide 35 text

Gateway Login Buy Sell Search Shipping ... User Item Photo Notification Address ...

Slide 36

Slide 36 text

Gateway Login Buy Sell Search Shipping ... User Item Photo Notification Address ...

Slide 37

Slide 37 text

Gateway Login Buy Sell Search Shipping ... User Item Photo Notification Address ... BFF Layer DAO Layer notf db user db item db s3

Slide 38

Slide 38 text

In reality...

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

How to migrate to microservices ● Implement new features in microservices ○ API Gateway ○ Technical stack: Kubernetes, Go, gRPC, etc. ● Break down existing API into multiple services ○ Start from UX features ○ Analyze fundamental components ○ Split them into microservices

Slide 41

Slide 41 text

Challenges

Slide 42

Slide 42 text

Challenges: Testing With so many services, how do we test them?

Slide 43

Slide 43 text

Challenges: Testing With so many services, how do we test them? ● Enforce unit tests coverage ● Automated integration tests ● Reduce manual QA process

Slide 44

Slide 44 text

Challenges: DB DB is in Sakura but microservices are in GCP.

Slide 45

Slide 45 text

Challenges: DB DB is in Sakura but microservices are in GCP. ● Run DAO layer services in Sakura ● Migrate DB from Sakura to GCP ● Create Tokyo DC and host DB / services in it