Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MTC2018 - Web Application as a Microservice

mercari
PRO
October 04, 2018

MTC2018 - Web Application as a Microservice

Speaker: Sota Sugiura

Mercari Web was developed to support Mercari’s growth, and it has grown in tandem with Mercari for approximately two years. The Japan Web Re-architecture Team was created to improve Mercari Web by staying on top of the constantly evolving web-related technology and preparing for a growing frontend team. In this talk, Sugiura will talk about the details of the new architecture the Web Re-architecture Team is implementing, and what’s in store for Mercari Web after the new architecture is in place.

mercari
PRO

October 04, 2018
Tweet

More Decks by mercari

Other Decks in Technology

Transcript

  1. Web Application as a Microservice Sota Sugiura Tech Lead (Backend)

  2. Tech Lead Backend Sota Sugiura

  3. Web Application as a Microservice

  4. https://www.mercari.com/jp/

  5. JP Web Re-architect

  6. Improve scalability of the development team Goals A flexible architecture

    that is resilient to change 1 2
  7. Flexible architecture that is resilient to change • Keeping up

    to date with trends • Scrap and Build
  8. None
  9. Improving scalability of the development team • Architecture that can

    handle the increasing number of engineers • Providing freedom for each team to choose their own technology stack
  10. Single PHP Server Te m ar Te m ar Box

    Te m ar G i Now Mercari Mercari Box Mercari Guide
  11. SSR GraphQL SPA REST API Simple HTTP server Mic r

    i s Te m ar Mic r i s Te m ar Box Mic r i s Te m ar G i Goal
  12. How to Carry Out the Re-architecture

  13. 1 Monolithic Service to 4 Microservices

  14. Wait, can you migrate all features from a Monolithic Service

    at once? :P
  15. 1 Monolithic Service to with 4 Microservices

  16. Monolithic Service Feture ・ ・ ・ Monolithic Service Feature ・

    ・ ・ Microservice Microservice Microservice Microservice Microservice Microservice
  17. Why? • This is a re-architecture, not a redesign •

    Limiting the scope of migration at first • We will start by taking on many small challenges and failures
  18. Limiting the migration scope

  19. Monolithic Service Feature ・ ・ ・ /jp/*

  20. Monolithic Service Feature ・ ・ ・ Microservice Microservice /jp/* /jp/top

  21. How we decided the order of migration • Difficulty •

    Clashes with other tasks and changes • Amount of traffic
  22. Challenges

  23. Challenges Monolithic Service + Microservices 1 2 Separating Frontend and

    Backend 3 Session synchronization
  24. GraphQL SSR mercari-web mercari-api Web Gateway Session service CDN

  25. GraphQL SSR mercari-web mercari-api Web Gateway Session service CDN 1

    2 3
  26. GraphQL SSR mercari-web mercari-api Web Gateway Session service CDN 1

    2 3
  27. Web Gateway • Balancing for each service • Initially for

    both new and old architectures to co-exist • In the future, for Web Microservices
  28. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others CDN
  29. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others CDN /hoge=”Service A” /moge=”Service B”
  30. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others 0% 100% WI
  31. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others 10% 90% WI
  32. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others 100% 0% WI
  33. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others 10% 90% A B WI
  34. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others 10% 90% A B WI
  35. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others 10% 90% A B WI
  36. • Mapping with path control • Scoped release • Session

    persistence Web Gateway mercari-web Service A Service B /hoge /moge /others 10% 90% A B WI
  37. GraphQL SSR mercari-web mercari-api Web Gateway Session service CDN 1

    2 3
  38. How is frontend affected by the backend migration to Microservices?

  39. mercari-web mercari-api JSON over HTTPs

  40. mercari-web mercari-api JSON over HTTPs SSR JSON over HTTPs New!

  41. mercari-web mercari-api JSON over HTTPs SSR Offer service JSON over

    HTTPs gRPC New!
  42. mercari-web mercari-api JSON over HTTPs SSR Offer service JSON over

    HTTPs gRPC Listing service gRPC New!
  43. mercari-web mercari-api JSON over HTTPs SSR Offer service JSON over

    HTTPs gRPC Listing service gRPC New! Buying service New! Live service New! User service
  44. mercari-web mercari-api JSON over HTTPs SSR Offer service JSON over

    HTTPs gRPC Listing service gRPC New! Buying service New! Live service New! User service • We need to keep up to date with the specs for every new microservice • We need to write high-performance code
  45. mercari-web mercari-api JSON over HTTPs BFF Listing Service JSON over

    HTTPs gRPC Offer Service gRPC Our strategy SSR
  46. mercari-web mercari-api JSON over HTTPs BFF Listing Service JSON over

    HTTPs gRPC Offer Service gRPC Our strategy SSR Frontend Engineer Backend Engineer
  47. Backends for Frontends in our JP Web Re-architecture • Our

    definition of BFF: “a layer to aggregate backend microservices” • To improve the development experience for frontend engineers
  48. What to implement as BFF • GraphQL - works well

    with SPAs • a
  49. GraphQL SSR mercari-web mercari-api Web Gateway Session service CDN 1

    2 3
  50. Session synchronization issues • Several microservices will exist in the

    same domain • Need to maintain consistency with the longin state of the Monolithic Service
  51. mercari-web Logged-out user SSR server

  52. mercari-web POST /login SSR server Logged-out user

  53. mercari-web Logged-in user Success! SSR server

  54. mercari-web Logged-in user GET / SSR server

  55. mercari-web Should show page for logged in users SSR server

    Logged-in user
  56. Microservice for managing session • Create a microservice to manage

    sessions • Maintaining session consistency and data preservation, etc. • Assume multiple microservices will use this
  57. How we will manage synchronization

  58. mercari-web Session service Request Call to get session every time

    Response Session data SSR server
  59. mercari-web Session service Request Call to get session every time

    Response Session data SSR server
  60. https://gist.github.com/jboner/2841832 Latency Numbers Every Programmer Should Know

  61. mercari-web Session service Q. What’s the distance here? SSR server

  62. About 800km ※だいたいです

  63. Data transmission between Hokkaido and Tokyo • We use GKE

    in Tokyo region • The exisiting Monolithic Service will run from Hokkaido • We need to design the architecture with latency in mind
  64. • Making Storage to  store cache data • Minimize amount

    of back-and-forth data transmission in Japan mercari-web Session service New! Cache storage
  65. mercari-web Session service New! Cache storage • We chose gRPC

    via Node.js • Based on technical expertise of our members
  66. mercari-web Session service mercari-api Cache storage SSR server

  67. mercari-web Session service mercari-api Cache storage Firstly, check the Cache

    and DB at hand to check for existing session data SSR server
  68. mercari-web Session service mercari-api Cache storage If no session data

    can be found, check for the session in Monolithic Service SSR server
  69. mercari-web Session service mercari-api Cache storage With the session data,

    request authentication data to the Monolithic API SSR server
  70. mercari-web Session service mercari-api Cache storage Save session data SSR

    server
  71. • Use cache data if it exists, but if not,

    get from Hokkaido • No access to Tokyo from Hokkaido • This ensures security and consistency Simple is the best!
  72. Conclusion

  73. Sowing the seeds of success • Implement a supreme architecture

    • Make it possible to develop at explosive speeds • And provide the ultimate developer experience
  74. So many challenges! • Tech stack • Re-architecture • Evolving

    and scaling our development structure
  75. Al or One All for the ultimate Web

  76. Thank you :) Feel free to ask me any questions

    at the “Ask the speaker” booth
  77. None