MTC2018 - Why Are We Trying to Turn Payment Processing Into a Microservice?

Speaker: Yuichiro Saito

Merpay will provide users with the means to use their Mercari sales balance as a form of electronic money. This feature will of course be available for shopping on Mercari itself, but in order to do that, we have to face the issue of processing payments over Mercari. A monolithic program that has been built up over years now finds itself confronted with the challenge of implementing a new method of payment. To resolve this, we are taking a Go Bold approach of moving to microservices, but many difficulties lie within. In this session, Saito will talk about:
1) Why we decided to make the move to microservices
2) Deciphering the control flow of complex intertwined switch statements
3) The battle with potential errors caused by distributed transactions
4) How to reduce the effect on accounting systems"

Transcript

1. Why Are We Trying to Turn Payment Processing Into a Microservice? Yuichiro Saito Software Engineer (Backend)
2. Why Are We Trying to Turn Payment Processing Into a Microservice?
3. Distribution Migration Understanding the current situation
4. Payment Service Growth Loose Coupling Maintainability
5. Software Engineer Backend Yuichiro Saito
6. Why we’re taking the leap to Microservices
7. Mercari is like an old hot spring inn, built floor upon floor. =
8. Specifically, with regard to payment processes... • Existing payment processes aren’t scalable • New electronic payments need to be added • Only a limited number of people understand the payment processes This could hinder our growth.
9. Let’s take a closer look at this convoluted switch statement of a workflow...
10. First... Take a look at the way payment methods are being added (in pseudocode).
15. SAY WHAT?
16. Problems • The current structure of payment processes is not scalable. • It’s become necessary to develop loose coupling between the electronic payment services in development. Solutions • Migrate to microservices using the grpc interface • Reimplement payment processing (PaymentService) • At the same time, move toward a sparse organizational structure
17. Refactoring Microservices Solutions Options! etc.
18. Results Achieved • Better maintainability • Ability to separate responsbilities • Easier to add new payment methods • Increased number of people who understand payment processing through reimplementation
19. The battle with potentials errors of distributed transactions
21. But it’s still too early to celebrate! Let’s take another look at the current payment processing.
22. If the transaction fails, we just call the rollback function.
23. After payments migrate to microservices... Payment processing becomes a distributed transaction.
24. Problem 1: Processed results exist independently in each system.
25. payment->consume() is executed as an independent process. If point payment and card payment occur simultaneously and one of them fail, the result will be incorrect.
27. Solution 1: Separate 1. Authorize payment method. 2. Continue the purchase process in a state where payment abilities are guaranteed. 3. If no errors, finalize payment. 4. If there are errors, release authorization of payment method. Idempotency of the process is guaranteed.
28. Problem 2: What happens when a partial failure occurs within PaymentService?
29. This process can’t begin. When payment->consume() is slow...
31. Characteristics • Asynchronous processing is possible. • Automatic re-queuing and retry upon failure. High availability. • Can revert the state of unrecoverable failures. The successive addition of new payment methods also becomes possible. compensating transaction Solution 2: State Machine retry retry success point consump- tion card payment succ- ess start fail
32. How to reduce the influence on accounting systems
33. Forgetting something?
34. 1. When migrating payment systems, logs are processed in parallel in the current and new systems. 2. Parallel processing ends after the migration to PaymentService is complete. Payment Business Payment Service Current System Done DB DB Pub/ Sub
35. To conclude
36. Better maintainability →
37. Separating responsibilities allows for easier development and maintainability. Separation of responsibilities Flexible expansion Eliminating dependence As the system scales, reorganizing the organizational structure allows for independent teams. Through redesign, re-implementation and documentation, we can ensure that each task can be understood and completed by anyone.
38. Understand the nature of distributed transactions during implementation of payment processing. Ensure that there is a method to revert to previous state.
39. Prepare for the migration of accounting systems using parallel processing.
40. Now in the last spurt before release! Images from いらすとや (https://www.irasutoya.com/)