How LINE Securities Integrates with External Services

Di Ai (LINE Fukuoka / Securities Trade Dev2 / Server-Side Engineer )

https://tech-verse.me/ja/sessions/8
https://tech-verse.me/en/sessions/8
https://tech-verse.me/ko/sessions/8

Transcript

2. Agenda - Self introduction - About LINE Securities - Project: Money Forward ME ☓ LINE Securities └ How the security issue is resolved. - Project: LINE app wallet tab integration └ How the test issue is resolved. - Future challenges and prospects
3. Self Introduction 2015/04 ~ ⎯ Name: Ai Di ⎯ From China・Guizhou (famous for hot chili) ⎯ Got master degree from Kyushu University, Information Intelligence Engineering. ⎯ Joined LINE Fukuoka as a server-side engineer. ⎯ Living in Fukuoka. Favourite food is とんこつRamen Profile GUIZHOU FUKUOKA My dog named “Java” ⎯ Border collie is the smartest dog in the world, I think he may become a Java programmer like me.
4. About LINE Securities
5. 2022/04 2021/02 2020/02 2021/06 2020/06 2019/11 About LINE Securities Timeline related to me LINE Securities service start Recurring fund investment service start IPO service start Project join MoneyForward ME integration start STO (security token offering) service start
6. JSDA (Japan Securities Dealers Association) JASDEC (Japan Securities Depository Center) Intertrade JPX (Japan Exchange Group) Refinitiv (Renamed from Reuter) LINE pay QUICK LINE Securities Internal network Internet Buy/Sell LINE Securities server pool NRI system Price, market situation Buy/Sell Price information News Price Big data traffic, No data loss, less delay execution data Securities buy/sell records Off-exchange report
7. +4%" ೔ຊূ݊ۀڠձ +"4%&
   ΄;Γ ূ݊อ؅ৼସػߏ Intertrade +19 ೔ຊऔҾॴάϧʔϓ 3FGJOJUJW
   3FVUFS͔Β໊ࣾมߋ LINE pay QUICK LINE Securities Internal network Internet Buy/Sell LINE Securities server pool NRI system Price, market situation Buy/Sell Price information News Price Big data traffic, No data loss, less delay execution data Securities buy/sell records Off-exchange report Various external services - Big transaction among systems. - Big data traffic between Refinitiv(rename from Reuter) and LINE Securities. - Combine data sources to show.
8. Big Data Traffic Over 30K/s peak data traffic(user requests 5k/s) at the beginning of morning session(9:00) ⎯ Use Spring WebFlux to do non-blocking I/O processing. ⎯ Use Kafka as a hub to get data. Please refer to LINE Engineering Blog for more detail: ⎯ https://engineering.linecorp.com/ja/blog/on-the-use-of-stock-price-information-from-feeds/ No data lose, less delay
9. About LINE Securities Current system structure, an example of getting securities asset LINE Securities cosmos NRI system LINE Securities microservices internal network LINE Securities front-end LINE Securities old server pool get asset info get fund/stock detail
10. LINE Securities Cosmos Origin of name exosphere The services which users can touch directly, e.g., stock buy/sell API Services control the core logic and business logic, e.g., master data management system asteroid
11. LINE Securities Money Forward ME ☓ LINE Securities
12. Money Forward ME Integration Code name: comet LINE Securities cosmos NRI system LINE Securities microservices internal network LINE Securities old server pool Money Forward ME
13. LINE Securities Cosmos Code name: comet exosphere asteroid The part we are going to add. For integrating external system. comet
14. Money Forward ME Integration Issues Fintech service common issue Issue 1 Separation of internal network and external network The first time to expose API to external. Issue 2 The authentication and authorization
15. Solutions to Issue 1 Separation of securities network and external network DMZ ngnix internal network LINE Securities cosmos NRI system LINE Securities microservices LINE Securities old server pool Money Forward ME Server
16. internal network DeMilitarized Zone A network between public network and internal network, to protect the internal network. Enable Money Forward servers to access LINE securities internal network . DMZ DMZ firewall public network Comet-web Use ngnix in DMZ named comet-web. comet-web (nginx)
17. Solutions to Issue 1 Separation of internal network and public network DMZ comet-web (ngnix) internal network ACL LINE Securities cosmos NRI system LINE Securities microservices LINE Securities old server pool Money Forward ME Server
18. Plan A Create a new authentication system. Use the API in the existing web server. Plan B Create a new authentication system. Create an API in a new microservice. Plan C Use LINE login to authenticate. Create an API in a new microservice. Solutions to Issue 2 Authentication
19. Why LINE Login The benefit - Security ensured. - Many services in LINE use LINE Login including LINE securities. - Rich document for client to implement easily.
20. Goal To integrate external services by providing asset API. wallet-handler Core microservice of integration A simple and flexible scheduler and orchestrator to deploy and manage containers and non- containerized applications. Spring Boot (Kotlin) Microservice Nomad Recently most used in LINE Securities Easy for multiple teams to take responsibility. Tech stack wallet-handler get /asset
21. Solutions to Issue 2 Authorization Provide authorization API for client to access via access token. curl –H “Authorization: Bearer {access_token}” ¥ -X POST ¥ 'https://{domain}/{path-to-wallet-handler}/authorize { ”result": ”OK" } We can know canceling association from LINE by calling access token verification API. curl -X GET ¥ 'https://api.line.me/oauth2/v2.1/verify?access_token =eyJhbGciOiJIUzI1NiJ9.UnQ_o-GP0VtnwDjbK0C8E_NvK...’ { "error": "invalid_request", "error_description": "access token expired" } Also auto delete authorization when user has no access for a specified period.
22. Money Forward ME Integration System structure DMZ comet-web (ngnix) LINE Securities cosmos internal network ACL comet-router wallet-handler NRI system LINE Securities old server pool LINE Login Money Forward ME Server
23. LINE App Wallet Tab Integration
24. LINE app Wallet Tab Integration To become a hub to fintech DPOUFOUT securities asset FX asset
25. LINE app Wallet Tab Integration Issues Less then 10 req/s Over 400 req/s 0 50 100 150 200 250 300 350 400 450 Issue 2 How to test with FX service while it is still in developing. Issue 1 40 times increasing in traffic securities wallet-handler FX
26. Cache Create account info API Solutions to Issue 1 Increasing request amount ⎯ In Wallet tab side, cache 1 min by FX/securities. ⎯ Call account info API first to reduce unnecessary API call. Global rate limit & API rate limit Global rate limit API rate limit API rate limit FX securities wallet tab wallet-handler account info
27. Use WireMock to mock FX API server WireMock is a flexible tool for building mock APIs. Solutions to Issue 2 How to test with FX service while it is still in developing securities wallet tab wallet-handler FX FX mock server
28. Motivation Do you have same experience? - Only need to mock 1 API from a system. - Need to wait some APIs which is being developed. - Mock server is always not easy to build or use. ? ? ? ? ? ?
29. Why WireMock 1 json file for 1 API. Easy to use We only need to mock 1 API, it means 1 json file in WireMock. Modify mock API without restarting. Dynamic management Use admin APIs to manage mock APIs. Matches many use cases. Suitable for testing For 1 API, we can use request mapping to define different response patterns.
30. WireMock Example #1 Easy to use. Modify mock API without restarting. { "request": { "method": "GET", "url": "/something" }, "response": { "body": "Hello world!", "headers": { "Content-Type": "text/plain" }, "status": 200 } } > java -jar wiremock-jre8-standalone-2.34.0.jar --port 8080 > curl -X POST --data {json String} http://localhost:8080/__admin/mappings > curl -X POST http://localhost:8080/__admin/mappings/save > curl http://localhost:8080/__admin/ { "id" : "746e0a7c-84b3-4837-830a-f228ce381a41", "request" : { "url" : "/something", "method" : "GET" }, ... > curl http://localhost:8080/something Hello world! mockAPI.json
31. WireMock Example #2 Use WireMock Studio UI 1 1. For 1 API, use request mapping to define different response patterns.
32. WireMock Example #2 Use WireMock Studio UI 2 1. For 1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case.
33. WireMock Example #2 Use WireMock Studio UI 3 1. For 1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case. 3. Mock response body to test UI.
34. WireMock Example #2 Use WireMock Studio UI 1. For 1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case. 3. Mock response body to test UI. 4. Proxy all requests to the real system, only simulate APIs that don't exist yet. 4
35. Future Challenges and Prospects
36. wallet-handler Challenges FX securities cache How to improve rate limit? - Need to config rate limit by clients. - Otherwise, if one service reaches rate limit, other services will be unavailable. Where to cache and how long we can cache? - Currently only wallet tab server has cache. - When provide to more services, cache can reduce the access to NRI system. wallet tab cache
37. Prospects Easy to expand Easily and safely add new asset sources as well as new clients. Built a foundation for integration with external services FX securities ？？？ wallet-handler
38. Summary Money Forward ME Integration LINE App Wallet Tab Integration Future Challenges and Prospects About LINE Securities WireMock
39. Thank you!