Agenda - Self introduction - About LINE Securities - Project: Money Forward ME ☓ LINE Securities └ How the security issue is resolved. - Project: LINE app wallet tab integration └ How the test issue is resolved. - Future challenges and prospects
Self Introduction 2015/04 ~ ⎯ Name: Ai Di ⎯ From China・Guizhou (famous for hot chili) ⎯ Got master degree from Kyushu University, Information Intelligence Engineering. ⎯ Joined LINE Fukuoka as a server-side engineer. ⎯ Living in Fukuoka. Favourite food is とんこつRamen Profile GUIZHOU FUKUOKA My dog named “Java” ⎯ Border collie is the smartest dog in the world, I think he may become a Java programmer like me.
2022/04 2021/02 2020/02 2021/06 2020/06 2019/11 About LINE Securities Timeline related to me LINE Securities service start Recurring fund investment service start IPO service start Project join MoneyForward ME integration start STO (security token offering) service start
JSDA (Japan Securities Dealers Association) JASDEC (Japan Securities Depository Center) Intertrade JPX (Japan Exchange Group) Refinitiv (Renamed from Reuter) LINE pay QUICK LINE Securities Internal network Internet Buy/Sell LINE Securities server pool NRI system Price, market situation Buy/Sell Price information News Price Big data traffic, No data loss, less delay execution data Securities buy/sell records Off-exchange report
LINE pay QUICK LINE Securities Internal network Internet Buy/Sell LINE Securities server pool NRI system Price, market situation Buy/Sell Price information News Price Big data traffic, No data loss, less delay execution data Securities buy/sell records Off-exchange report Various external services - Big transaction among systems. - Big data traffic between Refinitiv(rename from Reuter) and LINE Securities. - Combine data sources to show.
Big Data Traffic Over 30K/s peak data traffic(user requests 5k/s) at the beginning of morning session(9:00) ⎯ Use Spring WebFlux to do non-blocking I/O processing. ⎯ Use Kafka as a hub to get data. Please refer to LINE Engineering Blog for more detail: ⎯ https://engineering.linecorp.com/ja/blog/on-the-use-of-stock-price-information-from-feeds/ No data lose, less delay
About LINE Securities Current system structure, an example of getting securities asset LINE Securities cosmos NRI system LINE Securities microservices internal network LINE Securities front-end LINE Securities old server pool get asset info get fund/stock detail
LINE Securities Cosmos Origin of name exosphere The services which users can touch directly, e.g., stock buy/sell API Services control the core logic and business logic, e.g., master data management system asteroid
Money Forward ME Integration Code name: comet LINE Securities cosmos NRI system LINE Securities microservices internal network LINE Securities old server pool Money Forward ME
Money Forward ME Integration Issues Fintech service common issue Issue 1 Separation of internal network and external network The first time to expose API to external. Issue 2 The authentication and authorization
Solutions to Issue 1 Separation of securities network and external network DMZ ngnix internal network LINE Securities cosmos NRI system LINE Securities microservices LINE Securities old server pool Money Forward ME Server
internal network DeMilitarized Zone A network between public network and internal network, to protect the internal network. Enable Money Forward servers to access LINE securities internal network . DMZ DMZ firewall public network Comet-web Use ngnix in DMZ named comet-web. comet-web (nginx)
Solutions to Issue 1 Separation of internal network and public network DMZ comet-web (ngnix) internal network ACL LINE Securities cosmos NRI system LINE Securities microservices LINE Securities old server pool Money Forward ME Server
Plan A Create a new authentication system. Use the API in the existing web server. Plan B Create a new authentication system. Create an API in a new microservice. Plan C Use LINE login to authenticate. Create an API in a new microservice. Solutions to Issue 2 Authentication
Why LINE Login The benefit - Security ensured. - Many services in LINE use LINE Login including LINE securities. - Rich document for client to implement easily.
Goal To integrate external services by providing asset API. wallet-handler Core microservice of integration A simple and flexible scheduler and orchestrator to deploy and manage containers and non- containerized applications. Spring Boot (Kotlin) Microservice Nomad Recently most used in LINE Securities Easy for multiple teams to take responsibility. Tech stack wallet-handler get /asset
Solutions to Issue 2 Authorization Provide authorization API for client to access via access token. curl –H “Authorization: Bearer {access_token}” ¥ -X POST ¥ 'https://{domain}/{path-to-wallet-handler}/authorize { ”result": ”OK" } We can know canceling association from LINE by calling access token verification API. curl -X GET ¥ 'https://api.line.me/oauth2/v2.1/verify?access_token =eyJhbGciOiJIUzI1NiJ9.UnQ_o-GP0VtnwDjbK0C8E_NvK...’ { "error": "invalid_request", "error_description": "access token expired" } Also auto delete authorization when user has no access for a specified period.
Money Forward ME Integration System structure DMZ comet-web (ngnix) LINE Securities cosmos internal network ACL comet-router wallet-handler NRI system LINE Securities old server pool LINE Login Money Forward ME Server
LINE app Wallet Tab Integration Issues Less then 10 req/s Over 400 req/s 0 50 100 150 200 250 300 350 400 450 Issue 2 How to test with FX service while it is still in developing. Issue 1 40 times increasing in traffic securities wallet-handler FX
Cache Create account info API Solutions to Issue 1 Increasing request amount ⎯ In Wallet tab side, cache 1 min by FX/securities. ⎯ Call account info API first to reduce unnecessary API call. Global rate limit & API rate limit Global rate limit API rate limit API rate limit FX securities wallet tab wallet-handler account info
Use WireMock to mock FX API server WireMock is a flexible tool for building mock APIs. Solutions to Issue 2 How to test with FX service while it is still in developing securities wallet tab wallet-handler FX FX mock server
Motivation Do you have same experience? - Only need to mock 1 API from a system. - Need to wait some APIs which is being developed. - Mock server is always not easy to build or use. ? ? ? ? ? ?
Why WireMock 1 json file for 1 API. Easy to use We only need to mock 1 API, it means 1 json file in WireMock. Modify mock API without restarting. Dynamic management Use admin APIs to manage mock APIs. Matches many use cases. Suitable for testing For 1 API, we can use request mapping to define different response patterns.
WireMock Example #2 Use WireMock Studio UI 2 1. For 1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case.
WireMock Example #2 Use WireMock Studio UI 3 1. For 1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case. 3. Mock response body to test UI.
WireMock Example #2 Use WireMock Studio UI 1. For 1 API, use request parameters mapping to define different response patterns. 2. Mock response status to check error case. 3. Mock response body to test UI. 4. Proxy all requests to the real system, only simulate APIs that don't exist yet. 4
wallet-handler Challenges FX securities cache How to improve rate limit? - Need to config rate limit by clients. - Otherwise, if one service reaches rate limit, other services will be unavailable. Where to cache and how long we can cache? - Currently only wallet tab server has cache. - When provide to more services, cache can reduce the access to NRI system. wallet tab cache
Prospects Easy to expand Easily and safely add new asset sources as well as new clients. Built a foundation for integration with external services FX securities ？？？ wallet-handler
techverse_2022
kawajiritakayuki
yamahiro
pf_businessdivision
mploed
orinbou
coconala_engineer
nunukim
kawagoi
ewolff
coe
yamasatimi
dahatake
gr2m
eileencodes
bcantrill
robhawkes
maltzj
shlominoach
chrisshort
panda_program
quramy
zakiyama
destraynor
kneath