Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How LINE Securities Integrates with External Services

How LINE Securities Integrates with External Services

Di Ai (LINE Fukuoka / Securities Trade Dev2 / Server-Side Engineer )

https://tech-verse.me/ja/sessions/8
https://tech-verse.me/en/sessions/8
https://tech-verse.me/ko/sessions/8

Tech-Verse2022
PRO

November 18, 2022
Tweet

More Decks by Tech-Verse2022

Other Decks in Technology

Transcript

  1. View Slide

  2. Agenda
    - Self introduction
    - About LINE Securities
    - Project: Money Forward ME ☓ LINE Securities
    └ How the security issue is resolved.
    - Project: LINE app wallet tab integration
    └ How the test issue is resolved.
    - Future challenges and prospects

    View Slide

  3. Self Introduction
    2015/04 ~
    ⎯ Name: Ai Di
    ⎯ From China・Guizhou (famous for hot chili)
    ⎯ Got master degree from Kyushu University,
    Information Intelligence Engineering.
    ⎯ Joined LINE Fukuoka as a server-side engineer.
    ⎯ Living in Fukuoka. Favourite food is とんこつRamen
    Profile
    GUIZHOU
    FUKUOKA
    My dog named “Java”
    ⎯ Border collie is the smartest dog in the world, I think
    he may become a Java programmer like me.

    View Slide

  4. About LINE Securities

    View Slide

  5. 2022/04
    2021/02
    2020/02
    2021/06
    2020/06
    2019/11
    About LINE Securities
    Timeline related to me
    LINE Securities
    service start
    Recurring fund
    investment
    service start
    IPO
    service start
    Project join MoneyForward ME
    integration start
    STO
    (security token offering)
    service start

    View Slide

  6. JSDA
    (Japan Securities
    Dealers Association)
    JASDEC
    (Japan Securities Depository Center)
    Intertrade
    JPX
    (Japan Exchange Group)
    Refinitiv
    (Renamed from Reuter)
    LINE pay
    QUICK
    LINE Securities
    Internal network
    Internet
    Buy/Sell
    LINE Securities server pool
    NRI system
    Price, market situation
    Buy/Sell
    Price information
    News
    Price
    Big data traffic,
    No data loss, less delay
    execution data Securities buy/sell records
    Off-exchange report

    View Slide

  7. +4%"
    ೔ຊূ݊ۀڠձ
    +"4%&΄;Γ
    ূ݊อ؅ৼସػߏ

    Intertrade
    +19
    ೔ຊऔҾॴάϧʔϓ
    3FGJOJUJW
    3FVUFS͔Β໊ࣾมߋ

    LINE pay
    QUICK
    LINE Securities
    Internal network
    Internet
    Buy/Sell
    LINE Securities server pool
    NRI system
    Price, market situation
    Buy/Sell
    Price information
    News
    Price
    Big data traffic,
    No data loss, less delay
    execution data Securities buy/sell records
    Off-exchange report
    Various external services
    - Big transaction among systems.
    - Big data traffic between Refinitiv(rename from Reuter) and LINE
    Securities.
    - Combine data sources to show.

    View Slide

  8. Big Data Traffic
    Over 30K/s peak data traffic(user requests 5k/s) at the beginning of morning session(9:00)
    ⎯ Use Spring WebFlux to do non-blocking I/O processing.
    ⎯ Use Kafka as a hub to get data.
    Please refer to LINE Engineering Blog for more detail:
    ⎯ https://engineering.linecorp.com/ja/blog/on-the-use-of-stock-price-information-from-feeds/
    No data lose, less delay

    View Slide

  9. About LINE Securities
    Current system structure, an example of getting securities asset
    LINE Securities cosmos
    NRI system
    LINE Securities
    microservices
    internal network
    LINE Securities front-end
    LINE Securities
    old server pool
    get asset info
    get fund/stock detail

    View Slide

  10. LINE Securities Cosmos
    Origin of name
    exosphere
    The services which users
    can touch directly,
    e.g., stock buy/sell API
    Services control the core
    logic and business logic,
    e.g., master data
    management system
    asteroid

    View Slide

  11. LINE Securities
    Money Forward ME

    LINE Securities

    View Slide

  12. Money Forward ME Integration
    Code name: comet
    LINE Securities cosmos
    NRI system
    LINE Securities
    microservices
    internal network
    LINE Securities
    old server pool
    Money Forward
    ME

    View Slide

  13. LINE Securities Cosmos
    Code name: comet
    exosphere
    asteroid
    The part we are going to add.
    For integrating external system.
    comet

    View Slide

  14. Money Forward ME Integration
    Issues
    Fintech service common issue
    Issue 1
    Separation of
    internal network and
    external network
    The first time to expose API to external.
    Issue 2
    The
    authentication
    and authorization

    View Slide

  15. Solutions to Issue 1
    Separation of securities network and external network
    DMZ
    ngnix
    internal network
    LINE Securities cosmos
    NRI system
    LINE Securities
    microservices
    LINE Securities
    old server pool
    Money Forward
    ME Server

    View Slide

  16. internal network
    DeMilitarized Zone
    A network between public network and internal
    network, to protect the internal network.
    Enable Money Forward servers to access
    LINE securities internal network .
    DMZ
    DMZ
    firewall
    public network
    Comet-web
    Use ngnix in DMZ named comet-web.
    comet-web
    (nginx)

    View Slide

  17. Solutions to Issue 1
    Separation of internal network and public network
    DMZ
    comet-web
    (ngnix)
    internal network
    ACL
    LINE Securities cosmos
    NRI system
    LINE Securities
    microservices
    LINE Securities
    old server pool
    Money Forward
    ME Server

    View Slide

  18. Plan A
    Create a new
    authentication system.
    Use the API in the
    existing web server.
    Plan B
    Create a new
    authentication system.
    Create an API in a new
    microservice.
    Plan C
    Use LINE login to
    authenticate.
    Create an API in a new
    microservice.
    Solutions to Issue 2
    Authentication

    View Slide

  19. Why LINE Login
    The benefit
    - Security ensured.
    - Many services in LINE use LINE Login including LINE securities.
    - Rich document for client to implement easily.

    View Slide

  20. Goal
    To integrate external services
    by providing asset API.
    wallet-handler
    Core microservice of integration
    A simple and flexible scheduler
    and orchestrator to deploy and
    manage containers and non-
    containerized applications.
    Spring Boot
    (Kotlin)
    Microservice Nomad
    Recently most
    used in LINE
    Securities
    Easy for multiple
    teams to take
    responsibility.
    Tech stack
    wallet-handler
    get /asset

    View Slide

  21. Solutions to Issue 2
    Authorization
    Provide authorization API for client to access
    via access token.
    curl –H “Authorization: Bearer {access_token}” ¥
    -X POST ¥
    'https://{domain}/{path-to-wallet-handler}/authorize
    {
    ”result": ”OK"
    }
    We can know canceling association from LINE
    by calling access token verification API.
    curl -X GET ¥
    'https://api.line.me/oauth2/v2.1/verify?access_token
    =eyJhbGciOiJIUzI1NiJ9.UnQ_o-GP0VtnwDjbK0C8E_NvK...’
    {
    "error": "invalid_request",
    "error_description": "access token expired"
    }
    Also auto delete authorization when user
    has no access for a specified period.

    View Slide

  22. Money Forward ME Integration
    System structure
    DMZ
    comet-web
    (ngnix)
    LINE Securities cosmos
    internal network
    ACL
    comet-router
    wallet-handler
    NRI system
    LINE Securities
    old server pool
    LINE Login
    Money Forward
    ME Server

    View Slide

  23. LINE App Wallet Tab
    Integration

    View Slide

  24. LINE app Wallet Tab Integration
    To become a hub to fintech DPOUFOUT
    securities asset
    FX asset

    View Slide

  25. LINE app Wallet Tab Integration
    Issues
    Less then 10 req/s
    Over 400 req/s
    0
    50
    100
    150
    200
    250
    300
    350
    400
    450
    Issue 2
    How to test with FX service while it is
    still in developing.
    Issue 1
    40 times increasing in traffic
    securities
    wallet-handler
    FX

    View Slide

  26. Cache
    Create account info API
    Solutions to Issue 1
    Increasing request amount
    ⎯ In Wallet tab side, cache 1 min by
    FX/securities.
    ⎯ Call account info API first to reduce
    unnecessary API call.
    Global rate limit & API rate limit
    Global rate limit
    API rate limit
    API rate limit
    FX
    securities
    wallet tab
    wallet-handler
    account info

    View Slide

  27. Use WireMock to mock FX API server
    WireMock is a flexible tool for building mock APIs.
    Solutions to Issue 2
    How to test with FX service while it is still in developing
    securities
    wallet tab
    wallet-handler
    FX
    FX mock server

    View Slide

  28. Motivation
    Do you have same experience?
    - Only need to mock 1 API from a system.
    - Need to wait some APIs which is being developed.
    - Mock server is always not easy to build or use. ?
    ?
    ?
    ?
    ? ?

    View Slide

  29. Why WireMock
    1 json file for 1 API.
    Easy to use
    We only need to mock
    1 API, it means 1 json
    file in WireMock.
    Modify mock API without restarting.
    Dynamic management
    Use admin APIs
    to manage mock
    APIs.
    Matches many use cases.
    Suitable for testing
    For 1 API, we can use
    request mapping to
    define different
    response patterns.

    View Slide

  30. WireMock Example #1
    Easy to use. Modify mock API without restarting.
    {
    "request": {
    "method": "GET",
    "url": "/something"
    },
    "response": {
    "body": "Hello world!",
    "headers": {
    "Content-Type": "text/plain"
    },
    "status": 200
    }
    }
    > java -jar wiremock-jre8-standalone-2.34.0.jar --port
    8080
    > curl -X POST --data {json String}
    http://localhost:8080/__admin/mappings
    > curl -X POST
    http://localhost:8080/__admin/mappings/save
    > curl http://localhost:8080/__admin/
    {
    "id" : "746e0a7c-84b3-4837-830a-f228ce381a41",
    "request" : {
    "url" : "/something",
    "method" : "GET"
    },
    ...
    > curl http://localhost:8080/something
    Hello world!
    mockAPI.json

    View Slide

  31. WireMock Example #2
    Use WireMock Studio UI
    1
    1. For 1 API, use request mapping to define different
    response patterns.

    View Slide

  32. WireMock Example #2
    Use WireMock Studio UI
    2
    1. For 1 API, use request parameters mapping to
    define different response patterns.
    2. Mock response status to check error case.

    View Slide

  33. WireMock Example #2
    Use WireMock Studio UI
    3
    1. For 1 API, use request parameters mapping to
    define different response patterns.
    2. Mock response status to check error case.
    3. Mock response body to test UI.

    View Slide

  34. WireMock Example #2
    Use WireMock Studio UI
    1. For 1 API, use request parameters mapping to
    define different response patterns.
    2. Mock response status to check error case.
    3. Mock response body to test UI.
    4. Proxy all requests to the real system, only simulate
    APIs that don't exist yet.
    4

    View Slide

  35. Future Challenges and
    Prospects

    View Slide

  36. wallet-handler
    Challenges
    FX
    securities
    cache
    How to improve rate limit?
    - Need to config rate limit by clients.
    - Otherwise, if one service reaches rate
    limit, other services will be unavailable.
    Where to cache and how long we
    can cache?
    - Currently only wallet tab server has cache.
    - When provide to more services, cache can
    reduce the access to NRI system.
    wallet tab
    cache

    View Slide

  37. Prospects
    Easy to expand
    Easily and safely add new asset sources as well
    as new clients.
    Built a foundation for integration with
    external services
    FX
    securities
    ???
    wallet-handler

    View Slide

  38. Summary
    Money Forward ME Integration
    LINE App Wallet Tab Integration Future Challenges and Prospects
    About LINE Securities
    WireMock

    View Slide

  39. Thank you!

    View Slide