Accelerating Fraud Detection Using Data on Fraud Risk of Potential Transactions / 発生可能な取引の属性データを用いた素早い不正検知

In this session, we will introduce a new mechanism for faster fraud detection.
Fraud detection using batch inference carries the risk that fraudulent activities may conclude before the machine learning model detects them, resulting in response delays. Therefore, in this session, we will present a mechanism that calculates fraud risk using attribute data of potential transactions. By combining near-real-time feature data and an risk inference platform, we will recalculate the risk every time the attribute data is updated. This enables the detection of fraudulent activities before they occur.

本セッションでは、素早く不正検知するための新しい仕組みについて紹介します。
従来のバッチ推論による不正検知では、機械学習モデルが推論する前に不正が終わることがあり、対応に遅れが生じる場合がありました。
そこで、本セッションでは、発生可能な取引の属性データを使って不正リスクを計算するシステムを紹介します。さらに、リアルタイム特徴量と推論基盤を組み合わせ、属性データが更新されるたびにリスクを再計算します。これにより、不正行為が発生する前に検知することが可能となります。

------
Merpay & Mercoin Tech Fest 2023は3日間のオンライン技術カンファレンスです。
IT企業で働くソフトウェアエンジニアおよびメルペイ・メルコインの技術スタックに興味がある方々を対象に2023年8月22日（火）、23日（水）、24日（木）の3日間、開催します。 Merpay & Mercoin Tech Fest は事業との関わりから技術への興味を深め、プロダクトやサービスを支えるエンジニアリングを知ることができるお祭りです。

今年のテーマは「Unleash Fintech」。 メルペイ・メルコインのこれまでの技術的な取り組みはもちろん、メルカリグループのFintech事業における新たな挑戦をお伝えします。 セッションでは事業を支える組織・技術・課題などへの試行錯誤やアプローチなど多面的にご紹介予定です。

メルペイ・メルコインが今後どのようにUnleash（解放）していくのか、ぜひ見に来てください。

■イベント関連情報
- 公式ウェブサイト：https://events.merpay.com/techfest-2023/
- 申し込みページ：https://mercari.connpass.com/event/286670/
- Twitterハッシュタグ： #MerpayMercoinTechFest
■リンク集
- メルカリ・メルペイイベント一覧：https://mercari.connpass.com/
- メルカリキャリアサイト：https://careers.mercari.com/
- メルカリエンジニアリングブログ：https://engineering.mercari.com/blog/
- メルカリエンジニア向けTwitterアカウント：https://twitter.com/mercaridevjp
- 株式会社メルペイ：https://jp.merpay.com/

Transcript

1. Session Title Accelerating Fraud Detection Using Data on Fraud Risk of Potential Transactions Liu Songjie / Victoria Li Merpay Fraud prevention Team
2. Liu Songjie Joined Merpay in October of 2019 • Fraud prevention Team ◦ Machine Learning Engineer Merpay Machine Learning Engineer
3. Victoria Li Joined Yahoo! JAPAN in April of 2017 • Software Engineer • 2019/1~: Machine Learning Engineer Joined Merpay in September of 2021 • Machine Learning Platform • Feature Store • GraphDB Merpay Machine Learning Engineer
4. Fraud Prevention Models • Alert Filtering (multiple ML models) ◦ ref: https://engineering.mercari.com/blog/entry/alertfiltering-ml/ • ChargeBack Detection (ML model) ◦ ref: https://engineering.mercari.com/en/blog/entry/chargeback-ml/ • Suspicious Action Detection (complex network) ◦ ref: https://engineering.mercari.com/blog/entry/complex-network-ml/ • Potential Transaction Risk Detection • (New) Graph Based Fraud Detection • etc.
5. Past Events • Merpay Tech Fest 2021 ◦ Using Feature Store and Vertex Pipelines in Fraud Prevention System ▪ ref: https://speakerdeck.com/mercari/using-feature-store-and-vertex-pipelines-in-fraud-prevention-system • Merpay Tech Fest 2022 ◦ Machine Learning infrastructure using Feature Store and Vertex AI ▪ ref: https://speakerdeck.com/mercari/machine-learning-infrastructure-using-feature-store-and-vertex-ai ◦ Graph Theory and Anti-Fraud Measures Unravel connections between data ▪ ref: https://speakerdeck.com/mercari/graph-theory-and-anti-fraud-measures-unravel-connections-between-data
6. Content Background 01 Mechanism 02 System Architecture 03 Feature Store 04 04 05 Summary
7. Background 2 Types of Fraud Detection • Afterward Detection ◦ Stop the delivery of the product, after the fraudulent transaction has been made ◦ Risk of missing out on fraud • Immediate Detection ◦ Take immediate action upon discovering a fraudulent transaction ◦ Low latency requested ref: https://mercan.mercari.com/articles/31223/
8. Background Immediate Fraud Detection • Judge service: An immediate fraud prevention system with low latency (0.1s) ◦ provide by TnS backend team • ML solutions: from daily batch to 1 minute latency ◦ provide by ML team Judge Service: https://mercan.mercari.com/articles/31223/
9. Content Background 01 Mechanism 02 System Architecture 03 Feature Store 04 04 05 Summary
10. Mechanism Provide Potential Risk Rank, before transaction has been made
11. Mechanism Some features (attribute data) are already existed / updated before the transaction takes place: • item information • seller information • buyer information • … Features mentioned above help us evaluating the risk of a potential transaction before its completion
12. Mechanism
13. Mechanism Process • ML solutions (ML model) calculate the risk of a potential transaction after the attributes data created/updated • Potential risk sent to the Judge Service, and saved in database • Judge Service assess the risk of transactions in real-time as they occur
14. Mechanism Batch prediction pain point
15. Mechanism Latency of ML solutions
16. Mechanism To accelerate the process, add a Streamlined solution to the existing Batch solution, utilizing: • Feast Online Store • Stream prediction system
17. Content Background 01 Mechanism 02 System Architecture 03 Feature Store 04 04 05 Summary
18. System Architecture
19. System Architecture 4 Parts • Feast (Feature Store) ◦ Introduce in the next part • Batch system ◦ Using Vertex Pipelines • Stream system ◦ Microservices • Publish API ◦ A service for publishing results to Judge Service
20. Batch System
21. Batch System • Cloud Scheduler ◦ daily / hourly • Pub/Sub • Cloud Run ◦ Vertex Pipelines Executor • Vertex Pipelines • Spanner • BigQuery
22. Stream System
23. Stream System
24. Stream System
25. Publish API
26. Publish API Outbox Table CREATE TABLE outbox ( id STRING(64) NOT NULL, information_1 INT64 NOT NULL, information_2 STRING(64) NOT NULL, …, pubsub_topic STRING(64) NOT NULL, created TIMESTAMP NOT NULL OPTIONS ( allow_commit_timestamp = true ), updated TIMESTAMP NOT NULL OPTIONS ( allow_commit_timestamp = true ), is_published BOOL NOT NULL, schedule TIMESTAMP, ) PRIMARY KEY(id); Information used in Pub/Sub message Pub/Sub topic Is this record published Don’t publish until this time
27. Publish API • Deployed to Cloud Run • Publish messages to Judge Service • Triggered by ◦ Cloud Scheduler (every minute) for batch system ◦ Pub/Sub for stream system
28. Publish API
29. Content Background 01 Mechanism 02 System Architecture 03 Feature Store 04 04 05 Summary
30. System Architecture
31. Feature Store In addition to prediction, serving features at near real-time and low latency is also crucial To address this, we have implemented Feast Online Store, a powerful tool that allows us to store / serve features
32. Life Before a Feature Store https://www.hopsworks.ai/post/feature-store-the-missing-data-layer-in-ml-pipelines
33. Life After a Feature Store https://www.hopsworks.ai/post/feature-store-the-missing-data-layer-in-ml-pipelines
34. Feature Store Architecture Online Store and Offline Store Offline Store Online Store Moderate to high latency Low latency Store historical features for each entity Store only the latest features for each entity Mainly used to train model and batch prediction Mainly used to serve model
35. Feature Store Architecture
36. Feature Store Which features should be materialized? Which features should be ingested by stream? • stream ingestion ◦ real-time features • materialization ◦ not real-time features (aggregated features etc.) ▪ window features etc. ▪ https://engineering.mercari.com/en/blog/entry/chargeback-ml/
37. Scheduled/On-demand materialization
38. FeatureView between Online/Offline Store https://speakerdeck.com/mercari/machine-learning-infrastructure-using-feature-store-and-vertex-ai?slide=12 FeatureView1 FeatureView2 FeatureView3 Feature Service1 Feature Service2 Feature Service3 FeatureView4 … … Model A Training Model B Training Model C Training … BigQuery GCS FEAST
39. How to manage FeatureView between Online/Offline Store • Let’s say there’s a FeatureView M at the offline store, have the features below: ◦ Feature A (realtime feature) ◦ Feature B (not realtime feature) • Need to choose stream ingest or materialize to ingest data to the online store. FeatureView between Online/Offline Store
40. Here comes the problem: • Feature A (realtime feature) ◦ could be ingested from Kafka (stream) • Feature B (not realtime feature) ◦ could be ingested by materialization Overwriting can occur in the same feature view, due to simultaneous ingestion. FeatureView between Online/Offline Store
41. FeatureView between Online/Offline Store
42. FeatureView between Online/Offline Store
43. Solution • Create FeatureView for each stream ingest and materialization We’ll have 3 feature views: • Original FeatureView ◦ include all features • FeatureView for materialization ◦ only contain features which could be ingested by materialization • FeatureView for stream ingest ◦ only contain features which could be stream ingested FeatureView between Online/Offline Store
44. Content Background 01 Mechanism 02 System Architecture 03 Feature Store 04 04 05 Summary
45. New mechanism for enhancing fraud detection speed Calculate & Provide Potential Risk Rank before transaction has been made, to immediate fraud prevention system
46. Implementation of the mechanism Adding stream system to existing batch system, we have taken one step closer to a comprehensive fraud prevention solution.
47. Coming soon • Graph database in fraud prevention system! • Graph-based approaches demonstrate remarkable effectiveness in detecting fraud. Please stay tuned for upcoming blogs post for more details!