Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Anonymize Large-scale Sparse User Features at LINE Corp
Search
LINE Developers
PRO
March 07, 2019
Technology
2
3.4k
Anonymize Large-scale Sparse User Features at LINE Corp
2019/3/7 Machine Learning Production Pitch #1
Yeo Chaerim
LINE Developers
PRO
March 07, 2019
Tweet
Share
More Decks by LINE Developers
See All by LINE Developers
LINEスタンプのSREing事例集:大きなスパイクアクセスを捌くためのSREing
line_developers
PRO
1
1.4k
Java 21 Overview
line_developers
PRO
6
800
Code Review Challenge: An example of a solution
line_developers
PRO
1
830
KARTEのAPIサーバ化
line_developers
PRO
1
370
著作権とは何か?〜初歩的概念から権利利用法、侵害要件まで
line_developers
PRO
5
1.8k
生成AIと著作権 〜生成AIによって生じる著作権関連の課題と対処
line_developers
PRO
3
1.7k
マイクロサービスにおけるBFFアーキテクチャでのモジュラモノリスの導入
line_developers
PRO
9
2.4k
A/B Testing at LINE NEWS
line_developers
PRO
2
630
LINEのサポートバージョンの考え方
line_developers
PRO
2
790
Other Decks in Technology
See All in Technology
ここが嬉しいABAC ここが辛いよABAC #再解説+補足編
masahirokawahara
1
270
リテール金融(キャッシュレス・ネット銀行・ネット証券)の競争環境と経済圏
8maki
0
930
Azure Container Apps + Bicep 〜 こんな感じで運用しています
kaz29
2
470
VSCodeの拡張機能を作っている話
ebarakazuhiro
1
380
チームでロジカルシンキングに改めて向き合っている話 〜学習環境と実践⽅法〜
sansantech
PRO
2
2.2k
Cracking the KubeCon CfP
inductor
2
240
アクセシビリティを考慮したUI/CSSフレームワーク・ライブラリ選定
yajihum
2
1k
Python と Snowflake はズッ友だょ!~ Snowflake の Python 関連機能をふりかえる ~
__allllllllez__
1
120
Databricks における 『MLOps』
databricksjapan
2
170
Além do else! Categorizando Pokemóns com Pattern Matching no JavaScript
wmsbill
0
550
DevOpsメトリクスとアウトカムの接続にトライ!開発プロセスを通して計測できるメトリクスの活用方法
ham0215
2
240
Compose Compiler Metricsを使った実践的なコードレビュー
tomorrowkey
1
220
Featured
See All Featured
Building Better People: How to give real-time feedback that sticks.
wjessup
355
18k
Sharpening the Axe: The Primacy of Toolmaking
bcantrill
17
1.4k
The Language of Interfaces
destraynor
151
23k
Six Lessons from altMBA
skipperchong
21
3k
In The Pink: A Labor of Love
frogandcode
138
21k
Imperfection Machines: The Place of Print at Facebook
scottboms
260
12k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
357
22k
From Idea to $5000 a Month in 5 Months
shpigford
377
45k
Facilitating Awesome Meetings
lara
42
5.6k
The Pragmatic Product Professional
lauravandoore
25
5.8k
How to train your dragon (web standard)
notwaldorf
73
5.2k
実際に使うSQLの書き方 徹底解説 / pgcon21j-tutorial
soudai
121
39k
Transcript
ANONYMIZE LARGE-SCALE SPARSE USER FEATURES AT LINE CORP CHAERIM YEO,
LINE CORPORATION MACHINE LEARNING PRODUCTION PITCH #1, 2019/03/07
ABOUT ME l Chaerim Yeo(呂 彩林) l 2018.12 ~ LINE
Corporation l Account Platform Development Dept. l Ad performance optimization
Agenda • Z-Features • Y-Features • Evaluation • Conclusion
Z-FEATURES
WHAT ARE Z-FEATURES
WHAT ARE Z-FEATURES
WHAT ARE Z-FEATURES
WHAT ARE Z-FEATURES
WHAT ARE Z-FEATURES
BENEFIT OF Z-FEATURES Reusable Flexible
LIMITATION OF Z-FEATURES Human Interpretable Extremely Sparse
Y-FEATURES
BEYOND Z-FEATURES Obfuscation Dimensionality Reduction
BEYOND Z-FEATURES Obfuscation Dimensionality Reduction With keeping information as far
as possible
BEYOND Z-FEATURES Obfuscation Dimensionality Reduction SCDV https://arxiv.org/abs/1612.06778
OVERVIEW OF SCDV
INTEGRATE Z-FEATURES WITH SCDV
SYSTEM OVERVIEW
EVALUATION
DATA DIMENSION RELATIVE TO Z-FEATURES (LOG-SCALE) 0.0001 0.0010 0.0100 0.1000
1.0000 10.0000 100.0000 type1 type2 type3 type4 type5 type6 type7 type8 type9
DATA DENSITY LOG-SCALE 0.0000001 0.0000010 0.0000100 0.0001000 0.0010000 0.0100000 0.1000000
1.0000000 type1 type2 type3 type4 type5 type6 type7 type8 type9 z-features y-features
DATA SIZE RELATIVE TO Z-FEATURES 0.00 5.00 10.00 15.00 20.00
25.00 30.00 35.00 40.00 45.00 50.00 type1 type2 type3 type4 type5 type6 type7 type8 type9
USER DEMOGRAPHICS ESTIMATION MATRICS (RELATIVE TO Z-FEATURES) 0.95 0.96 0.97
0.98 0.99 1.00 1.01 1.02 gender age-group region precision recall f1-score
USER DEMOGRAPHICS ESTIMATION RUNNING TIME (RELATIVE TO Z-FEATURES) 0.00 0.05
0.10 0.15 0.20 0.25 0.30 0.35 0.40 0.45 0.50 gender age-group region training prediction
CONCLUSION
CONCLUSION l Anonymize user features based on SCDV l Enough
to use in ML l Future works l Add workflow to production l Apply further dimensionality reduction l Auto encoders, PCA, …
THANK YOU