Building a smart recommender system across LINE services

Building a smart recommender system across LINE services

Jun Namikawa
LINE Machine Learning Team Fellow
https://linedevday.linecorp.com/jp/2019/sessions/C1-7

Be4518b119b8eb017625e0ead20f8fe7?s=128

LINE DevDay 2019

November 20, 2019
Tweet

Transcript

  1. 1.

    2019 DevDay Building a Smart Recommender System Across LINE Services

    > Jun Namikawa > LINE Machine Learning Team Fellow
  2. 5.
  3. 9.

    > Day2: C2-1 12:00-12:40 "LINE-Like" Product Management > Poster Session

    13:40-14:20/15:30-16:10 (2days) > Day1: B1-2 14:30-15:10 The Art of Smart Channel Continuous Improvements in Smart Channel Platform/Contents Related Sections
  4. 12.

    Many Recommender Systems Exist in LINE Each system has a

    different > Implementation > Algorithm > Objective
  5. 15.

    Recommender System Architecture Recommende r System 
 for Service Recommender

    System 
 for Service Recommender System 
 for Service Recommender System 
 for Service Recommended
 Items (Candidates) Ranker Trainer Events
 (imp, click, etc) LINE App User ID Items Model 
 parameter Item Request Top k items 
 for each user
  6. 16.

    Ranker Item A 0.7 Current Expected 
 Score 0.4 Current

    Expected 
 Score 0.6 Current Expected 
 Score 0.1 Current Expected 
 Score Item B Item C Item D > Ranker chooses an item from candidates A, B, C … by using contextual bandits > Each expected score is computed by a prediction model corresponding to the item
  7. 17.

    Prediction Model > Imp: 0.5, Click: 1.0, Mute: 0.0 >

    Balance Exploration-Exploitation Tradeoff > Laplace Approximation Bayesian Factorization Machine (FM) as an Arm of Contextual Bandits Output User ID Item ID User Features
 (Gender, Age, …) Other Features
 (Timestamp, …) Bayesian FM Embedding Embedding
  8. 18.

    Parameter Server for Distributed ML Events LINE App Trainer Worker

    Model Worker Model Parameter Server Ranker Executor Model Executor Model Δw W W Request Contents
  9. 19.

    Example of asynchronous communications between the parameter server and trainers.

    In the situation, learning doesn't work well just by accumulating the gradient in the parameter server. Asynchronous Distributed Online Learning
  10. 20.

    Asynchronous distributed learning algorithm Example of asynchronous communications between the

    parameter server and trainers. In the situation, learning doesn't work well just by accumulating the gradient in the parameter server. Asynchronous Distributed Online Learning Deceleration Backtrack
  11. 23.

    Primary Performance Metric > Consistent with user satisfaction trends obtained

    from questionnaire research > Easy to calculate > Stable under temporary fluctuations due to user's unfamiliarity Why score is used as main indicator?
  12. 24.

    Primary Performance Metric > Consistent with user satisfaction trends obtained

    from questionnaire research > Easy to calculate > Stable under temporary fluctuations due to user's unfamiliarity Why score is used as main indicator? Release new types of contents, or expand target users
  13. 27.

    Offline Test Off-policy Evaluation We use the More Robust Doubly

    Robust (MRDR) algorithm to estimate the performance of a new logic from the data generated by other logics. Framework of Offline Test To Evaluate New Logic Offline Test Environment Parameter server and trainers are clones of the production system. We use the event logs stored in DataLake by using PySpark. Trainer Parameter Server (Offline) Ranker DataLake
  14. 30.
  15. 31.

    LinUCB To Bayesian FM CTR +4.8% Score +5.8% -1.0% xCTR

    > Linearity: Easy To Parallelize LinUCB > Explicit Feature Interactions Bayesian FM
  16. 34.

    User and Item Embeddings 16 User ID Item ID User

    Features
 (Gender, Age, …) Other Features
 (Timestamp, …) Bayesian FM Embedding Embedding
  17. 39.