Upgrade to Pro — share decks privately, control downloads, hide ads and more …

竟然有人說AI不能拿來做行銷

 竟然有人說AI不能拿來做行銷

Speaker: Johnson
Event: 台大AI社

LINE Developers Taiwan
PRO

November 01, 2022
Tweet

More Decks by LINE Developers Taiwan

Other Decks in Technology

Transcript

  1.  __ɼᰈવ༗ਓ㘸AIෆೳ፤ိ၏ߦ᭖ Data Scientist /Johnson Wu @LINE Taiwan

  2. Hello! - Johnson Wu - CSIE (B01, R05) - Intern

    @ Microsoft Research Asia - Poetry book writer (indirect) - Data Scientist @ LINE Taiwan 2019 @LINE Fukuoka
  3. Imagination about Digital Marketing in LINE: *OUFSOFU DFMFCSJUZ 4PDJBM NFEJB

    "EWFSUJTFNFOU
  4. Our actual thought of Marketing: Content & Experience Social &

    Relationship Commerce & Sales Management Data Advertising & Promotion • Talent Mgt. • Finance Mgt. • Quality Mgt. • CRM • Channels & Partners • Sales Automation & Intelligence • CDP/DMP • Business Intelligence • Data Governance • Customer Exp. • KOL • Community & Review • Content • Experience Mgt. • SEO • Marketing • Advertising • PR Align with user journey!
  5. Look into our problems with… (A)AARRR Funnel ! How to

    let people visit my content? How to let people have action/buy? How to make people come back and buy? How to make people start to buy? How to let people refer new users to your content? Acquisition Activation Retention Revenue Referral 1.25 2.5 3.75 5
  6. Commercial problems with the AARRR Real Cases in LINE Taiwan

    How to recommend OA push to users with limited number of pushes? How to attract most possibly new users to the service? How to value customers and Recommend based on values? How to deliver ad promotion with the best cost-profit balance? How do we understand users for AD targeting? Acquisition Activation Retention and activation AARRR Revenue
  7. Using knowledge graph solution for new user acquisition User Acquisition

  8. What are active users in the service? Where should I

    find the new users? How to connect new users with active users? Acquisition
  9. Users in LINE Correlate users with behaviors of other services

    -*/& 50%": -*/& 
 4)011*/( -*/& .64*$ ? Acquisition 0"
  10. Attracting New users Via SmartChannel with knowledge graph technique. smart

    channel And graph? Acquisition
  11. Graph embedding helps Service knowledge graph provides abundant information -

    GraphSAGE (inductive), GCN - Graph Attention network with deepwalk - Semantic matching based: ComplEX - Algorithm choice depends on data traits https://arxiv.org/pdf/1710.10903.pdf https://arxiv.org/pdf/1706.02216.pdf 1 2 1 2 Acquisition
  12. How does graph work? Model predict the probability of the

    “link”. - Example: Graph Attention network with deepwalk 1 2 Acquisition - Users are represented as edge connections between entity with meaningful actions - Users1: article1 read -> 
 shop item2 -> join campaign 3…. - Each node (entity) is aggregated 
 by the weighted sum of neighbors.
  13. New acquired users Case study on LINE POINTS wall :

    how to acquire new users? -*/&10*/54XBMMVTFST OPO-*/&10*/54XBMMVTFST 0 ffi DJBM "DDPVOU GPMMPXJOH TUBUVT ,OPXMFEHF HSBQINPEFMJOH 10*/54 0" $MVTUFSJOH ,OPXMFEHF HSBQIJOGFSFODF 
 $MVTUFSJOH 0" OPO-*/&10*/54 XBMMVTFST NPTUMJLFMZUP KPJO-*/&SFXBSE Compose triples Compose triples Top k similar users
  14. - Flexible and complex decision making - Applied to LINE

    SHOPPING, SPOT, POINTS, MUSIC - Embeddings show clustering effect w.r.t campaign Performance Case study on LINE POINTS : effectiveness and fully data orientation - ≥ 25% of new user growth comes from AI’s decision. - Easily fit for different scenario : 
 How about recommend OA Push message to non-new users? - ≥200% CTR lift Acquisition Activation
  15. Apply uplift model to target the persuadable Retention: How to

    Maximize Campaign Profit?
  16. Who should receive OAP? How many OAP I should deliver?

    How to evaluate performance? Retention
  17. How to deliver ad promotion with the best cost-profit balance?

    $PVOUPGDBNQBJHO      .POUIPG:FBS             3FDFJWF 0"NFTTBHF 1VSDIBTF JODBNQBJHOQFSJPE &BSO -*/&$0*/10*/54 Boost the Profits with Incentives Retention
  18. User Segments by Uplift High Low Low High Buy if

    treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  19. User Segments by Uplift EFMJWFSDPTU JODFOUJWFDPTU High Low Low High

    Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  20. User Segments by Uplift EFMJWFSZDPTU SFWFOVF High Low Low High

    Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  21. User Segments by Uplift EFMJWFSZDPTU High Low High Low Buy

    if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  22. User Segments by Uplift  EFMJWFSZDPTU  *ODFOUJWFDPTU  SFWFOVF

    High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
  23. User Segments by Uplift QPTJUJWFVQMJGU OFHBUJWFVQMJGU OPVQMJGU OPVQMJGU Retention High

    Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Uplift M odel
  24. User Segments by Uplift Retention High Low Low High Buy

    if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable 3FTQPOTF.PEFM Uplift M odel
  25. A/B Testing to Collect Lookalike User Groups Treatment Group Control

    Group Retention
  26. #Control Group #Treatment Group Leverage Lookalike Groups to Estimate Uplift

  27. User 360 Degree Features Campaign Period Observation Period, 30 days

    Purchase or not Purchase or not Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections Treatment Group Control Group Retention
  28. User 360 Degree Features Campaign Period Observation Period, 30 days

    Purchase or not Purchase or not Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections Treatment Group Control Group If the campaign effect is significant  If the sure thing exist  Retention
  29. Meta-learners, S-Learner Campaign Period Observation Period, 30 days Browsing behavior

    Demographics Purchase history OA engagement Ad event Sticker collections Purchase probabilities Treatment Group Control Group $MBTTJ fi FS treatment = 1 treatment = 0 Purchase or not ≈  *OUVJUJWFBOEFBTZUPUSBJO  -PXWBSJBODFBOE fl FYJCMF Retention
  30. Estimate Individual Treatment Effects Campaign Period Observation Period, 30 days

    Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections $MBTTJ fi FS Purchase probability Purchase probability treatment = 1 treatment = 0 candidate Retention
  31. Estimate Individual Treatment Effects Campaign Period Observation Period, 30 days

    Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections $MBTTJ fi FS treatment = 1 treatment = 0 Uplift Purchase probability candidate Retention
  32. Uplifts by Deciles 6QMJGU      

             The Persuadable Sure Thing/Lost Cause Sleeping Dog Retention
  33. Reduce Delivery Cost and Mute Rate in the Future 6TFS-JTU

    %JTDPVOU 6TFS-JTU $PJO1PJOU #BDL 6TFS-JTU -VDLZESBX Day 3 Day 2 Day 1 Campaign Period 4UJDLFS %BUB %FW Retention
  34. Actionable and Explainable User Segmentation from RFM to CLV CRM:

    How to identify valuable customers?
  35. Know Your Customers F R M Historical Unknown Time Now

    From Buying Profile Retention Revenue
  36. Measure Your Customer Lifetime Value From RFM to CLV Past

    Now Future Measure Former Value Predict Future Value RFM CLV Retention Revenue
  37. Measure Your Customer Lifetime Value From RFM to CLV Past

    Now Future RFM CLV Measure Former Value Predict Retention Revenue
  38. Measure Former Value - RFM Modeling RFM Scores Historical data

    R F M Segments Retention Revenue
  39. Measure Former Value - RFM Modeling R F M RFM

    Segments Historical data Clustering Retention Revenue
  40. RFM Modeling - Clustering R F Retention Revenue

  41. RFM Modeling - Clustering R F Retention Revenue

  42. Measure Former Value - RFM Modeling R F M Historical

    data Clustering Retention Revenue
  43. Measure Your Customer Lifetime Value From RFM to CLV Past

    Now Future RFM CLV Measure Former Value Predict Future Value Retention Revenue
  44. Predict Future Value - CLV Modeling Demographic Browsing history Buying

    pro fi le OA interaction Feature Engineering Modeling Tags CLV High ↑ CLV Median CLV Low ↓ { > 50% High valued customers <= 50% Medium valued customers 0, Churned customers 450 days 180 days Now Cascade model Retention Revenue
  45. Predict Future Value - CLV Modeling Feature Engineering Modeling Tags

    CLV High ↑ CLV Median CLV Low ↓ { > 50% High valued customers <= 50% Medium valued customers 0, Churned customers 450 days 180 days Now User Embedding Demographic Browsing history Buying pro Retention Revenue
  46. Predict Future Value - CLV Modeling User Embedding Time of

    View Content Views from customers AD 1 AD n Product 1 Article 1 Article n Product n … … … Context window Retention Revenue
  47. Explicit and explainable user segment R↑F↑M↑ R↑F↑M↓ R↑F↓M↑ R↑F↓M↓ R↓F↑M↑

    R↓F↑M↓ R↓F↓M↑ R↓F↓M↓ ࠷ۙ༗ိత౔߽ৗ٬ ࠷ۙ༗ိతখࢿৗ٬ ࠷ۙ༗ိత౔߽ك٬ ࠷ۙ༗ိతখࢿك٬ ফࣦత౔߽ৗ٬ ফࣦతখࢿৗ٬ ফࣦత౔߽ك٬ ফࣦతখࢿك٬ CLV Low CLV Median CLV High ະိߴফඅ ະိதফඅ ະိෆফඅ RFM CLV Deliverables: Tags Retention Revenue
  48. ೗Կར༻໛ܕඪតʁ R↑F↑M↓ Upselling ! RFM RFM 最近有來的⼩資常客 R↑F↑M↑ RFM 最近有來的⼟豪常客

    Retention Revenue
  49. R↑F↑M↑ CLV Low Retain ! ೗Կར༻໛ܕඪតʁ RFM + CLV RFM

    CLV 最近有來的⼟豪常客 未來不消費 Retention Revenue
  50. How to adopt CLV service? Overview of CLV Service Retention

    Revenue
  51. Wait… is there…

  52. Any model that could handle everything? A good start of

    user understanding ! #MBDLCPY Who among them are more likely to come? Which content is more likely be click by whom? Who among them are more likely persuadable? How to estimate user values? How to refer friends of users to us?
  53. Foundation models Pertaining on user behaviors, which come from our

    family services log! Click prediction User segment prediction Sales/revenue prediction
  54. Combined with MLOps: 
 Scalable Automated MarTech Solution Data Source

    %BUB 8BSFIPVTF Data Integration 'FBUVSF4UPSF Application "DRVJSFOFXVTFS "%5BSHFUJOH #PPTUSFWFOVF *NQSPWF3FUFOUJPO … ,(4PMVUJPO1JQFMJOF $-74PMVUJPO1JQFMJOF 6QMJGU 
 4PMVUJPO1JQFMJOF AI Solution 6TFSTFHNFOUBUJPO 
 4PMVUJPO1JQFMJOF … LINE Family Services Demographic Log
  55. To sum up Swifter Scalable and Flexible Solution Faster Speed

    of Delivery Data-oriented Decision Making AI solves (almost) every Martech problem Developer Marketer YOU
  56. Thank you  資料科學家大解密 - LINE資料工程部門介紹 - 資料科學生活 - 職場眉眉角角

    Closing the Distance Share easily with friends in LINE Timeline
  57. Thank you Closing the Distance Official Account Ads 圖⽚來源︓https://hub.line.me/

  58. Thank you Closing the Distance Official Account Ads 圖⽚來源︓https://hub.line.me/ 如何推薦用戶會感興趣的商家?

    如何確保新聞品質? 如何判斷假新聞? 如何判斷潛在的訂閱制流失戶?
  59. Thank you AI-enable Applications Business Intelligence Data Dev LINE Family

    Services LINE TODAY LINE SHOPPING LINE SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Official Account Fact Checker LINE HELP TW LINE Travel Ads 獨立的資料工程部門,提供資料科學解決方案 LINE TODAY
  60. Thank you Data Dev LINE Family Services LINE SHOPPING LINE

    SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Fact Checker LINE HELP TW LINE Travel NLP Knowledg e Graph Uplift Modeling NER Classifier Duplication Detector Auto completion Keyword Extraction Related Search Text Generation User Tagging Data Analytics Recom- mendation CLV 從報表、分析洞見到預測模型 LINE TODAY
  61. Thank you 成員組成 • Build and optimize da ta pipeline

    architectur e • Assemble large, com plex data sets that m eet requirements Data Engineer Data Analyst Big data infra, SQL, ET L, message queuing • Interpret data, analyz e results using statisti cal techniques • Identify, analyze, and interpret trends or pat terns in complex data sets Statistics, Data Visualiz ation, Business Knowle dge SKILL RESPONSIBILITY • Select appropriate da tasets and data repre sentation methods • Research and imple ment appropriate ML algorithms Data Scientist Machine learning, deep learning, CV, NLP, Spe ech ML Svc Engineer • Build and scale mach ine learning infrastruc ture • Monitor model perfor mance System infrastructure d esign, DevOps
  62. Thank you Subtitle 成員組成 • Build and optimize da ta

    pipeline architectur e • Assemble large, com plex data sets that m eet requirements Data Engineer Data Analyst Big data infra, SQL, ET L, message queuing • Interpret data, analyz e results using statisti cal techniques • Identify, analyze, and interpret trends or pat terns in complex data sets Statistics, Data Visualiz ation, Business Knowle dge SKILL RESPONSIBILITY Pipeline Biz • Select appropriate da tasets and data repre sentation methods • Research and imple ment appropriate ML algorithms Data Scientist Machine learning, deep learning, CV, NLP, Spe ech Model ML Svc Engineer • Build and scale mach ine learning infrastruc ture • Monitor model perfor mance System infrastructure d esign, DevOps Service
  63. Thank you