竟然有人說AI不能拿來做行銷

Speaker: Johnson
Event: 台大AI社

Transcript

1.  __ɼᰈવ༗ਓ㘸AIෆೳ፤ိ၏ߦ᭖ Data Scientist /Johnson Wu @LINE Taiwan
2. Hello! - Johnson Wu - CSIE (B01, R05) - Intern @ Microsoft Research Asia - Poetry book writer (indirect) - Data Scientist @ LINE Taiwan 2019 @LINE Fukuoka
3. Imagination about Digital Marketing in LINE: *OUFSOFU
   
   DFMFCSJUZ 4PDJBM
   NFEJB "EWFSUJTFNFOU
4. Our actual thought of Marketing: Content & Experience Social & Relationship Commerce & Sales Management Data Advertising & Promotion • Talent Mgt. • Finance Mgt. • Quality Mgt. • CRM • Channels & Partners • Sales Automation & Intelligence • CDP/DMP • Business Intelligence • Data Governance • Customer Exp. • KOL • Community & Review • Content • Experience Mgt. • SEO • Marketing • Advertising • PR Align with user journey!
5. Look into our problems with… (A)AARRR Funnel ! How to let people visit my content? How to let people have action/buy? How to make people come back and buy? How to make people start to buy? How to let people refer new users to your content? Acquisition Activation Retention Revenue Referral 1.25 2.5 3.75 5
6. Commercial problems with the AARRR Real Cases in LINE Taiwan How to recommend OA push to users with limited number of pushes? How to attract most possibly new users to the service? How to value customers and Recommend based on values? How to deliver ad promotion with the best cost-profit balance? How do we understand users for AD targeting? Acquisition Activation Retention and activation AARRR Revenue
7. Using knowledge graph solution for new user acquisition User Acquisition
8. What are active users in the service? Where should I find the new users? How to connect new users with active users? Acquisition
9. Users in LINE Correlate users with behaviors of other services -*/&
   50%": -*/& 
 4)011*/( -*/&
   .64*$ ? Acquisition 0"
10. Attracting New users Via SmartChannel with knowledge graph technique. smart channel And graph? Acquisition
11. Graph embedding helps Service knowledge graph provides abundant information - GraphSAGE (inductive), GCN - Graph Attention network with deepwalk - Semantic matching based: ComplEX - Algorithm choice depends on data traits https://arxiv.org/pdf/1710.10903.pdf https://arxiv.org/pdf/1706.02216.pdf 1 2 1 2 Acquisition
12. How does graph work? Model predict the probability of the “link”. - Example: Graph Attention network with deepwalk 1 2 Acquisition - Users are represented as edge connections between entity with meaningful actions - Users1: article1 read -> 
 shop item2 -> join campaign 3…. - Each node (entity) is aggregated 
 by the weighted sum of neighbors.
13. New acquired users Case study on LINE POINTS wall : how to acquire new users? -*/&
    10*/54
    XBMM
    VTFST OPO-*/&
    10*/54
    XBMM
    VTFST 0 ffi DJBM
    "DDPVOU
    GPMMPXJOH
    TUBUVT ,OPXMFEHF
    HSBQI
    NPEFMJOH
    10*/54
    
    0" $MVTUFSJOH ,OPXMFEHF
    HSBQI
    JOGFSFODF
     
 $MVTUFSJOH
    0" OPO-*/&
    10*/54
    XBMM
    VTFST
    NPTU
    MJLFMZ
    UP
    KPJO
    -*/&
    SFXBSE Compose triples Compose triples Top k similar users
14. - Flexible and complex decision making - Applied to LINE SHOPPING, SPOT, POINTS, MUSIC - Embeddings show clustering effect w.r.t campaign Performance Case study on LINE POINTS : effectiveness and fully data orientation - ≥ 25% of new user growth comes from AI’s decision. - Easily fit for different scenario : 
 How about recommend OA Push message to non-new users? - ≥200% CTR lift Acquisition Activation
15. Apply uplift model to target the persuadable Retention: How to Maximize Campaign Profit?
16. Who should receive OAP? How many OAP I should deliver? How to evaluate performance? Retention
17. How to deliver ad promotion with the best cost-profit balance? $PVOU
    PG
    DBNQBJHO      .POUI
    PG
    :FBS
                 3FDFJWF
    
    0"
    NFTTBHF 1VSDIBTF
    
    JO
    DBNQBJHO
    QFSJPE &BSO
    -*/&
    $0*/10*/54 Boost the Profits with Incentives Retention
18. User Segments by Uplift High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
19. User Segments by Uplift 
    EFMJWFS
    DPTU
    
    JODFOUJWF
    DPTU High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
20. User Segments by Uplift 
    EFMJWFSZ
    DPTU
    
    SFWFOVF High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
21. User Segments by Uplift 
    EFMJWFSZ
    DPTU High Low High Low Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
22. User Segments by Uplift  EFMJWFSZ
    DPTU
     *ODFOUJWF
    DPTU
    
    
    
    SFWFOVF High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Retention
23. User Segments by Uplift QPTJUJWF
    VQMJGU OFHBUJWF
    VQMJGU OP
    VQMJGU OP
    VQMJGU Retention High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable Uplift M odel
24. User Segments by Uplift Retention High Low Low High Buy if treated Buy if not treated Sure Thing Sleeping Dog Lost Cause The Persuadable 3FTQPOTF
    .PEFM Uplift M odel
25. A/B Testing to Collect Lookalike User Groups Treatment Group Control Group Retention
26. #Control Group #Treatment Group Leverage Lookalike Groups to Estimate Uplift
27. User 360 Degree Features Campaign Period Observation Period, 30 days Purchase or not Purchase or not Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections Treatment Group Control Group Retention
28. User 360 Degree Features Campaign Period Observation Period, 30 days Purchase or not Purchase or not Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections Treatment Group Control Group If the campaign effect is significant  If the sure thing exist  Retention
29. Meta-learners, S-Learner Campaign Period Observation Period, 30 days Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections Purchase probabilities Treatment Group Control Group $MBTTJ fi FS treatment = 1 treatment = 0 Purchase or not ≈  *OUVJUJWF
    BOE
    FBTZUPUSBJO
     -PX
    WBSJBODF
    BOE
    fl FYJCMF Retention
30. Estimate Individual Treatment Effects Campaign Period Observation Period, 30 days Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections $MBTTJ fi FS Purchase probability Purchase probability treatment = 1 treatment = 0 candidate Retention
31. Estimate Individual Treatment Effects Campaign Period Observation Period, 30 days Browsing behavior Demographics Purchase history OA engagement Ad event Sticker collections $MBTTJ fi FS treatment = 1 treatment = 0 Uplift Purchase probability candidate Retention
32. Uplifts by Deciles 6QMJGU                The Persuadable Sure Thing/Lost Cause Sleeping Dog Retention
33. Reduce Delivery Cost and Mute Rate in the Future 6TFS
    -JTU
    %JTDPVOU 6TFS
    -JTU
    $PJO1PJOU
    #BDL 6TFS
    -JTU
    -VDLZESBX Day 3 Day 2 Day 1 Campaign Period 4UJDLFS %BUB
    %FW Retention
34. Actionable and Explainable User Segmentation from RFM to CLV CRM: How to identify valuable customers?
35. Know Your Customers F R M Historical Unknown Time Now From Buying Profile Retention Revenue
36. Measure Your Customer Lifetime Value From RFM to CLV Past Now Future Measure Former Value Predict Future Value RFM CLV Retention Revenue
37. Measure Your Customer Lifetime Value From RFM to CLV Past Now Future RFM CLV Measure Former Value Predict Retention Revenue
38. Measure Former Value - RFM Modeling RFM Scores Historical data R F M Segments Retention Revenue
39. Measure Former Value - RFM Modeling R F M RFM Segments Historical data Clustering Retention Revenue
40. RFM Modeling - Clustering R F Retention Revenue
41. RFM Modeling - Clustering R F Retention Revenue
42. Measure Former Value - RFM Modeling R F M Historical data Clustering Retention Revenue
43. Measure Your Customer Lifetime Value From RFM to CLV Past Now Future RFM CLV Measure Former Value Predict Future Value Retention Revenue
44. Predict Future Value - CLV Modeling Demographic Browsing history Buying pro fi le OA interaction Feature Engineering Modeling Tags CLV High ↑ CLV Median CLV Low ↓ { > 50% High valued customers <= 50% Medium valued customers 0, Churned customers 450 days 180 days Now Cascade model Retention Revenue
45. Predict Future Value - CLV Modeling Feature Engineering Modeling Tags CLV High ↑ CLV Median CLV Low ↓ { > 50% High valued customers <= 50% Medium valued customers 0, Churned customers 450 days 180 days Now User Embedding Demographic Browsing history Buying pro Retention Revenue
46. Predict Future Value - CLV Modeling User Embedding Time of View Content Views from customers AD 1 AD n Product 1 Article 1 Article n Product n … … … Context window Retention Revenue
47. Explicit and explainable user segment R↑F↑M↑ R↑F↑M↓ R↑F↓M↑ R↑F↓M↓ R↓F↑M↑ R↓F↑M↓ R↓F↓M↑ R↓F↓M↓ ࠷ۙ༗ိత౔߽ৗ٬ ࠷ۙ༗ိతখࢿৗ٬ ࠷ۙ༗ိత౔߽ك٬ ࠷ۙ༗ိతখࢿك٬ ফࣦత౔߽ৗ٬ ফࣦతখࢿৗ٬ ফࣦత౔߽ك٬ ফࣦతখࢿك٬ CLV Low CLV Median CLV High ະိߴফඅ ະိதফඅ ະိෆফඅ RFM CLV Deliverables: Tags Retention Revenue
48. ೗Կར༻໛ܕඪតʁ R↑F↑M↓ Upselling ! RFM RFM 最近有來的⼩資常客 R↑F↑M↑ RFM 最近有來的⼟豪常客 Retention Revenue
49. R↑F↑M↑ CLV Low Retain ! ೗Կར༻໛ܕඪតʁ RFM + CLV RFM CLV 最近有來的⼟豪常客 未來不消費 Retention Revenue
50. How to adopt CLV service? Overview of CLV Service Retention Revenue
51. Wait… is there…
52. Any model that could handle everything? A good start of user understanding ! #MBDL
    CPY Who among them are more likely to come? Which content is more likely be click by whom? Who among them are more likely persuadable? How to estimate user values? How to refer friends of users to us?
53. Foundation models Pertaining on user behaviors, which come from our family services log! Click prediction User segment prediction Sales/revenue prediction
54. Combined with MLOps: 
 Scalable Automated MarTech Solution Data Source %BUB
    8BSFIPVTF Data Integration 'FBUVSF
    4UPSF Application "DRVJSF
    OFX
    VTFS "%
    5BSHFUJOH #PPTU
    SFWFOVF *NQSPWF
    3FUFOUJPO … ,(
    4PMVUJPO
    1JQFMJOF $-7
    4PMVUJPO
    1JQFMJOF 6QMJGU 
 4PMVUJPO
    1JQFMJOF AI Solution 6TFS
    TFHNFOUBUJPO 
 4PMVUJPO
    1JQFMJOF … LINE Family Services Demographic Log
55. To sum up Swifter Scalable and Flexible Solution Faster Speed of Delivery Data-oriented Decision Making AI solves (almost) every Martech problem Developer Marketer YOU
56. Thank you  資料科學家大解密 - LINE資料工程部門介紹 - 資料科學生活 - 職場眉眉角角 Closing the Distance Share easily with friends in LINE Timeline
57. Thank you Closing the Distance Official Account Ads 圖⽚來源︓https://hub.line.me/
58. Thank you Closing the Distance Official Account Ads 圖⽚來源︓https://hub.line.me/ 如何推薦用戶會感興趣的商家？ 如何確保新聞品質？ 如何判斷假新聞? 如何判斷潛在的訂閱制流失戶？
59. Thank you AI-enable Applications Business Intelligence Data Dev LINE Family Services LINE TODAY LINE SHOPPING LINE SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Official Account Fact Checker LINE HELP TW LINE Travel Ads 獨立的資料工程部門，提供資料科學解決方案 LINE TODAY
60. Thank you Data Dev LINE Family Services LINE SHOPPING LINE SPOT LINE MUSIC LINE Sticker LINE VOOM LINE Reward Fact Checker LINE HELP TW LINE Travel NLP Knowledg e Graph Uplift Modeling NER Classifier Duplication Detector Auto completion Keyword Extraction Related Search Text Generation User Tagging Data Analytics Recom- mendation CLV 從報表、分析洞見到預測模型 LINE TODAY
61. Thank you 成員組成 • Build and optimize da ta pipeline architectur e • Assemble large, com plex data sets that m eet requirements Data Engineer Data Analyst Big data infra, SQL, ET L, message queuing • Interpret data, analyz e results using statisti cal techniques • Identify, analyze, and interpret trends or pat terns in complex data sets Statistics, Data Visualiz ation, Business Knowle dge SKILL RESPONSIBILITY • Select appropriate da tasets and data repre sentation methods • Research and imple ment appropriate ML algorithms Data Scientist Machine learning, deep learning, CV, NLP, Spe ech ML Svc Engineer • Build and scale mach ine learning infrastruc ture • Monitor model perfor mance System infrastructure d esign, DevOps
62. Thank you Subtitle 成員組成 • Build and optimize da ta pipeline architectur e • Assemble large, com plex data sets that m eet requirements Data Engineer Data Analyst Big data infra, SQL, ET L, message queuing • Interpret data, analyz e results using statisti cal techniques • Identify, analyze, and interpret trends or pat terns in complex data sets Statistics, Data Visualiz ation, Business Knowle dge SKILL RESPONSIBILITY Pipeline Biz • Select appropriate da tasets and data repre sentation methods • Research and imple ment appropriate ML algorithms Data Scientist Machine learning, deep learning, CV, NLP, Spe ech Model ML Svc Engineer • Build and scale mach ine learning infrastruc ture • Monitor model perfor mance System infrastructure d esign, DevOps Service
63. Thank you