Improvement in Information Reception Experience of LINE Official Account of LINE Pay by Data Science

Koya Sugimoto
LINE / Financial Data Science Team / Data Scientist

https://linedevday.linecorp.com/2021/ja/sessions/76
https://linedevday.linecorp.com/2021/en/sessions/76
https://linedevday.linecorp.com/2021/ko/sessions/76

Transcript

2. Agenda - Background & Mission - challenge 1 : Quantification of the value of official account messages - challenge 2 : Improvement of user targeting accuracy by message contents - Session Summary
3. Background & Mission
4. - Communication channel that delivers information on new LINE Pay features, campaigns, etc. “LINE Pay” LINE Official Account
5. The experience of receiving messages was not good Users were receiving a lot of valueless information Messages per user per month Over 30 ※ as of August 2019
6. - Users will only receive valuable messages Mission
7. Challenge 1 : Quantification of the value of official account messages Challenge 2 : Improvement of user targeting accuracy by message contents Two challenges
8. Challenge1 : Quantification of the value of official account messages
9. How to quantify the value of the official account messages Message click rate Non-block rate （1- Block rate） valuable valueless High Low High Low
10. Problem Block rate cannot be measured easily - There is no log data about the official account has blocked that caused the message - Block rate cannot be measured by a simple method time Number of blocks : Message is delivered
11. Solution - Estimate the number of blocks caused by message delivery by fitting a statistical model time Number of blocks : Number of blocks caused by message : Number of blocks not caused by message Fit a statistical model to measure the block rate Message is delivered
12. The number of blocks has periodicity by day of the week and time The number of blocks after message delivery is approximated by power curve Message clicks will occur within 6 hours Elapsed time Observed value Fitted value Number of blocks Number of blocks Day of week Time Elapsed time Cumulative percentage of clicks 00 23 Mon Sun Solution Analysis of user blocking behavior
13. : Number of blocks caused by message time t Number of blocks : Number of blocks not caused by message Solution Overview of the fitted statistical model
14. time Number of blocks : Message is delivered Solution Statistical model fits well : Actual : Fitted
15. Result Message click rate Non-block rate（1- Block rate） valuable valueless High Low High Low - Valuable messages are delivered to many followers - Valueless messages are reduced the number of delivery, and improve user targeting • LINE financial services information • LINE Family Service Information • LINE Pay Coupon • LINE Pay new features
16. Challenge2 : Improving the accuracy of user targeting
17. Goal 1. Increase the number of new users 2. Reduce the number of deliveries
18. - Lookalike engine takes a seed user set as input and output a set of similar audience - By setting existing users to seed, we expected to improve the accuracy of user targeting for new acquisitions All Users Similar Audience Seed users Seed users Lookalike Engine Similar Audience z features Solution Lookalike Audience Targeting
19. Is it possible for Lookalike Audience targeting to maintain the number of new acquisitions even if the number of delivery is reduced? < Approach > - Verified by AB test Solution Key Question
20. Result Existing methods（ Targeting based on estimated user demographics ) vs Lookalike Audience Targeting the number of delivery 1/10 the number of new acquisitions 1/2 0 20 40 60 80 100 Existing methods Lookalike Audience Targeting 0 20 40 60 80 100 Existing methods Lookalike Audience Targeting
21. System integration of Lookalike Audience Targeting planner Query to extract seed users API to upload user list CMS Lookalike Engine Lookalike Targeting API seed users similar audience Target List - The planner can execute from extraction of seed user to message delivery settings - UDF / API was prepared, and the cost of system development was low
22. Session Summary
23. Session Summary Using data science techniques to support decision making for a better user experience - Quantified block action for which data is not easily available by statistical analysis - Reduced the number of delivery of valueless messages - Lookalike Audience Targeting showed that even if the number of delivery is reduced to 1/10, the number of new acquisitions can be maintained at 1/2. - Enabled self-service targeting by Lookalike Audience Targeting Using data science techniques, we did the following so that users will only receive valuable message from “LINE Pay” LINE official account :
24. Thank you