Agenda - Background & Mission - challenge 1 : Quantification of the value of official account messages - challenge 2 : Improvement of user targeting accuracy by message contents - Session Summary
The experience of receiving messages was not good Users were receiving a lot of valueless information Messages per user per month Over 30 ※ as of August 2019
Challenge 1 : Quantification of the value of official account messages Challenge 2 : Improvement of user targeting accuracy by message contents Two challenges
Problem Block rate cannot be measured easily - There is no log data about the official account has blocked that caused the message - Block rate cannot be measured by a simple method time Number of blocks : Message is delivered
Solution - Estimate the number of blocks caused by message delivery by fitting a statistical model time Number of blocks : Number of blocks caused by message : Number of blocks not caused by message Fit a statistical model to measure the block rate Message is delivered
The number of blocks has periodicity by day of the week and time The number of blocks after message delivery is approximated by power curve Message clicks will occur within 6 hours Elapsed time Observed value Fitted value Number of blocks Number of blocks Day of week Time Elapsed time Cumulative percentage of clicks 00 23 Mon Sun Solution Analysis of user blocking behavior
: Number of blocks caused by message time t Number of blocks : Number of blocks not caused by message Solution Overview of the fitted statistical model
Result Message click rate Non-block rate（1- Block rate） valuable valueless High Low High Low - Valuable messages are delivered to many followers - Valueless messages are reduced the number of delivery, and improve user targeting • LINE financial services information • LINE Family Service Information • LINE Pay Coupon • LINE Pay new features
- Lookalike engine takes a seed user set as input and output a set of similar audience - By setting existing users to seed, we expected to improve the accuracy of user targeting for new acquisitions All Users Similar Audience Seed users Seed users Lookalike Engine Similar Audience z features Solution Lookalike Audience Targeting
Is it possible for Lookalike Audience targeting to maintain the number of new acquisitions even if the number of delivery is reduced? < Approach > - Verified by AB test Solution Key Question
Result Existing methods（ Targeting based on estimated user demographics ) vs Lookalike Audience Targeting the number of delivery 1/10 the number of new acquisitions 1/2 0 20 40 60 80 100 Existing methods Lookalike Audience Targeting 0 20 40 60 80 100 Existing methods Lookalike Audience Targeting
System integration of Lookalike Audience Targeting planner Query to extract seed users API to upload user list CMS Lookalike Engine Lookalike Targeting API seed users similar audience Target List - The planner can execute from extraction of seed user to message delivery settings - UDF / API was prepared, and the cost of system development was low
Session Summary Using data science techniques to support decision making for a better user experience - Quantified block action for which data is not easily available by statistical analysis - Reduced the number of delivery of valueless messages - Lookalike Audience Targeting showed that even if the number of delivery is reduced to 1/10, the number of new acquisitions can be maintained at 1/2. - Enabled self-service targeting by Lookalike Audience Targeting Using data science techniques, we did the following so that users will only receive valuable message from “LINE Pay” LINE official account :
line_devday2021
sbtechnight
you5805
kaznishi
cembasaranoglu
teckl
hamakou108
ryomaru0825
superbrothers
amarelo_n24
yoshinagaiwnl
nutslove
holman
3n
cherdarchuk
andyhume
searls
jrom
ammeep
jacobian
philhawksworth
revolveconf
rocio
mthomps