Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Improvement in Information Reception Experience of LINE Official Account of LINE Pay by Data Science

Improvement in Information Reception Experience of LINE Official Account of LINE Pay by Data Science

3a35402ba4f955e81dce2c7a22609d5c?s=128

LINE DEVDAY 2021
PRO

November 10, 2021
Tweet

More Decks by LINE DEVDAY 2021

Other Decks in Technology

Transcript

  1. None
  2. Agenda - Background & Mission - challenge 1 : Quantification

    of the value of official account messages - challenge 2 : Improvement of user targeting accuracy by message contents - Session Summary
  3. Background & Mission

  4. - Communication channel that delivers information on new LINE Pay

    features, campaigns, etc. “LINE Pay” LINE Official Account
  5. The experience of receiving messages was not good Users were

    receiving a lot of valueless information Messages per user per month Over 30 ※ as of August 2019
  6. - Users will only receive valuable messages Mission

  7. Challenge 1 : Quantification of the value of official account

    messages Challenge 2 : Improvement of user targeting accuracy by message contents Two challenges
  8. Challenge1 : Quantification of the value of official account messages

  9. How to quantify the value of the official account messages

    Message click rate Non-block rate (1- Block rate) valuable valueless High Low High Low
  10. Problem Block rate cannot be measured easily - There is

    no log data about the official account has blocked that caused the message - Block rate cannot be measured by a simple method time Number of blocks : Message is delivered
  11. Solution - Estimate the number of blocks caused by message

    delivery by fitting a statistical model time Number of blocks : Number of blocks caused by message : Number of blocks not caused by message Fit a statistical model to measure the block rate Message is delivered
  12. The number of blocks has periodicity by day of the

    week and time The number of blocks after message delivery is approximated by power curve Message clicks will occur within 6 hours Elapsed time Observed value Fitted value Number of blocks Number of blocks Day of week Time Elapsed time Cumulative percentage of clicks 00 23 Mon Sun Solution Analysis of user blocking behavior
  13. : Number of blocks caused by message time t Number

    of blocks : Number of blocks not caused by message Solution Overview of the fitted statistical model
  14. time Number of blocks : Message is delivered Solution Statistical

    model fits well : Actual : Fitted
  15. Result Message click rate Non-block rate(1- Block rate) valuable valueless

    High Low High Low - Valuable messages are delivered to many followers - Valueless messages are reduced the number of delivery, and improve user targeting • LINE financial services information • LINE Family Service Information • LINE Pay Coupon • LINE Pay new features
  16. Challenge2 : Improving the accuracy of user targeting

  17. Goal 1. Increase the number of new users 2. Reduce

    the number of deliveries
  18. - Lookalike engine takes a seed user set as input

    and output a set of similar audience - By setting existing users to seed, we expected to improve the accuracy of user targeting for new acquisitions All Users Similar Audience Seed users Seed users Lookalike Engine Similar Audience z features Solution Lookalike Audience Targeting
  19. Is it possible for Lookalike Audience targeting to maintain the

    number of new acquisitions even if the number of delivery is reduced? < Approach > - Verified by AB test Solution Key Question
  20. Result Existing methods( Targeting based on estimated user demographics )

    vs Lookalike Audience Targeting the number of delivery 1/10 the number of new acquisitions 1/2 0 20 40 60 80 100 Existing methods Lookalike Audience Targeting 0 20 40 60 80 100 Existing methods Lookalike Audience Targeting
  21. System integration of Lookalike Audience Targeting planner Query to extract

    seed users API to upload user list CMS Lookalike Engine Lookalike Targeting API seed users similar audience Target List - The planner can execute from extraction of seed user to message delivery settings - UDF / API was prepared, and the cost of system development was low
  22. Session Summary

  23. Session Summary Using data science techniques to support decision making

    for a better user experience - Quantified block action for which data is not easily available by statistical analysis - Reduced the number of delivery of valueless messages - Lookalike Audience Targeting showed that even if the number of delivery is reduced to 1/10, the number of new acquisitions can be maintained at 1/2. - Enabled self-service targeting by Lookalike Audience Targeting Using data science techniques, we did the following so that users will only receive valuable message from “LINE Pay” LINE official account :
  24. Thank you