Using Data to Drive Platform Growth

Transcript

1. Sharing on how to manage data from different sources
   to perform analysis and how to use data to optimize
   product features and drive platform growth.
   Piyawat Amnuayphoncharoen (Trust)
   Data Analyst, LINE Thailand
   Using Data to Drive
   Platform Growth
   

   View Slide

2. ➢ What is LINE SHOPPING?
   ➢ How to manage data from different sources?
   ○ What are steps needed to create efficient data platform?
   ➢ How to use data to optimize product features?
   ○ How to turn data into insights?
   ○ What should be aware when
   interpreting A/B testing results?
   Agenda
   

   View Slide

3. Keychron Thailand
   on LINE SHOPPING
   NuPhy Thailand
   on LINE SHOPPING
   

   View Slide

4. LINE SHOPPING
   Sellers
   - Create order via chat.
   - List products via storefront.
   - Manage stocks and orders
   from both sources in 1 place.
   - Send out new product launch
   or promotion using broadcast
   messages.
   - Build their own brand.
   - Create fan base using LINE
   Official Account.
   Buyers
   - Discover brands they love.
   - Follow or Add Friend with
   shops to get promotion and
   follow their contents.
   - Exclusive promotion on LINE
   SHOPPING : LINE POINTS /
   Coupon / LINE MAN Discount
   

   View Slide

5. Usage of Data Inside LINE SHOPPING
   Business
   Operation
   Predictive
   Model
   Product Feature
   Optimization
   • Monitor platform performance.
   • Optimize campaign.
   • Customer segmentation.
   • Monitor user journey.
   • Monitor seller performance.
   • Plan seller supporting
   packages.
   • Recommend products / shops
   on search engine.
   • Recommend contents on feed.
   • Suggest product category.
   • Retarget customers.
   • Detect fraud / abuse cases.
   • Monitor product features
   performance.
   • Optimize / create data-driven
   features (apart from strategy-
   driven features).
   

   View Slide

6. Reference : https://www.datamesh-architecture.com
   2. Ingestion
   1. Data Source 3. Storage 4. Processing
   5. Consumers
   

   View Slide

7. Data Platform at LINE
   Ingestion
   Data
   Source
   Storage Processing
   Orchestration
   Consumers
   

   View Slide

8. Reference : https://www.datamesh-architecture.com
   

   View Slide

9. Thoughtful Execution Framework
   - Anna Koskinen, Principal Designer, Spotify
   Reference : https://spotify.design/article/from-gut-to-plan-the-thoughtful-execution-framework
   1.Goal 2. Data / Insights
   3. Problems /
   Opportunities
   4. Hypothesis 5. Solution 6. Learning
   

   View Slide

10. Reference : https://spotify.design/article/from-gut-to-plan-the-thoughtful-execution-framework
    Goal
    Data / Insights
    Problems / Opportunities
    Hypothesis
    Solution
    Learning
    Why are we optimizing this product ?
    What metrics are we using ?
    Insights from data or customer interviews related to the goal
    From insights gathered, what are the current problems to solve
    or the opportunities we have ?
    How can we tackle each problem / opportunity?
    Design real action and estimate impacts based on hypothesis.
    Review whether the hypothesis is correct or not.
    

    View Slide

11. Goal
    Data / Insights
    Problems /
    Opportunities
    Hypothesis
    Solution
    Learning
    Increase LINE SHOPPING visit frequency.
    Click navigation
    bar (NB).
    Receive LINE
    POINTS (LP).
    Favourite shops release
    many collections.
    

    View Slide

12. Behavioural Cohorts
    Group users based on their actions :
    - G1 : Click navigation bar 1 time
    - G2 : Buy 1 order
    - G3 : Receive LINE POINTS
    Method #1 : Find effects of each group :
    - G1 : Visit Frequency 3.43 vs. 1.21
    - G2 : Visit Frequency 1.98 vs. 1.94
    - G3 : Visit Frequency 2.87 vs. 1.33
    Method #2 : Calculate PPV and NPV of each group :
    PPV = 1,345 / (1,345 + 13) = 99%
    NPV = 135 / (135 + 7) = 95%
    - G1 : PPV = 99% / NPV = 95%
    - G2 : PPV = 76% / NPV = 24%
    - G3 : PPV = 94% / NPV = 82%
    * Suitable for retention or 0/1 calculation
    Visit > 1 (Yes) Visit > 1 (No)
    Click NB (Yes) 1,345 (TP) 13 (FP)
    Click NB (No) 7 (FN) 135 (TN)
    Reference : https://amplitude.com/blog/find-the-key-to-your-apps-growth-without-an-army-of-data-scientists
    

    View Slide

13. Goal
    Data / Insights
    Problems /
    Opportunities
    Hypothesis
    Solution
    Learning
    Increase LINE SHOPPING visit frequency.
    Click navigation
    bar (NB).
    Receive LINE
    POINTS (LP).
    Favourite shops release
    many collections.
    How might we make
    more users use NB?
    How might we make users
    use NB more frequent?
    How might we make users
    see LP products easier?
    Adding features to NB will lead
    to higher LS visit frequency.
    Adding notifications to NB will lead
    to higher LS visit frequency.
    Add coupon aggregation page to NB.
    Does new coupon aggregation page boost any metrics ?
    

    View Slide

14. Top Banner
    Wow Item
    Collection
    Search
    Feed
    

    View Slide

15. Goal
    Data / Insights
    Problems /
    Opportunities
    Hypothesis
    Solution
    Learning
    Increase HOME page performance.
    Feed has highest CTR.
    How might we increase visibility of Feed?
    Moving Feed up to top most position will lead
    to better HOME page performance.
    Design new HOME page with Feed as first placement.
    A/B Testing
    

    View Slide

16. Control (A) Variance (B)
    

    View Slide

17. A/B Testing Results
    Pros :
    - CTR increases 19%.
    - Add to Cart Rate increases 4%.
    - GMV increases 20%.
    Cons :
    - Shop Visit decreases 8%.
    - Time Spend decreases 8%.
    Key Findings :
    - CTR and CVR of Design A is higher or
    equal in promotion day.
    - Top Banner and Collection that were
    removed has high Shop Visit.
    - Business team has no placement to
    promote campaigns using Design B.
    A/B Testing Analysis Steps
    1.Basic Analysis : Define Winners
    2.Secondary Metrics Analysis
    3.Audience Breakdown Analysis
    Reference : https://www.dynamicyield.com/lesson/ab-testing-analysis
    

    View Slide

18. Roll Out Design
    

    View Slide

19. Key Takeaways
    Thoughtful Execution Framework
    1.Goal
    2. Data /
    Insights
    3. Problems /
    Opportunities
    4. Hypothesis 5. Solution 6. Learning
    Data Mesh Architecture
    Operational
    Data
    Raw
    Data
    Aggregation
    Data
    Analytics
    Data Product
    

    View Slide

20. Data really EMPOWERS
    everything that we do.
    - Jeff Weiner, Chief Executive Officer of LinkedIn
    

    View Slide