Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Using Data to Drive Platform Growth

Using Data to Drive Platform Growth

LINE Developers Thailand

September 04, 2022
Tweet

More Decks by LINE Developers Thailand

Other Decks in Technology

Transcript

  1. Sharing on how to manage data from different sources
    to perform analysis and how to use data to optimize
    product features and drive platform growth.
    Piyawat Amnuayphoncharoen (Trust)
    Data Analyst, LINE Thailand
    Using Data to Drive
    Platform Growth

    View full-size slide

  2. ➢ What is LINE SHOPPING?
    ➢ How to manage data from different sources?
    ○ What are steps needed to create efficient data platform?
    ➢ How to use data to optimize product features?
    ○ How to turn data into insights?
    ○ What should be aware when
    interpreting A/B testing results?
    Agenda

    View full-size slide

  3. Keychron Thailand
    on LINE SHOPPING
    NuPhy Thailand
    on LINE SHOPPING

    View full-size slide

  4. LINE SHOPPING
    Sellers
    - Create order via chat.
    - List products via storefront.
    - Manage stocks and orders
    from both sources in 1 place.
    - Send out new product launch
    or promotion using broadcast
    messages.
    - Build their own brand.
    - Create fan base using LINE
    Official Account.
    Buyers
    - Discover brands they love.
    - Follow or Add Friend with
    shops to get promotion and
    follow their contents.
    - Exclusive promotion on LINE
    SHOPPING : LINE POINTS /
    Coupon / LINE MAN Discount

    View full-size slide

  5. Usage of Data Inside LINE SHOPPING
    Business
    Operation
    Predictive
    Model
    Product Feature
    Optimization
    • Monitor platform performance.
    • Optimize campaign.
    • Customer segmentation.
    • Monitor user journey.
    • Monitor seller performance.
    • Plan seller supporting
    packages.
    • Recommend products / shops
    on search engine.
    • Recommend contents on feed.
    • Suggest product category.
    • Retarget customers.
    • Detect fraud / abuse cases.
    • Monitor product features
    performance.
    • Optimize / create data-driven
    features (apart from strategy-
    driven features).

    View full-size slide

  6. Reference : https://www.datamesh-architecture.com
    2. Ingestion
    1. Data Source 3. Storage 4. Processing
    5. Consumers

    View full-size slide

  7. Data Platform at LINE
    Ingestion
    Data
    Source
    Storage Processing
    Orchestration
    Consumers

    View full-size slide

  8. Reference : https://www.datamesh-architecture.com

    View full-size slide

  9. Thoughtful Execution Framework
    - Anna Koskinen, Principal Designer, Spotify
    Reference : https://spotify.design/article/from-gut-to-plan-the-thoughtful-execution-framework
    1.Goal 2. Data / Insights
    3. Problems /
    Opportunities
    4. Hypothesis 5. Solution 6. Learning

    View full-size slide

  10. Reference : https://spotify.design/article/from-gut-to-plan-the-thoughtful-execution-framework
    Goal
    Data / Insights
    Problems / Opportunities
    Hypothesis
    Solution
    Learning
    Why are we optimizing this product ?
    What metrics are we using ?
    Insights from data or customer interviews related to the goal
    From insights gathered, what are the current problems to solve
    or the opportunities we have ?
    How can we tackle each problem / opportunity?
    Design real action and estimate impacts based on hypothesis.
    Review whether the hypothesis is correct or not.

    View full-size slide

  11. Goal
    Data / Insights
    Problems /
    Opportunities
    Hypothesis
    Solution
    Learning
    Increase LINE SHOPPING visit frequency.
    Click navigation
    bar (NB).
    Receive LINE
    POINTS (LP).
    Favourite shops release
    many collections.

    View full-size slide

  12. Behavioural Cohorts
    Group users based on their actions :
    - G1 : Click navigation bar 1 time
    - G2 : Buy 1 order
    - G3 : Receive LINE POINTS
    Method #1 : Find effects of each group :
    - G1 : Visit Frequency 3.43 vs. 1.21
    - G2 : Visit Frequency 1.98 vs. 1.94
    - G3 : Visit Frequency 2.87 vs. 1.33
    Method #2 : Calculate PPV and NPV of each group :
    PPV = 1,345 / (1,345 + 13) = 99%
    NPV = 135 / (135 + 7) = 95%
    - G1 : PPV = 99% / NPV = 95%
    - G2 : PPV = 76% / NPV = 24%
    - G3 : PPV = 94% / NPV = 82%
    * Suitable for retention or 0/1 calculation
    Visit > 1 (Yes) Visit > 1 (No)
    Click NB (Yes) 1,345 (TP) 13 (FP)
    Click NB (No) 7 (FN) 135 (TN)
    Reference : https://amplitude.com/blog/find-the-key-to-your-apps-growth-without-an-army-of-data-scientists

    View full-size slide

  13. Goal
    Data / Insights
    Problems /
    Opportunities
    Hypothesis
    Solution
    Learning
    Increase LINE SHOPPING visit frequency.
    Click navigation
    bar (NB).
    Receive LINE
    POINTS (LP).
    Favourite shops release
    many collections.
    How might we make
    more users use NB?
    How might we make users
    use NB more frequent?
    How might we make users
    see LP products easier?
    Adding features to NB will lead
    to higher LS visit frequency.
    Adding notifications to NB will lead
    to higher LS visit frequency.
    Add coupon aggregation page to NB.
    Does new coupon aggregation page boost any metrics ?

    View full-size slide

  14. Top Banner
    Wow Item
    Collection
    Search
    Feed

    View full-size slide

  15. Goal
    Data / Insights
    Problems /
    Opportunities
    Hypothesis
    Solution
    Learning
    Increase HOME page performance.
    Feed has highest CTR.
    How might we increase visibility of Feed?
    Moving Feed up to top most position will lead
    to better HOME page performance.
    Design new HOME page with Feed as first placement.
    A/B Testing

    View full-size slide

  16. Control (A) Variance (B)

    View full-size slide

  17. A/B Testing Results
    Pros :
    - CTR increases 19%.
    - Add to Cart Rate increases 4%.
    - GMV increases 20%.
    Cons :
    - Shop Visit decreases 8%.
    - Time Spend decreases 8%.
    Key Findings :
    - CTR and CVR of Design A is higher or
    equal in promotion day.
    - Top Banner and Collection that were
    removed has high Shop Visit.
    - Business team has no placement to
    promote campaigns using Design B.
    A/B Testing Analysis Steps
    1.Basic Analysis : Define Winners
    2.Secondary Metrics Analysis
    3.Audience Breakdown Analysis
    Reference : https://www.dynamicyield.com/lesson/ab-testing-analysis

    View full-size slide

  18. Roll Out Design

    View full-size slide

  19. Key Takeaways
    Thoughtful Execution Framework
    1.Goal
    2. Data /
    Insights
    3. Problems /
    Opportunities
    4. Hypothesis 5. Solution 6. Learning
    Data Mesh Architecture
    Operational
    Data
    Raw
    Data
    Aggregation
    Data
    Analytics
    Data Product

    View full-size slide

  20. Data really EMPOWERS
    everything that we do.
    - Jeff Weiner, Chief Executive Officer of LinkedIn

    View full-size slide