Using Data to Drive Platform Growth

Transcript

1. Sharing on how to manage data from different sources to perform analysis and how to use data to optimize product features and drive platform growth. Piyawat Amnuayphoncharoen (Trust) Data Analyst, LINE Thailand Using Data to Drive Platform Growth
2. ➢ What is LINE SHOPPING? ➢ How to manage data from different sources? ◦ What are steps needed to create efficient data platform? ➢ How to use data to optimize product features? ◦ How to turn data into insights? ◦ What should be aware when interpreting A/B testing results? Agenda
3. Keychron Thailand on LINE SHOPPING NuPhy Thailand on LINE SHOPPING
4. LINE SHOPPING Sellers - Create order via chat. - List products via storefront. - Manage stocks and orders from both sources in 1 place. - Send out new product launch or promotion using broadcast messages. - Build their own brand. - Create fan base using LINE Official Account. Buyers - Discover brands they love. - Follow or Add Friend with shops to get promotion and follow their contents. - Exclusive promotion on LINE SHOPPING : LINE POINTS / Coupon / LINE MAN Discount
5. Usage of Data Inside LINE SHOPPING Business Operation Predictive Model Product Feature Optimization • Monitor platform performance. • Optimize campaign. • Customer segmentation. • Monitor user journey. • Monitor seller performance. • Plan seller supporting packages. • Recommend products / shops on search engine. • Recommend contents on feed. • Suggest product category. • Retarget customers. • Detect fraud / abuse cases. • Monitor product features performance. • Optimize / create data-driven features (apart from strategy- driven features).
6. Reference : https://www.datamesh-architecture.com 2. Ingestion 1. Data Source 3. Storage 4. Processing 5. Consumers
7. Data Platform at LINE Ingestion Data Source Storage Processing Orchestration Consumers
8. Reference : https://www.datamesh-architecture.com
9. Thoughtful Execution Framework - Anna Koskinen, Principal Designer, Spotify Reference : https://spotify.design/article/from-gut-to-plan-the-thoughtful-execution-framework 1.Goal 2. Data / Insights 3. Problems / Opportunities 4. Hypothesis 5. Solution 6. Learning
10. Reference : https://spotify.design/article/from-gut-to-plan-the-thoughtful-execution-framework Goal Data / Insights Problems / Opportunities Hypothesis Solution Learning Why are we optimizing this product ? What metrics are we using ? Insights from data or customer interviews related to the goal From insights gathered, what are the current problems to solve or the opportunities we have ? How can we tackle each problem / opportunity? Design real action and estimate impacts based on hypothesis. Review whether the hypothesis is correct or not.
11. Goal Data / Insights Problems / Opportunities Hypothesis Solution Learning Increase LINE SHOPPING visit frequency. Click navigation bar (NB). Receive LINE POINTS (LP). Favourite shops release many collections.
12. Behavioural Cohorts Group users based on their actions : - G1 : Click navigation bar 1 time - G2 : Buy 1 order - G3 : Receive LINE POINTS Method #1 : Find effects of each group : - G1 : Visit Frequency 3.43 vs. 1.21 - G2 : Visit Frequency 1.98 vs. 1.94 - G3 : Visit Frequency 2.87 vs. 1.33 Method #2 : Calculate PPV and NPV of each group : PPV = 1,345 / (1,345 + 13) = 99% NPV = 135 / (135 + 7) = 95% - G1 : PPV = 99% / NPV = 95% - G2 : PPV = 76% / NPV = 24% - G3 : PPV = 94% / NPV = 82% * Suitable for retention or 0/1 calculation Visit > 1 (Yes) Visit > 1 (No) Click NB (Yes) 1,345 (TP) 13 (FP) Click NB (No) 7 (FN) 135 (TN) Reference : https://amplitude.com/blog/find-the-key-to-your-apps-growth-without-an-army-of-data-scientists
13. Goal Data / Insights Problems / Opportunities Hypothesis Solution Learning Increase LINE SHOPPING visit frequency. Click navigation bar (NB). Receive LINE POINTS (LP). Favourite shops release many collections. How might we make more users use NB? How might we make users use NB more frequent? How might we make users see LP products easier? Adding features to NB will lead to higher LS visit frequency. Adding notifications to NB will lead to higher LS visit frequency. Add coupon aggregation page to NB. Does new coupon aggregation page boost any metrics ?
14. Top Banner Wow Item Collection Search Feed
15. Goal Data / Insights Problems / Opportunities Hypothesis Solution Learning Increase HOME page performance. Feed has highest CTR. How might we increase visibility of Feed? Moving Feed up to top most position will lead to better HOME page performance. Design new HOME page with Feed as first placement. A/B Testing
16. Control (A) Variance (B)
17. A/B Testing Results Pros : - CTR increases 19%. - Add to Cart Rate increases 4%. - GMV increases 20%. Cons : - Shop Visit decreases 8%. - Time Spend decreases 8%. Key Findings : - CTR and CVR of Design A is higher or equal in promotion day. - Top Banner and Collection that were removed has high Shop Visit. - Business team has no placement to promote campaigns using Design B. A/B Testing Analysis Steps 1.Basic Analysis : Define Winners 2.Secondary Metrics Analysis 3.Audience Breakdown Analysis Reference : https://www.dynamicyield.com/lesson/ab-testing-analysis
18. Roll Out Design
19. Key Takeaways Thoughtful Execution Framework 1.Goal 2. Data / Insights 3. Problems / Opportunities 4. Hypothesis 5. Solution 6. Learning Data Mesh Architecture Operational Data Raw Data Aggregation Data Analytics Data Product
20. Data really EMPOWERS everything that we do. - Jeff Weiner, Chief Executive Officer of LinkedIn