Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A/B Testing in LINE TODAY – Recommendation Tab Case @ TECHPULSE 2023

A/B Testing in LINE TODAY – Recommendation Tab Case @ TECHPULSE 2023

- Speaker: Wilson Wang
- Event: http://techpulse.line.me/

好的推薦功能讓用戶在看 LINE TODAY 時能夠看到關注的內容。本次將以 LINE TODAY 的推薦頁面為案例,帶大家了解 LINE TODAY 是如何透過 A/B Testing ,分析用戶對於不同功能的接受度,進一步了解用戶需求與體驗。

LINE Developers Taiwan

February 21, 2023
Tweet

More Decks by LINE Developers Taiwan

Other Decks in Technology

Transcript

  1. 1

  2. 5 User Research De fi ne Specs & Test Plan

    Development Conduct A / B Test Analyze Results A B A/B Testing in LINE TODAY 1 2 3
  3. User Research Identify our goal 7 Research method Expected results

    • Better understand usage of our app • Explore effectiveness of UI layout and response • A/B testing of different layout • User survey to different test groups • Get user feedback • How will the test results help future planning
  4. Define Specs & Test Plan 8 Derive details from the

    previous step and decide what data to collect and how to collect Define how many percent of people should be included and how to segment the group Decide when to start and how long should we run the test Define Configuration Plan Schedule Test Specification 8
  5. Conduct A/B Test 11 Con fi gure A/B test groups

    settings (optional) A/A test Start A/B test
  6. Analyze Results 13 Quantitative Analysis •Monitor metrics such as Clicks,

    PV, CTR through internal dashboards •Validate and review our hypothesis from previous steps Collect User Metrics Generate Report
  7. Analyze Results 14 Qualitative Analysis •Understand the reason why through

    user’s point of view •Analyze survey results to draw some conclusions about this test