Slide 1
Slide 1 text
1
Slide 2
Slide 2 text
Recommendation Tab Case
2
Slide 3
Slide 3 text
Recommendation Tab Case
3
auto play video or not
Slide 4
Slide 4 text
Recommendation Tab Case
4
auto play video or not
thumbnail to the left or right
Slide 5
Slide 5 text
5
User
Research
De
fi
ne Specs
& Test Plan
Development
Conduct A / B Test
Analyze
Results
A B
A/B Testing in
LINE TODAY
1
2
3
Slide 6
Slide 6 text
User Research
Identify our
goal
Expected
results
Research
method
6
Slide 7
Slide 7 text
User Research
Identify
our goal
7
Research
method
Expected
results
• Better understand usage
of our app
• Explore effectiveness of
UI layout and response
• A/B testing of different
layout
• User survey to different
test groups
• Get user feedback
• How will the test results
help future planning
Slide 8
Slide 8 text
Define Specs & Test Plan
8
Derive details from the
previous step and decide
what data to collect and
how to collect
Define how many percent
of people should be
included and how to
segment the group
Decide when to start and
how long should we run
the test
Define
Configuration
Plan
Schedule
Test
Specification
8
Slide 9
Slide 9 text
Development
Internal Setting Flow
Configure
Group Settings
Create A/B Test
Groups
9
Slide 10
Slide 10 text
Development
User Request Flow
10
Get User
Information
Identify User
Group
Respond with
Page A or B
Slide 11
Slide 11 text
Conduct A/B Test
11
Con
fi
gure A/B test groups settings (optional) A/A test Start A/B test
Slide 12
Slide 12 text
Analyze Results
12
Quantitative
Analysis
Qualitative
Analysis
Slide 13
Slide 13 text
Analyze Results
13
Quantitative
Analysis
•Monitor metrics such as Clicks, PV, CTR
through internal dashboards
•Validate and review our hypothesis from
previous steps
Collect User
Metrics
Generate
Report
Slide 14
Slide 14 text
Analyze Results
14
Qualitative
Analysis
•Understand the reason why through
user’s point of view
•Analyze survey results to draw some
conclusions about this test
Slide 15
Slide 15 text
Thank you
15