Data Driven UX Design

Introducing a process for translating UX strategy to quantifiable metrics at every layer of the elements of the user experience.

Transcript

1. DATA DRIVEN UX DESIGN
2. 2 AGENDA I.  UX Design Philosophy II.  Metrics to track •  Quantifying performance •  AARRR Funnel Framework III.  Tools and software IV.  Implement Data Driven Process •  Data driven process •  Set target success metrics •  Isolate causality relationships
3. 3 I. UX DESIGN PHILOSOPHY Data Informed Design Data Driven Design
4. 4 The difference between User Experience and Usability http://www.uie.com/brainsparks/2007/03/16/the-difference-between-usability-and-user-experience/ Usability answers the question, “Can the user accomplish their goal?” In the case of our camera shopper, from the perspective of the site’s design, she did accomplish the goal, being very satisfied with the result. User experience answers the question, “Did the user have as delightful an experience as possible?” The store portion of the experience canceled out the online portion.
5. 5 LOCAL MAXIMUM http://52weeksofux.com/post/694598769/the-local-maximum “The local maximum occurs frequently when UX practitioners rely too much on a/b testing or other testing approaches to make improvements”
6. 6 Role of Analytics in UX Design Single action level Page level Feature level User level Business KPI CTR, Subscribe Time on site, Bounce Task completion Register, Checkout 7 day actives Cost per customer acq. Revenue, ARPU Net promoter score Life-time value
7. http://www.slideshare.net/bokardo/metricsdriven-design-4317168/9 Joshua Porter - Metrics Driven Design
8. http://www.lukew.com/ff/entry.asp?1336 Julie Zhou - Data & Design at Facebook Decisions at Facebook have to balanced between: ! ü  qualitative data ! ü  quantitative data ! ü  strategy! ü  user! ü  network interests! ü  competition! ü  regulatory bodies! ü  business interests! “Facebook is wary of being too data driven. Data alone might look good but could have a bad impact on brand.”
9. When to Use Which User Experience Research Methods
10. 10 II. METRICS TO TRACK Which Metrics?
11. 11 II. METRICS TO TRACK What are your KPIs? Key Performance Indicators help teams track performance and set measurable targets.
12. 12 OBJECTIVES AND KEY RESULTS How Google sets goals: OKRs •  Objectives are ambitious •  Key Results are measurable •  OKRs are transparent companywide •  The “sweet spot” for an OKR grade is .6 – .7 https://www.youtube.com/watch?v=mJB83EZtAjc
13. 13 AIDA FRAMEWORK A - Attention (Awareness): attract the attention of the customer. I - Interest: raise customer interest by focusing on and demonstrating advantages and benefits (instead of focusing on features, as in traditional advertising). D - Desire: convince customers that they want and desire the product or service and that it will satisfy their needs. A - Action: lead customers towards taking action and/or purchasing https://en.wikipedia.org/wiki/AIDA_(marketing) AIDA
14. 14 AARRR FUNNEL FRAMEWORK Startup Metrics for Pirates: AARRR! http://500hats.typepad.com/500blogs/2007/09/startup-metrics.html Acquisition Activation Retention Referral Revenue
15. 15 AARRR FUNNEL FRAMEWORK http://youtu.be/irjgfW0BIrw Dave McClure
16. 16 AARRR FUNNEL FRAMEWORK http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version
17. 17 III. TOOLS AND SOFTWARE Software Tutorials
18. 18 TOOLS AND SOFTWARE Unique Pageviews Traffic sources Devices A/B Testing User segmentation Cohort analysis
20. 20 GOOGLE ANALYTICS USER FLOW DIAGRAM User Flow Diagram Derived from Pageview Traffic
22. 22 GOOGLE ANALYTICS CAMPAIGNS TRACKING WITH UTM TAGS http://blog.kissmetrics.com/how-to-use-utm-parameters/ Google UTM tags allow you to add extra information to you the link you create.
23. Optimizely http://support.optimizely.com/customer/portal/articles/935621-running-and-interpreting-an-a-a-
24. 24 COHORT ANALYSIS WITH MIXPANEL http://www.e-nor.com/blog/web-analytics/cohort-analysis-using-cross-platform-data-web-mobile-and-offline User retention over time
25. 25 IV. IMPLEMENTATION PLAN Data Driven Process
26. 26 DATA INFORMED DESIGN http://andrewchen.co/2012/05/29/know-the-difference-between-data-informed-and-versus-data-driven/ Metrics are merely a reflection of the product strategy that you have in place
27. 27 UX Layers http://www.neospot.se/usability-vs-user-experience/ Which layer of the user experience are you measuring? How would you optimize its performance?
28. 28 DATA DRIVEN DESIGN PROCESS data analysis Run Experiment Deliver value for biz hypotheses Step 1 Select Funnel stage to target Conduct exploratory data analysis Step 2 Formulate hypothesis Design and setup experiment Step 3 Hypothesis validated Launch new design
29. 29 5 STEPS TO DATA DRIVEN BUSINESS 1.  Create clear, measurable goals 2.  Make an uber-model that breaks down key variables 3.  Collect both quantitative and qualitative data 4.  Generate hypotheses around key variables and variable combinations 5.  Execute test and control methods, and don't confuse correlation with causality! http://andrewchen.co/2008/06/04/5-steps-towards-building-a-metrics-driven-business/ Andrew Chen’s process
30. 30 STATISTICAL INFERENCE exploratory data analysis https://en.wikipedia.org/wiki/Exploratory_data_analysis 1.  Do you see any trends? 2.  Are there any correlation or causality relationships? 3.  Based on this data, are you able to make any forecasts? http://bit.ly/data_ux_sample
31. 31 ACTIVITY: Exploratory data analysis for YouTube KPIs ANALYZE THIS SAMPLE DATASET OF YOUTUBE METRICS 10 MINUTES WORK IN PAIRS 10 MINUTES DEBRIEF Answer the follow questions: 1.  Can you detect any trends? 2.  Is there any correlation or causality relationships? 3.  Based on this data, are you able to make any forecasts? OBJECTIVES
32. 32 ACTIVITY: DESIGN CALL TO ACTION FOR UPWORTHY http://bit.ly/upworthy_may2012
33. 33 ACTIVITY: Upworthy Page call to action GO TO THIS UPWORTHY PAGE 10 MINUTES OF INDIVIDUAL WORK 5 MINUTE DISCUSS IN GROUPS OF 3 10 MINUTE DEBRIEF 1.  Redesign the location and interaction for the “share” buttons 2.  Add a Facebook Like button 3.  Add a call-to-action to go to another article OBJECTIVES http://www.slideshare.net/Upworthy/how-to-make-that-one-thing-go-viral-just-kidding/38
34. 34 RESOURCES http://blog.hubspot.com http://blog.kissmetrics.com http://unbounce.com/blog Product Marketing for Pirates: AARRR! Andrew Chen blog Designing for the Next Step 12 Surprise A/B Test Results Metrics-Driven Design How to choose the right UX metrics for your product
35. 35 Q&A DISCUSSION