DATA DRIVEN UX DESIGN 

2 AGENDA I.  UX Design Philosophy II.  Metrics to track •  Quantifying performance •  AARRR Funnel Framework III.  Tools and software IV.  Implement Data Driven Process •  Data driven process •  Set target success metrics •  Isolate causality relationships 

3 I. UX DESIGN PHILOSOPHY Data Informed Design Data Driven Design 

4 The difference between User Experience and Usability http://www.uie.com/brainsparks/2007/03/16/the-difference-between-usability-and-user-experience/ Usability answers the question, “Can the user accomplish their goal?” In the case of our camera shopper, from the perspective of the site’s design, she did accomplish the goal, being very satisfied with the result. User experience answers the question, “Did the user have as delightful an experience as possible?” The store portion of the experience canceled out the online portion. 

5 LOCAL MAXIMUM http://52weeksofux.com/post/694598769/the-local-maximum “The local maximum occurs frequently when UX practitioners rely too much on a/b testing or other testing approaches to make improvements” 

6 Role of Analytics in UX Design Single action level Page level Feature level User level Business KPI CTR, Subscribe Time on site, Bounce Task completion Register, Checkout 7 day actives Cost per customer acq. Revenue, ARPU Net promoter score Life-time value 

http://www.slideshare.net/bokardo/metricsdriven-design-4317168/9 Joshua Porter - Metrics Driven Design 

http://www.lukew.com/ff/entry.asp?1336 Julie Zhou - Data & Design at Facebook Decisions at Facebook have to balanced between: ! ü  qualitative data ! ü  quantitative data ! ü  strategy! ü  user! ü  network interests! ü  competition! ü  regulatory bodies! ü  business interests! “Facebook is wary of being too data driven. Data alone might look good but could have a bad impact on brand.” 

When to Use Which User Experience Research Methods 

10 II. METRICS TO TRACK Which Metrics? 

11 II. METRICS TO TRACK What are your KPIs? Key Performance Indicators help teams track performance and set measurable targets. 

12 OBJECTIVES AND KEY RESULTS How Google sets goals: OKRs •  Objectives are ambitious •  Key Results are measurable •  OKRs are transparent companywide •  The “sweet spot” for an OKR grade is .6 – .7 https://www.youtube.com/watch?v=mJB83EZtAjc 

13 AIDA FRAMEWORK A - Attention (Awareness): attract the attention of the customer. I - Interest: raise customer interest by focusing on and demonstrating advantages and benefits (instead of focusing on features, as in traditional advertising). D - Desire: convince customers that they want and desire the product or service and that it will satisfy their needs. A - Action: lead customers towards taking action and/or purchasing https://en.wikipedia.org/wiki/AIDA_(marketing) AIDA 

14 AARRR FUNNEL FRAMEWORK Startup Metrics for Pirates: AARRR! http://500hats.typepad.com/500blogs/2007/09/startup-metrics.html Acquisition Activation Retention Referral Revenue 

15 AARRR FUNNEL FRAMEWORK http://youtu.be/irjgfW0BIrw Dave McClure 

16 AARRR FUNNEL FRAMEWORK http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version 

17 III. TOOLS AND SOFTWARE Software Tutorials 

18 TOOLS AND SOFTWARE Unique Pageviews Traffic sources Devices A/B Testing User segmentation Cohort analysis 

No content

20 GOOGLE ANALYTICS USER FLOW DIAGRAM User Flow Diagram Derived from Pageview Traffic 

No content

22 GOOGLE ANALYTICS CAMPAIGNS TRACKING WITH UTM TAGS http://blog.kissmetrics.com/how-to-use-utm-parameters/ Google UTM tags allow you to add extra information to you the link you create. 

Optimizely http://support.optimizely.com/customer/portal/articles/935621-running-and-interpreting-an-a-a- 

24 COHORT ANALYSIS WITH MIXPANEL http://www.e-nor.com/blog/web-analytics/cohort-analysis-using-cross-platform-data-web-mobile-and-offline User retention over time 

25 IV. IMPLEMENTATION PLAN Data Driven Process 

26 DATA INFORMED DESIGN http://andrewchen.co/2012/05/29/know-the-difference-between-data-informed-and-versus-data-driven/ Metrics are merely a reflection of the product strategy that you have in place 

27 UX Layers http://www.neospot.se/usability-vs-user-experience/ Which layer of the user experience are you measuring? How would you optimize its performance? 

28 DATA DRIVEN DESIGN PROCESS data analysis Run Experiment Deliver value for biz hypotheses Step 1 Select Funnel stage to target Conduct exploratory data analysis Step 2 Formulate hypothesis Design and setup experiment Step 3 Hypothesis validated Launch new design 

29 5 STEPS TO DATA DRIVEN BUSINESS 1.  Create clear, measurable goals 2.  Make an uber-model that breaks down key variables 3.  Collect both quantitative and qualitative data 4.  Generate hypotheses around key variables and variable combinations 5.  Execute test and control methods, and don't confuse correlation with causality! http://andrewchen.co/2008/06/04/5-steps-towards-building-a-metrics-driven-business/ Andrew Chen’s process 

30 STATISTICAL INFERENCE exploratory data analysis https://en.wikipedia.org/wiki/Exploratory_data_analysis 1.  Do you see any trends? 2.  Are there any correlation or causality relationships? 3.  Based on this data, are you able to make any forecasts? http://bit.ly/data_ux_sample 

31 ACTIVITY: Exploratory data analysis for YouTube KPIs ANALYZE THIS SAMPLE DATASET OF YOUTUBE METRICS 10 MINUTES WORK IN PAIRS 10 MINUTES DEBRIEF Answer the follow questions: 1.  Can you detect any trends? 2.  Is there any correlation or causality relationships? 3.  Based on this data, are you able to make any forecasts? OBJECTIVES 

32 ACTIVITY: DESIGN CALL TO ACTION FOR UPWORTHY http://bit.ly/upworthy_may2012 

33 ACTIVITY: Upworthy Page call to action GO TO THIS UPWORTHY PAGE 10 MINUTES OF INDIVIDUAL WORK 5 MINUTE DISCUSS IN GROUPS OF 3 10 MINUTE DEBRIEF 1.  Redesign the location and interaction for the “share” buttons 2.  Add a Facebook Like button 3.  Add a call-to-action to go to another article OBJECTIVES http://www.slideshare.net/Upworthy/how-to-make-that-one-thing-go-viral-just-kidding/38 

34 RESOURCES http://blog.hubspot.com http://blog.kissmetrics.com http://unbounce.com/blog Product Marketing for Pirates: AARRR! Andrew Chen blog Designing for the Next Step 12 Surprise A/B Test Results Metrics-Driven Design How to choose the right UX metrics for your product 

35 Q&A DISCUSSION