Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Johnny Ross_Rebuild Better with Experimentation_UX Y'all 2022

UX Y'all
September 27, 2022

Johnny Ross_Rebuild Better with Experimentation_UX Y'all 2022

UX Y'all

September 27, 2022
Tweet

More Decks by UX Y'all

Other Decks in Design

Transcript

  1. …Test your theories using experiments! Point B: Desired Reality Point

    A: Ideas Test #1 Test #2 Test #3 Before launching costly solutions… $$$$$
  2. The Hitchhiker’s Guide to Experimentation Discovery Experiments Optimization Experiments How

    to Start Experimenting Gain early product direction about what problems to solve & what to build. Test different designs to inform how best to build the solution. Some simple tips to help you start your first experiment.
  3. Discovery Experiments Goal = Prioritize: Avoid weeks/months of effort on

    solutions that don’t resonate. Types of Tests: • Concept Videos • Interest Signups • “Fake Door” Tests Note: This is to supplement generative research; NOT replace it!!
  4. Discovery example: Prioritize through concept video & signups KPI TEST

    CONTROL DECISION Beta Signups & Perceived Usefulness Concept video of both solutions + Survey/Signup 500 users (restaurants) shown both options View-Only had significantly more signups & prioritized
  5. Discovery Example: Concept Videos x Beta Signups Design: • Presented

    Concept Video to 500 dine-in focused customers (restaurants). • Surveyed for usefulness, feedback, and beta interest. Results: • View-Only Menus had significantly higher signup rates (82 vs. 38 Waitlist signups). • Digital Ordering team works on View-Only Menus. • Waitlist goes to New Venture backlog.
  6. Optimization Experiments Goal = Optimize: (you don’t say!) Tweak designs

    of existing solutions to improve desired metrics. Types of Tests: • A/B Tests (code) • A/B Tests (no or low-code) Note: This is to supplement usability testing; NOT replace it!!
  7. Optimization example: Boosting adoption with low-code A/B test KPI TEST

    CONTROL DECISION Segment creation on product data pages New Create Segment button in a visible place Original button in drop down menu Productionalize new Create Segment button Old Option
  8. Optimization Example: Boost Adoption with Low-Code A/B Test Design: •

    Created “Segment” button using Pendo. • Ran with 50% of new users. Results: • Increased “Segment” adoption by 127%! Previous Insight: • Users create target “segments” to filter product data. • Users (especially new ones) rarely use the dropdown option. A: Old (Only) Option B: New Option
  9. Tips for Getting Started with Experiments Start with the end

    in mind Just get started, scale later Build & share with others Know your impact metrics and develop hypotheses on how to influence them. Focus on getting experiments out and then figure out how to scale a program. Partner with other teams to plan & run experiments. Share your results broadly!
  10. Resources to Get Started Start at the End Matt Wallaert

    Testing Business Ideas Alexander Osterwalder David J. Bland Guide: How-To Create Experiments Johnny Ross | UX Researcher Link
  11. CREDITS: This presentation template was created by Slidesgo, including icons

    by Flaticon, and infographics & images by Freepik Thanks! Reach out on LinkedIn! Johnny Ross UX Research @ Pendo.io https://www.linkedin.com/in/johnny-c-ross/