Up a tree: Using tree tests to evaluate your IA

By Dave O'Brien at UX New Zealand 2013

You run usability tests on your designs, but do you test the structure of your site? How do you know it’s better than the old site? Online tree testing lets you test several proposed site structures against each other to see which is most effective. You get quick objective feedback from real users early in the design process. We start with tree-testing concepts and run a live tree test, then dive into the how-to’s of running a good study and the lessons we’ve learned from several high-profile projects. It’s a great chance to add another practical technique to your design toolkit.

Transcript

1. Up a Tree
   Dave O’Brien Nov 2013
   Using tree tests to evaluate your IA
   

   View Slide

2. Easy to find
   UXD easy to use
   IA easy to find
   

   View Slide

3. Bad navigation
   

   View Slide

4. 4
   Bad Labeling
   

   View Slide

5. Bigger = harder
   “Getting to the right page within a
   website or intranet is the inevitable
   prerequisite to getting anything done.”
   - Jakob Nielsen
   

   View Slide

6. How do we make it easier to find stuff?
   How do we
   make it easier
   to find stuff?
   

   View Slide

7. We research our target users
   

   View Slide

8. We design up a bunch of ideas
   • French
   • Italian
   • German
   • Japanese
   – Sushi
   – Teppanyaki
   – Yakitori
   • Chinese
   • Breakfast
   – Hot
   – Cold
   • Lunch
   • Dinner
   • Snacks
   • Beef
   • Poultry
   – Chicken
   – Turkey
   – Duck
   • Pork
   • Vegetarian
   Recipes
   

   View Slide

9. We test our ideas to find out which work
   best
   

   View Slide

10. Revise and re-test…
    

    View Slide

11. …until we get something that works really
    well
    

    View Slide

12. Test early
    Test early
    

    View Slide

13. Today we’re going to look at tree testing
    

    View Slide

14. Doing a real tree test
     Volunteer?
    

    View Slide

15. So, what did we just see?
    

    View Slide

16. Just like a real site
    Like a
    real
    site…
    

    View Slide

17. But not really
    …but not
    really
    

    View Slide

18. So why test this way?
    So, why test
    this way?
    

    View Slide

19. Quick to set up
    

    View Slide

20. It isolates two basic elements
    

    View Slide

21. It makes analysis easy
    

    View Slide

22. Don’t need to design a website first
    

    View Slide

23. Just a spreadsheet
    

    View Slide

24. What do we find out?
    What do we
    find out?
    

    View Slide

25. What we look for
    Did they succeed?
    Did they backtrack?
    How fast did they click?
    Which sections need work?
    

    View Slide

26. Some data, please!
    

    View Slide

27. At a glance
    

    View Slide

28. Low-scoring task
    

    View Slide

29. Low-scoring pie tree
    

    View Slide

30. High-scoring task
    

    View Slide

31. High-scoring pie tree
    

    View Slide

32. It’s not about the tool
    Are there other
    ways to test this?
    

    View Slide

33. Closed card sorting
    

    View Slide

34. Card-based classification evaluation
    Your 9-year-old son asks
    for a new belt with a
    cowboy buckle
    

    View Slide

35. Other ways to test trees
    

    View Slide

36. Best bang for the buck
     Online TT tools = quick and easy
     Fast to get up and running
     Easy for participants, on their own time
     The analytical gruntwork is mostly done for you
     Choice of commercial tools…
    

    View Slide

37. C-Inspector (Steffen Schilb)
    

    View Slide

38. UserZoom (tree-testing module)
    

    View Slide

39. Treejack (Optimal Workshop)
    

    View Slide

40. It’s not about the tool
    It’s not about
    the bike tool
    

    View Slide

41. Lessons learned
    Some lessons
    we’ve learned
    

    View Slide

42. When to test
    When to run
    a tree test?
    

    View Slide

43. Phases in design process
    Research
    Open card sorts
    Baseline tree test
    Design
    Test tree 1, 2
    Revise, retest
    Testing
    Usability test
    

    View Slide

44. Test against an existing tree
    Test
    new vs. old
    

    View Slide

45. ACC old tree
    30%
    

    View Slide

46. ACC new tree
    67%
    

    View Slide

47. test alternative trees
    Test
    new vs. new
    

    View Slide

48. recipes
    • French
    • Italian
    • German
    • Japanese
    – Sushi
    – Teppanyaki
    – Yakitori
    • Chinese
    • Breakfast
    – Hot
    – Cold
    • Lunch
    • Dinner
    • Snacks
    • Beef
    • Poultry
    – Chicken
    – Turkey
    – Duck
    • Pork
    • Vegetarian
    Recipes
    

    View Slide

49. DIA example
    

    View Slide

50. test iteratively
    Lather,
    rinse,
    repeat
    

    View Slide

51. Typical test schedule (design phase)
    Week 1
    Create tree 1
    Create tree 2
    Week 2
    Run test 1
    Run test 2
    Week 3
    Analyse results
    Revise best tree
    Week 4
    Run revised test
    Analyse results
    Round 1
    Round 2
    

    View Slide

52. Revising the best tree
    + =
    

    View Slide

53. Different tasks for different groups
    Different strokes tasks
    for different folks
    

    View Slide

54. Shimano example
    

    View Slide

55. How many?
    How many?
    50
    Users/group
    10
    tasks/user
    

    View Slide

56. Analysis – what should you be looking for?
    What should we
    look for?
    

    View Slide

57. Low success scores
    

    View Slide

58. Where did they go?
    

    View Slide

59. Lots of backtracking
    

    View Slide

60. Evil attractors
    

    View Slide

61. Current limitations of the method
    All sunshine
    and lollipops?
    

    View Slide

62. Only works with traditional tree structures
    

    View Slide

63. WebMD
    

    View Slide

64. Only tests certain factors
    Organisation
    Labeling
    Navigation (e.g. menus, see-also links)
    Content (e.g. supporting text)
    Visual design
    • Good and bad
    

    View Slide

65. How good are your tasks?
    Too many tasks
    Badly worded tasks
    

    View Slide

66. Don’t give away the answer!
    “You need to get reimbursed
    for expenses. Find the form.”
    •Library
    •Expenses/Reimbursements
    •Rules for claiming expenses
    •Expense forms
    •Who to contact
    •Travel
    

    View Slide

67. No perfect tool (yet)
    

    View Slide

68. Not a replacement for usability testing
    

    View Slide

69. What next?
    What next?
    

    View Slide

70. Better visualisation of results
    

    View Slide

71. Testing navigation too
    

    View Slide

72. Summary
     Tree testing = organisation and labeling
     Gives you a rough answer, very early
     Brings some objective data to the discussion
     Reveals IA problems to u-test later
     One more tool for your UX toolkit
    

    View Slide

73. Tree testing – further reading
     Donna Spencer: paper testing
    http://www.boxesandarrows.com/view/card_based_classification_evaluation
     Dave O’Brien: online tree testing
    http://www.boxesandarrows.com/view/tree-testing
     Step Two Designs: tree testing overview
    http://www.steptwo.com.au/papers/kmc_treetesting
     Measuring Usability: tree testing overview
    http://www.measuringusability.com/blog/tree-testing-ia.php
    

    View Slide

74. Let’s talk IA!
    Dave O’Brien
    dave@optimalexperience.com
    

    View Slide

75. Make like a tree…and leave
    

    View Slide