Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Usability Testing as Design Research

Usability Testing as Design Research

A presentation given to WVU University Relations' Professional Technologist Co-Hort about why web designers should be conducting usability testing, and how to start to do so.

Tim Broadwater

March 21, 2018
Tweet

More Decks by Tim Broadwater

Other Decks in Technology

Transcript

  1. Usability Testing as Design Research

    View full-size slide

  2. http://cache.reelz.com/assets/content/repFrame/58917/multiRolesHtt.jpg
    Some Call
    Me Tim

    View full-size slide

  3. Art/Design
    Graphic Design
    Animation
    Multimedia Design
    Social Media &
    Web Design
    Instructional Design
    & Web Development
    UI/UX Designer
    Front-End
    Development
    2000 2003 2008






    View full-size slide

  4. I thought you were a Professional Technologist?
    UI/UX Designer for WVU Libraries? Say what?!?

    View full-size slide

  5. Lead Mobile Front End Design Strategist
    Chief Mobile GUI Researcher
    Middleweight User Centered Design Developer
    Black Belt Interaction Architect
    Senior IxD Expert
    Principal Front End Design Technologist
    Junior Human Factors Manager
    Chief Information Hero
    Head Content Unicorn
    Heavyweight GUI Extraordinaire
    {
    http://www.aaronweyenberg.com/uxgenerator/
    Hey, Titles Are Titles!

    View full-size slide

  6. +
    UX Designer
    • Conducting usability tests
    • Face-to-face interviews
    • Field research
    • Creating user personas
    • Studying detailed analytics
    • Prototyping Experiences
    UI Designer
    • Interface layout
    • Interface design
    • Visual design
    • Interaction design
    • Establishing visual guidelines
    https://www.usertesting.com/blog/2014/11/10/dont-hire-ui-ux-designer/
    But Seriously...

    View full-size slide

  7. http://alistapart.com/article/usability-testing-demystified
    “Every time a person has a great experience with a website, a web app, a
    gadget, or a service, it’s because a design team made excellent decisions
    about both design and implementation—decisions based on data about how
    people use designs. And how can you get that data? Usability testing.”
    Usability Testing Demystified, A List Apart
    by Dana Chisnell

    View full-size slide

  8. What is Usability Testing?
    Usability Testing is just a type of Design Research...

    View full-size slide

  9. Qualitative Research
    Answers “why” and “how”
    Quantitative Research
    Answers “how much” and
    “how many”
    Peer/Visual Research
    Answers “what are others doing”
    https://www.slideshare.net/secret/b1eaHjusqSn43K

    View full-size slide

  10. It’s Important to Note
    Design Research Isn’t Marketing Research
    https://www.slideshare.net/secret/b1eaHjusqSn43K

    View full-size slide

  11. http://www.usability.gov/how-to-and-tools/methods/usability-testing.html
    ...and the Benefits of Usability Testing Far Outweigh
    Those of Other Design Research Methods:
    • Learn if participants are able to complete specified tasks successfully and
    • Identify how long it takes to complete specified tasks
    • Find out how satisfied participants are with your Web site or other product
    • Identify changes required to improve user performance and satisfaction
    • And analyze the performance to see if it meets your usability objectives

    View full-size slide

  12. Have You Ever Watched A User Try To Use Your Website?
    https://twitter.com/DesignUXUI/status/576432203560685568

    View full-size slide

  13. “Pay Attention To What Users Do, Not What They Say.”
    -Jakob Nielsen, Nielsen Norman Group

    View full-size slide

  14. So, I Work
    Here

    View full-size slide

  15. Library, Downtown Library, HSC Library, Mountainlynx, Databases,
    eJournals, Course Reserves, Research Help”>
    libraries and their collections, host 30 digital collections, offer courses, and
    provide research help to students, faculty, & staff.”>

    View full-size slide

  16. <
    What We Do: Service
    • Websites
    - WVU Libraries
    - West Virginia & Regional History Center
    - Art in the Libraries
    - Electronic Thesis & Dissertations
    - Open Access Textbooks (coming soon)...
    • Digital Collections
    - Storer College Digital Collections
    - Pearl S. Buck Digital Collection
    - Pierpont Cival War Telegrams...
    • WebApplications
    - Databases
    - Room Reservation
    - Collections
    - eJournals...
    • Integrate 3rd Party Apps...

    View full-size slide

  17. <
    • Ask A Librarian
    • Crazy Egg
    • EZProxy Data
    • Google Analytics
    • Office of the University Registrar Data
    • Qualtrics Surveys
    • PageFair
    • QR Code Generator
    • Server Statistics
    • ShareThis
    • SiteImprove
    • Student Services
    • Summon Results
    • Wave
    • WebApp User Data
    ...and more...
    Design Research We Use + and -

    View full-size slide

  18. Design Research Informs Our In-House Usability Testing
    Design
    Research
    What Should
    We Test?

    View full-size slide

  19. We Don’t Rely On Out-House Usability Testing
    (AKA Peek, Five Second Test, UserFeel, etc.)
    Instead, we asked ourselves:
    1. Is the site you want to test relevant to the “random” participant?
    2. Are the participant’s answers reliable / representative?
    http://peek.usertesting.com/result/957593343602
    3. How is task success measured? (Or is there a task?)
    4. If we can’t get a really specific usability tests, or we have to pay a lot of
    money to get great usability test, why are we not doing this ourselves
    or hiring a position?
    https://copyhackers.com/2014/03/get-peek-sites-usability-review/
    I like the
    colors!

    View full-size slide

  20. So Knowing That Our Primary Target Audience Is:
    73%*of WVU Libraries users are undergraduate
    students who are 48% female or 52% male, are
    primarily Caucasian, African American, Hispanic, or
    Asian in ethnicity, are United States citizens or are
    international students, are 21 years in their average
    age, and are 49% from the state of West Virginia or 51%
    from other United States, territories, or countries.
    Target audience numbers are populated from WVU Data, survey polls, Ask A Librarian, and server statistics.

    View full-size slide

  21. ...And That Our Audience Wanted To Use:
    ...We Decided To Run Some Tests
    Target audience numbers are populated from WVU Data, survey polls, Ask A Librarian, and server statistics.

    View full-size slide

  22. Usability Test:
    Examples of User Tasks
    If WVU Libraries doesn’t have a specific book, where
    would you go on the website to request it from another
    library?
    https://testing.lib.wvu.edu/usability2/Task2.wmv
    Reserve a study room at the Downtown Library for a
    group that has 10 people.
    http://community.wvu.edu/~twbroadwater/usability3/
    t24.wmv
    Where would you find a book at the Downtown Campus
    Library with a call number that starts with the letter D?
    https://testing.lib.wvu.edu/usability2/Task7.wmv
    Find the business database “MRI MediamarkReporter.”
    https://testing.lib.wvu.edu/usability2/Task15.wmv
    Find information about borrowing a laptop from the
    Downtown Library and the Evansdale Library.
    https://testing.lib.wvu.edu/usability/Task11.wmv
    At the Downtown Library, what is the maximum number
    of hours for one reservation?
    http://community.wvu.edu/~twbroadwater/usability3/
    t32.wmv

    View full-size slide

  23. http://content.moviery.com/89/003089/13.jpg
    We Can Do Better

    View full-size slide

  24. So, We Made Some Changes...
    And Re-Tested!

    View full-size slide

  25. 0 0.5 1 1.5 2 2.5 3
    Go to the WVU Libraries website.
    Find today’s hours for Eliza’s Coffee Shop.
    Where can you find information about the kinds of cameras you can check…
    Find the database Project Muse.
    What is the loan period for books for undergraduate students?
    Where would you find a book at the Downtown Campus Library with a call…
    See if study room 1028 is available at 5pm tomorrow.
    Find a peer-reviewed article on global warming.
    Find information about borrowing a laptop from the Downtown Library and…
    Where can you find information about how to print in color?
    I need help writing citations in APA.
    Can you find Martin Dunlap’s phone number on the WVU Libraries website?
    Usability Test 2 - Overall Time
    Usability Test 1 - Overall Time
    Crossover Tasks:
    Time Comparisons

    View full-size slide

  26. http://assets3.thrillist.com/v1/image/1483377/size/tmg-facebook_share
    #winning

    View full-size slide

  27. It’s Simple Really!
    This Is What You Can Start With For A Low-Budget:
    Desktop/Laptop Test:
    • 1 Test Administrator
    • 1 User
    • 1 Desktop Computer/Laptop
    • 1 Screen & Audio Recording
    Software
    • 1 Video Camera/Tripod
    ... For A Mobile/Tablet Test, Add:
    • Phone-Mounted/Standing Camera

    View full-size slide

  28. 1. Set up the camera/software to record the user’s
    screen and spoken comments as they complete
    the testing. Start up the laptop/desktop computer,
    launch a web browser, and navigate to your webpage/
    webapp.
    2. Print out a sheet for the test administrator’s use with
    your tasks/questions.
    ...And It Took A Little Preparation

    View full-size slide

  29. http://www.wired.com/wp-content/uploads/2015/10/fast_five01.jpg
    That’s It!

    View full-size slide

  30. Mr Tappy:
    https://vimeo.com/37421265
    TechSmith Morae
    https://www.techsmith.com/morae.html
    This Is What WVU Libraries Uses:
    ...And We’re Building A Usability Lab.

    View full-size slide

  31. So, If You Are Interested In Usability Testing
    Here Are 5 Tips:

    View full-size slide

  32. #1. Choose Usability Software Wisely
    http://kristyhodgson.ca/wp-content/up-
    loads/2014/11/last_crusade_choose_wisely.jpg
    TechSmith Morae, Silverback, Ovo Solo, etc.

    View full-size slide

  33. #2. Administrators Must Be Non-Bias
    http://vignette3.wikia.nocookie.net/harrypotter/images/c/c4/Detention_dolores.jpg/revision/latest?cb=20150824223903
    WVU Office of Research Integrity & Compliance - CITI Training Program: Conflict of Interest in Research course and the Social and Behavioral Research Investigators
    I Must Not
    Tell Lies

    View full-size slide

  34. #3. Record How A Person Uses A Device
    http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobile-devices.php
    Let students use their own devices when possible, Record the thumbs

    View full-size slide

  35. #4. Develop Phrasing Vs. Leading, And Tread Softly
    When developing questions for the test, be sure that the questions are designed
    to test the functionality of the website, rather than the individual users’
    knowledge of the website.
    Some users may become frustrated if they can’t find the information they are
    looking for easily during the test. The test administrator should help put them at
    ease and assure them that the tests are designed to test the website not them.
    https://i.ytimg.com/vi/Gi_S2HGpOJA/maxresdefault.jpg
    #RageQuit

    View full-size slide

  36. #5. Visualize Data... Re-Test, Re-Test, Re-Test
    https://filmkijker.files.wordpress.com/2011/01/aground1.jpg
    The real value of usability testing is to see if previously tested tasks have improved: Error Rate, Overall Task Time, Percentage of Success.
    It’s Groundhog Day!

    View full-size slide

  37. Going Forward Here Are Some Resources:
    UX Degrees
    Master of Professional Studies
    Human-Computer Interaction
    UX Certificate UX Workshops

    View full-size slide

  38. http://www.everydaynodaysoff.com/wp-content/uploads/2013/03/Thumbs-And-Ammo-Pulp-Fiction.jpg
    THANK YOU

    View full-size slide

  39. ...And We’re Hiring A Design Graduate Student:
    MountaineerTRAK: 32953

    View full-size slide