Breaking Down the Distinctions Evaluating AI generated Web Designs - Laura Ruel

This presentation involves an interactive journey where participants will assess the power of AI-generated designs versus human-generated designs. Using eye tracking technology and usability testing, participants will uncover how each design influences the user experience. The participants will consider the clients' goals, including target audience, content strategy, and KPIs, and share their input on the potential success of each prototype. By initially concealing their origin, we ensure unbiased thoughts about the impact of each mock-up. At the end, we reveal the source of each and explore the contrasts between AI-generated and conventional designs, unraveling insightful differences and future challenges. Participants will gain a balanced perspective on the capabilities and implications of integrating AI in UX design.

UX Y'all 2023

Transcript

1. AI vs. human-generated SITE DESIGN An evaluation of websites through usability and eye tracking techniques – and what that means for the future Laura Ruel, Associate Professor, UNC-Chapel Hill October 6, 2023
2. Today’s talk inspired by…. Images generated with Bing AI chat
3. But first…
4. Let’s DO this… • Fire up your laptop, tablet or phone • Go to one of these trending AI site creators HOCOOS.com MIXO.io DURABLE.co 10WEB.io
5. USE some version of this prompt if you’d like… Discover our dental practice in Tauranga, New Zealand, offering comprehensive care for adults and children. Experience our flexible hours and passionate service, ensuring a comfortable dental visit for all ages. Your smile is our priority. TIP: use AI to write your AI prompt. Bing chat utilizes ChatGPT 4. It is free and utilizes the web for reference as well.
6. And WIX.ai is on it too…
7. Impressed?
8. Let’s test…
9. Human-created design
10. Human created design Time Cost
11. AI-generated design Created site with three questions and a name generator using durable.co:
12. AI-generated design • Created logo with this prompt in bing.com: Please create a logo for a business called Smile Street Dental with an orange, yellow and black color pallette • Created menu items with one prompt in Chat GPT 4 Please let me know the five or six most essential menu items for a dental practice website • Replaced 4-5 related images Images from Shutterstock and Unsplash via durable.co • Added 3 testimonials written using bing.com • Copied three bios from Duncan site for team page.
13. AI site design TOTAL TIME: Less than 20 minutes
14. Two testing methods
15. Usability testing Nielsen/Norman group
16. Eye tracking Eye tracking is a powerful tool to evaluate the effectiveness of web designs. By analyzing eye movements, we can determine which areas of a website are most engaging. In addition RTA (retrospective think-aloud protocol) can give insight from user’s narration of their thoughts while watching their eye movements. Image generated by wwpix ai
17. Synopsis
18. Human created site User concerns No search function within site No objective measures | rankings of dentists No representation of minoritized groups Too much text Eye movement First fixation on headline in upper left (same as AI-designed site) Gaze patterns - read more text, viewed fewer images Image generated by wwpix ai
20. Human design: Gaze patterns Notice that site gaze patterns are similar. Text, not images are viewed heavily.
21. AI site User concerns No phone number to contact dental office Too many promotion-type photos No representation of minoritized groups Navigation tough to find Eye movement First fixation on headline in upper left (same as Human-designed site) Gaze patterns as predicted Users spent more time time viewing photos Image generated by wwpix ai
22. AI design - first fixation, gaze pattern Notice that site gaze patterns are similar. Text, not images are viewed first.
23. AI design: Images viewed, yet disliked? AI templates are all about the images. Did AI learn from research? 4 out of 5 viewers of this site said they disliked the large number of photos.
24. Both sites - inclusivity concern “When I see all white people on a site, I start to look at the testimonials and try to see if there is a name that sounds as if it is from a minoritized group. As a member of a minoritized group, It really affects my thoughts about the inclusivity of the site -- and my overall feelings about the site’s awareness of its customer base.” ~ test participant four Unprompted, the lack of Inclusivity only mentioned in comments by only two out of 11 participants.
25. Tauranga, New Zealand ethnic breakdown The ethnic breakdown of people living in Tauranga, New Zealand, according to the 2018 Census, is as follows* • European/Pākehā: 81.7% • Māori: 18.2% • Pacific peoples: 2.9% • Asian: 7.6% • Other ethnicities: 2.2% *Please note that these percentages add up to more than 100% because people may identify with more than one ethnicity Getty images
26. By the numbers…
27. AI design VISUAL APPEAL Human design 4 1= poor | 5 = great 4 1= poor | 5 = great Images generated with Bing AI chat
28. AI design NAVIGATION Human design 4 1= poor | 5 = great 3. 6 1= poor | 5 = great Images generated with Bing AI chat
29. AI design CONTENT QUALITY Human design 4. 2 1= poor | 5 = great 3. 4 1= poor | 5 = great Images generated with Bing AI chat
30. AI design INTERACTIVITY Human design 2. 8 1= poor | 5 = great 3. 4 1= poor | 5 = great Images generated with Bing AI chat
31. AI design CONSISTENCY Human design 4. 7 1= poor | 5 = great 4. 6 1= poor | 5 = great Images generated with Bing AI chat
32. AI design CREDIBILITY Human design 4. 5 1= poor | 5 = great 4. 4 1= poor | 5 = great Images generated with Bing AI chat
33. AI design USER SATISFACTION Human design 3. 5 1= poor | 5 = great 3. 8 1= poor | 5 = great Images generated with Bing AI chat
34. Now what?
36. Future of Digital Design AI-generated web designs are just the beginning. As technology advances, we can expect even more innovative and creative designs. Embrace this future of web design, while remembering what essential human skills are needed. Image generated by wwpix ai
37. Communication Collaboration Empathy Problem solving Humans needed for UX designers should focus on developing their soft skills as well as the AI prowess. SOFT SKILLS
38. Human Effort Understanding Problems User-Centered Design Perception inconsistency Humans needed for Usability testing is an essential part of UX design, and it is unlikely to be replaced by AI in the near future. USABILITY TESTING
39. “First we shape our tools, thereaſter they shape us.” Marshall McLuhan
40. Thanks! Do you have any questions? lruel@unc.edu Presentation: go.unc.edu/UXyall-23