AI vs. human-generated SITE DESIGN An evaluation of websites through usability and eye tracking techniques – and what that means for the future Laura Ruel, Associate Professor, UNC-Chapel Hill October 6, 2023

Today’s talk inspired by…. Images generated with Bing AI chat

But first…

Let’s DO this… ● Fire up your laptop, tablet or phone ● Go to one of these trending AI site creators HOCOOS.com MIXO.io DURABLE.co 10WEB.io

USE some version of this prompt if you’d like… Discover our dental practice in Tauranga, New Zealand, offering comprehensive care for adults and children. Experience our flexible hours and passionate service, ensuring a comfortable dental visit for all ages. Your smile is our priority. TIP: use AI to write your AI prompt. Bing chat utilizes ChatGPT 4. It is free and utilizes the web for reference as well.

And WIX.ai is on it too…

Impressed?

Let’s test…

Human-created design

Human created design Time Cost

AI-generated design Created site with three questions and a name generator using durable.co:

AI-generated design ● Created logo with this prompt in bing.com: Please create a logo for a business called Smile Street Dental with an orange, yellow and black color pallette ● Created menu items with one prompt in Chat GPT 4 Please let me know the five or six most essential menu items for a dental practice website ● Replaced 4-5 related images Images from Shutterstock and Unsplash via durable.co ● Added 3 testimonials written using bing.com ● Copied three bios from Duncan site for team page.

AI site design TOTAL TIME: Less than 20 minutes

Two testing methods

Usability testing Nielsen/Norman group

Eye tracking Eye tracking is a powerful tool to evaluate the effectiveness of web designs. By analyzing eye movements, we can determine which areas of a website are most engaging. In addition RTA (retrospective think-aloud protocol) can give insight from user’s narration of their thoughts while watching their eye movements. Image generated by wwpix ai

Synopsis

Human created site User concerns No search function within site No objective measures | rankings of dentists No representation of minoritized groups Too much text Eye movement First fixation on headline in upper left (same as AI-designed site) Gaze patterns - read more text, viewed fewer images Image generated by wwpix ai

No content

Human design: Gaze patterns Notice that site gaze patterns are similar. Text, not images are viewed heavily.

AI site User concerns No phone number to contact dental office Too many promotion-type photos No representation of minoritized groups Navigation tough to find Eye movement First fixation on headline in upper left (same as Human-designed site) Gaze patterns as predicted Users spent more time time viewing photos Image generated by wwpix ai

AI design - first fixation, gaze pattern Notice that site gaze patterns are similar. Text, not images are viewed first.

AI design: Images viewed, yet disliked? AI templates are all about the images. Did AI learn from research? 4 out of 5 viewers of this site said they disliked the large number of photos.

Both sites - inclusivity concern “When I see all white people on a site, I start to look at the testimonials and try to see if there is a name that sounds as if it is from a minoritized group. As a member of a minoritized group, It really affects my thoughts about the inclusivity of the site -- and my overall feelings about the site’s awareness of its customer base.” ~ test participant four Unprompted, the lack of Inclusivity only mentioned in comments by only two out of 11 participants.

Tauranga, New Zealand ethnic breakdown The ethnic breakdown of people living in Tauranga, New Zealand, according to the 2018 Census, is as follows* ● European/Pākehā: 81.7% ● Māori: 18.2% ● Pacific peoples: 2.9% ● Asian: 7.6% ● Other ethnicities: 2.2% *Please note that these percentages add up to more than 100% because people may identify with more than one ethnicity Getty images

By the numbers…

AI design VISUAL APPEAL Human design 4 1= poor | 5 = great 4 1= poor | 5 = great Images generated with Bing AI chat

AI design NAVIGATION Human design 4 1= poor | 5 = great 3. 6 1= poor | 5 = great Images generated with Bing AI chat

AI design CONTENT QUALITY Human design 4. 2 1= poor | 5 = great 3. 4 1= poor | 5 = great Images generated with Bing AI chat

AI design INTERACTIVITY Human design 2. 8 1= poor | 5 = great 3. 4 1= poor | 5 = great Images generated with Bing AI chat

AI design CONSISTENCY Human design 4. 7 1= poor | 5 = great 4. 6 1= poor | 5 = great Images generated with Bing AI chat

AI design CREDIBILITY Human design 4. 5 1= poor | 5 = great 4. 4 1= poor | 5 = great Images generated with Bing AI chat

AI design USER SATISFACTION Human design 3. 5 1= poor | 5 = great 3. 8 1= poor | 5 = great Images generated with Bing AI chat

Now what?

No content

Future of Digital Design AI-generated web designs are just the beginning. As technology advances, we can expect even more innovative and creative designs. Embrace this future of web design, while remembering what essential human skills are needed. Image generated by wwpix ai

Communication Collaboration Empathy Problem solving Humans needed for UX designers should focus on developing their soft skills as well as the AI prowess. SOFT SKILLS

Human Effort Understanding Problems User-Centered Design Perception inconsistency Humans needed for Usability testing is an essential part of UX design, and it is unlikely to be replaced by AI in the near future. USABILITY TESTING

“First we shape our tools, thereaſter they shape us.” Marshall McLuhan

Thanks! Do you have any questions? lruel@unc.edu Presentation: go.unc.edu/UXyall-23