When Personas Talk to You (Presentation at ACM DIS 2025)

When Personas Talk to You (Presentation at ACM DIS 2025)

Persona Team: https://personateam.xyz

Transcript

1. When Personas Talk to You: Evaluating the Evolution of User Personas from Static Profiles to Conversational User Interfaces Ilkka Kaate, Joni Salminen, Soon-Gyo Jung, Trang Thi Thu Xuan, Jinan Y. Azem, João M. Santos, Bernard J. Jansen ACM Designing Interactive Systems (DIS2025), Funchal, Madeira, Portugal. Corresponding author: iokaat@utu.fi
2. Currently: • 10 PhD students • 2 professors • 7 nationalities • See website: https://personateam.xyz Persona team
3. Survey2Personas https://s2p.qcri.org Automatic Persona Generation (APG) https://persona.qcri.org Personas (ca. 1999) ➔ Data-driven personas (ca. 2008) ➔ Interactive persona systems (ca. 2016)
4. From documents to dialogue Personas are traditionally served as profiles, but chat personas developed using LLMs and RAG are becoming more common.
5. The current study • LLM-based persona systems provide a possibility for end users to interact with personas in natural language. • In a randomized controlled experiment, we compare two persona interaction modalities, document and dialogue personas, both generated using AI approaches from a real survey dataset (Pew Research, N = 10k). • Participants (N = 54) were researchers and engineers (44.4% female, avg. age 33 years) and the task was about locating specific information about the persona. • We measured participant perceptions of personas (Persona Perception Scale), their perceptions of the persona system’s usability (System Usability Scale), and task success rate.
7. Findings (statistically significant) Document Personas Dialogue Personas • Harder to use • Less enjoyable experience • More complete • More transparent • Were found less helpful for the task • Higher task success rate • Easier to use • More enjoyable experience • Less complete • Less transparent • Were found more helpful for the task • Lower task success rate
8. Implications • Both persona modalities involve interaction challenges, but of different types; for example: • Document personas caused users to struggle with information overload (“high information personas”) • Chat personas imposed a ‘fourth wall’ effect in that people experienced the interaction as if they were talking to a representative of the persona rather than the persona itself (prompting matters!) • Task success rate is not necessarily connected with better usability in the context of using personas (false confidence?).
9. Thank you! Happy to take questions! • Email: joolsa@utu.fi • Web: • https://personateam.xyz (Persona Team website) • https://jonisalminen.com (personal blog) • https://s2p.qcri.org (Survey2Persona system) 2nd edition coming out 2026! (Slides will be made online at: https://jonisalminen.com/dis2025- personas/) Heads up: