Creating and Evaluating Personas Using Generative AI: A Scoping Review of 81 Articles

As generative AI (GenAI) is increasingly applied in persona development to represent real users, understanding the implications and limitations of this technology is essential for establishing robust practices. This scoping review analyzes how 81 articles (2022-2025) use GenAI techniques for the creation, evaluation, and application of personas. The articles exhibited good level of reproducibility, with 61% of articles sharing resources (personas, code, or datasets). Furthermore, conversational persona interfaces are increasingly provided alongside traditional profiles. However, nearly half (45%) of the articles lack evaluation, and the majority (86%) use only GPT models. In some articles, GenAI use creates a risk of circularity, in which the same GenAI model both generates and evaluates outputs. Our findings also suggest that GenAI seems to reduce the role of human developers in the persona-creation process. To mitigate the associated risks, we propose actionable guidelines for the responsible integration of GenAI into persona development.

Transcript

1. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Creating and Evaluating Personas Using Generative AI A Scoping Review of 81 Articles presented at CHI '26 in Barcelona Danial Aminᵃ | Joni Salminenᵃ | Farhan Ahmedᵃ | Sonja M.H. Tervolaᵇ | Sankalp Sethiᶜ | Bernard J. Jansenᵈ a: University of Vaasa, Finland b: University of Cambridge, Cambridge, United Kingdom c: University of Arizona, Tuscan, Arizona, USA d: Hamad Bin Khalifa University, Doha, Qatar
2. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] What is a persona? Persona is a fictious representation of a group of people, (usually the target group) Persona is a way to present different user data in a humanized way to create empathy Persona is useful in User- Centric Design and helps designers in decision-making HCI Persona is a method to provide personality to a chatbot (NLP) or an LLM (GenAI). Persona can be created from no data or to represent even an individual person. Persona is used to contextualize the output of the LLMs/chatbots to a specific domain NLP Persona is a method to provide personality to a chatbot (NLP) or an LLM (GenAI) Persona can be created from no data or to represent even an individual person Persona is used to contextualize the output of the LLMs/chatbots to a specific domain NLP
3. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Why is this important? Personas help designers empathize with the data and make decisions for the target users. Personas need to be evaluated to ensure that they represent the needs of real users. Amin, Danial, Joni Salminen, Bernard J. Jansen, Ilkka Kaate, and Waleed Akhtar. 2026. Large language models (LLMs) in human-computer interaction: Using LLM-generated personas to model everything from minority views to entire ecosystems. In Artificial Intelligence & Large Language Models: A Scientific Perspective. CRC Press. https://doi.org/10.1201/9781003492252-10
4. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Why is this important? 04 But there is no systematic understanding of how GenAI is used, evaluated, or what risks it introduces Researchers are applying GenAI to persona development without shared standards. 01 Personas are fictional characters representing user groups, central to User Centered Design An AI-generated persona from Survey2Persona system [1] [1] Jung, S.-G., Salminen, J., Aldous, K. K., & Jansen, B. J. (2025). PersonaCraft: Leveraging language models for data-driven persona development. International Journal of Human-Computer Studies, 197, 103445. 03 GenAI now automates much of this process, compressing timelines from weeks to hours 02 Traditionally persona generation follows qualitative data, manual analysis, domain expertise
5. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] We analyzed 81 articles across five databases using a 35-variable coding scheme A PRISMA-guided review of academic literature from 2022 to 2025 1-Identification 885 identified across ACM DL, IEEE, Scopus, Web of Science, and arXiv (2022–2025) 2-Screening 3-Coding 4 coders utilized a 35-variable coding scheme with Fleiss' κ = 0.86 4-Research questions Analyzing usage (RQ1), evaluation (RQ2), and ethical considerations (RQ3) 885 to 81 articles (9%) Methodology
6. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] RQ1: How is GenAI used in personas? 65% used only one LLM without any cross-model comparison 21% presented conversational personas, not profiles 86% only used GPT models [2] Choi, Y., Kang, E. J., Choi, S., Lee, M. K., & Kim, J. (2025). Proxona: Supporting Creators’ Sensemaking and Ideation with LLM-Powered Audience Personas. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, 1–32. Proxona: A system creating an interactive persona that is enabled by GenAI [2]
7. Four innovations of GenAI personas Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] 1.Input Sources Use of unstructured text, visual data, and expert knowledge 2.Process Redesign Automated pipelines; empirical clustering often bypassed 3.Prompt Engineering Two-stage prompts and few-shot example-giving techniques 4.Output Formats Narratives, multimodal, and conversational interfaces RQ1: How is GenAI used in personas?
8. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Evaluation is largely human-driven, but nearly half of articles skip it. RQ2: How are GenAI personas evaluated? Evaluation Types Human-driven: 59% (PPS, expert analysis) Computational: 21% (NLP, stereotype detection) Benchmark-based: 21% (Real-world ground truth) Critical Gaps Only 11.5% address hallucination detection Circularity risk exists as same LLM generates and evaluates
9. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] RQ2: How are GenAI personas evaluated? 14% Research relies on LLMs to evaluate the personas 45% Articles that provide no evaluation framework for their personas 42% Research reporting minimal human involvement in persona evaluation Persona-L: A system including different tasks of personas generation pipeline automated using Generative AI [3] [3] Sun, L., Qin, T., Hu, A., Zhang, J., Lin, S., Chen, J., Ali, M., & Prpa, M. (2025). Persona-L has Entered the Chat: Leveraging LLMs and Ability-based Framework for Personas of People with Complex Needs. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems, CHI ’25, 1–31.
10. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] More than 40% of articles raise no ethical considerations . Out of those that do, here are the themes they raise: RQ3: What ethical considerations are considered? 01 Bias and representation (37%) stereotyping and algorithmic othering 02 Societal harms (22%) unvalidated personas informing decisions 03 Trust and validity (20%) ‘'Potemkin personas'’ lacking user grounding 04 Reduced oversight (21%) lack of community validation
11. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Expanded Access & Sharing 61% of articles share at least one research artefact, representing a big improvement over prior eras of persona research Automated Pipeline Growth 42% of persona generation pipelines are now fully automated, bypassing human review. Reduced Evaluation Rigor Easy generation lowers evaluation pressure, leading to significantly lower rigor. Human-AI collaborative workflows for persona generation [4] [4] Shin, J., Hedderich, M. A., Rey, B. J., Lucero, A., & Oulasvirta, A. (2024). Understanding Human-AI Workflows for Generating Personas. Designing Interactive Systems Conference, 757–781. The Good, the Bad, and the Ugly
12. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Three takeaways for the HCI community GenAI has expanded what personas can do, from static profiles to interactive, conversational personas that designers can question in real time. Easier generation leads to weaker evaluation, and nearly half of articles provide no evaluation framework at all. Fully automated pipelines, with synthetic data, an LLM evaluates and generates the profiles completely, cutting the connection of personas to real users completely.
13. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Four guidelines to address the most persistent gaps in current practice: Guidelines Validate with at least two GenAI systems to surface model-specific biases Document prompts, parameters, and model versions for transparency Validate against real user data or demographic benchmarks Apply human oversight specifically at the evaluation stage
14. Classification INTM : Public [X] Interne [ ] Restreint [ ] Confidentiel [ ] Thanks! Questions? Danial Amin University of Vaasa, Finland danial.amin@uwasa.fi academic.danialamin.com personateam.xyz Read more about Persona Team’s research: Read the paper online. PS. Stay tuned for the updated GenAI persona book in 2027!