Share
Embed
Chasing Engaging Ingredients in Design
Talk at the BIG Bristol Interaction Group on April 3, 2025.
Sebastian Deterding
April 08, 2025
Transcript
90 slides
-
Chasing Engaging Ingredients in Design Prof Sebastian Deterding Dyson School
of Design Engineering Imperial College London @dingstweets @[email protected] -
low patisserieness medium patisserieness high patisserieness N=128 eaters, inherent tastiness
inventory (ITI) Patisserieness positively affects tastiness, p<0.05 -
12.2 Implications for Cooking 12.2 Implications for Cooking … “For
more tasty baked goods, consider a higher degree of patisserieness, such as cupcakes or cakes.” … -
Hi, I’m a pastry chef and read your paper. Can
I ask you a question? Sure, my pleasure. How do I bake a delicious cake? Our results show that people find high patisserieness very tasty. So how do I do that? Have you thought about baking a cake? Yes. I’m already making a cake. How do I make a delicious cake? Well, make it … very caky I guess? -
Science episteme Analyse and abstract real multitudes into decontextualised, universal
building blocks, described in data and diagrams. -
Science episteme Design techne/phronesis Analyse and abstract real multitudes into
decontextualised, universal building blocks, described in data and diagrams. Synthesise and specify a single functioning whole within the local context, crafted from objects and actions. ? -
One solution, many names: Active ingredients in design! Leaderboard sparks
competition Points reward activity Progress bar creates sense of progression Michie et al., 2013 -
Löwgren, 2013; Dalsgaard & Dindler, 2014; Colusso et al., 2017,
2019 Active ingredients are “intermediate knowledge forms” bridging theory and artefact -
Game design elements Nudges Behaviour change techniques Psychostructural characteristics Motivational
affordances Rhetorical tropes Collative variables Persuasive patterns Dark patterns Patterns This has been an intuition in many domains beyond games -
It’s hard. We need formal models of motivational affordances –
and games are a great way to get them Wise old me, ca. 2025 -
What I mean with formal models* Turning verbal theories into
executable formulas * Yes, modelling cognition is itself an intractable problem (van Rooj et al., 2021), but “all models are wrong; some are useful”. Or: “Models are stupid, and we need more of them” (Smaldino, 2017) -
Theorising is a generating and constraining cycle between observing, formalising,
analysing, and testing Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation Heavily inspired by Conceptual analysis Formal analysis Simulation Severe, quantitative hypothesis-testing Phenomenology Ethnography Grounded theory -
#1 Psychology: How does motivation work? #2 Design: What affords
motivation? How do we create systems that reliably motivate? 2 problems, 2 opportunities for formal theory -
#1 Psychology: How does motivation work? The Replication Crisis hits
most research findings … 2005 theory: Most findings are false 2015+ data: 39% of studies replicate, effect sizes half -
Oberauer & Lewandowsky, 2019; Eronen & Bringman, 2021; Fried, 2021;
Muthukrishna & Henrich, 2019 … pointing toward a deeper theory crisis • New open science practices don’t suffice for cumulative progress because … • Current theories are mostly ‘weak’ verbal theories that don’t allow unambiguous derivation of hypotheses or severe testing • Verbal constructs cannot resolve jingle-jangle • Most verbal theories label effects instead of explaining them -
Take Balanced Challenge and Competence/Mastery/Flow An effect widely recognised in
practice and research “Challenge is consistently identified as the most important aspect of good game design. Games should be sufficiently challenging, match the player’s skill level” Sweetser & Wyeth, 2005, 6 -
Challenge as difficulty-skill balance Enjoyment Balanced challenge is a core
affordance in self-determination theory, yet poorly specified »Within SDT, then, optimal challenge means facing demands that most often one can master, rather than ones that are continuously at the leading edge of one’s capabilities. That type of high difficulty challenge should, however, be an intermittent element, in which case it can enhance and heighten intrinsic motivation« Deci & Ryan, 2017, 153 -
Challenge as difficulty-skill balance Enjoyment • How do you quantify
“difficulty” and “skill”? • Linear or log scales or …? • What shape is that curve? • What equation describes the shape of this curve? • Evidence is mixed, operationalisations are “too heterogeneous to aggregate” Fong et al, 2015, Cutting et al, 2023 Balanced challenge is a core affordance in self-determination theory, yet poorly specified -
Cutting et al., 2022 We found this when we tested
the effect of “balanced challenge” on enjoyment and engagement -
No significant difference in enjoyment or engagement, no ‘goldilocks’ optimum
around balanced challenge (n=311) Cutting et al., 2023 -
Players perceive draw as most balanced, enjoy easy, and seek
hard – which fits and contradicts SDT?!? Deterding & Cutting, 2023 Deterding & Cutting, 2023 -
• Effectance: Causing effects (even unintended, must be novel) •
Task performance: Attaining some defined goal • Capacity growth: Increasing quality or range of skills Deterding et al, 2025 Competence as defined in SDT has at least 3 meanings that need not empirically co-occur »Competence refers to feeling effective in one’s interactions with the social environment—that is, experiencing opportunities and support for the exercise, expansion, and expression of one’s capacities and talents. Where individuals are prevented from developing skills, understanding, or mastery, the competence need will be unmet« Deci & Ryan, 2017, 86 -
Deterding et al, 2025 Present self-report scales and experiments mostly
reflect only task performance Intrinsic Motivation Inventory (IMI) 1. I think I am pretty good at this activity. 2. I think I did pretty well at this activity, compared to other students. 3. After working at this activity for a while, I felt pretty competent. 4. I am satisfied with my performance at this task. 5. I was pretty skilled at this activity. 6. This was an activity that I couldn’t do very well. (R) Basic Psychological Need Satisfaction and Frustration Scale (BPNSFS) 1. I feel confident that I can do things well. 2. I feel capable at what I do. 3. I feel competent to achieve my goals. 4. I feel I can successfully complete difficult tasks. Basic Psychological Need Satisfaction Scale (BPNSS) 1. Often, I do not feel very competent. (R) 2. People I know tell me I am good at what I do. 3. I have been able to learn interesting new skills recently. 4. Most days I feel a sense of accomplishment from what I do. 5. In my life I do not get much of a chance to show how capable I am. (R) 6. I often do not feel very capable. (R) -
Deterding et al, 2025 SDT claims success feedback increases competence
and competence generates challenge-seeking: a contradiction -
Verbal theory + standard factor analytic methods can’t resolve illusory
essences, lexical fallacies, jingle-jangle • Immersion ≈ engrossment ≈ flow ≈ absorption ≈ involvement ≈…? • Curiosity ≈ knowledge-seeking ≈ novelty-seeking ≈ interest ≈ …? • Competence ≈ mastery ≈ effectance ≈ agency ≈ flow ≈ …? -
Verbal theories often merely state and label effects instead of
providing mechanisms that explain them • “Engines cause movement” • “Dormative properties cause sleep” • “Grit causes behavioural persistence” • “Competence causes challenge- seeking” • … don’t explain which constituent mechanisms work how to bring this effect about -
If you want to know more, read these preprints osf.io/preprints/psyarxiv/n6x8s_v2
arxiv.org/abs/2502.07423 -
#2 Design: What affords motivation? Leaderboard sparks competition Points reward
activity Progress bar creates sense of progression Michie et al., 2013 -
Kristal & Whilans, 2020; Bird et al., 2021; Löschel et
al., 2020 No, again, the replication crisis rears its head -
Longitudinal (6m) RCT, n=69, with major app Fitocracy Intrinsic motivation
does increase activity But: No significant differences in physical activity, body fat %, intrinsic motivation between gamification and control group Main issue: Non-use/quick disengagement Gummelt, 2017 … and they can not work (or even backfire) -
“A muddle of things we here call ‘design elements X,Y,Z’
affect a muddle of things we here call ‘engagement’, unless when they don’t. More research is needed.” The default contribution -
Goveia, Pereira, Karapanos et al., 2016 #1 Design detail matters:
Patterns remain underspecified -
Codecademy van Roy, Deterding & Zaman, 2019 Khan Academy #2
Different understandings, different uses and effects -
Deterding, 2016 Standard wisdom: (Narrative) choice supports autonomy Game journalist,
faced with seeing game endings under deadline “Sometimes, you just have to play, you have to get further, and then, … playing is indeed work” #3 Context shapes effects -
In short: Motivation is afforded by whole functional person-environment mechanisms
Lewin, 1936; Gibson, 1977; Warren, 1984; Deterding, 2011 B=f(P,E) -
Yet current practice mostly identifies surface patterns, imputing but not
specifying or validating causal mechanisms Deterding, 2015 -
Slack onboarding bot ≈ Early bot levels in WoW Game-inspired
bot caused Slack’s success Yet current practice mostly identifies surface patterns, imputing but not specifying or validating causal mechanisms Deterding, 2015 -
This results in cargo cult design that copies non-functional or
brittle, underspecified surface patterns Deterding, 2011, 2014 -
To get a better analytic and practical grip, we need
affordances as dynamic person-environment relations … Deterding, 2011 Kidd, Piantadosi & Allan, 2012 Warren, 1984 -
… at the grain of stable necessary and sufficient actor-
environment elements that make a functioning mechanism -
You probably guessed it Theory Formal Verbal Empirical testing Analytic
testing Naturalistic observation -
Engaging creative practitioners and creative practice tunes us into professional
vision, concepts, material ‘grain’ Wang & Deterding, under review; Wang & Deterding, in preparation Löwgren, 2013 -
• Task performance • Doing better/worse than expected Observation surfaces
constraints, likely elements, mechanisms Ballou & Deterding, 2023; Petralito et al., 2017 Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation -
Conceptual analysis foregrounds inconsistencies, possible fits Deterding et al., 2022
Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation • If competence = doing well, people will choose games with certain success • Better than expected progress doesn’t fit SDT, but PP -
Formalisation forces us to specify necessary & sufficient components to
render them severely testable Kidd, Piantadosi & Allan, 2012 Warren, 1984 Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation -
Computational intrinsic motivation offers ready formalisms - for simulation and
working RL agents Oudeyer & Kaplan, 2007; Lintunen, Ady, Deterding, & Guckelsberger, 2025 -
They run in games and virtual environments and include and
exceed formalisms in, e.g., active inference/PP Biehl et al., 2018; Guckelsberger et al., 2017 -
Games and virtual environments can be excellent petri dishes for
intrinsically motivated behaviours Theory Formal Verbal Empirical testing Analytic testing Naturalistic observation -
#1 Games bring rich longitudinal telemetry of behaviour and environment
telemetry for model exploration & testing Zendle et al., 2023 – even better when fused with psychometrics -
All game trace data is a GDPR request away –
and the Smart Data Donation Service will make it easy sdds.ac.uk -
#2 Games allow large-scale human/model agent comparison in dynamic, rich,
ecologically valid environments King, 2021; Microsoft Research, 2017 -
#2 Games allow large-scale human/model agent comparison in dynamic, rich,
ecologically valid environments King, 2021; Microsoft Research, 2017 -
E.g., testing surprise formalisms to explain novelty/variety effects on engagement
within and across game levels Modirshanechi, Brea, & Gerstner (2022): A taxonomy of surprise definitions -
#3 Large online (multi-cultural) metastudies allow determining limits of generalisability
– and more Baribault et al., 2018 -
Generative design lets us systematically create and dynamically adapt hypervariate
design spaces Agent behaviour Procedural content generation Industrial design Narrative Architecture Visual art Music -
Hypervariate online studies using generated large n design spaces can
test actual parameter variants and interactions -
#4 Every game AI requires/embodies some implicit or explicit model
of what motivates players Procedural content generation Difficulty adjustment/matchmaking Automated playtesting AI NPCs AI-assisted design tools AI directors -
Kristal & Whilans, 2020; Bird et al., 2021; Löschel et
al., 2020 But current research and practice remain hit and miss -
… because theories and design elements are underspecified Challenge as
difficulty-skill balance Enjoyment -
Better theorising can help overcome this crisis … Theory Formal
Verbal Empirical testing Analytic testing Naturalistic observation -
… employing formal modelling informed by computational intrinsic motivation …
Oudeyer & Kaplan, 2007; Lintunen, Ady, Deterding, & Guckelsberger, 2025 -
… and making use of today’s generative game design tools
and online environments Procedural content generation Difficulty adjustment/matchmaking Automated playtesting AI NPCs AI-assisted design tools AI directors