Reality Check: Gamification 10 Years Later

Slide 1

Slide 1 text

Reality Check: Gamification 10 years later Sebastian Deterding (@dingstweets) University of York, Digital Creativity Labs

Slide 2

Slide 2 text

chapter 1 Time travel

Slide 3

Slide 3 text

early 2000s: fear of exodus into the virtual

Slide 4

Slide 4 text

exemplary: second life

Slide 5

Slide 5 text

late 2000s: fears of the virtual invading

Slide 6

Slide 6 text

“gamification” examplary: gamification

Slide 7

Slide 7 text

gamification The use of game design elements in non-game contexts

Slide 8

Slide 8 text

business

Slide 9

Slide 9 text

health

Slide 10

Slide 10 text

education

Slide 11

Slide 11 text

a heated debate

Slide 12

Slide 12 text

»Gamification presents the best tools humanity has ever invented to create and sustain engagement in people. [...] It’s a proven approach using breakthroughs in design and technology to vastly improve the world as we know it.« – Gabe Zichermann, 2012

Slide 13

Slide 13 text

»Gamification is bullshit.« – Ian Bogost, 2011

Slide 14

Slide 14 text

so 10 years later: who is right?

Slide 15

Slide 15 text

chapter 1 Does it work?

Slide 16

Slide 16 text

My empire of crud

Slide 17

Slide 17 text

first, a quick detour into the replication crisis 2005 theory: Most findings are false 2015+ data: 39% of studies replicate, mean effect sizes half

Slide 18

Slide 18 text

behaviour change, behavioral economics fraught with fraud

Slide 19

Slide 19 text

behaviour change, behavioral economics fraught with fraud

Slide 20

Slide 20 text

technology making us addicted and depressed?

Slide 21

Slide 21 text

a.k.a. “let’s run all the analyses!” technology making us addicted and depressed?

Slide 22

Slide 22 text

a.k.a. “let’s run all the analyses!” Average effect is equal to wearing glasses or eating potatoes technology making us addicted and depressed?

Slide 23

Slide 23 text

games and phones are addicting dopamine pumps?

Slide 24

Slide 24 text

games and phones are addicting dopamine pumps? 0 empirical studies directly testing the relation

Slide 25

Slide 25 text

100k+ papers later, we know next to nothing

Slide 26

Slide 26 text

gamification *can* work

Slide 27

Slide 27 text

gamification *can* work

Slide 28

Slide 28 text

butt #1 it’s not easily replicable

Slide 29

Slide 29 text

e.g. there’s pokémon go! … adding 144bn steps in the us in 1 year alone!

Slide 30

Slide 30 text

5 easy steps to gamification success, pokémon go-style 1. Create a 100+ year old trusted brand for quality family entertainment (Nintendo) 2. Create the world’s largest media franchise with hundreds of millions of fans across the globe (Pokémon) 3. Get Google $, talent, and infrastructure to build your alpha (Ingress) 4. Find a game mechanic that fits hand in glove with the franchise’s core fantasy and runs on the world’s largest install base OSs 5. Make a game and don’t care 1 second about gamification or behaviour change

Slide 31

Slide 31 text

butt #2 we don’t know what works when and how

Slide 32

Slide 32 text

we mostly have just-so stories without data Slack onboarding bot ≈ Early bot levels in WoW Game-inspired onboarding caused Slack’s success How alike are they, really? Was Slack really game-inspired? Are WoW’s bot levels working? Is Slack’s bot working? And IFF all that were true: how generalisable is that?

Slide 33

Slide 33 text

most of this is not empirically tested

Slide 34

Slide 34 text

specific understandings and uses differ – and matter codecademy van Roy, Deterding & Zaman, 2018 khan academy

Slide 35

Slide 35 text

specific contexts differ – and matter 500 steps 8th day without cycling – you really should step it up! What about a 5 minute ride today? C’mon, your friends in California did it! Frank & Engelke, 2001, Reeve 1996

Slide 36

Slide 36 text

specific designs differ – and matter Goveia, Pereira, Karapanos et al., 2016

Slide 37

Slide 37 text

butt #3 gamification ignores sweating the details, a.k.a. (game) design

Slide 38

Slide 38 text

Goveia, Pereira, Karapanos et al., 2016

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

12 months in …

Slide 44

Slide 44 text

A skilled team following standard UX and game design best practices* is more predictive of success than any design element or gamification method. contention *User research, (re)framing, abduction, iteration, playcentric design, polish, …

Slide 45

Slide 45 text

chapter 3 Did it conquer the world?

Slide 46

Slide 46 text

In 2010, gamification is the new black.* * “black” = Opens the door to innovation departments and hearts of bored or desperate middle managers. 2008 2017 2016 2018 2019 big data 2005 web 2.0 machine learning Blockhain, VR/XR chatbots tech ethics, 5G

Slide 47

Slide 47 text

literally disappeared from the map in 2015 … … and today?

Slide 48

Slide 48 text

need a job?

Slide 49

Slide 49 text

need a job?

Slide 50

Slide 50 text

we see some areas that ‘work’ areas with heavy adoption • Self-care/mgmt apps (Headspace) • Private learning (Khan Academy, MindWare) • Corporate health programmes (Virgin Health Miles) • Corporate training (Supercell) • Crowdsourcing (Zooniverse) areas with low/slow adoption • Medical interventions • Formal schooling • Factory floors

Slide 51

Slide 51 text

hypothesis: a matter of market fit incumbents regulation buyers users task structure production adoption factors • Incumbents don’t have buyers, regulators in capture • Little regulation • Buyer is individual consumer (b2c) • User is game literate • Task is unengaging, repetitive, piecemeal, easily assessable, digital opportunity Incumbents (edu publishers, insurers, health tech, …) enter market and buy in products/teams.

Slide 52

Slide 52 text

(not talking about ‘premium’ applied games offerings)

Slide 53

Slide 53 text

game design elements behaviour change techniques ≈ A behaviour change technique is “an active component of an intervention designed to change behaviour ... the smallest component compatible with retaining the postulated active ingredients” (Michie & Johnson, 2013) behaviour change (design) is the new gamification

Slide 54

Slide 54 text

this is coming from health …

Slide 55

Slide 55 text

spreading across design

Slide 56

Slide 56 text

not just old wine in new bottles Engagement design Gamification Behavioural design

Slide 57

Slide 57 text

chapter 4 Summary

Slide 58

Slide 58 text

gamification *can* work

Slide 59

Slide 59 text

but most current research findings are crud 2005 theory: Most findings are false 2015+ data: 39% of studies replicate, mean effect sizes half

Slide 60

Slide 60 text

successes are not necessarily replicable

Slide 61

Slide 61 text

we have mostly stories not data on what works when & why

Slide 62

Slide 62 text

good design process likely matters more than gamification

Slide 63

Slide 63 text

in fact, gamification has disappeared from the map

Slide 64

Slide 64 text

and been replaced by behaviour change design

Slide 65

Slide 65 text

though there are some potential markets incumbents regulation buyers users task structure production

Slide 66

Slide 66 text

So if you want to get into gamification today?

Slide 67

Slide 67 text

if you want to get into gamification today … 1. Don’t trust the science. 2. Read up on and consider reframing your work as behaviour change design. 3. Practice your fundamentals: game and UX design. 4. Consider your market fit: go in-house with an incumbent (educational publisher, health tech, energy provider, …) or startup if you have an uncrowded market niche that is not captured by incumbents, and a partner with deep domain expertise, or consultant in an agency that looks for behaviour change expertise.

Slide 68

Slide 68 text

[email protected] @dingstweets codingconduct.cc thank you.