UXA2022 Day 2; Joel Perlgut - The Narrative Design of Everyday Things: Storytelling and Digital Products

In a complex and uncertain world, businesses have focused on the importance of 'telling a good story' to cut through the noise.

But how can designers use storytelling to inform their day-to-day work? Not just when crafting presentations or communicating to clients, but within the DNA of our digital products.

In this talk, we'll take proven narrative structures and apply them to the new world of websites, apps and emerging technologies.

Transcript

1. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. www.captionslive.com.au | captionslive@outlook.com | 0447 904 255 UX Australia UX Australia 2022 Friday, 26 September 2022 Captioned by: Carmel Downes & Kasey Allen
2. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 89 JOEL PERIGUT: Thank you. STEVE BATY: You're welcome. JOEL PERIGUT: You blew up my spot. I was trying to keep it on the downlow to remove any pressure. Hello everyone. Welcome to my talk. The narrative design of everyday things, storytelling and digital products. So today we will be talking about storytelling but particularly narrative structure. To think about technology. Once we have introduced some of these ideas then we will dig a little deeper and apply this concept of story structure to apps and immersive technology. Finally we will round out some of this thinking with a series of three storytelling principles for designers. We have had a few principles in this conference but you will get a few more from me. So who am I? I work as a UX designer and content strategist at a company owned by Microsoft. I had a background as a screenwriter and film maker. I made a few short films and web series and worked as an assistant in scripted development for film and TV. It was doing this work in writing and reading a lot of screen plays that I became obsessed with this idea around narrative structure. Narrative structure is the idea that underneath all of the things that make a story so compelling, that is the character, the dialogues, the twists and the turns, is an underlying system that the author is using to tell the story. The way I like to think about it is as a model for taking a complex system of human feeling, like a film or a TV show, and breaking it down into smaller, digestible chunks. On this slide you will see what is commonly referred to as a story arc broken down in a composition, climax and resolution, a begin, middle and end. You are probably familiar with something that looks like this from a
3. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 90 storytelling perspective but this is one way of seeing how stories work. That is because writers and theorists have been going back to the time of Aristotle and the Greeks have been mapping out their own system of meaning and emotions. Some you might have heard of like the heroes journey but others like five act or Sparkline you may not be familiar with yet. Story structure has its own industry of gurus who write books and get up to speak at conferences. In the USA researchers at Georgia tech even have AI machines in on the action and this is an algorithm that was asked to watch episodes of Game of Thrones and plot out its own storytelling structures. These storytelling issues and now AI algorithms have done a lot of work to explain how story structure functions in the world of data. That is literature, film and TV. How can we think about how these processes in meaning making and storytelling work in these new technologies. Enter this happy looking guy here, Don Norman, I know he has been mentioned already in these talks. The first guy to hold the title of UX designer at Apple in 2003. He was working in Silicon Valley in the late 1980s holding degrees in electrical engineers and psychology and was helping IBM design one of the first ever laptops, the PC convertible. When it came time to test the product he noticed something strange which is that the team had meticulously worked on the physical build of the product but the users, they did care about how easy it was to open and close and they didn't care about how big the handle was. What they cared about was how frustratingly long it took to boot up and how confusing it was to navigate the interface, which is to say that he had stumbled upon the idea that would become central to UX design, which you are all familiar with, but it is that humans process their experience as a series of feelings, as a story, and it is the designers job to make a product that enables that story. In other words, users don't care about products or services they care about how that technology or product or
4. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 91 service makes them feel. And this became the thesis of his book the design of everyday things which I ripped off for the title of this talk today and where you will find the word story referenced some 30 times. So Norman's story reminds us that design research storytellers and that part of our job is to create a series of feelings in our users, to tell them a story using our designs. How do we go about telling those stories? Today we will be using the old to understand the new. We will be applying these universal concepts around narrative structure, how stories are constructed to map how users experience the modern world of information technology. We are going to start by looking at the humble website, specifically we will look at how we might think about websites and apps through the lens of the Sparkline, a storytelling structure that I think has a lot of utility for designers. When it comes to designing experiences but I think also when it comes to showcasing designs and you know playing that design back to stakeholders. So what is the sparkline? Well Nancy Duerte is a writer known for the film An Inconvenient Truth. When she was writing Al Gore's speech for that film she looked at great speeches and tried to identify if the speeches followed a particular pattern. The core the end it of these speeches was contrasting events and narrative beats between the world that it is, that is our current reality and the world as it could be, a vision of the future. Until finally asking the audience to commit to that new police, that state of change or a call to action. This was the case whether she looked at Steve Jobs' address at the launch of the iPhone or look at Martin Luther King's I have a dream speech. I don't know if you have seen the iPhone launch speech but it is really interesting. Jobs sets up the story by talking about the average mobile handset of the day, a chunky Nokia brick that could call and text but not too much else. Then he provides a vision of the future, a device
5. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 92 that combines an MP3 player, a touch screen and an iPod. An MP3 player, touch screen and mobile. He continues to compare the two devices until finally he lets the audience know that this mythical device, the iPhone is available and it is shipping today, that call to action, that vision of the new bliss. But what about when we're thinking about designs them save, not just talking, not just playing those back. Many, but not all websites and apps are about persuading our users to do something, whether that is enrolling to vote, making a reservation at a restaurant or if you are me buying a dehumidifier on eBay.com.au. What you are looking at here is my eBay home page you can see that I've recently bought a dehumidifier as well as a pair of Reebok sneakers. So let's use the Sparkline to map that journey of persuasion, that story, how eBay.com sucked in yet another returning customer. So our story starts with two months of nonstop rain if Sydney, hence the dehumidifier and my shoe, my current Reeboks get very wet. Sad stuff, the world as it is. But I know that a new world is possible and this's because you might not be able to see the text very clearly here but I spot Zuma wearing some fresh Yeezies and I know that these Reeboks are not doing the trick for me. A vision of the future. Then that rain continues so I browse for speakers online until finally I click buy now on eBay and become the proud owner of some white Reeboks. I encourage you to play around with the story format like the sparkline. When you are thinking about persuasive design, particularly how you may be able to plot your users' feelings in a structure like this between what is and what could be leading to towards a call to action. Not all websites aim to persuade us to do something. Sometimes an experience is only meant to grab our attention for only a few short moments. So I want a talk a little about micro sites, small, targeted online experiences created as part of an online campaign or experiment by a creative technologist. First one is Guns2swords. The
6. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 93 website offers the users the opportunity to mail in a firearm and receive in its place an ornamental sword. Another experience I really enjoyed was Spotify's listening together, accessible to any user with a Spotify account allowing you to play the same song at the exact same time with a total stranger anywhere in the world by browsing along a 3D map. So two very different micro sites but both short, sharp and playful gestures. Now you have to excuse me in advance because I am about to do one of the least funny things you can do is to talk very, very seriously about jokes. The joke is a limited but often deeply complex format for telling stories because of its brevity every word and its placement within a sentence has outside importance. So in this way a joke is very similar to a smaller digital experience or interaction like a micro site. Now Jerry Seinfeld of the hit TV show Seinfeld has a great process when it comes to writing Volkswagen and he details it in a video from 2012 in the New York times. He rewrites the joke often taking up 10 pages for a single 20-word joke structuring them very carefully in a set up, the beginning of the joke, the punch line, the moment where the joke lands and a tag or a series of tags, the moments after the punch line riffing on the joke. I told you it wouldn't be funny talking about jokes. So each step escalates notably and feeds into the next. What can the structure of a joke tell us about how micro sites work? Let's think about our Guns2swords example. What is the set-up. The set-up really is America's obsession with fire arms. There are 1.2 guns in America for every person. Without this fact the website doesn't really make that much sense in the context of Australia, who is using this website. The punchline is the website itself and the service that it offers allowing you to send in that gun and melt it down in a medieval furnace into an ornamental sword. The tag on the website is a strange and cheeky manifesto titled are guns a phallic symbol. So next time you are
7. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 94 working on a micro site or a small digital interaction you can think about how a joke lifelike structure might apply, how you might be able to surprise and delight your user, to excite them, to make them laugh by making use of every available element to tell a short escalating and punchy story. So we have talked about persuasive design in websites and talked about micro sites but what about those websites and apps that operate on a bigger scale. I have clumsily called these mega sites but what I really mean by that is large-scale apps and websites like YouTube, Microsoft Teams or Tinder. What kind of June eye fees these completely different experiences is that all these services are actively trying to keep you on platform and engaged for as long as possible. So whether that's catching yourself in an endless scroll down the feed watching reel after reel or whether it is 1am and you are buzzing on Diet Coke and you are cruising through Wikipedia clicking on article after article. The creator or of the TV shows Community and Rick & Morty has a great model for thinking about this type of cyclical narrative called the power circle. Its power rests in its connection to our deep human needs embedded in our ancestry and biology. To leave the home, steps one and two you need to hunt and to gather and to explore the terrain, steps three, four, five and six go, search, find and take and to finally return home changed by the experience, steps 7 and 8, return and change. In thinking about how the story circle might apply to UX I thought about how certain social media sites incentivise us to return again and again, incentivise users to come back for that next hit of dopa mean. So here a user feeling bored, ventured out into the world by picking up a their phone. And there the user is, scrolling through the news feed until eventually they take a screen shot of a people and they send to it a friend and they laugh and they put down their phone thinking maybe that's the
8. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 95 end of it until finally they get a notification, a change, a trigger for that next hit that kicks off the whole process again. Now the story circle teaches us how powerful but also in this case potentially addictive good storytelling can be in UX. But I'd encourage those of you working on larger applications to think about how you can create a less nefarious version of this story and how you can use all of the different digital touch points available to you in order to create a circular narrative. So finally we will take a look at immersive experiences and by this I mean augmented reality also known as mixed reality, that is computer images projected onto physical space and virtual reality completely fabricated 3D worlds. I get really excited an this stuff but I really not everyone does. A lot of designers aren't getting necessarily hands on in this in their day-to-day but I think even though these technologies are still in their emergent stages they have a lot to teach us. I really think that - yeah, that, you know, there's a lot of hype around them but beyond that I think they are doing something really interesting from a storytelling perspective. Now it was written in the 60s that the medium is the message and then he was talking about television and other theorists who looked at TV focussed on the content, they looked at the advertisements and it is comes and news bulletins and he would two into people's comes and observe them watching TV, when they watched it, how many times and at what hour, where the TV was placed within the home so this is what is really meant when we say the medium is the message and why immersive technology is so exciting from this angle. As technology has evolved we have gone from being able to explore ideas over time in a book to being able to replicate sensory experience in film and TV until finally in our world have been able to integrate it into video games. This creates awe sorts of new possibilities both for designers and for
9. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 96 storytellers. So six years on Pokemon Go remains one of the most popular augmented reality apps on the app store entertaining millions of gamers on the daily. And on its face it is a very simple story. You, a hero, step out into the world, your neighbourhood, to collect as many cute creatures as possible and sometimes you battle them and get experience points and medals. But in practice the real story of Pokemon Go was complicated by millions of players moving around in physical space. Whilst parents were initially received that their kids were finally getting some exercise the game became responsible for countless traffic accidents and controversies and Pokemon and gips gyms were placed over all sorts of places including Auschwitz and war memorials. Only a few brave souls have stuck their map out and tried to map out immersive storytelling. This lady was a theatre director and inspired by the lacklustre job market in the arts she went back to school and studied computer science in the late 90s and got involved in the early VR scene and it was there that she became captivated by the technology and she said that more than web or film or video games, it reminded her of theatre, of the stage. So in her book the end of storytelling she writes when there is no frame we look where we place, act on what we are compelled to respond to and respond to our environment and it in turn responds us to. This is an interesting framing particular leer as we move into a world where these technologies will become more and more popular. When I read that quote I was reminded of David O'Reilly who you might know for the film Hoerr but he makes these cool Instagram augmented reality experiences but particularly his 2017 video game Everything. In Everything you can play as everything. You can play as a cow, you can play as a tree, you can play as a single-cell bacteria, you can play as a continent and you zoom in and out chaining perspective and shifting scope and it gets very psychedelic very quickly but more than
10. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 97 anything else that I have scene the video game everything captures the way we can experience immersive technology. It has helped me to think of the idea of immersive storytelling as a fractal experience. People have long talked about how these patterns in nature can help us conceptualise complex human systems but what it does mean in the context of narrative structure? In a fractal story structure the player or user is just one part of a layered, nonlinear and spatial narrative. In the storytelling the player relinquishes some of their control over the flow of the story and hands that back to the audience and this can be an interesting proposition and that's why in Pokemon Go even though it is not the game developer's intentions sometimes a player will enter a Buddhist temple and start tossing virtual balls into the ether. The first is that different technologies demand different approaches to story, the second is that your design story is co-created with your user and the third is that different stories create different relationships between you and the user. First up, different technologies demand different approaches to story. So now as we've seen we can plot out different forms, different technologies using these different narrative instructors but how exactly do they differ from one another? Well the Sparkline, which we used when talking about persuasive design is linear and finite ending with that final call to action for the user. The story circle when we looked at when mapping out some of those bigger experiences like that social media example is also a linear journey but here it is repeating and there is the possibility and maybe even the intention on the part of the designer to create an experience where the user sticks around for as long as possible. And finally we talked about the fractal which we can use to understand immersive technology and the direction in which digital story telling is going. Here the user's journey is more free form. It is nonlinear and can be repeating or finite.
11. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 98 Number 2, your designs story is co-created with your user. So in the same way that we talk about our designs being co-created with our user so too are your stories but in a slightly different way. We can think about that level of story co-creation depending on the technology, each of them tend towards different spaces. I call this the autership to co-authorship scale. So from autership, the idea of a single creator that has complete control. I can throw my copy of Harry Potter across the room but when I pick it up again the words in the book have still been determined by JK Rowling. As we move forward to co-authorship and websites and video gapes and immersive experience, the interaction the user has with that object, that awareness and choice, bringing that into the equation that creates a new factor that can completely change the type of experience and the type of story we are talking about. There are exceptions to the rule like in the case of something like fan faction where, you know, the user is doing a lot of co-creation and there are immersive experiences that are very much artistic, like a lot of David O'Reilly's work fits into that and they are highly controlled by the artist. But I think it is an interesting conception. Number 3, these different technologies and stories create a different relationship between you, the designer, and your user. So we can map that on a scale from a level of passive engagement where we can use words like an audience to consumer to getting more active words like user or customer or words that capture this idea of an active co-creator or an active partner in the story. We are no longer telling the story, telling them how to feel but rather actively generating that story together in a bit more of a dungeons and Dragons kind of way. I will leave you on this final note, a reminder of something I said earlier that designers are story tellers and to bring your own story and humanity to the story, whether you are a musician who loves playing the
12. Note that this is an unedited transcript of a live event and therefore may contain errors. This transcript is the joint property of CaptionsLIVE and the authorised party responsible for payment and may not be copied or used by any other party without authorisation. Page 99 piano and you can find a way to bring that playfulness or creativity into those designs or a parent whose deep concern poor the future can help inform your work. Your story ads to the rich tapestry of the thing we are building together because I really believe that designs are more than code in a database, they are sense-making tool, they are emotional, they are story, part of the deep interconnected fabric of our world. Thanks. (APPLAUSE) STEVE BATY: Thanks very much, Joel. Do we have - we have time for a question for Joel if anyone in the room has one? Okay. Happy birthday and thank you very much. JOEL PERIGUT: Thanks. (APPLAUSE) STEVE BATY: Our next speaker for the afternoon who will pick up on this writing theme and this storytelling theme, Matt Fenwick. Welcome. Over to you. (APPLAUSE) MATT FENWICK: Cool, good to be with you all. Who speaks for your work when you are not in the room? So when you want to explain the design - sure there are the people you have met one-on-one through the workshops and the stand-ups and the showcases and the road houses and the agile pantheon of things, what about when that person needs to explain it to someone else. So that is where writing comes in and today I'm not going to be talking about content design and content strategy and how content and UX fit together. STEVE BATY: Our next speaker for the afternoon, who will pick up on this writing and story-telling theme, Matt Fenwick. Matt, welcome. Over to