Sunday, June 9, 13 This is Failing our Way to Victory, a talk about user research and the impact it can have on game and software design. User research! YEAH! Seriously I’m really grateful to you guys for coming here when you could be lining up at the beer tent outside. I’m impressed by your dedication to knowledge. It’s dedication to knowledge right? You did know there’s a beer tent outside? Ok thank you for staying!
My name is Erica Firment. I work as an Experience Designer at a weird and awesome company called Linden Lab. In my daily life, I attend meetings with tiny robots, cartoon rabbits, and occasionally a large chambered Nautilus. We make Second Life, a virtual world. All of our content was made by our residents. Second life has become one of the largest hubs of user-generated content in the world. We have a robust and unique platform, and support a gigantic world full of hilarious and strange content. I have the challenge of designing interfaces and user experiences for a very broad audience. So that’s me.
YOU. You are the creator of a thing. A webthing, A softwarething. Perhaps a websoftwarething. This thing you have made or are thinking about making... Has an interface. A front end with some graphics buttons and knobs and error messages and, hopefully users that you want to make happy and extract money from. The interface you make might be totally different from the person next to you. This woman might focus on the keyboard commands for his casual game, this guy may be working on the cool animations, and someone else might be trying to think up a way to make you forget your console is busy loading up. My husband works on software for a finance firm. They don’t really care if you have fun at all, as long as they get the math right. The thing all of our interfaces have in common, video games and tax software is that they will be used by humans.
With big thumbs. Monkeys who click everywhere, don’t read a thing. Humans tend to be different from one another. They do the unexpected. They shock you by not following the carefully groomed path you have laid out for them.
you one thing from cold hard experience. The people who use your interface don’t care about your tutorial. They might not even speak your language, or use your character set, or enjoy the same range of motion as you. Which is to say, YOUR USERS ARE GOING TO FAIL.
In ways you absolutely can not anticipate. The good news is, if your content is compelling enough, many users will get up, brush themselves off, and continue. They will madly click, try, get lost, not read, until they fail again. Most times they won’t even notice a fail point. The missed step. The unread instruction. A label you thought was so clear. But, each misstep, every time they become confused, insulted, bored, or frustrated it deducts a bit from their stock of goodwill.
then they are gone. So. YOUR USERS Will FAIL. Always. Somehow. In ways you absolutely can not anticipate. Embrace the User FAIL. It is the Inevitable result of human interaction. Don’t pretend it’s not there. Don’t convince yourself that your users are just like you. Don’t assume that because your thing makes sense, that it won’t cause people to become confused. One you acknowledge that it will happen, how do you find the places where users will fail? How can you identify the collection of insults, injuries, and misunderstandings and try to minimize them? guesses? it’s in the title of the talk...
have limited patience I’m going to try and describe why finding a problem early in the user flow gets more people to the good stuff with their patience intact I’m going to use an awesome chart I made called THE FUNNEL.
start at the top with a ton of users. Innocent, bright eyed people who just want to use the thing you have made. As they travel through the steps required to reach their goal, they will as discussed previously, FAIL. For example, say I forget that users might want to turn off the sound effects. Because of this, some folks get annoyed and leave. A smaller group of users continue along the path, but say I don’t explain a key move very well. People get annoyed and the group gets smaller. At this point, only a subset of the people who WANTED TO PLAY MY GAME actually arrive at their goal. The rest were thrown away due to some dumb mistakes. Let’s make it even worse. Say players have trouble progressing because I didn’t test the difficulty with enough novices. Who is left after a series of problems? Who will endure a bad experience and persevere? There’s only one person.
proud of you. Look at you, here at a conference looking all grown up. Good Job! You have a nametag! But, if you could find and address even one of the Fail points experienced earlier, yer mom wouldn’t be the only one remaining. In fact, the people remaining would be exponentially larger because the more people who DON’T encounter a problem early on, the more are able to SURVIVE long enough to enter my next awesome chart describing… FLOW STATE
many here are familiar w/flow state or have heard the term? Flow is the mental state in which the person is fully immersed in what she is doing by a feeling of energized focus, full involvement, and success in the process of the activity. ESPN calls it Being in The Zone.
as a balance between the time it takes to do something, and the difficulty. A balance between being too hard, and too boring. Somewhere between a Nuclear Submarine and a Fisher Price toy Not impossible but Not insulting either
goal. It’s the point you reach where you feel so engaged that you forget to eat or go to the bathroom. So. How do you find the things in your system that drop users out of flow state? oh yeah. USER RESEARCH again. What do I mean by user research? Several things...
monitor user chatter, listen to the support people, nd top search queries, pretend you are new, championing change, look i’m a frog! Sunday, June 9, 13 user research is an entire discipline, so I’m not going to be able to do it justice here, but I’m going to try and cover: user observation, use your own product, read your own documentation, monitor user chatter, listen to the support people, find top search queries pretend you are new
of the best way to find things that cause users to fail is to Watch them. Spend as much time as you can observing actual users performing actual tasks with your designs. If you do nothing else, v. early in your design process sit and watch three real people encounter your stuff for the first time. While you are watching, Keep your trap shut. Ask them to talk aloud and do what comes naturally. You hear a lot about user observation as a research tech. It can be as complicated as you make it. You can use a fancy setup with microphones and one way glass. You can hire staff and make it part of the design and QA cycle. These are all good things.
13 I’m not selling this book. It’s just good. Krug book, a how-to book that explains exactly how to do your own discount usability testing. Hallway usability - For the rest of us However you do it, do it. Impose on your loved ones. They can’t sue.
SXSW. When you are analyzing the results of a user observation. You are looking for fail. What does it look like? Will you know it when you see it? It can be too many features, bad performance, arbitrary limits You are looking for any thing that you think will cause people to become frustrated and wander off. Unlike the imaginary users in your head who look suspiciously familiar, real people insist on doing what makes sense to them. They bring their own understanding and background to the interaction. Whatever it looks like on an individual basis, you are looking for the collection of problems that makes these users say things like “I suck at this”. People will insult themselves when they probably should be insulting your interface.
of the value you can get from user observation. A year and a half ago we ran a series of user observations involving our registration process. 1. Our intrepid User wants to register for Second Life. 2. Like every user on earth, she doesn’t read the instructions. Instead, she clicks something that leads her to a registration form…in French. 3. user does not speak French. 4. obligingly ATTEMPTS TO FILL OUT THE FORM ANYWAY
13 [wait for recording to play] We did not plan for this poor woman to end up with a French webpage. We did not guess the series of steps that let her there. But, because of a quick, CHEAP user observation, we fixed a problem we didn’t know we had.
fail points, the less traumatic the discovery will be to everyone involved. So test early and often. We recently ran a user test on a new training area we were considering for SL. There was a virtual parrot in the room that was supposed to teach people how to chat. After learning how to sit on chairs in SL, one user went back and tried sitting on the parrot. You can make as many chairs as you want to, but sometimes people are just going to sit on the parrot.
yourself. Watching people try to use your stuff is the BEST way to learn what crazy things people who aren’t you will do to your clever design. Another option is to pretend you are new. Delete your settings. Start from the Start. Go through the experiences as a new user. See what frustrates you, or what you might not have expected. It helps if you do this with someone standing next to you. This ensures you won’t take shortcuts. I suggest everyone in a company do this, regularly, even the CEO or the office managers. Following the new user experience keeps you seeing your interface with fresh eyes. And it reminds everyone to think of the user’s experience and not their own. An internal motto: Nobody can afford to be so precious that they won’t walk a mile in the users shoes.
that FAQ sitting around on your website? Users shouldn't have to frequently ask those Questions. If the problem is common enough to have documentation around it, THAT’S your software TELLING itself something Does somebody blog about your software? Did someone write a book? READ IT.
13 Consultants are great, but there’s a lot of information out there that is just sitting around waiting to be gathered. Don’t hire someone to do a Google search for you Find out what happens when you Google “YOUR THING HERE + problem” or “confusing” or “help” or “bug” Set up Google Alerts - when someone talks about their experience online, you can read it and learn. I set up an RSS feed to watch several SL-related Flickr streams, including one called “Second Life Interface” where folks sometimes post screenshots of strange/funny things they’ve seen.
or bug RSS related Flickr streams What are your top search terms? Talk to Support Sunday, June 9, 13 I’ve found more bugs by listening to users grumble to each other than I could have by only listening to people motivated enough to contact us. Also, keep in mind that the things users bring to your attention are often not the top problems that everyone faces. It’s easy to think up a new preference that will address an edge case, but it isn’t easy to step back and see the big picture. You can get in trouble by being too reactive, and not addressing the end-to-end user experience. Big picture stuff can’t be easily summarized in a bug ticket or a forum post. Speaking of which, Find the top search terms in your website or forums - tailor the results accordingly Talk to the support people - who works the phones? who answers the angry emails? They have your user data.
June 9, 13 Big changes take big time, but small changes can have a huge impact Numbers are very alluring in an engineering organization. In the absence of data, engineers and designers are forced to guess how people will act, and to treat every problem as a crisis. If you have the numbers to prove that a feature isn’t being used, or is causing more crashes than joy, you will have just won yourself some friends. Here are some techniques for getting your message to the people who need it...
a formal Bug triage process for design problems. Give them a place to live in your tracking system, and give them an owner. Design bugs are different from the “it doesn’t work” kind, and if you don’t acknowledge that, you will have Random engineers making design decisions because they were the one assigned to “Fix the bug”. Bug triage is also a great way to train up a new designer. Nothing makes you more familiar with an application than spending an hour a day listening to the most extreme edge cases and determining expected behavior.
BEHOLD AS I DEMONSTRATE THE POWER OF A COMPELLING NAME In SL, if you had a certain video card with certain settings you would encounter a bug we called “Jay Leno Jaw”. Everyone wanted to work on Jay Leno Jaw. Even saying it was funny.
When I ran our design bug triage a year ago I made a bunch of pictures that summarized the reason something was important. Interestingly, the issues with pictures assigned seemed to get fixed more quickly, because their value had been made clear in a visual and funny way. THE COMPELLINGLY NAMED PROBLEM GETS SOLVED FIRST
THERE WILL BE FAILURE, ALLOW YOURSELF TO SEE IT (DON’T ASSUME EVERYONE IS YOU) FIGHT TO KEEP YOUR USER IN FLOW STATE OBSERVE USERS FAILING AND DOCUMENT EACH PROBLEM USE YOUR OWN SOFTWARE READ YOUR OWN DOCUMENTATION READ TO WHAT USERS ARE SAYING TO EACH OTHER CHAMPION CHANGE WITH THE DATA YOU GATHER TRIAGE DESIGN PROBLEMS GIVE THEM A SOLID NAME HAPPY USERS = VICTORY! THANK GOOPYMART FOR CARTOONS - ON FLICKR
librarianavenger
helemarudesu
narizuka
hirokazu_oda
ncdc
yasuhiroyokota
kenijam
transit_kix
shingo2000
hopin
putchomsmartbank
tomo1215
xxxkinu
smashingmag
hatefulcrawdad
jnunemaker
iamctodd
cromwellryan
mongodb
destraynor
tanoku
mojombo
jakevdp
jeffersonlam
zakiyama
