Michelle Pickrell & Ruth Ellison

Transcript

1. 1 www.captionslive.com.au | captionslive@outlook.com | 0425 904 255 UX AUSTRALIA Design Research 2020 Day 2 Friday, 20 March 2020 Captioned by: Gail Kearney & Rebekah Goulevitch
2. 2 RUTH ELLISON: Hello, everyone, nice to see your research family. Michelle and I would like to start with the acknowledgement of the land on which we are zooming in from Ngunawal people in Canberra. And Michelle is muted. MICHELLE PICKRELL: Sorry! RUTH ELLISON: We wish to acknowledge and respect the culture it makes to our cities. We acknowledge and welcome any other abl and Torres Strait Island people who may be listening to our presentation today. OK, so, for those we haven't met before, hi, my name is Ruth Ellison. I have been working in design and research fields for 18 years in the government and private sector and I am passionate about growing good design cultures. MICHELLE PICKRELL: I'm Michelle, a research candidate and working in UX for eight years and doing my PhD as long as that. It focuses on interactive equipment for stroke rehabilitation. RUTH ELLISON: My Internet is unstable because of the NBN. I never thought we would get to the day with these types of headlines in our media. I went into my local supermarket, Woollies, to find and get baby wipes for my little one and paper towels and this is what I saw. I went back again in another week and again yesterday evening and it was exactly the same, like this. So, what is actually going on over here? So, these are cognitive biases in action. For those that are listening for the first time, a cognitive bias is a mental short cut, a psychological tendency that causes us to use strategies to make judgments and decisions because there is so much information out there. So, these rules are often really useful in helping us deep with complexity and ambiguity but it also leads to faulty judgments. MICHELLE PICKRELL: You're probably all aware of this, but thinking fast and slow by Daniel Kahneman. If you haven't, please give it a read, it is tough and super dense but you will learn how the breaks works. He proposes the brain has two systems, system one and system two. System one creates automatic responses to things we have created before in the world. It pays no attention to what it does not know. System two is the deliberate and logical part of the mind that deals with more difficult problems, by nature is slow and where possible will leave answering questions to system one. Now, using it is very effortful. However, we do tend to identify with system two as being who we are, as being our identity of someone who is intelligent and deep thinking. However, most of the time system one is in control. Where is where our cognitive biases come in to play. RUTH ELLISON: So back to the great toilet paper apocalypse in action. We were unsure how to behave as individuals we look around us for assurance. I looked around going wow! Toilet paper is going? This week it was mince at the shops and Michelle? MICHELLE PICKRELL: Yes, eggs in Sydney. RUTH ELLISON: Social proof to get an idea how to behave from others. The second is FOMO, the fear of missing out. We fear we might miss out
3. 3 on a social occasion, investment or experience or event and the final one is scarcity, we place a higher value on scarce objects. Now there is hardly anything so now it is as valuable as toilet paper. Not quite but close. What does it all mean? Cognitive biases are lurking everywhere because we are human. We want research to provide reliable input and help stakeholders assess what we are working on. But we are human and we and our stakeholders are susceptible to many cognitive biases at any stage of our projects. They are unavoidable. Being good researchers but stating our inherent biases of ourselves and our stakeholders and how to minimise the effects. When communicating research findings it can be detrimental to a project and can provide false confidence about decisions. You would have seen this the last few days, Buster Benson, 180 different biases. I love the simplification of this. But what he covers are the four different ways that biases can help us address the four key problems. First one is there is too much information out there so we filter. We pick out the bits and pieces of information we feel might be most useful to us. The second problem our brains are trying to address is the world's very confusing and we only see bits and pieces of it. We need to make some sense of it to survive. So lack of meaning is confusing. We fill in the gaps with stuff we think we already know. The third problem trying to address is need to act fast unless we lose chances and we jump to conclusions. Finally, there is too much information out there, so when we try to run the important bits we have to make constant tradeoffs in what we remember and what we forget. And the decisions inform our mental models of the world. MICHELLE PICKRELL: This afternoon we'll explore six main themes of biases and share our experiences, identifying the biases and give tips to manage them when you identify them in stakeholders. The first is confirmation bias, which is considered one of the most dangerous. The tendency to search for or interpret information in a way that confirms our beliefs. Common factors such as internal politics, personal goals or simply a lack of knowledge can turn into a cherry picking exercise I researchers or stakeholders may consider some results and ignore others. This is common when we're googling to find information. The fact that we have an idea of the information we are wanting to find means we are likely to structure our search terms around that. So, for example, if I Googled will Trump will the next election? I'm most likely to see the web sites that confirm this idea and a less general overview of the situation at hand. Now, it's common at researchers that we form a hypothesis and test it to be true rather than looking more broadly to test to see if it is false. As humans we are wanting to confirm what we believe is true. So testing is going to enable our brains to grip on to the belief. As James Clear explains we look for facts which confirm our beliefs not viewing the overall set of facts. An example of this in a stakeholder context. I was once presenting research findings back to a group of stakeholders where they passionate about the product and its success. There were 10 interviews with people who used the existing website and the feedback wasn't reflecting well on the product. I had framed the findings in a way that would soften the blow a bit by making sure following every negative finding there was a positive finding. In the case of the findings, the negative findings were more meaty and shallower were shallower. The negative came down like a tonne of bricks, the research methods were questioned? How can we be confident they are consistent outside the
4. 4 research group? Alternatively, the positive were celebrated with no questioning. At the end of the discussion it was around the positive findings. I slowly started to reintroduce the negative to have a discussion how we might include them in the design going forward. Now, I am sure everyone's probably come across this situation in the past. In varying ways. Where the stakeholders have questioned the research and its methods because the results don't confirm their view on the world. So how might be deal with this? Firstly, we list out our assumptions with our stakeholders. Where I currently work we have a research canvas which includes working as a team to list out the team's assumptions of what they believe to be true. By having it out on the table we keep a note of them. It is not something you do just at the beginning of a project, it is something, a bit of a living list, adding to it before every round of research. Now, it's also important to take an opposing view. This leaves you open to new evidence. So try and structure your research questions and your hypothesis from the opposing direction sometimes, or to try and prove your hypothesis false instead. Sometimes it can be good to try and disprove rather than prove those hypothesis. NATALIE ROWLAND: OK. The next group of biases, forgive me, I'm not a French speaker, is about perspective. This one is called I think deformation professionelle. One that is very closely related is the ego centric biases which rely too heavily on your own perspective. Ideas and experiences are recalled when they match your own causing an ego centric outlook. These biases give people ability to empathise with others. It is important when dealing with stakeholders who cannot empathise as clearly with users. Just due to the nature of what they do, their specialty. We spend the majority of time seeing things from our own perspective. We should adjust our perspectives to see things through other people's eyes and anchor this to our own and we often fail to adjust our visual view point to assess how others may think or feel about a situation. I like the image with the six and nine showing their perspectives. Quite a few years ago, I was working with an environmental organisation looking how we shift behaviours of mind sets. Our research at the time indicated for some people getting short, sharp messages through social media was a way to pique interest in a set of tips. We were presenting our findings back to stakeholder with one saying very dismissive, "That would never work" when I asked why, she said, "I would never use social media for that type of information". There you go. In that situation, how would you deal with it? The first way is to consider alternate view points. And helping to ask whys. See things from other's points of view and a generalised perspective of our users. We want to take it back to user needs. Having conversations around the actual user needs rather than being an opinion-based piece. We want our stakeholders to come with us on research sessions where possible. Often when I suggest this, I get stakeholders pushing back, that's not my job, it is why we hire researchers to do stuff. But when they do, the change of mind set is amazing. It went from talking about their perspective and their views to here is what our users are dealing with or how they're feeling. A useful tip. The fourth one is to encourage our stakeholders to use self-distancing languages and not thinking about themselves our ourselves, what should I do, it will help think about the second person pronoun, for example, what should you do or John do or Michelle do?
5. 5 MICHELLE PICKRELL: So, we saw that there was a lot of similarity between the three biases we thought were relevant in our stakeholders. They are the irrational escalation, the IKEA effect that was mentioned earlier this morning. Firstly, irrational escalation is investing increased investment in a decision, based on prior investment despite new evidence the decision was probably wrong. Similarly, the sum cost fallacy is where a behaviour continues as a result of previously invested resources, for example, time, money, effort. And finally, the IKEA effect, the idea that consumers place a disproportionately high value on products they partially created, the name deriving from the furniture retailer, IKEA. Where we align with our stakeholders, if stakeholders had been part of the process of making something they are more likely to feel that attachment to it. The sum cost fallacy is most found in gambling, when someone is at a loss they keep investing with the justification they are closer to winning. Now, Kahneman and Tversky show it in this diagram. A is starting from scratch and B is starting from the point they are at. As you can see, there is the journey to get from B to A before you start making those games and adding value. So, to share a bit of a story, I was working on a large-scale project on a very complex platform, to use this platform you needed to start on the company website and then move through two other platforms before returning back to the company website. So, it's a little bit like what happens when you are buying a product using pay pal on eBay where you purchase the product on eBay and then get sent to PayPal to make the payment before coming back to eBay, however, imagine two layers within that. It was really complex. Now we came in started research at the beta phase of the project. In the first week we conducted feasibility testing and of the six participants not one could get through the process end to end unassisted. We told the team and they said, "We can make the little changes that will help with the usability but we can't change the over all model. We have been working on that forever". We made small design changes which we hoped would fix the usability issues of people not getting from end to end and then took it out for usability testing again and again people couldn't get through the task end to end. They became so confused. So we did usability testing week after week, finding the same thing over and over. Each time the research buries part of the process but during all the weeks of research, not one participant was able to get through. We did a cross analysis to ensure the product manager could see the massive impact on the usability of the product in the case that we pushed the product live. The product manager told us that we're under such tight deadline and he spent too much time and money to not deliver what we have. This is a clear representation of sum cost fallacy where delivering a product came over the needs and usability. How might we support our stakeholders around this? We need to encourage our stakeholders to take a long-term view. It may include offering alternatives or giving stakeholders an understanding of the time and cost something might take to fix, which may be more than starting from scratch or pivoting. We want to identify the value from learning from mistakes. We have all heard the phrase, to fail fast. We need to encourage our stakeholders that failing fast and learning is such an important factor in successful design and not something to be scared of. We also need to provide a way to pivot to a more useful pathway. So help to support our stakeholders to see a pathway away from the current situation. Over to you
6. 6 RUTH ELLISON: This is one of my favourite biases for stakeholders. I am not sure how many hit the HiPPO, the highest individually paid person's opinion. Preferences whether something is worth sharing or a programme or service is worth implementing. The first person coined the term. When a hippo is in the room and a decision needs to be made but there is no data or analysis to determine the cause one way or the other, the group in the room refer to the judgment of the hippo. The most experienced and powerful person in the room, generally the senior executive. Once they voiced an opinion, opposing views are shut down and in some cases, particularly organisations where people feel psychologically unsafe, they feel fear speaking out even if they disagree with it. We have identified two hippo scenarios, the first where the hippo makes the decision and the second is the hippo by proxy. The person who works closely with the HiPPO. You are unable to talk directly and negotiate with them. A school of management found they fail more often compared to projects led by junior managers which were more likely to be successful and the benefit is the juniors critique plans and others help build a stronger approach to the project. Employees felt they couldn't give critique to a hippo. The problems when the person wants it done, there is no data or research to back it up. It is what I love, you give the information for option A and hippo wants option B. It wastes time and lowers confidence of the team and stakeholders but it could be the wrong action to take. Back in my career, I was working on a new digital service with many stakeholders across a large organisation. I designed a lot of design research. We had to deal with the feedback of users and dealt with expectation of management and a lot of stakeholder management. It felt like everyone had an opinion. Often we felt like we were dealing with many levels of hippos in every meeting and it was a big committee exercise. When we got to the end, it first like a huge Herculean effort to get ready to sign off from the executive for this project. We did the outline of the findings. When we presented the findings to the hippo, we discovered the hippo had a lot of opinion on the design, the colour, he didn't like the shade of green, the order of the screen presented, had to change it many times previously due to stakeholders and feedback from participants and he had additional functionality he wanted to see even though the research didn't show it was a user need. It was back to the drawing board and we had to factor in the hippo's opinions. I wish I knew then what I knew how on how to deal with it. How to deal with it, we need to understand our hippo's motivations and goals. If we can understand what drives them or what is their performance metrics we can have insights and evidence that speaks to that. Have the data that speaks to the concern of the highest paid person it will just help to drive decisions. The second is to take the hippo on the journey. A hippo needs to attend as many showcases and where possible, use user's voices directly so the hippo can hear straight from them rather than think it is our opinion of the research analysis. Thirdly, where we can, we want to build consensus, so before heading to a meeting with the hippo, we want to try to inform the larger group about the facts, or get consensus of stakeholders, the more prepared and informed they are, and the larger the groups for the decision, the more likely they will be able to challenge the highest paid person in the room, particularly if that person to challenge has a bit of kudos with the hippo. So that way, challenging the hippo won't seem as daunting if there is evidence to support the position. And finally, use a proxy to communicate to the hippo. And what happens
7. 7 if you are the hippo in the room? It could be lucky, once you work your way up in the career you could be the hippo. You want to create a culture to seek multiple opinions and dissenting opinions and ask someone to play the devil's advocate prior to an important decision being made. The hippo's job is to help embrace other people's perspective. MICHELLE PICKRELL: Moving on to the curse of knowledge. The curse of knowledge is a cognitive bias that occurs when an individual communicating with other individuals unknowingly assumes that the others have the background to understand. Now, this is the failure to understand how things affect people who don't have the knowledge we have about a particular subject. There is such a thing as being too close to an issue. And this is why independent research is necessary to identify problem areas for users of a product or service. We see this all the time in user research, particularly in web sites or apps developed by engineers who are unable to put themselves in the user's shoes. We need to encourage stakeholders to understand the large amounts of corporate and domain knowledge they have about a problem. Knowledge their users do not have. To share a story, we have done the right thing and invited our stakeholders along to observe usability testing sessions through a one-way mirror. One of my colleagues was doing the research and I was sitting in the observation room with a group of stakeholders. The screen on the test computer was in front of us - sorry, another screen, the screen on the test computer was in front of us so we could observe the way the participant was interacting with the website. It was 20 minutes in to the first session of the day, we learned about the participant, their background and two tasks in. The participant started to find the task particularly difficult. He wasn't able to navigate through the system due to some of the terminology being used. The stakeholders that was sitting next to me whispered under their breath, the button is just there, how can they see it. I looked over, not sure if I heard them correctly but didn't want to cause a fuss and looked back and observed the screen and wrote up my notes. A few minutes later, the stakeholder said, "This guy is just dumb. How can he not finds the button?". I stopped and explained to the stakeholder by doing research like this we are able to understand which parts of a website are easy to use and which need to be redesigned and if one was having a problem, then others will probably during the day. I hope you have never been put in that position but if you are, it is important to remind stakeholders research is about learning. We want to understand perspectives and ways of understanding people, that is why we do research in the first place. We want them to keep an open mind. We need to ask our to stakeholders to leave their ego and judgment at the door and observe what is going on in the session. Finally, we want them to practice detachment from the product which we know can be really hard when they have sunk a lot of time, effort and money into it. Again, we're all here to learn and that's the purpose of the research. RUTH ELLISON: Thanks. And the final one is hyperbolic discounting, which is choosing the long term gain over short term reward. We discount the value of waiting. Back in late 1960s, a Stanford university professor presented preschoolers with a plate of marshmallows and were told they could have one now or two later. It was to test the levels of instant gratification in children. So the hyperbolic waiting, one now or
8. 8 two later. This bias might show up when taking stakeholders through insights and making a decision. We get to the point of the decision where we realise we never will have enough information to make the complex decision. We end up choosing, as a result, the option that allows us to move along and kick the ball along and present something. It gives us the shiny thing now versus give us something solid later. This focus on quick wins or instant gratification means we can miss out on the chance that creates something that is amazing for users. I was working on a four-month phase of a project, doing research every week. As you can imagine when you conduct research every week, it means you gather a lot of data. Each week we planned, conducted, analysed and gathered research. The stories and around the findings and concepts we were exploring with each participant. Each team would listen to the share back and be interested in the stories. From the share back, the product manager would implement the team to implement the quick simple wins, they rushed off and made changes, and it was like making a button choice more obvious or replace a word with an image. When it came for the next prototype, none of the larger issues which came out in the research were being addressed. Issues such as trust and security, which were the deeper insights that impacted the overall product strategy and vision and direction. When we spoke with the product manager they commented the findings would impact the project too much. This is an example of hyper discounting where we need quick wins for progress versus changes that need to be made. How do we deal with this kind of situation? We have to work out how to balance quick wins versus long term rewards. There are questions to ask your stakeholders, we seem to be anchored in a particular short term quick win, how would we do a longer term win instead. What is stopping it? What are the barriers? We can understand that and help us to work out strategies to move to a longer term thinking. And break down the longer term rewards into smaller chunks. Big lofty goals yield big rewards but they take a long time to achieve. Smaller chunks mean you get it quicker and you get a gratification each week or each sprint. By doing something smaller it means the return is guaranteed and immediate. And objectively, valuing options. We write down each objective or option or idea and get everyone to evaluate it on a case-by-case basis. Atalassian has a concept canvas and you can do it on Google. The final one is about sharing the risk of not incorporating your longer term findings in the overall success of the product. We want our stakeholders to understand how the preferred option aligns with the strategic goals they are aiming for. Will it help them meet it? If not, what will? What they are uncertain about and remove the uncertainty. The more we can align to a strategic goal, it more it helps us come and deal with this particular bias. MICHELLE PICKRELL: So, what are the main things to remember from all of those different biases and tips? Firstly, we want to take our stakeholders on the journey. To start with, we want to explain the value of user research to them. We want to plan with them, so make sure they're involved in the planning sessions as well as involved in the research. Inviting them to be an observer in the user research. We finally want to ensure they come to share backs and showcases so they are understanding the analysed findings and insights as well. We want to understand our stakeholders' point of view, their goals and motivation, what is making them tick on this project. Why are they working on this
9. 9 project, what do they want to get out of it? We want to offer that strategic advice aligned with user needs. Don't just give the what we found in research, also offer the how it's going to impact the project. Encourage that longer-term thinking, that strategic thinking in your stakeholders. And also, identify the risk of not implementing those longer term findings on the overall vision and strategy of the product. We also want to teach our stakeholders about research. Things about disproving rather than proving, list the assumptions at the beginning of the project and ensure there is a living list of assumptions throughout the project for the whole team. And to be neutral but have that inquisitive mind set when we go out and do user research. One important thing to remember, our biases are part of what makes us human. It is not that they are a bad thing. They are just something we need to work with. In all different areas of our life. So, on that note, does anyone have any questions about anything we've presented today?