DR2023 Debbie Levvit - Don't Democratize UX Research - And Governance For Those Doing It

Transcript

1. ©2023 Delta CX deb@deltacx.com 1 Don't Democratize UX Research – And Governance For Those Doing It Ⓒ 2023. All Rights Reserved. Please credit Debbie Levitt and Delta CX if you are using/reusing/creating derivatives/variations of any of this. Thanks! Debbie Levitt
2. ©2023 Delta CX deb@deltacx.com 2 Connect and join! Debbie Levitt Delta CX DeltaCX.link/joinslack No sponsorships or affiliate links. Ad revenue donated to charity monthly. DeltaCX.link/discord
3. ©2023 Delta CX deb@deltacx.com 6 6 Why not “democratize?”
4. ©2023 Delta CX deb@deltacx.com 7 Defining “democratization” • It originally meant breaking down silos and making sure that research artifacts, reports, and actionable suggestions were “accessible” and shared with others. Yes, please! • Somewhere along the way, “democratization” started meaning that research should be done by anybody and everybody. Training and oversight are optional. “Anybody can talk to customers!” “Anybody can do what UX Researchers do!” No, please! • To me, “democratization” isn’t when we have Designers and Researchers work together on testing. Democratization is when people from outside of the CX or UX dept want to do CX/UX tasks, research studies, designs, etc. (often without partnering, oversight, or standards).
5. ©2023 Delta CX deb@deltacx.com 9 Critical thinking questions 1. What job exists at our company where the quality of work, outputs, and outcomes doesn’t matter? 2. Which other domains will also democratize their work? 3. What is the ROI of having anybody or everybody do research or design work? 4. If we’re struggling with a bottleneck, should we stop specialists’ mission critical work to try to teach newbies to do mission critical work? 5. How long will it take for someone to learn some or all of a specialized full-time job? 6. If the person learning is doing poor work, do we keep that work? Ban that person from future CX work? How much risk and waste are we willing to accept? 7. Aren’t experienced specialists better and more efficient at strategic and tactical CX or UX work than a newbie guessing at CX work? 8. Did we ask CX leaders what they need so that we can best support them?
6. ©2023 Delta CX deb@deltacx.com 11 Perceived vs actual skills People think UX skills are: • Talking to people. • Making a Customer Journey Map. • Putting sticky notes on a board. • Using Figma. • Laying out screens. Skills include: • Studies of human behavior and cognitive psychology. • Problem finding and solving. • Critical thinking. • Deductive reasoning and logic. • Put your preferences and biases aside to design for user needs. • For research: Planning the research. Choosing the best method(s). Planning the correct questions and the tasks we should observe. Choosing the right quantity and types of participants. Executing sessions with neutrality and a good interviewing style. Observing and noticing things others miss. Being a mini-detective. Analyzing the data. Bringing it together to report on insights, pain points, and opportunities. Delivering actionable suggestions around pain points, opportunities, strategy, and direction.
7. ©2023 Delta CX deb@deltacx.com 12 “But PMs won’t have empathy!” No empathy unless I do CX/UX work. I’ll also need to answer support calls, handle Marketing, and run QA tests. I have empathy. I know CX is specialized, and they don’t want me doing their work. I’ll be the best partner I can be, support them, and help them get what they need. I care about our customers. I’m going to watch most of the research sessions, live or recorded. I don’t have to be the session moderator to gain first-hand knowledge. I was a CX/UX pro who took a PM job so I could do the CX/UX work I was trying to do. Now I can also do the product strategy work I wanted to do. ©2023 Delta CX deb@deltacx.com 12
8. ©2023 Delta CX deb@deltacx.com 14 “I Found a Pro-Democratization UX Expert” “Anyone can cook!” “Elitist gatekeepers who want silos!” “Anyone with a clear, relevant question.” “Teach and then influence.”
9. ©2023 Delta CX deb@deltacx.com 16 16 Quality over speed
10. ©2023 Delta CX deb@deltacx.com 17 Cost of Poor Quality (COPQ) Internal costs: • Bad partners/suppliers • Delays • Failure/root cause analysis • Re-designing • Rework • Re-testing • Downtime • Lack of flexibility/adaptability • Morale • Training External costs: • Complaints (support tickets, etc.) • Customers’ trust for you • Fewer sales, fewer repeat sales • Bad word-of-mouth (ratings, tweets, “they suck,” etc.) • Environmental costs (physical objects) • Repairing goods and redoing services (customer time & frustration even if free) • Returns/recalls/exchanges • Safety incidents/lawsuits
11. ©2023 Delta CX deb@deltacx.com 19 Bad evidence is a root cause Symptoms During Project • No or unclear strategy. • No or only business-centric success criteria or KPIs. • No/incorrect/customer-peripheric problem statements. • Hard to make decisions. Symptoms After Release • Support utilization. • Negative VOC. • Lost revenue and customers. • Failed A/B test. • KPIs/OKRs not met. • Blame the Designer for “bad design.” Root Cause • We didn’t have enough or the right evidence to make better decisions, design what users need, or solve user problems.
12. ©2023 Delta CX deb@deltacx.com 20 Unanswered questions
13. ©2023 Delta CX deb@deltacx.com 21 We solve bottlenecks by hiring
14. ©2023 Delta CX deb@deltacx.com 22 22 Governance
15. ©2023 Delta CX deb@deltacx.com 23 Governance model 1. Executive support and an enforcement mechanism are necessary components. 2. Root causes and problem statement. 3. Success criteria. 4. Accountability for small or large failures. 5. Training 6. Priorities 7. Process 8. Work and Quality 9. Costs of Time and Salaries. 10. Costs of Poor Quality. 11. Costs of Attrition and Worker Dissatisfaction.
16. ©2023 Delta CX deb@deltacx.com 24 24 Think critically and take action
17. ©2023 Delta CX deb@deltacx.com 25 Your company believes in research • Surveys • Talk to customers • Analytics data • Focus groups • VOC (Voice of the Customer) • Customer Support and call centers are data rich. • Market research and predictions based on it
18. ©2023 Delta CX deb@deltacx.com 26 Sample cost-benefit analysis Alternative: • 20 people & 10% = 2-3 FT CXR Efficiency: • Experienced specialists better at strategy and tactics. • Decreases when work is guessed at by lightly-trained newbies. • Time and cost of delays, lost revenue, customer trust, poor user outcomes, support utilization. UX Research “Coach” $130K/yr 6 FT qualified Researchers 1 Lead, 1 Sr, 4 Juniors 20 PMs as part-time UXers $190K/yr Mistakes & lost revenue $200K/yr “Democratization” costs $520K/yr OR
19. ©2023 Delta CX deb@deltacx.com 28 Who does CX/UX tasks Consider our interview & assessment process. How many interviews CX/UX went through, who assessed them, design/research challenges/assignments? Qualified for a full- time CX/UX job. If you wouldn’t qualify for a CX/UX job at our company, you shouldn’t be doing our work. Full-time under CX/UX mgmt & standards. People choosing CX/UX as their career path.
20. ©2023 Delta CX deb@deltacx.com 30 Beware “democratization” and “decentralization” Why does CX/UX rush to give up power, then complain about low maturity, and evangelize how they need power? ©2023 Delta CX deb@deltacx.com 30
21. ©2023 Delta CX deb@deltacx.com 34 Thanks for attending! • CustomerCentricity.com is in progress but has fresh things! • Check out my new book, “Customers Know You Suck.” https://cxcc.to/ckys • Free 30 min of coaching for each of you. Book “30 Minutes Free” at deltacx.link/coaching • Questions? deb@deltacx.com
22. ©2023 Delta CX deb@deltacx.com 36 36 Appendix: Governance
23. ©2023 Delta CX deb@deltacx.com 37 Answer, document, and monitor: 1. Executive support and an enforcement mechanism are necessary components. Have we identified the execs or leaders sponsoring this experiment? How will we compel compliance with this model when people might want to “break the rules?” 2. Root causes and problem statement. What problem are we solving? What do we know about that problem? What’s causing the problem? What solutions have we considered? Have we analyzed pros and cons of various solutions? Why are we diluting CX? Is dilution the best solution? ▪ For example, if we are diluting CX because “Product Managers would like to do UX tasks,” is that a good enough reason, and are we allowing similar experiments in other domains? ▪ If we are diluting CX because CX is a bottleneck, and we’re willing to burn partial non-CX salaries to “get the work done,” wouldn’t hiring CX professionals make more sense?
24. ©2023 Delta CX deb@deltacx.com 38 Answer, document, and monitor: 3. Success criteria. Before embarking on the democratization experiment, we must establish success criteria to know when it’s succeeding or failing. Failure signals changing or ending the experiment. Our success criteria can’t be any variation of “we did a thing and it got done.” Having more people do research tasks might lead to more research tasks being done, but at what quality and cost? Our goal should be outstanding research that brings us closer to customer-centricity and high-quality evidence that will be used in strategies and decisions, not just any research. The success criteria should be tied to quality and Customer Experience outcomes. It should use measurable metrics, not opinions or emotions. “People are enjoying doing research” is not a success criterion.
25. ©2023 Delta CX deb@deltacx.com 39 Answer, document, and monitor: 3. Success criteria. Before embarking on the democratization experiment, we must establish success criteria to know when it’s succeeding or failing. Failure signals changing or ending the experiment. Our success criteria can’t be any variation of “we did a thing and it got done.” Having more people do research tasks might lead to more research tasks being done, but at what quality and cost? Our goal should be outstanding research that brings us closer to customer-centricity and high-quality evidence that will be used in strategies and decisions, not just any research. The success criteria should be tied to quality and Customer Experience outcomes. It should use measurable metrics, not opinions or emotions. “People are enjoying doing research” is not a success criterion.
26. ©2023 Delta CX deb@deltacx.com 40 Answer, document, and monitor: 4. Accountability for small or large failures. If there are no consequences for the people who drove this initiative, then there is no incentive to do better and no reason to reduce, change, or stop a failing program. We said we would hold people responsible or accountable, but what does that look like? Will we block that decision-maker from leading an experiment in the future? Do we reduce their available budget? How about a performance improvement plan and coaching? Could they be demoted? I have seen companies demote managers and leaders; don’t exclude it as a possibility.
27. ©2023 Delta CX deb@deltacx.com 41 Answer, document, and monitor: 5. Training. Consider Bloom’s Taxonomy, a pedagogical framework visualized as a pyramid. In many cases, democratization assumes that non-CX staff will jump right in at the highest level, which is the ability to produce new or original work. Does that non-CX worker recall facts and basic concepts about CX, the lowest level in this model? Can they explain CX ideas or concepts (the second level) or use CX knowledge in new situations (the third level)? We cannot expect quality work based on the application of extensive knowledge if we are not providing that knowledge and taking people through these pedagogical stages.
28. ©2023 Delta CX deb@deltacx.com 42 Answer, document, and monitor: 5. Training (continued). Most companies experimenting with democratization expect a research expert to provide training. Given that the Researcher’s full-time job is doing mission critical research tasks, we must create appropriate expectations for the length, breadth, and depth of training. Some Researchers are great teachers. Some are not. Have we checked which of our Researchers is also great with training and pedagogy? Let’s not also dilute education. CX research is highly specialized and, on average, takes people years to be good at. We might see low-quality results through no fault of the trainer, especially since there may not have been a selection process with standards for who will be trying CX work. Speaking of which, is there a selection or screening process for non-CX individuals? Can anybody be trained, or do we declare some people not a match to CX work? Some students will never be good at research tasks, no matter how much training they get. There will be people who just don’t get it. How much training will there be? What types of training? Will learners have time to practice on non-urgent projects? When does our Researcher have time to shift into Instructional Designer mode and create a training program? Are we sending our non-Researchers to an outside training course or bringing in a corporate trainer? How did we investigate and vet the quality of that course or that trainer? Will you utilize formative and summative assessments of the students and the trainer to measure process and effectiveness?
29. ©2023 Delta CX deb@deltacx.com 43 Answer, document, and monitor: 6. Priorities. Have we codified which work should go to the professional Researchers and which work should go to the non-Researchers? How will we prioritize the incoming work, and match work to practitioners? What are the priorities for the people being asked to train others? Which is more important: the work they currently have on their plate or the training and work reviews for those who are learning? Do not say, “They can do both.” People with packed schedules need clear priorities. What are the priorities for the people trying to learn or do research tasks? Which is more important: the work they currently have on their plate or the training and research work? Do not say, “They can do both.” 7. Process. A good research process includes planning, recruiting participants, executing the sessions, analyzing the data, synthesizing findings, and arriving at actionable insights. Will non-Researchers be required to execute a correct research process? If not, why not?
30. ©2023 Delta CX deb@deltacx.com 44 Answer, document, and monitor: 8. Work and Quality. Will non-Researchers doing research tasks be required to meet ISO or other standards we have documented? If not, why not? If we find that someone’s work doesn’t meet our quality standards, what do we do about that? Do we ban them from future research? Do we invest more time and money in trying to train them further? Do we include the bad research or deliver it to a client? Do we scrap the research and pretend it never happened? Some well-paid trainers say that research work done by non-Researchers does not need to be checked. We can let it go and assume it’s “good enough.” If we are not checking this work, that should be documented as high risk. All work that is in progress and completed should be documented and archived. This includes documentation of every step of our research process. We should archive our session recordings, research plans, and artifacts relating to how this data was analyzed. 9. Costs of Time and Salaries. Calculate the time and salary a Researcher spends training, overseeing, coaching, reviewing work, correcting work, etc. Calculate the time and salary a non- Researcher spends learning and doing research. Did work have to be redone by someone with higher skill? That took time and salary.
31. ©2023 Delta CX deb@deltacx.com 45 Answer, document, and monitor: 10. Costs of Poor Quality. Are we monitoring the projects using non-Researchers for research? Have we checked if research quality set a later stage of the project up for mistakes or failure? For example, was usability testing highly flawed, invalid, or didn’t deliver the information Designers needed to improve their designs? Did something that research “told” us turn out to not be accurate? Did we release something to the public or deliver it to a client, but it’s partially or wholly failing? What did that cost us in reputation, stock price, customer trust, Customer Support utilization, negative word of mouth, or customers downgrading or leaving? Did we make a big enough mistake where we had to roll back a software release, undo a price increase, or negate something we delivered to the public? What are the costs of that mistake and the efforts to fix or undo it? 11. Costs of Attrition and Worker Dissatisfaction. Did anybody quit our company partially or wholly due to democratization? It would be a serious loss if that knowledge and capability walked out the door. Researchers might not want to stay in a workplace where anybody can do their job regardless of proficiency. Non- Researchers might not like more tasks dumped on them, especially if they already have no spare time. Your workers might stay, but are they happy? If you opened a job in their domain, would they recommend that their friends work here? Are they telling their network that this is a great place to work? We must monitor all of these. And with some effort, we could calculate the time and money spent on most, if not all, of these. We should be able to quantify what this experiment costs, and compare that to the salaries that we could have paid professionals to do this work.