You are not your user - EdSurge Immersion 2019

“You are not your user” is a common adage in the world of product and design, and is particularly true when building products for K-12 schools. In the fall of 2019, I gave a talk at EdSurge Immersion on my favorite “you are not your user” insights from five years of working in edtech.

Transcript

1. You are not your user Understanding the teacher experience to build products that really work Before we jump into the meat of today’s topic, I want you to imagine logging out of a website. It’s a pretty simple feature, most websites have it. You can think about what it looks like to log out of your own product, or maybe something you use like LinkedIn. I’d wager that what you’re thinking of looks something like this:
2. There’s some sort of icon in the top right, or maybe your name. You click it, there’s a dropdown, you click the thing that says “Log out.” Makes sense, you see this pattern everywhere. Now I want to show you what it looks like to log out of Lexia, a product built for education:
3. A big red “X” - there’s no words here, no dropdown. I love the Lexia design here, because it recognizes that the world you and I live in, the world that leads to LinkedIn’s dropdown menus, is very different than the world of students and teachers. For example:
4. • I am the only one using my device • Students swap devices often • Logout is a primary action • I can read English • May not be able to read complex words • Bright visuals with symbols are important • I have experience using a mouse and keyboard • May be used to tapping, but using a mouse for the first time • Large click areas; don’t use complex patterns Me on LinkedIn Young Student On Lexia Product Implications This is a small interaction, but I love it as an example because it surfaces how different we all are from the users that use our products, and how those differences surface themselves when products are thoughtfully designed.
5. You are not your user. That’s the takeaway here, and what we’ll be digging into today, is that you are not your user. To build great products, especially in K12, you have to be willing to leave your own assumptions at the door and deeply understand the customer context.
6. Brett van Zuiden Product Lead Discovery I’m Brett van Zuiden, a PM lead @ Clever, and for the last 5 years have focused on building new products for our teachers and schools.
7. At Clever, we’ve been lucky to get to work lots of people not like us 55% of U.S. K-12 schools 11M students and 750k active teachers 1 in 3 K-2 students using Clever Badges 10% U.S. teachers using Clever Library For those who don’t know, Clever has been around about 8 years, and along the way we’ve experimented a lot and learned a lot. For example, when we built Badges, we had a key insight about how young students struggled to type usernames and passwords, and badges are now used by 1 in 3 K-2 students in the U.S. I’d like to share some insights that helped me understand how different I am from the people who use our products and what that means for the products we build.
8. 1 “You are not your user”: uncovering insights to build great businesses 2 Insights from how teachers use technology in the classroom 3 Insights from how teachers relate to their organization Today’s agenda In our time together this morning, we’ll first dive a bit more into this idea of “you are not your user”,. I’ll then walk you through a collection of high-level takeaways about teachers, especially the ways that teachers use technology differently than you and I, and the ways that teachers related to their organization differently than I relate to Clever.
9. 1. Design for yourself, assuming users are the same as you. 2. Design for yourself, knowing users are different than you. 3. Design for users that are different than you. Becoming a great designer I had a colleague share this insight with me a while back: 1. First you design for yourself and assume everyone else is exactly the same as you. 2. Then you design for yourself, knowing your users will actually be quite different. 3. Then you design for users who are quite different than you. But it’s unclear if anyone actually gets to stage 3! It’s far more difficult than you imagine.
10. Deeply understanding customer context is essential for every leader across every function And this is not just for your design team. If you want to be successful in impact or dollars, the decisions you make about how your product works, how the features come together, even the business model need to be grounded in a realization that you are not your user.
11. Warm up: District IT Let’s take another look at this concept in practice - how many people here have ever been a school district IT administrator? How many people here have ever met a school district IT administrator?
12. IT admin’s workspace My workspace Clever works with these folks all the time, in fact our founder was one! They’re great people, here’s what their typical workspace looks like vs. mine at Clever. Right away you can start seeing (and probably imagining) a lot of differences about what our work style might be like. But I think one of the most interesting “you are not your user” moments for me when learning about IT administrators is by comparing our workflows.
13. Idea Goal Process Research Success Employee Like many of you all, my work is largely determined by goals that we set on a quarterly or yearly basis. Beyond that, I have a lot of latitude to figure out the best way to achieve the goal, and so will come up with ideas and put them on my TODO list, but if a direction isn’t looking promising, I’ll try another approach.
14. For a typical IT admin, their workflow is based on a ticket queue. People submit tickets, and the admin goes through them in priority order, working through them one by one until they are resolved.
15. I am not my user - whereas I have quite a bit of flexibility to work on whatever I think is best to hit my goals, there’s not a ton of agency for the IT admin when it comes to what to work on. On the other hand, if I run into a challenging issue I might punt it to deal with it later, but you can bet that if there’s a ticket at the top of the queue, as long as it’s possible to be done, it’ll get done. Your turn: how might some of these differences impact the design or functionality of your products? • Flexibility to choose what I work on to hit goals • Frequently takes tasks based on queue • Route around or punt on challenging issues • If it’s possible to get done, it’ll get done Me IT Admin How might these differences impact the design or functionality of product?
16. For me, the truest example of this was when we built a feature that allowed kids to log into a chromebook with a Clever Badge. The setup instructions were about 5 pages of very detailed instructions, and if any step was skipped or done incorrectly, the whole thing wouldn’t work. I worried that no one would be able to do this correctly, or that everyone would bail out after the 2nd step, but no - people wanted the feature, a ticket was created, and IT admins actually commented that this was relatively easy compared to some of their other tasks, the documentation was clear and it only took an hour. I think if I was using a product and it took me more than 5 minutes to complete a task I would bail out, let alone an hour! But, I am not my user! “Clever Badge setup was relatively easy compared to some of their other tasks, the documentation was clear and it only took an hour.” District IT Admin in CA
17. Now that you’ve seen two examples, one for students, one for IT administrators, I’d love for you all to reflect on your own experience and share out with your table. We’ll take 10 minutes or so, and I’d like for each table to share out some insights or stories that you have experienced where you noticed how different your users were from yourself. I’d also ask that each table nominate someone who can share out some of the highlights of the discussion from their table. Discuss Aha moments when you’ve noticed how different your users are from yourself. 10 minutes
18. Thank you! Some of you have touched on things like this in what you shared, but I wanted to dig a bit deeper into what I think are two of the most important differences between ourselves and our users: how we differ from teachers in our relationship with technology, and how we differ from teachers in our relationship with our organization. Teachers with technology
19. Let’s start on the technology side. Let’s look again at a picture of how I use technology at work, vs. a teacher How I work with technology How a teacher works with technology
20. I want to call out three important differences between me and that teacher. • Sitting with a screen and monitor • Moving around the room • Only I can see my screen • In front of an audience Me Teacher • Quiet, headphones on • Using tech while presenting
21. I think this is probably the single clearest takeaway that I’d encourage you write down - unlike how you use technology at work, if your product being used by a teacher in class, it’s not on a normal laptop screen! Teachers have a “public” projected screen and a “private” phone screen
22. If your product is meant to be student-facing, you should expect and design for the screen to be projected. Build your product as if you were building a slide deck - make sure it can be seen from the back of the class, make it fun with vibrant colors, and not a lot of text. Kahoot is a great example of this. Also, Kahoot plays music, it’s super fun, and teachers and kids love it, in large part because it’s designed to be projected.
23. Also, make sure there is nothing sensitive on the screen! Be thoughtful about things like showing student scores on the teacher’s screen, having notifications or popups fly in at inopportune times, or even things like showing the teacher’s first name in the UI - many teachers don’t want students knowing their first name, so if you put the full name in the header, you’ll see less willingness from teachers to use your product in class with students.
24. If your product is more teacher-facing or a classroom management tool, rather than the big, public, projected screen, you’ll want to design for mobile - teachers are walking around their classroom, phone in hand. This is a diagram one of our designers made showing the movement of a teacher throughout a classroom in a 30 minute period.
25. “Remote Control” Public Leaderboard Design your product like a remote control, with big buttons and quick shortcuts. Not a lot of menus or complexity - teachers are using this with one hand when walking around and half paying attention. Simple rule of thumb for teacher tools: if you can’t use it while holding a conversation, teachers probably won’t use it in class. ClassDojo does a great job with this.
26. This is so different than how I use technology! And I have to keep reminding myself of this. The bottom line here: You are not your user! You spend your time quietly in front of a personal laptop or device, a teacher in class has a projected “public” screen and a phone as their “personal” screen, build your products accordingly. Teachers have a “public” projected screen and a “private” phone screen. You are not your user.
27. Let’s go back to this photo - we talked about the teacher using technology while “presenting,” there’s another implication of that besides how you design your interface, and that’s what happens when something goes wrong. If you’re on your laptop and you run into a bug, it’s annoying, but if you can figure out a way around it, it’s not too bad. If a service is totally down, maybe you grab a coffee or go work on something else for a while. For teachers it’s a different story.
28. You know that feeling when you’re about to do a live demo? You know that feeling you get when you’re about to do a live demo? You’ve tested it 100 times but you’re still really nervous that you’ll run into some small hiccup, stumble a bit, and lose the audience? Or god forbid that the demo breaks right in a critical moment and you have to scramble to salvage the presentation? That fear, that feeling of being “on stage” and that everything needs to go 100% perfect - that’s the experience of teachers using technology every day.
29. For this reason, it’s so important that your product be stable, resilient, and predictable. We have a team at Clever that’s solely related to reliability - we count every minute of downtime, because we know each of those minutes is a teacher who’s watching their plan for the day start unravelling. At Clever’s current scale, one minute of downtime represents 150 days of lost learning time across the county. It may sound dramatic, but remember, these teachers are basically on-stage when they’re using your product. It’s also why things like Single Sign-On and automated rostering are so important, so that the experience just works. Stable Resilient Predictable
30. One strategy to consider is how your product can minimize the impact when something does go wrong - for instance, we have separate servers for running our “critical path” features from those that support the rest of our functionality, so that students and teachers can always at least log in to applications. We also build features like backup login codes - if a student’s password isn’t working, the teacher can generate a temporary “backup login code” that they can give to the student and get them in. If your product doesn’t work, it’s going to be disruptive to a teacher’s lesson plan, but if you can give them a straightforward workaround, you can soften the blow.
31. Another thing you can do to help a teacher be more comfortable using your product with students is to allow the teacher to see what the students see! Teachers want to run through the experience ahead of time so they know exactly what students will see and can help them out. To the point of “on question 6, it asks for the simplified form, make sure you put parentheses or it will mark it as wrong”
32. (Teacher in-product pop-up placeholder) Finally, if you’re trying to get a teacher’s attention (email, popups, etc.) do it after school hours or on Sundays! Don’t interrupt a teacher during the middle of class. And if you’re doing user research, don’t make the mistake I’ve made by trying to schedule user research calls during the day - I can take a 30 minute call during the middle of the day, but no teacher is going to spend 30 minutes giving you product feedback while they’re in front of students!
33. The bottom line here: while you use products quietly and privately, teaching is like Live TV! Teachers using your product in class feel the impact of bugs and outages much more acutely than you do. I use technology privately. When something breaks, I can go around it. Teaching is like live TV. You are not your user.
34. So two really important takeaways here - how teachers use technology is so different than how you or I use technology, especially in that: 1. My primary screen is a laptop - Teachers in class have a public projected screen and a private phone screen. 2. I use technology privately, and if something breaks I can route around it - Teaching is like Live TV Design for a public screen & private screen Stability, resiliency, predictability Teachers use technology very differently than I do.
35. Teachers’ relationship with organization Now I want to zoom out a bit and talk about another set of differences that relate a little less to technology specifically, but can make a huge impact on your business and go-to-market, and that’s how a teacher relates to their school and district.
36. Teachers’ relationship with organization Quick show of hands - raise your hand if you work in an organization of more than 100 people. Raise your hand if you did not play a part in setting the quarterly or annual goals for your organization. Raise your hand if you don’t know the name of most if not every person in your organization?
37. Teachers’ relationship with organization What is normal for nearly all of us in this room - a close-knit group of individuals, probably all located in the same office, who collaborate around setting and then hitting goals - is very, very different than what is normal for a teacher.
38. DC Public Schools #111 This is a map of DC public schools, the 111th largest district in the country - there are plenty much bigger.
39. 116 schools 3,700 teachers 3,300 staff Spread across a city From the central office to Shepherd elementary school here at the top is a 25 minute drive, up to an hour in traffic. And there are 116 schools in DCPS, 3,700 teachers plus another 3,300 staff members. DCPS is a 7,000 person org, spread out across 120 buildings throughout an entire city.
40. • Regularly talk with company leadership • Rarely or never meet people from the district office • Frequently share my work and get exec feedback • Central office has little visibility or say in the classroom Me Teacher • Participate in company goal setting • Initiatives handed down with little buy-in When I first got to Clever, I assumed that just as I work regularly with our VP of Product, a high school math teacher must of course work with the head of high school math at the central office? But you are not your user, and teachers feel much less connection to and cohesion with the school district than I feel with my company:
41. • Regularly talk with company leadership • Rarely or never meet people from the district office • Frequently share my work and get exec feedback • Central office has little visibility or say in the classroom Me Teacher • Participate in company goal setting • Initiatives handed down with little buy-in Let’s talk about these last two in a bit more detail, and what they mean for your business.
42. Your job Ensure the 300+ teachers in 100+ schools do a good job teaching social studies. Good luck! Say you’re that head of the high school social studies curriculum - your boss says that your job is to ensure every one of the 300+ of social studies teachers - most who you’ve never met - across 100+ buildings you’ve never been to, are day in and day out following the recommended curriculum and doing a good job teaching social studies to their students. It’s impossible.
43. Selling to a district administrator is selling leverage. And so if you are selling a technology product to a district administrator, you are selling leverage. I see all these edtech products that have this really awesome set of features for teachers, and their district version is “you get reporting.” Reporting is like 2% of the value you can provide to a district. Yes, it gives them visibility into what is going on in their classrooms, but really what they want is the ability to impact what’s going on.
44. I think what Newsela did here was masterful.
45. I mean look at this, it just screams leverage. And as far as I understand, it’s been incredibly lucrative for them. Because they can go to the head of social studies at a district and say “hey, we know all your teachers are already using Newsela, now you can pick the content that they use and the scope and sequence and see that teachers are actually following the guidelines” and the administrator says “thank god - how much money do you want?”
46. I also think this is the whole reason why districts buy LMSs and assessment management systems, because someone at the central office was tasked with managing some process or outcome across all their schools, and needed technology to give them leverage in order to do so.
47. It’s also one of the reasons why districts love Clever - they can click a button and deploy an application directly onto the screen for 10 students or 1000 teachers without leaving their desk, and get analytics on how those programs are being used. We also build features like in-product announcements, so that district administrators can broadcast information to students and teachers.
48. And this is very different than it is at Clever - at Clever, we adopted slack in a day because our CTO said let’s try it and we said okay. There’s not much need for leverage if you know everyone and see them every day - but you are not your user, and for a district administrator dealing with hundreds or thousands of classrooms, leverage is essential.
49. • Regularly talk with company leadership • Rarely or never meet people from the district office • Frequently share my work and get exec feedback • Central office has little visibility or say in the classroom Me Teacher • Participate in company goal setting • Initiatives handed down with little buy-in We examined this from the district administrator side, but this diffuse relationship between the teacher and the organization has another impact: teachers often have very little buy-in for district initiatives. Again with Slack - our CTO said “let’s try it,” we said “okay.” In a district, even if the superintendent says that adoption of a program is the #1 priority and is super critical and please please please participate, you’ll see a number of teachers shrug and say “this too shall pass”.
50. A median of only 30% of licenses are used And, a median of 97.6% of licenses are never used intensively Source: BrightBytes 2018 Annual Insights Report Here’s a report from BrightBytes that says the same thing with data. Simply put, you can’t rely on the district to roll out your program if you want significant usage. So what can you do?
51. 1. Make it as easy as possible for a teacher to start using your product. 1) Make it as easy as possible for a teacher to start using your program. Invest a ton in your new user onboarding flow, progressively train teachers in the product to first do the basic things, then expand to more of the complex functionality. Also, this is self serving, but SSO & automated rostering significantly lower the barrier to the teacher deciding to adhere to district policy. If a teacher has to set up a bunch of new accounts and teach new passwords, it’s far too easy for them to let it slip.
52. 1. Make it as easy as possible for a teacher to start using your product. 2. Look for ways to drive bottoms-up adoption. 2) Look for ways to drive bottoms-up adoption: you can increase adoption if you get one teacher in a school to start using your product, and then lean into a teacher’s natural tendency to spread the word to others in their school if they think the product is helpful. At Clever, we provide PD to the district, but we also use our login data to find “lead teachers,” and then send them training & PD materials to them to encourage them to spread the word at the grassroots level
53. Sell leverage Make it super easy for teachers to get started Drive bottoms-up adoption The gap between teacher and district is much wider than the gap between you and your organization. The bottom line here: the gap between a teacher and their district is much, much wider than the gap between you and your organization. As a result: 1) People in the central office want leverage (and you should sell it to them), and 2 )If you want teachers to adopt your product, you have to complement top-down district deployments by making it easy for teachers to get started and drive your own bottoms-up adoption.
54. Making it stick Now, if you let me, I could stand up here all day and share stories about how you are not your user, but I want to allow you to get on with the rest of the conference. To summarize what we talked about here:
55. Design for yourself, assuming users are the same as you. Design for yourself, knowing users are different than you. Design for users that are different than you. Becoming a great designer First, and this is the key - remember: 1. First you design for yourself and assume everyone else is exactly the same as you. 2. Then you design for yourself, knowing your users will actually be quite different. 3. Then you design for users who are quite different than you.
56. You are not your user. You are not your user. To build great products, especially in K12, you have to be willing to leave your own assumptions at the door and deeply understand the customer context.
57. 1 Young students switch devices, may not be able to read or use technology fluently 2 IT administrators work out of a ticket queue; more persistence, less agency 3 Teachers have a public projected screen and a private phone screen We saw this in a number of different areas of K12: 1. Young students switch devices a lot and may not be able to read or use technology like you can 2. IT administrators work out of a ticket queue, and so have less agency but more persistence 3. Teachers have a “public” projected screen and a “private” phone screen
58. 1 Young students switch devices, may not be able to read or use technology fluently 4 Teaching is like live TV 2 IT administrators work out of a ticket queue; more persistence, less agency 5 District administrators have little influence in the classroom; offer them leverage! 3 Teachers have a public projected screen and a private phone screen 6 Teachers don’t feel connected to district priorities; encourage bottoms up usage 4. Teaching is like Live TV - how can you make your product more reliable, stable, and predictable? 5. District administrators have far less influence over classrooms than you might expect, how can you offer them leverage? 6. Teachers feel much less connection to district priorities than you might expect, how can you encourage usage anyway?
59. Discuss What changes do you want to make to your product, process, or company to design for your users? 10 minutes For the last 10 minutes here, I’d like to return to a similar exercise we did in the beginning. In your tables again, I’d like you to reflect on what changes you want to make to your process, product, or company as a result of some of the things we’ve talked about today. Again, I’d ask that each table nominate someone who can share out some of the highlights of the discussion from their table.
60. 1 Young students switch devices, may not be able to read or use technology fluently 4 Teaching is like live TV 2 IT administrators work out of a ticket queue; more persistence, less agency 5 District administrators have little influence in the classroom; offer them leverage! 3 Teachers have a public projected screen and a private phone screen 6 Teachers don’t feel connected to district priorities; encourage bottoms up usage What changes do you want to make to your process, product, or company as a result of these insights? Hopefully you’ve learned something today, but I want to make it stick by asking you to think about this in the context of your organization. When you return home, this may all be a bit of a blur, so I want you to write down some action items, right now, before you forget.
61. You are not your user Brett van Zuiden, PM Lead at Clever brett.vanzuiden@clever.com http://brettcvz.com Thank you all - I appreciate your time, enjoy the rest of the conference, and come find me if you want to talk more about building products in K12