Understanding what students really need from a virtual learning environment

The University of Edinburgh's User Experience Service has undertaken a comprehensive programme of user research supporting Learn Foundations, a project aimed at improving students' experience accessing course materials digitally. Find out how we developed a programme of multiple user research methods to understand what students really need.

For more detailed blog posts see https://edin.ac/uxlf

Presented at Edinburgh UX Meetup.

Transcript

1. Duncan Stephen Understanding what students really need from a virtual learning environment User Experience Service The University of Edinburgh duncanstephen.net — @DuncanBSS →
2. Learn Foundations A project to ensure students and staff find Learn easy to use duncanstephen.net — @DuncanBSS
3. Students are frustrated by inconsistencies across digital systems duncanstephen.net — @DuncanBSS
4. Answer = “Make it consistent” …Obviously duncanstephen.net — @DuncanBSS
5. Consistent how? duncanstephen.net — @DuncanBSS
6. Solutions →Checklist →Terminology →Templates →Training →Automation →QA and audit duncanstephen.net — @DuncanBSS
7. A comprehensive programme of user research, adopting multiple methods duncanstephen.net — @DuncanBSS
8. Understanding what students really need Understanding how to support staff to better meet students’ needs duncanstephen.net — @DuncanBSS
9. Qualitative research with students and staff duncanstephen.net — @DuncanBSS
10. A semester in the life of students and staff using Learn duncanstephen.net — @DuncanBSS
11. duncanstephen.net — @DuncanBSS Students’ experience Frustrations around being made to use templates Love using Learn as a comms tool Time consuming to give good feedback Staff members’ experience Lack of trust in existing templates Overwhelmed by notifications Inconsistencies around assignments and feedback
12. Programme of targeted quantitative research duncanstephen.net — @DuncanBSS
13. Quantitative research specifically designed to inform the development of the new template duncanstephen.net — @DuncanBSS
14. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
15. Phenomenal response rates Almost 5,000 responses across five studies duncanstephen.net — @DuncanBSS
16. The elements of user experience duncanstephen.net — @DuncanBSS Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design
17. Students’ top tasks duncanstephen.net — @DuncanBSS Lecture slides Lecture notes/outlines/handouts Lecture recordings Past exam papers Top 3 tasks >25% of votes Next 4 tasks Almost half Next 7 tasks Final 31 tasks 1. Lecture slides 2. Lecture notes / outlines / handouts 3. Lecture recordings 4. Past exam papers 5. Grades 6. Reading lists / study resources 7. Assignment deadlines 8. Assignment feedback / feed forward
18. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Ongoing usability testing
19. duncanstephen.net — @DuncanBSS Course materials Assessment Course information Help and support
20. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
21. Gerry McGovern on testing a new structure for the first time: “You will have maybe a 60% success rate if you do it really well.” duncanstephen.net — @DuncanBSS
22. Our success rates ranged from 67% to 93% duncanstephen.net — @DuncanBSS
23. Worst performing task You need to contact a teacher. Where would you find their email address? 67% success 59% directness duncanstephen.net — @DuncanBSS
24. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
25. Research has suggested that when users get the first click right, 87% ultimately succeed in a task Versus 46% who get the first click wrong Jeff Sauro, Getting the first click right: https://measuringu.com/first-click/ duncanstephen.net — @DuncanBSS
26. We A/B tested the new prototype against existing Learn courses duncanstephen.net — @DuncanBSS
27. These results were less positive But we learned some really valuable information duncanstephen.net — @DuncanBSS
28. It’s coming up to exam time and you’d like to see some mock exam papers. Where will you find them? 30% selected Course materials 21% selected Resource list 42% selected Assessment duncanstephen.net — @DuncanBSS
29. Past papers problem Past papers were the 4th top task duncanstephen.net — @DuncanBSS
30. Past papers problem But they were not clearly grouped in the card sort duncanstephen.net — @DuncanBSS
31. Past papers problem We placed it in the Assessment section. But students were not finding it. duncanstephen.net — @DuncanBSS
32. Method triangulation to the rescue duncanstephen.net — @DuncanBSS
33. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Ongoing usability testing
34. duncanstephen.net — @DuncanBSS
35. By watching just four students, we understood that they don’t look for past papers in Learn They Google for them duncanstephen.net — @DuncanBSS
36. Without using a mixture of research methods, we would never have understood this duncanstephen.net — @DuncanBSS
37. A service design approach duncanstephen.net — @DuncanBSS Illustration by Megan Erin Miller: https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c/
38. duncanstephen.net — @DuncanBSS Illustration by Megan Erin Miller: https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c/ Students Lecturers School-based learning technologists Learn service, central support We’d like to make it easier to post announcements We’re rolling out enhanced announcements functionality! I have an announcement! Not another announcement…
39. By conducting research across this entire ecosystem, we are better placed to improve students’ experience duncanstephen.net — @DuncanBSS
40. Find out more on our blog: edin.ac/uxlf duncanstephen.net — @DuncanBSS
41. Duncan Stephen Thank you edin.ac/uxlf duncanstephen.net — @DuncanBSS
42. Duncan Stephen Questions? edin.ac/uxlf duncanstephen.net — @DuncanBSS