Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Understanding what students really need from a virtual learning environment

08911a018fe9e8412434d165d8474187?s=47 Duncan Stephen
September 02, 2019

Understanding what students really need from a virtual learning environment

The University of Edinburgh's User Experience Service has undertaken a comprehensive programme of user research supporting Learn Foundations, a project aimed at improving students' experience accessing course materials digitally. Find out how we developed a programme of multiple user research methods to understand what students really need.

For more detailed blog posts see https://edin.ac/uxlf

Presented at Edinburgh UX Meetup.

08911a018fe9e8412434d165d8474187?s=128

Duncan Stephen

September 02, 2019
Tweet

Transcript

  1. Duncan Stephen Understanding what students really need from a virtual

    learning environment User Experience Service The University of Edinburgh duncanstephen.net — @DuncanBSS →
  2. Learn Foundations A project to ensure students and staff find

    Learn easy to use duncanstephen.net — @DuncanBSS
  3. Students are frustrated by inconsistencies across digital systems duncanstephen.net —

    @DuncanBSS
  4. Answer = “Make it consistent” …Obviously duncanstephen.net — @DuncanBSS

  5. Consistent how? duncanstephen.net — @DuncanBSS

  6. Solutions →Checklist →Terminology →Templates →Training →Automation →QA and audit duncanstephen.net

    — @DuncanBSS
  7. A comprehensive programme of user research, adopting multiple methods duncanstephen.net

    — @DuncanBSS
  8. Understanding what students really need Understanding how to support staff

    to better meet students’ needs duncanstephen.net — @DuncanBSS
  9. Qualitative research with students and staff duncanstephen.net — @DuncanBSS

  10. A semester in the life of students and staff using

    Learn duncanstephen.net — @DuncanBSS
  11. duncanstephen.net — @DuncanBSS Students’ experience Frustrations around being made to

    use templates Love using Learn as a comms tool Time consuming to give good feedback Staff members’ experience Lack of trust in existing templates Overwhelmed by notifications Inconsistencies around assignments and feedback
  12. Programme of targeted quantitative research duncanstephen.net — @DuncanBSS

  13. Quantitative research specifically designed to inform the development of the

    new template duncanstephen.net — @DuncanBSS
  14. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  15. Phenomenal response rates Almost 5,000 responses across five studies duncanstephen.net

    — @DuncanBSS
  16. The elements of user experience duncanstephen.net — @DuncanBSS Top tasks

    surveys with students and staff Card sorting with students Tree testing with students First click testing with students Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design
  17. Students’ top tasks duncanstephen.net — @DuncanBSS Lecture slides Lecture notes/outlines/handouts

    Lecture recordings Past exam papers Top 3 tasks >25% of votes Next 4 tasks Almost half Next 7 tasks Final 31 tasks 1. Lecture slides 2. Lecture notes / outlines / handouts 3. Lecture recordings 4. Past exam papers 5. Grades 6. Reading lists / study resources 7. Assignment deadlines 8. Assignment feedback / feed forward
  18. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Ongoing usability testing
  19. duncanstephen.net — @DuncanBSS Course materials Assessment Course information Help and

    support
  20. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  21. Gerry McGovern on testing a new structure for the first

    time: “You will have maybe a 60% success rate if you do it really well.” duncanstephen.net — @DuncanBSS
  22. Our success rates ranged from 67% to 93% duncanstephen.net —

    @DuncanBSS
  23. Worst performing task You need to contact a teacher. Where

    would you find their email address? 67% success 59% directness duncanstephen.net — @DuncanBSS
  24. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  25. Research has suggested that when users get the first click

    right, 87% ultimately succeed in a task Versus 46% who get the first click wrong Jeff Sauro, Getting the first click right: https://measuringu.com/first-click/ duncanstephen.net — @DuncanBSS
  26. We A/B tested the new prototype against existing Learn courses

    duncanstephen.net — @DuncanBSS
  27. These results were less positive But we learned some really

    valuable information duncanstephen.net — @DuncanBSS
  28. It’s coming up to exam time and you’d like to

    see some mock exam papers. Where will you find them? 30% selected Course materials 21% selected Resource list 42% selected Assessment duncanstephen.net — @DuncanBSS
  29. Past papers problem Past papers were the 4th top task

    duncanstephen.net — @DuncanBSS
  30. Past papers problem But they were not clearly grouped in

    the card sort duncanstephen.net — @DuncanBSS
  31. Past papers problem We placed it in the Assessment section.

    But students were not finding it. duncanstephen.net — @DuncanBSS
  32. Method triangulation to the rescue duncanstephen.net — @DuncanBSS

  33. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Ongoing usability testing
  34. duncanstephen.net — @DuncanBSS

  35. By watching just four students, we understood that they don’t

    look for past papers in Learn They Google for them duncanstephen.net — @DuncanBSS
  36. Without using a mixture of research methods, we would never

    have understood this duncanstephen.net — @DuncanBSS
  37. A service design approach duncanstephen.net — @DuncanBSS Illustration by Megan

    Erin Miller: https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c/
  38. duncanstephen.net — @DuncanBSS Illustration by Megan Erin Miller: https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c/ Students

    Lecturers School-based learning technologists Learn service, central support We’d like to make it easier to post announcements We’re rolling out enhanced announcements functionality! I have an announcement! Not another announcement…
  39. By conducting research across this entire ecosystem, we are better

    placed to improve students’ experience duncanstephen.net — @DuncanBSS
  40. Find out more on our blog: edin.ac/uxlf duncanstephen.net — @DuncanBSS

  41. Duncan Stephen Thank you edin.ac/uxlf duncanstephen.net — @DuncanBSS

  42. Duncan Stephen Questions? edin.ac/uxlf duncanstephen.net — @DuncanBSS