Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A service design approach to improving student ...

Duncan Stephen
October 15, 2020

A service design approach to improving student experience

Research had shown us that University of Edinburgh students were frustrated by inconsistencies between different courses in their virtual learning environment. So we embarked on a programme of user research with students. However, for most of our findings about students, we found that something about staff was driving that experience. By adopting a service design approach, we have built a stronger understanding of this complex ecosystem of interactions, leaving us better placed to improve student experience.

For more detailed blog posts see https://edin.ac/uxlf

Presented at UCD Gathering virtual conference.
https://www.ucdgathering.net/programme/service-design-approach-improving-student-experience

Duncan Stephen

October 15, 2020
Tweet

More Decks by Duncan Stephen

Other Decks in Education

Transcript

  1. Duncan Stephen A service design approach to improving student experience

    User Experience Manager The University of Edinburgh duncanstephen.net — @DuncanBSS ⟶
  2. I will be happy to take questions at the end

    duncanstephen.net — @DuncanBSS
  3. Some schools already had their own templates These templates had

    168 different top-level menu items duncanstephen.net — @DuncanBSS Analysis by Paul Smyth
  4. 45,000 students 8,000 academic staff 24 schools and deaneries …all

    with different requirements duncanstephen.net — @DuncanBSS
  5. Overall goal to improve student experience But we knew that

    staff experience was also fundamental duncanstephen.net — @DuncanBSS
  6. A semester in the life of students using Learn duncanstephen.net

    — @DuncanBSS Week 1 Getting your head around Learn Week 6 Assignment submission Week 7 Different lecturers doing different things Week 8 Getting feedback
  7. duncanstephen.net — @DuncanBSS Students’ experience Frustrations around being made to

    use templates Love using Learn as a comms tool Time consuming to give good feedback Staff members’ experience Lack of trust in existing templates Overwhelmed by notifications Inconsistencies around assignments and feedback
  8. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional and content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  9. We learnt a lot about user research incentives Give people

    a small chance of a medium-sized incentive duncanstephen.net — @DuncanBSS
  10. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional and content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  11. duncanstephen.net — @DuncanBSS 0 100 200 300 400 500 600

    Students’ top tasks – the long neck and long tail
  12. Students’ top tasks duncanstephen.net — @DuncanBSS Lecture slides Lecture notes/outlines/handouts

    Lecture recordings Past exam papers Top 3 tasks >25% of votes Next 4 tasks Almost half Next 7 tasks Final 31 tasks 1. Lecture slides 2. Lecture notes / outlines / handouts 3. Lecture recordings 4. Past exam papers 5. Grades 6. Reading lists / study resources 7. Assignment deadlines 8. Assignment feedback / feed forward
  13. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional and content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  14. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional and content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  15. Gerry McGovern on testing a new structure for the first

    time: “You will have maybe a 60% success rate if you do it really well.” duncanstephen.net — @DuncanBSS
  16. Worst performing task You need to contact a teacher. Where

    would you find their email address? 67% success 59% directness duncanstephen.net — @DuncanBSS
  17. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional and content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students
  18. Research suggests that when users get the first click right,

    87% ultimately succeed in a task versus 46% who get the first click wrong Jeff Sauro, Getting the first click right https://measuringu.com/first-click/ duncanstephen.net — @DuncanBSS
  19. These results were less positive But we learned some really

    valuable information duncanstephen.net — @DuncanBSS
  20. It’s coming up to exam time and you’d like to

    see some mock exam papers. Where will you find them? 30% selected Course materials 21% selected Resource list 42% selected Assessment duncanstephen.net — @DuncanBSS
  21. But they were not clearly grouped in the card sort

    duncanstephen.net — @DuncanBSS
  22. We placed it in the Assessment section But students were

    not finding it duncanstephen.net — @DuncanBSS
  23. The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion

    Diagram adapted from Jesse James Garrett: www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional and content requirements Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Ongoing usability testing
  24. How usability testing is shaping the changes to our virtual

    learning environment duncanstephen.net — @DuncanBSS
  25. The most successful way to improve your design is to

    watch your users duncanstephen.net — @DuncanBSS
  26. “The closest thing we’ve found to a silver bullet when

    it comes to reliably improving the designs teams produce.” Jared Spool https://articles.uie.com/user_exposure_hours/ duncanstephen.net — @DuncanBSS
  27. “User research is a team sport” “All team members should

    watch real users interacting with your service and talking about it — ideally for at least 2 hours every 6 weeks.” Dipa Shah Government Digital Service User research community www.gov.uk/service-manual/user-research/how-user-research-improves-service-design duncanstephen.net — @DuncanBSS
  28. Usability testing is pretty easy to get your head around

    But there are still some challenges duncanstephen.net — @DuncanBSS
  29. Challenge 1 Getting the go-ahead from your bosses to spend

    time on usability testing duncanstephen.net — @DuncanBSS
  30. Challenge 2 Getting your colleagues to take on board what

    is uncovered by usability testing duncanstephen.net — @DuncanBSS
  31. Step 1 Conduct usability testing Each cycle we recorded four

    or five students or staff members attempting to complete some common, representative tasks in Learn duncanstephen.net — @DuncanBSS
  32. We test with as many people as we can —

    which is not many But that’s OK because usability testing is a qualitative approach “The most striking truth of the curve is that zero users give zero insights.” Jakob Nielsen www.nngroup.com/articles/ why-you-only-need-to-test-with-5-users/ duncanstephen.net — @DuncanBSS
  33. Step 2 Showcase the videos Open invite across the university

    to raise awareness among our community of learning technologists duncanstephen.net — @DuncanBSS
  34. Step 3 Attendees witness usability issues Session attendees note down

    the usability issues they see while watching the videos Focus on issues, not potential solution ideas yet duncanstephen.net — @DuncanBSS
  35. Step 4 Discuss each video in groups After each video,

    participants discuss for 5 minutes in small groups People decide on their top 3 issues from each video duncanstephen.net — @DuncanBSS
  36. Step 5 Collaboratively prioritise issues Across the whole room, people

    raise the usability issues they’ve seen Then together we work each issue through a prioritisation flowchart duncanstephen.net — @DuncanBSS
  37. Red route prioritisation How to prioritise usability problems David Travis

    www.userfocus.co.uk/articles/prioritise.html duncanstephen.net — @DuncanBSS
  38. Red route prioritisation Red routes are roads that are too

    important to get clogged up Think about key users and their top tasks in the same way duncanstephen.net — @DuncanBSS
  39. 3 yes/no questions Does the problem occur on a red

    route? Is the problem difficult for the user to overcome? Is the problem persistent? Yes to 3 questions = Critical usability issue Yes to 2 questions = Serious Yes to 1 question = Medium Yes to 0 questions = Low duncanstephen.net — @DuncanBSS
  40. Step 6 Impact/effort matrix Now we can start thinking about

    solutions, focusing on the most critical problems duncanstephen.net — @DuncanBSS
  41. Step 6 Impact/effort matrix duncanstephen.net — @DuncanBSS Minor issue for

    users Major issue for users Easy solution available No easy solution available Tackle these first
  42. “Running usability testing has been compared with taking a drink

    from a fire hydrant” David Travis www.userfocus.co.uk/articles/prioritise.html duncanstephen.net — @DuncanBSS
  43. The red route method moves us beyond prioritising by gut

    feel duncanstephen.net — @DuncanBSS
  44. 31 usability tests with students and staff 7 different Learn

    environments 9 showcase sessions 20–30 attendees at most sessions 102 usability issues identified and prioritised duncanstephen.net — @DuncanBSS
  45. “A really valuable and informative process that we are hoping

    to embed across the school and the upcoming projects we are undertaking.” Emma Forrest Student Services Project Officer, School of GeoSciences duncanstephen.net — @DuncanBSS
  46. “Being able to see what works well and what is

    more challenging for users enables us to prioritise what really matters and ensure our design is reflective of our users’ needs.” Lee-Ann Simpson Learn Foundations Project Manager duncanstephen.net — @DuncanBSS
  47. But they were not clearly grouped in the card sort

    duncanstephen.net — @DuncanBSS
  48. We placed it in the Assessment section But students were

    not finding it duncanstephen.net — @DuncanBSS
  49. By watching just 4 students through usability testing, we understood

    that they don’t look for past papers in Learn They Google for them duncanstephen.net — @DuncanBSS
  50. Without using a mixture of research methods, we would never

    have understood this duncanstephen.net — @DuncanBSS
  51. duncanstephen.net — @DuncanBSS Audience Front stage Backstage Behind-the- scenes Based

    on an illustration by Megan Erin Miller and Erik Flowers https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c
  52. duncanstephen.net — @DuncanBSS Students Lecturers School-based support staff Learn service,

    central support Based on an illustration by Megan Erin Miller and Erik Flowers https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c
  53. duncanstephen.net — @DuncanBSS Students Lecturers School-based support staff Learn service,

    central support We’d like to make it easier to post announcements Based on an illustration by Megan Erin Miller and Erik Flowers https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c
  54. duncanstephen.net — @DuncanBSS Students Lecturers School-based support staff Learn service,

    central support We’d like to make it easier to post announcements We’re rolling out enhanced announcements functionality! Based on an illustration by Megan Erin Miller and Erik Flowers https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c
  55. duncanstephen.net — @DuncanBSS Students Lecturers School-based support staff Learn service,

    central support We’d like to make it easier to post announcements We’re rolling out enhanced announcements functionality! I have an announcement! Based on an illustration by Megan Erin Miller and Erik Flowers https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c
  56. duncanstephen.net — @DuncanBSS Students Lecturers School-based support staff Learn service,

    central support We’d like to make it easier to post announcements We’re rolling out enhanced announcements functionality! I have an announcement! Not another announcement… Based on an illustration by Megan Erin Miller and Erik Flowers https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c
  57. By conducting research across this entire ecosystem, we are better

    placed to improve students’ experience duncanstephen.net — @DuncanBSS
  58. Schools that had adopted the Learn Foundations template early adapted

    to online delivery more easily Other schools needed more support duncanstephen.net — @DuncanBSS
  59. Learn Foundations rollout 2020/2021 Plan: 10 schools After coronavirus: 20

    schools (out of 24) duncanstephen.net — @DuncanBSS
  60. “Consistent templates paved the way for a seamless move into

    hybridity in a time of crisis.” Jon Jack Learning Technology Team Manager duncanstephen.net — @DuncanBSS