Duncan Stephen Understanding what students really need from a virtual learning environment User Experience Service The University of Edinburgh duncanstephen.net — @DuncanBSS → 

Learn Foundations A project to ensure students and staff find Learn easy to use duncanstephen.net — @DuncanBSS 

Students are frustrated by inconsistencies across digital systems duncanstephen.net — @DuncanBSS 

Answer = “Make it consistent” …Obviously duncanstephen.net — @DuncanBSS 

Consistent how? duncanstephen.net — @DuncanBSS 

Solutions →Checklist →Terminology →Templates →Training →Automation →QA and audit duncanstephen.net — @DuncanBSS 

A comprehensive programme of user research, adopting multiple methods duncanstephen.net — @DuncanBSS 

Understanding what students really need Understanding how to support staff to better meet students’ needs duncanstephen.net — @DuncanBSS 

Qualitative research with students and staff duncanstephen.net — @DuncanBSS 

A semester in the life of students and staff using Learn duncanstephen.net — @DuncanBSS 

duncanstephen.net — @DuncanBSS Students’ experience Frustrations around being made to use templates Love using Learn as a comms tool Time consuming to give good feedback Staff members’ experience Lack of trust in existing templates Overwhelmed by notifications Inconsistencies around assignments and feedback 

Programme of targeted quantitative research duncanstephen.net — @DuncanBSS 

Quantitative research specifically designed to inform the development of the new template duncanstephen.net — @DuncanBSS 

The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students 

Phenomenal response rates Almost 5,000 responses across five studies duncanstephen.net — @DuncanBSS 

The elements of user experience duncanstephen.net — @DuncanBSS Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design 

Students’ top tasks duncanstephen.net — @DuncanBSS Lecture slides Lecture notes/outlines/handouts Lecture recordings Past exam papers Top 3 tasks >25% of votes Next 4 tasks Almost half Next 7 tasks Final 31 tasks 1. Lecture slides 2. Lecture notes / outlines / handouts 3. Lecture recordings 4. Past exam papers 5. Grades 6. Reading lists / study resources 7. Assignment deadlines 8. Assignment feedback / feed forward 

The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Ongoing usability testing 

duncanstephen.net — @DuncanBSS Course materials Assessment Course information Help and support 

The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students 

Gerry McGovern on testing a new structure for the first time: “You will have maybe a 60% success rate if you do it really well.” duncanstephen.net — @DuncanBSS 

Our success rates ranged from 67% to 93% duncanstephen.net — @DuncanBSS 

Worst performing task You need to contact a teacher. Where would you find their email address? 67% success 59% directness duncanstephen.net — @DuncanBSS 

The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Visual design Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students 

Research has suggested that when users get the first click right, 87% ultimately succeed in a task Versus 46% who get the first click wrong Jeff Sauro, Getting the first click right: https://measuringu.com/first-click/ duncanstephen.net — @DuncanBSS 

We A/B tested the new prototype against existing Learn courses duncanstephen.net — @DuncanBSS 

These results were less positive But we learned some really valuable information duncanstephen.net — @DuncanBSS 

It’s coming up to exam time and you’d like to see some mock exam papers. Where will you find them? 30% selected Course materials 21% selected Resource list 42% selected Assessment duncanstephen.net — @DuncanBSS 

Past papers problem Past papers were the 4th top task duncanstephen.net — @DuncanBSS 

Past papers problem But they were not clearly grouped in the card sort duncanstephen.net — @DuncanBSS 

Past papers problem We placed it in the Assessment section. But students were not finding it. duncanstephen.net — @DuncanBSS 

Method triangulation to the rescue duncanstephen.net — @DuncanBSS 

The elements of user experience duncanstephen.net — @DuncanBSS Conception Completion Diagram adapted from Jesse James Garrett: http://www.jjg.net/elements/pdf/elements.pdf User needs and business objectives Functional/content requirements Content design and interface design IA and interaction design Top tasks surveys with students and staff Card sorting with students Tree testing with students First click testing with students Ongoing usability testing 

duncanstephen.net — @DuncanBSS 

By watching just four students, we understood that they don’t look for past papers in Learn They Google for them duncanstephen.net — @DuncanBSS 

Without using a mixture of research methods, we would never have understood this duncanstephen.net — @DuncanBSS 

A service design approach duncanstephen.net — @DuncanBSS Illustration by Megan Erin Miller: https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c/ 

duncanstephen.net — @DuncanBSS Illustration by Megan Erin Miller: https://blog.practicalservicedesign.com/the-difference-between-a-journey-map-and-a-service-blueprint-31a6e24c4a6c/ Students Lecturers School-based learning technologists Learn service, central support We’d like to make it easier to post announcements We’re rolling out enhanced announcements functionality! I have an announcement! Not another announcement… 

By conducting research across this entire ecosystem, we are better placed to improve students’ experience duncanstephen.net — @DuncanBSS 

Find out more on our blog: edin.ac/uxlf duncanstephen.net — @DuncanBSS 

Duncan Stephen Thank you edin.ac/uxlf duncanstephen.net — @DuncanBSS 

Duncan Stephen Questions? edin.ac/uxlf duncanstephen.net — @DuncanBSS