Student-centred design of LA systems

Presentation at the 11th UK Learning Analytics Network meeting, Aston University, Birmingham, 5th Sept 2017.

The interaction and interface design of Learning Analytics (LA) systems is often based upon the ability of the developer to extract information from disparate sources and not on the types of data and interpretive needs of the user. Current systems also tend to focus on the educator’s view and very rarely involve students in the development process. In this project we will be placing the learner at the centre by training student ambassadors in user-centred design techniques to find out what motivates their peers to study and how this can be incorporated into the design of a LA tool. Data Mining techniques will be used to build models of student behavior from VLE usage data and other relevant sources so that a LA tool can be developed and trialed on modules across the faculty. Students will be given access to a personalised representation of their progress in real-time, taking into account what motivates them to study. We will also investigate how LA can be incorporated into the delivery of modules with the key aims of increasing engagement, making the VLE a more active space for learning and teaching and bridging the current gap between physical and digital spaces.

Transcript

1. Student-centred design of LA systems Dr Ed de Quincey & Chris Briggs School of Computing and Mathematics, Keele University 11th UK Learning Analytics Network Meeting
2. Dr Ed de Quincey @eddequincey Senior Lecturer in Computer Science, UG and PG Course Director School of Computing and Mathematics, Keele University Senior Fellow of the HEA instagram.com/eddequincey
3. Chris Briggs @jooldesign Research Software Engineer in Learning Analytics School of Computing and Mathematics, Keele University instagram.com/jooldesign/
5. 
   
7. WP1. KLE Data Review
    • Inconsistent formats • Reports take a long time to run • No way to bulk download all “raw” data • Student Overview for a Single Course is the most useful BUT have to run for each student Around 11 “reports” available 9 Course Reports, a Performance Dashboard and a Retention Centre
8. Blackboard Analytics
9. Apple Lisa PRESENTING STUDENT DATA BACK TO STUDENTS and LECTURERS, USING USER CENTRIC QUERIES, FORMATS and METAPHORS
10. Identified and Reviewed 22 Learning Analytics Systems
11. Review of Existing Systems Few systems available for general use  Primarily targeted at educators. Only 5 of the 22 systems being designed purely for use by students Very few studies gathered the requirements for the systems directly from students Currently analysing visualisation techniques used and types/sources of data used
12. User-Centered Design Process Map http://www.usability.gov/how-to-and-tools/resources/ucd-map.html
13. WP3. Requirements Gathering Set as Coursework Case Study: Identify GUI metaphors that will engage and motivate them as learners and personalise their own learning experience 2nd Year Computing Module - 82 students in 14 groups Deliverables included: Sets of User Persona, analysed results of requirements elicitation sessions and annotated screen mock-ups of potential LA Dashboards (highlighting 5 key features along with objective justifications wherever possible).
14. 'My Timeline': Past events which have been completed successfully provide gratification to the user in the form of positive icons such as thumbs up or smiley faces. The past timeline balances with the upcoming events to try and alleviate future workload stress by demonstrating positive success at the same time.
15. Course/Module Progress Bar: a progress bar based upon the how far in the course users are. Indicating how much of the course they should know and how much time is left until the course finishes.
16. Score Indicator: The score indicator is a metric derived from an algorithm which would track a student’s engagement with course materials and other important areas of engagement that students should be utilising (comments, discussion).
17. Degree Classification Requirements: Panel showing the Percentage needed in Future Assignments to get certain Degree Classifications.
18. Comparison with peers: The chance for the user to see how they are comparing with the top 20% of the class and how they are doing compared with the average mark.
19. Goals and Trophies: The student will get a list of goals or tasks sent to their dashboard and there they can also add more tasks of their own.
20. HEFCE Catalyst Grant £99,790 (£49,988 from Catalyst Fund, £49,802 in matched funding from Keele) Title: Learner Centred Design for Learning Analytics This project aims to avoid the common problem in Learning Analytics (LA) of the technology and data driving the user experience, and therefore the ability to interpret and use the information. By sharing the data directly with students, using student-centred representations of their learning activity, this project aims to facilitate a common understanding of the learning experience between lecturers and students. Expanding on a successful teaching innovation project at Keele University interface metaphors for LA will be identified that motivate and personalise the learning experience of cohorts with differing levels of technical experience and levels of digital literacy. We will then produce appropriate visualisations of student activity based on the data available at Keele University and incorporate them into the delivery of relevant modules with the key aims of increasing engagement, making the VLE a more active space for learning and teaching and bridging the current gap between physical and digital spaces.
21. Outcomes for Students Students in lecture hall ©Jirka Matousek via Flickr • Access to personalised notifications and support e.g. highlighting/suggesting resources that have not been viewed. • Increased levels of engagement, in particular VLE usage. • Personalisation of cohort module delivery by the lecturer • Real-time feedback for students enabling them to judge their progress during a module using a different metric than current models of formative and summative feedback. • Direct involvement with the development of tools that support their learning.
22. Student Ambassadors
23. 1 Being the best you can be/Effort (ability to maintain effort) 2 Build self-confidence 3 Career/Vocation/Job prospects 4 Industry 5 Giving yourself options 6 Grades/Marks/Qualifications 7 Mastery of a subject/Interest in Subject/Stretch themselves intellectually 8 Mentoring/Family 9 Money 10 Part of a Professional community 11 Self-efficacy/ Helplessness (this might be the opposite of self-efficacy) 12 Sense of connectedness with others with similar goals/ Success as a group of peers 13 Social Prestige/Recognition Initial Identified Motivators for Studying in HE
26. Looks how they feel Shows how hard they are trying No point in spending £9,000 if you’re not going to try hard and do well A B ↑ ↑ “When thinking about what motivates you to study your degree, which of these do you prefer and why?”. “Why..?” “Why..?” Laddering
27. 14 Supporting family/Home life 15 Negative imagery 16 Rising above circumstances 17 Financial security 18 Controlling own destiny 19 Individuality 20 Opportunity to travel 21 Fear of wasting University opportunity Additional Identified Motivators for Studying in HE from Laddering Sessions
28. Revised Motivators (21 grouped into 9)
29. Timeline Formed basis of 6 Focus Groups organised and run by Student Ambassadors
34. Professional Theme
35. Personal/Personified Theme
36. Personal/Personified Theme
37. Mobile Optimised
38. Work in Progress
39. Learning Analytics in the classroom? Students in lecture hall ©Jirka Matousek via Flickr
40. Dr Ed de Quincey e.de quincey@keele.ac.uk Chris Briggs c.briggs@keele.ac.uk KEELE UNIVERSITY SCHOOL OF COMPUTING MOTIVATION METRICS V6 ALL DESIGNS ARE COPYRIGHT 2017 OF MORE THAN JUST DESIGN LIMITED WWW.MORETHANJUSTDESIGN.CO.UK CEO