Student-centred design of LA systems

Dafb8db748341892bda61117ecd16a43?s=47 Ed de Quincey
September 05, 2017

Student-centred design of LA systems

Presentation at the 11th UK Learning Analytics Network meeting, Aston University, Birmingham, 5th Sept 2017.

The interaction and interface design of Learning Analytics (LA) systems is often based upon the ability of the developer to extract information from disparate sources and not on the types of data and interpretive needs of the user. Current systems also tend to focus on the educator’s view and very rarely involve students in the development process. In this project we will be placing the learner at the centre by training student ambassadors in user-centred design techniques to find out what motivates their peers to study and how this can be incorporated into the design of a LA tool. Data Mining techniques will be used to build models of student behavior from VLE usage data and other relevant sources so that a LA tool can be developed and trialed on modules across the faculty. Students will be given access to a personalised representation of their progress in real-time, taking into account what motivates them to study. We will also investigate how LA can be incorporated into the delivery of modules with the key aims of increasing engagement, making the VLE a more active space for learning and teaching and bridging the current gap between physical and digital spaces.

Dafb8db748341892bda61117ecd16a43?s=128

Ed de Quincey

September 05, 2017
Tweet

Transcript

  1. Student-centred design of LA systems Dr Ed de Quincey &

    Chris Briggs School of Computing and Mathematics, Keele University 11th UK Learning Analytics Network Meeting
  2. Dr Ed de Quincey @eddequincey Senior Lecturer in Computer Science,

    UG and PG Course Director School of Computing and Mathematics, Keele University Senior Fellow of the HEA instagram.com/eddequincey
  3. Chris Briggs @jooldesign Research Software Engineer in Learning Analytics School

    of Computing and Mathematics, Keele University instagram.com/jooldesign/
  4. None
  5.  

  6. None
  7. WP1. KLE Data Review   • Inconsistent formats •

    Reports take a long time to run • No way to bulk download all “raw” data • Student Overview for a Single Course is the most useful BUT have to run for each student Around 11 “reports” available 9 Course Reports, a Performance Dashboard and a Retention Centre
  8. Blackboard Analytics

  9. Apple Lisa PRESENTING STUDENT DATA BACK TO STUDENTS and LECTURERS,

    USING USER CENTRIC QUERIES, FORMATS and METAPHORS
  10. Identified and Reviewed 22 Learning Analytics Systems

  11. Review of Existing Systems Few systems available for general use

     Primarily targeted at educators. Only 5 of the 22 systems being designed purely for use by students Very few studies gathered the requirements for the systems directly from students Currently analysing visualisation techniques used and types/sources of data used
  12. User-Centered Design Process Map http://www.usability.gov/how-to-and-tools/resources/ucd-map.html

  13. WP3. Requirements Gathering Set as Coursework Case Study: Identify GUI

    metaphors that will engage and motivate them as learners and personalise their own learning experience 2nd Year Computing Module - 82 students in 14 groups Deliverables included: Sets of User Persona, analysed results of requirements elicitation sessions and annotated screen mock-ups of potential LA Dashboards (highlighting 5 key features along with objective justifications wherever possible).
  14. 'My Timeline': Past events which have been completed successfully provide

    gratification to the user in the form of positive icons such as thumbs up or smiley faces. The past timeline balances with the upcoming events to try and alleviate future workload stress by demonstrating positive success at the same time.
  15. Course/Module Progress Bar: a progress bar based upon the how

    far in the course users are. Indicating how much of the course they should know and how much time is left until the course finishes.
  16. Score Indicator: The score indicator is a metric derived from

    an algorithm which would track a student’s engagement with course materials and other important areas of engagement that students should be utilising (comments, discussion).
  17. Degree Classification Requirements: Panel showing the Percentage needed in Future

    Assignments to get certain Degree Classifications.
  18. Comparison with peers: The chance for the user to see

    how they are comparing with the top 20% of the class and how they are doing compared with the average mark.
  19. Goals and Trophies: The student will get a list of

    goals or tasks sent to their dashboard and there they can also add more tasks of their own.
  20. HEFCE Catalyst Grant £99,790 (£49,988 from Catalyst Fund, £49,802 in

    matched funding from Keele) Title: Learner Centred Design for Learning Analytics This project aims to avoid the common problem in Learning Analytics (LA) of the technology and data driving the user experience, and therefore the ability to interpret and use the information. By sharing the data directly with students, using student-centred representations of their learning activity, this project aims to facilitate a common understanding of the learning experience between lecturers and students. Expanding on a successful teaching innovation project at Keele University interface metaphors for LA will be identified that motivate and personalise the learning experience of cohorts with differing levels of technical experience and levels of digital literacy. We will then produce appropriate visualisations of student activity based on the data available at Keele University and incorporate them into the delivery of relevant modules with the key aims of increasing engagement, making the VLE a more active space for learning and teaching and bridging the current gap between physical and digital spaces.
  21. Outcomes for Students Students in lecture hall ©Jirka Matousek via

    Flickr • Access to personalised notifications and support e.g. highlighting/suggesting resources that have not been viewed. • Increased levels of engagement, in particular VLE usage. • Personalisation of cohort module delivery by the lecturer • Real-time feedback for students enabling them to judge their progress during a module using a different metric than current models of formative and summative feedback. • Direct involvement with the development of tools that support their learning.
  22. Student Ambassadors

  23. 1 Being the best you can be/Effort (ability to maintain

    effort) 2 Build self-confidence 3 Career/Vocation/Job prospects 4 Industry 5 Giving yourself options 6 Grades/Marks/Qualifications 7 Mastery of a subject/Interest in Subject/Stretch themselves intellectually 8 Mentoring/Family 9 Money 10 Part of a Professional community 11 Self-efficacy/ Helplessness (this might be the opposite of self-efficacy) 12 Sense of connectedness with others with similar goals/ Success as a group of peers 13 Social Prestige/Recognition Initial Identified Motivators for Studying in HE
  24. None
  25. None
  26. Looks how they feel Shows how hard they are trying

    No point in spending £9,000 if you’re not going to try hard and do well A B ↑ ↑ “When thinking about what motivates you to study your degree, which of these do you prefer and why?”. “Why..?” “Why..?” Laddering
  27. 14 Supporting family/Home life 15 Negative imagery 16 Rising above

    circumstances 17 Financial security 18 Controlling own destiny 19 Individuality 20 Opportunity to travel 21 Fear of wasting University opportunity Additional Identified Motivators for Studying in HE from Laddering Sessions
  28. Revised Motivators (21 grouped into 9)

  29. Timeline Formed basis of 6 Focus Groups organised and run

    by Student Ambassadors
  30. None
  31. None
  32. None
  33. None
  34. Professional Theme

  35. Personal/Personified Theme

  36. Personal/Personified Theme

  37. Mobile Optimised

  38. Work in Progress

  39. Learning Analytics in the classroom? Students in lecture hall ©Jirka

    Matousek via Flickr
  40. Dr Ed de Quincey e.de quincey@keele.ac.uk Chris Briggs c.briggs@keele.ac.uk KEELE

    UNIVERSITY SCHOOL OF COMPUTING MOTIVATION METRICS V6 ALL DESIGNS ARE COPYRIGHT 2017 OF MORE THAN JUST DESIGN LIMITED WWW.MORETHANJUSTDESIGN.CO.UK CEO