Exploring the Efficacy of Learning Analytics Dashboards for Metacognitive Activity Support in Self-Learning

和訳: 自己調整学習におけるメタ認知的活動を支援するための学習分析ダッシュボードの有効性に関する検討

Transcript

1. Exploring the Efficacy of Learning Analytics Dashboards for Metacognitive Activity Support in Self-Learning Koki Saitoh, Chiemi Watanabe, and Kouhei Kikuchi Tsukuba University of Technology 1
2. Introduction 2
3. Issue of MOOCs Self-learning through MOOCs leads to dropout. Massive Open Online Courses (MOOCs) have become widespread. But Dropout rates over 90% [1]. [1] Bezerra, L.N., Silva, M.T.: A review of literature on the reasons that cause the high dropout rates in the moocs. Revista Espacios 38(05) (2017) 3
4. Reason of the issue Difficulty of appropriate metacognition. Self-Regulated Learning (SRL): Learners actively regulating their own learning through metacognition, motivation, and behavior [2]. Performance (monitor and control process) Model of SRL cycle Forethought (set goals, make a plan) Monitor, control, evaluate are hard for me! Self-reflection (evaluate performance) [2] Zimmerman, B.J.: Self-regulated learning and academic achievement: An overview. Educational psychologist 25(1), 3–17 (1990) 4
5. Purpose Support for better self-learning and SRL. No support for metacognitive activities Support success for metacognitive activities Monitor, control, evaluate are hard for me! I can do monitor, control, and evaluate my learning! Learning Analytics Dashboard Learning Analytics Dashboard ort supp 5
6. Research questions RQ1: How does our LAD affect learners’ metacognitive activities in self-learning environments? RQ2: Under what conditions can LADs provide effective support for metacognitive control? RQ3: What are the key factors that determine the effectiveness of LADs in supporting self-regulated learning? 6
7. LAD-Based SRL Support System 7
8. Support system Our system is based on the learning process as below. Learners watch lecture video by video viewing platform Learners take a quiz Learners reflection with a LAD 8
9. Video viewing platform Learner can view videos. These controls enable learner to them. • Play • Pause • Skip • Seek They are recorded with timestamps. 9
10. Learning analytics dashboard LAD for visualised learning behavior. I can recognise my own unconscious learning behaviors! 10
11. Operation Count by Type Visualize learner’s unconscious learning behavior. I can recognise my own unconscious behavior while viewing videos! 11
12. Operation Count by Time Interval Visualize learner’s focus areas in the video. I can understand which parts of the video I found challenging! 12
13. Estimated Learning Behavior Type Suggest the learner’s learning strategy type. I can reflect about my learning style and how effective my learning! 13
14. Quiz Results Show the learner’s outcomes. I can evaluate my learning process and outcomes! 14
15. Methodology 15
16. Experimental flow 1 month experiment with meta-activities scores. 6 participants, 1 month, 4 sessions per participant. Participants (learners) Experimenters Answering questionnaire 4 session of learn phrase Comparing pre- and post-questionnaire Analyze interviews Answering questionnaire Provide an interview 16
17. Questionaire Metacognitive Awareness Inventory (MAI) [3]. • Selected 22 items from MAI’s 55-item questionnaire. • Modified original a two-point scale to a four-point scale. Try to find change of metacognitive activities easier. [3] Schraw, Gregory, and Rayne Sperling Dennison. "Assessing metacognitive awareness." Contemporary educational psychology 19.4 (1994): 460-475. 17
18. interviews Exploring the reasons behind the data. 1. What changes, if any, happened in your learning throughout the research period, and what were the reasons for these changes? 2. For metacognitive activities where large differences were observed between pre- and post-scores, how do you perceive these changes? 3. How did you utilize the LAD? 18
19. Results 19
20. Result of the questionnaire score 3 types of score changes. Type 1: Scores is small change Type 2: Score is 14-point decrease Type 3: Score is 10-point increase Pre- and Post-self-assessment questionnaire score 20
21. Type 1 The learning content was not challenging enough. The 4 participants divided into 2 sub-types. Minor change in all items Polarized changes in items “I see, result of my quiz is good… And then?” Increase items: Goal setting and reflection Decrease items: Debugging stragetegies and monitoring 21
22. Type 2 The learning content was too difficult, causing give up. Content was too hard, lost motivation. Almost gave up, too difficult. “The content was difficult for me, and I couldn’t feel motivated to understand it.” “The content was so difficult that I partially gave up.” 22
23. Type 3 LAD triggered metacognitive awareness. LAD prompted reflection on methods. Good content motivated self-summary. “Looking at LAD and reflecting made me think about whether other learning methods were possible.” “The lecture video was good, so I wanted to summarize it myself. This was unrelated to LAD.” 23
24. Discussion 24
25. Discussion of type 1 LAD demotivates the SRL cycle on too easy. Performance: “Easily solved and finished” “Nothing to think about” Forethought: “not need plan revision” “not need goal revision” Self-reflection: “There isn’t problem” “It was easy” 25
26. Discussion of type 2 LAD failed to support the SRL cycle on too difficult. Performance: “I can’t understand about the learn content” Forethought: “I don’t know how to setting goal and plan” Self-reflection: “I couldn’t all things” “It was too difficult” 26
27. Discussion of type 3 LAD drives the SRL cycle with optimal challenge. Performance: “I try to how other solution” “I let solve to archive the goal” Forethought: “I set higher goals than previous learning” Self-reflection: “I felt motivated to find an easier solution” 27
28. Summary of discussion Optimal challenge is key to LAD's success. High Optimal challenge, succeed support Effectiveness of LAD Support Too easy, fail support Low easy Too difficult, fail support Challenge Level difficult 28
29. Conclusion and Future Work 29
30. Contribution of this Study Not just data, optimal challenge drives self-learn. RQ1: How does our LAD affect learners’ metacognitive activities in self-learning environments? Its effect is conditional on the task level. RQ2: Under what conditions can LADs provide effective support for metacognitive control? Only under conditions of optimal challenge. RQ3: What are the key factors that determine the effectiveness of LADs? The most important factor is the level of challenge. 30
31. Future work Need validation study and adaptive support. Future work 1: We need to investigate by a large-scale validation study. Future work 2: We need to change from static data to adaptive support. This is all for my presentation. Thank you so much for your attention. 31