Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Stephan Krusche 18.05.2021 Chair: Prof. Dr. Tobias Nipkow, TUM Department of Informatics Examiner: Prof. Dr. Bernd Brügge, TUM Department of Informatics Prof. Dr. Maria Bannert, TUM School of Education Habilitation
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Current state of learning in universities 2 0 1000 2000 3000 2013 2014 2015 2016 2017 2018 2019 2020 2.508 2.312 2.208 2.005 1.840 1.580 1.362 1.110 First year students (Informatics TUM) • Large effort for instructors, especially in the correction of exercises and exams • Impossible to interact with each student on an individual level • However: individual feedback is important for the learning experience [Iro07] Year Students [LB64] % unable to express idea size of group never talked had ideas which they did not express 0 12 24 36 48 0 10 20 30 40
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Problems in larger courses 3 No or little involvement Too much focus on lower cognitive skills Learning goals Learning activities Constructive alignment Assessment Misaligned assessments Heterogeneous student groups
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Objectives and research process 4 Teaching philosophy Learning Teaching platform Application in case studies Dissemination Empirical evaluation
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Interactive learning* Definition: Instructors teach and exercise small chunks of content in short cycles using technology. They provide immediate feedback so that learners can reflect on the content and increase their knowledge incrementally. 5 “Tell me and I will forget. Show me and I will remember. Involve me and I will understand. Step back and I will act.” — Chinese Proverb Practice Example Feedback Student Reflection Theory * integrates aspects of active learning [BE91], blended learning [GK04] and experiential learning [Kol84] [KvFA17, KSBB17, KS18]
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Interactive learning (embedded in the syllabus) 6 Topic Topic Topic Topic Topic Course syllabus Practice Example Feedback Student Reflection Theory Learning sprint Knowledge increment Learning gain Learning goal Learning goal Lecture Learning goal ➡ Homework and tutor based exercises further deepen the knowledge (adapted from Scrum [Sch95] and experiential learning [Kol84]) based on constructive alignment [KS19]
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Artemis - interactive learning with individual feedback 7 Programming Modeling Text Quiz Team exercises | Lectures | Presentations | Exam mode | Questions and answers | Learning analytics Scalability: handle > 200 submissions per second Instant feedback: provide feedback in real time Usability: beginners are able to use it
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Automatic assessment of programming exercises 8 Student Version control server 1 submit Continuous integration server 2 notify 3 compile, test & analyze 4 notify student with feedback [KS18]
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Online editor with interactive instructions 10 [KS18] Open source https://github.com/ls1intum/Artemis and free to use on https://artemis.ase.in.tum.de
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Artemis system architecture (v1) 11 Artemis client Local Build Agent Local Build Agent Local build agent University data center Version control server Continuous integration server Artemis server Version control client Remote Build Agent Remote Build Agent Remote build agent User management Student computer LTI Interface
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Artemis - interactive learning with individual feedback 12 Team exercises | Lectures | Presentations | Exam mode | Questions and answers | Learning analytics Scalability: handle > 200 submissions per second Instant feedback: provide feedback in real time Usability: beginners are able to use it Programming Modeling Text Quiz
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Apollon: online modeling editor 13 Open source https://github.com/ls1intum/Apollon and free to use on https://apollon.ase.in.tum.de (without account) [KvFRB20]
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Assessments of UML models 16 Proposed assessment Example solution Grading criteria [KvFRB20]
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Artemis system architecture (v2) 17 Athene Artemis client University data center Version control server Continuous integration server Artemis Server Artemis Server Artemis server Version control client User management Student computer LTI Interface Apollon Compass Broker Discovery Local Build Agent Local Build Agent Local build agent Remote Build Agent Remote Build Agent Remote build agent Load balancer
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Application in case studies 18 Course Short Active students Program Instances Introduction to Software Engineering EIST up to 2,100 Bachelor (2nd sem) SS19 - SS21 Patterns in Software Engineering PSE up to 600 Bachelor + Master WS16/17 - WS20/21 Project Organization and Management POM up to 400 Bachelor + Master SS15 - SS19 MOOC: Software Engineering Essentials SEECx up to 700 Anyone SS17 - WS20/21
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Empirical evaluation: hypotheses H1 Scalability - Interactive learning can be used in large courses with more than 1,500 students participating in exercises at the same time H2 Engagement - Interactive learning increases the participation and motivation of students H3 Learning outcome - Interactive learning improves the learning outcome for students H4 Grading effort and feedback quality - Supervised machine learning reduces the grading effort while improving the feedback quality H5 Adaptability - Interactive learning adapts the difficulty of a course to each individual student by using machine learning 19
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses 78 % 25 % 54 % 55 % 87 % 81 % 16 % 52 % 29 % 69 % 2018 exam (n=1128) 2019 exam (n=1225) H3: Learning outcome 22 1) Functional model 2) Structural model 3) Dynamic model 4) Architecture model 5) Model refactoring Improvement: 26 % (p = 2.2e-16, = 0.01) Improvement: 87 % (p = 2.2e-16, = 0.01) Improvement: 55 % (p = 6.4e-15, = 0.01) [KvFRB20] Exam assignments with UML modeling in EIST Control group Experimental group Results of a 2 sample t-test (1 tailed) Average scores per assignment in the final exams Average scores per assignment in the final exams
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Contributions 24 Practice Example Feedback Student Reflection Theory 1) Interactive learning Team, Lectures, Presentations, Exam mode, Q&A, Analytics Scalability Instant feedback Usability Programming Modeling Text Quiz 2) Artemis 3) Application in case studies • EIST • POM • PSE • SEECx
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Future work 26 Learning analytics Adaptive learning Exam mode Modeling Programming Micro service 1 Micro service 2 Shared database Micro service 3 Micro services and micro frontends
Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Stephan Krusche 18.05.2021 Chair: Prof. Dr. Tobias Nipkow, TUM Department of Informatics Examiner: Prof. Dr. Bernd Brügge, TUM Department of Informatics Prof. Dr. Maria Bannert, TUM School of Education Habilitation Thank you! Artemis Apollon Compass Metis Athene Orion Ares
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses Relevant publications [KSBB17] Krusche, Seitz, Börstler, Bruegge: Interactive Learning: Increasing Student Participation through Shorter Exercise Cycles. ACE 2017. [KvFA17] Krusche, von Frankenberg, Afifi. Experiences of a Software Engineering Course based on Interactive Learning. SEUH 2017. [KBC+17] Krusche, Bruegge, Camilleri, Krinkin, Seitz, Wöbker: Chaordic Learning: A Case Study. ICSE 2017. [KS18] Krusche, Seitz: ArTEMiS: An Automatic Assessment Management System for Interactive Learning. SIGCSE 2018. [KDXB18] Krusche, Dzvonyar, Xu and Bruegge. Software Theater — Teaching Demo Oriented Prototyping. TOCE 2018 [KS19] Krusche, Seitz: Increasing the Interactivity in Software Engineering MOOCs - A Case Study. HICSS 2019. [LKvFB19] Laß, Krusche, von Frankenberg, Bruegge: Stager: Simplifying the Manual Assessment of Programming Exercises. SEUH 2019. [KvFRB20] Krusche, von Frankenberg, Reimer and Bruegge: An Interactive Learning Method to Engage Students in Modeling, ICSE 2020. [BKKB21] Bernius, Kovaleva, Krusche, Bruegge. Towards the Automation of Grading Textual Student Submissions to Open-ended Questions. ECSEE 2020. [BKB21] Bernius, Krusche, Bruegge. A Machine Learning Approach for Suggesting Feedback in Textual Exercises in Large Courses. [email protected] 2021. 29
Habilitation | Stephan Krusche | Interactive learning - A Scalable and Adaptive Learning Approach for Large Courses References [BEF+56] B. Bloom, M. Engelhart, E. Furst, W. Hill, and D. Krathwohl, “Taxonomy of educational objectives: The classification of educational goals,” 1956. [Big03] John Biggs. Aligning teaching and assessing to course objectives. Teaching and learning in higher education: New trends and innovations, 2:13–17, 2003. [PNI+18] Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. Deep contextualized word representations. arXiv preprint arXiv:1802.05365, 2018. [LB64] Harold J Leavitt and Bernard M Bass. Organizational psychology. Annual Review of Psychology, 15(1):371– 398, 1964. [BE91] Charles Bonwell and James Eison. Active Learning: Creating Excitement in the Classroom. ASHE-ERIC Higher Education Reports, 1991. [Kol84] David Kolb. Experiential learning: Experience as the source of learning and development, volume 1. Prentice Hall, 1984. [Sch95] Ken Schwaber. Scrum development process. In Proceedings of the OOPSLA Workshop on Business Object Design and Information, 1995. [GK04] R. Garrison and H. Kanuka. Blended learning: Uncovering its transformative potential in higher education. The internet and higher education, 2004 [Iro07] Alastair Irons. Enhancing learning through formative assessment and feedback. Routledge, 2007. 30
krusche
kitachan_black
kyoun
reisato12345
karakurist
kentaroy47
hiroshinakagawa
yumulab
lndata01
zacky1972
danielkatz
javiergs
addyosmani
carmenintech
reverentgeek
revolveconf
phodgson
morganepeng
wjessup
jmmastey
lauravandoore
mclloyd
jacobian