Outcomes of Using an Infinitely Explorable Online Learning System

A next generation online learning system has been developed at Merrimack College with the goal of revolutionizing online engineering education by “understanding a subject” rather than simply delivering content. The software delivers content in an interactive, three-dimensional environment. When presented with a problem, student input is not constrained in format or to only the final solution. Students have the ability to enter equations, diagrams, graphs, or text related to any portion of the problem including the solution, intermediate steps, or even extraneous aspects of the problem. These entries are evaluated by the software and feedback is provided regarding the correctness of the entry in the context of the presented problem. This grants the student freedom to explore the problem and receive instantaneous feedback as though they were interacting with the instructor in real-time. A number of studies have found that students provided with immediate feedback when working problems are more actively engaged in the learning process and demonstrate greater retention of the information. This feedback can take the form of either standard correct/incorrect responses or an answer-until-correct approach. However, some evidence suggests that the answer-until-correct approach has a more significant impact on information retention. Further, the use of immediate feedback in the setting of the online educational software allows the student to experience a learning environment more consistent with that of individual instruction while not requiring the direct input of the instructor.
In this presentation, results of a pilot study on the effectiveness of this new online learning system will be presented. The specific aims of this study include:
1) Test the effectiveness of using this software in enhancing learning in engineering courses.
2) Evaluate the underlying mechanisms used in the software (active learning, immediate feedback, and an interactive three-dimensional environment).
3) Study the usage scenarios when students are allowed to use the software with little instruction.

Cite: http://dx.doi.org/10.6084/m9.figshare.731683
See: https://www.stemsi.com/

Transcript

1. Outcomes of Using an Infinitely Explorable Online Learning System Franco Capaldi1, Devin R. Berg2 1Department of Civil Engineering, Merrimack College, North Andover, MA 2Engineering & Technology Department, University of Wisconsin – Stout, Menomonie, WI ASEE Annual Conference 26 June 2013 1
2. Students explore problems in a guided manner, promoting critical thinking skills and knowledge retention. 2 = 1 = ⃗ × ⃗ = ⃗ d
3. Problems can be formulated to provide 3D exploration of realistic situations. 3
4. The software provides flexibility through interpretation of student input. 4
5. For example, vector notation can be used to describe position, forces, etc. 5
6. The software provides “word” descriptions of equations. 6 ⃗/ = −25 “The position vector from point D to point A is equal to the negative of the scalar value 25 multiplied by the unit vector i with units of inches”. becomes
7. Student answers can include explanatory text... 7
8. … or figures such as free body diagrams. 8
9. Effectiveness was evaluated through a preliminary study at Merrimack College and UW-Stout. 9 At Merrimack: Two classes -1 treatment (n=12) -1 control (n=7) Evaluated through semester using exams At Stout: Two classes -1 treatment (n=21) -1 control (n=23) Evaluated using pre- and post-FCI
10. Student performance throughout semester was greater for treatment group. 10 Section Initial Assessment Midterm 1 Vectors, Forces Midterm 2 Moments, Free body diagrams Midterm 3 Moment of inertia, Centroids Final Exam Treatment Group (n=12) ± 𝟏 𝟖 ± 𝟏 𝟗𝟗 ± ± 𝟏 𝟖 ± Control Group (n=7) ± 𝟏 𝟕 ± 𝟐 𝟕 ± 𝟏𝟏 𝟕 ± 𝟏𝟏 𝟖 ± 𝟏
11. Students learned to use the software more effectively with time. 11 Homework 1 Homework 2 Homework 3 Percent of total answers which contained program syntax errors 12% 41% 52% Percent of non-syntax errors which were correct / incorrect statements 83%/17% 46%/54% 21%/79% Percent of correct answers which were required for solution 93% 67% 53% Percent of correct answers which were hypotheses 7% 33% 47%
12. The treatment group showed greater overall score improvement on FCI. 12 Treatment (n=21) Control (n=23)
13. Students want intuitive input with fewer restrictions. 13 … and they want the answers!
14. Handling of syntax errors was a significant concern during the semester. 14 Student Entry Category Σ = + 12 sin 27 − Required for solution ∑ = + 12 [N sin(27 [deg]) – FB Program syntax error: ‘12 [N’. 𝑠𝑠 27 = 0.454 Correct hypothesis: ∑ = 𝐹 + 5.448 − Incorrect statement:
15. Room for improvement through… 15 better handling of student input. making use of student training. an adaptive algorithm to enhance student interaction.
16. 16 3D interaction with problems was a popular feature
17. Things that worked well included… 17 testing of intermediate steps for difficult problems. 3D exploration helped with vector and FBD problems. better performance across a wider range of students.