Towards Building Your Own Autograder in NetsBlox

Autograding is a valuable mechanism for assessment which facilitates scaling a curriculum to a much broader audience. Perhaps the best example can be found in the case of massive open online courses through platforms like Coursera and edX. However, developing autograders for lively, blocks-based programming environments can pose a number of unique challenges. These include the lack of clearly defined start state, often visual program output, and required user-interaction. How can we automatically grade assignments in these sorts of environments? What types of assignments can be graded easily?

Transcript

1. Towards Building Your Own Autograder in NetsBlox Brian Broll
2. Autograding in Snap! and NetsBlox • As the name suggests, autograders enable assignments to be automatically graded providing students with immediate feedback. • Environments like Scratch, Snap!, and NetsBlox have a unique set of challenges for autograding: ◦ Lack of clearly defined start state. What if I write a program to draw a square, run it, and then delete all the blocks and submit? ◦ Visual program output. How do I specify invariants easily? ◦ Required user interaction. What if user interaction is required (like in a game)? ◦ Diverse, open-ended projects. These environments have wide walls and students are often given a lot of freedom to create what is important and meaningful to them. How would this be autograded? ◦ Required network interaction (and custom protocols). In NetsBlox, student programs may compose a distributed application. This results in another dimension of considerations for autograding (esp. when combined with the aforementioned challenges).
3. Building Your Own Autograder • What if we could build our own autograders for Snap! and NetsBlox and enable them via URL parameters (like opening projects)? • Autograders could be developed, shared, and remixed alongside curriculum (not tightly coupled with any educational platform!) • This could also facilitate MOOCs using Snap! and NetsBlox with more assessment options
4. Demo
5. Next Steps? • Feedback ◦ Give it a try here! ◦ Note: Since it is still experimental, things may change! • Extension support in Snap! so autograders can be used there, too? • There are many remaining opportunities around autograding: ◦ Can we grade more than just custom blocks? ▪ Distributed applications? ▪ Visual feedback? ▪ Interactive applications? ◦ Can we give feedback for partial solutions? ◦ Should we add support for submitting solutions to platforms like Coursera?
6. Questions/Comments? Reach out at brian.broll@vanderbilt.edu Try it out at https://dev.netsblox.org Thank you!