Autograding is a valuable mechanism for assessment which facilitates scaling a curriculum to a much broader audience. Perhaps the best example can be found in the case of massive open online courses through platforms like Coursera and edX. However, developing autograders for lively, blocks-based programming environments can pose a number of unique challenges. These include the lack of clearly defined start state, often visual program output, and required user-interaction. How can we automatically grade assignments in these sorts of environments? What types of assignments can be graded easily?
Towards Building Your
Own Autograder in
Autograding in Snap! and NetsBlox
● As the name suggests, autograders enable assignments to be automatically graded providing students
with immediate feedback.
● Environments like Scratch, Snap!, and NetsBlox have a unique set of challenges for autograding:
○ Lack of clearly defined start state. What if I write a program to draw a square, run it, and then delete all the
blocks and submit?
○ Visual program output. How do I specify invariants easily?
○ Required user interaction. What if user interaction is required (like in a game)?
○ Diverse, open-ended projects. These environments have wide walls and students are often given a lot of
freedom to create what is important and meaningful to them. How would this be autograded?
○ Required network interaction (and custom protocols). In NetsBlox, student programs may compose a
distributed application. This results in another dimension of considerations for autograding (esp. when combined
with the aforementioned challenges).
Building Your Own Autograder
● What if we could build our own autograders for Snap! and NetsBlox and enable them via URL
parameters (like opening projects)?
● Autograders could be developed, shared, and remixed alongside curriculum (not tightly coupled with
any educational platform!)
● This could also facilitate MOOCs using Snap! and NetsBlox with more assessment options
○ Give it a try here!
○ Note: Since it is still experimental, things may change!
● Extension support in Snap! so autograders can be used there, too?
● There are many remaining opportunities around autograding:
○ Can we grade more than just custom blocks?
■ Distributed applications?
■ Visual feedback?
■ Interactive applications?
○ Can we give feedback for partial solutions?
○ Should we add support for submitting solutions to platforms like Coursera?
Questions/Comments? Reach out at
Try it out at https://dev.netsblox.org