Towards Building Your Own Autograder in NetsBlox

Autograding is a valuable mechanism for assessment which facilitates scaling a curriculum to a much broader audience. Perhaps the best example can be found in the case of massive open online courses through platforms like Coursera and edX. However, developing autograders for lively, blocks-based programming environments can pose a number of unique challenges. These include the lack of clearly defined start state, often visual program output, and required user-interaction. How can we automatically grade assignments in these sorts of environments? What types of assignments can be graded easily?

Transcript

1. Towards Building Your
   Own Autograder in
   NetsBlox
   Brian Broll
   

   View Slide

2. Autograding in Snap! and NetsBlox
   ● As the name suggests, autograders enable assignments to be automatically graded providing students
   with immediate feedback.
   ● Environments like Scratch, Snap!, and NetsBlox have a unique set of challenges for autograding:
   ○ Lack of clearly defined start state. What if I write a program to draw a square, run it, and then delete all the
   blocks and submit?
   ○ Visual program output. How do I specify invariants easily?
   ○ Required user interaction. What if user interaction is required (like in a game)?
   ○ Diverse, open-ended projects. These environments have wide walls and students are often given a lot of
   freedom to create what is important and meaningful to them. How would this be autograded?
   ○ Required network interaction (and custom protocols). In NetsBlox, student programs may compose a
   distributed application. This results in another dimension of considerations for autograding (esp. when combined
   with the aforementioned challenges).
   

   View Slide

3. Building Your Own Autograder
   ● What if we could build our own autograders for Snap! and NetsBlox and enable them via URL
   parameters (like opening projects)?
   ● Autograders could be developed, shared, and remixed alongside curriculum (not tightly coupled with
   any educational platform!)
   ● This could also facilitate MOOCs using Snap! and NetsBlox with more assessment options
   

   View Slide

4. Demo
   

   View Slide

5. Next Steps?
   ● Feedback
   ○ Give it a try here!
   ○ Note: Since it is still experimental, things may change!
   ● Extension support in Snap! so autograders can be used there, too?
   ● There are many remaining opportunities around autograding:
   ○ Can we grade more than just custom blocks?
   ■ Distributed applications?
   ■ Visual feedback?
   ■ Interactive applications?
   ○ Can we give feedback for partial solutions?
   ○ Should we add support for submitting solutions to platforms like Coursera?
   

   View Slide

6. Questions/Comments? Reach out at
   brian.broll@vanderbilt.edu
   Try it out at https://dev.netsblox.org
   Thank you!
   

   View Slide