$30 off During Our Annual Pro Sale. View Details »

Towards Building Your Own Autograder in NetsBlox

Brian Broll
December 11, 2020

Towards Building Your Own Autograder in NetsBlox

Autograding is a valuable mechanism for assessment which facilitates scaling a curriculum to a much broader audience. Perhaps the best example can be found in the case of massive open online courses through platforms like Coursera and edX. However, developing autograders for lively, blocks-based programming environments can pose a number of unique challenges. These include the lack of clearly defined start state, often visual program output, and required user-interaction. How can we automatically grade assignments in these sorts of environments? What types of assignments can be graded easily?

Brian Broll

December 11, 2020

More Decks by Brian Broll

Other Decks in Programming


  1. Towards Building Your
    Own Autograder in
    Brian Broll

    View Slide

  2. Autograding in Snap! and NetsBlox
    ● As the name suggests, autograders enable assignments to be automatically graded providing students
    with immediate feedback.
    ● Environments like Scratch, Snap!, and NetsBlox have a unique set of challenges for autograding:
    ○ Lack of clearly defined start state. What if I write a program to draw a square, run it, and then delete all the
    blocks and submit?
    ○ Visual program output. How do I specify invariants easily?
    ○ Required user interaction. What if user interaction is required (like in a game)?
    ○ Diverse, open-ended projects. These environments have wide walls and students are often given a lot of
    freedom to create what is important and meaningful to them. How would this be autograded?
    ○ Required network interaction (and custom protocols). In NetsBlox, student programs may compose a
    distributed application. This results in another dimension of considerations for autograding (esp. when combined
    with the aforementioned challenges).

    View Slide

  3. Building Your Own Autograder
    ● What if we could build our own autograders for Snap! and NetsBlox and enable them via URL
    parameters (like opening projects)?
    ● Autograders could be developed, shared, and remixed alongside curriculum (not tightly coupled with
    any educational platform!)
    ● This could also facilitate MOOCs using Snap! and NetsBlox with more assessment options

    View Slide

  4. Demo

    View Slide

  5. Next Steps?
    ● Feedback
    ○ Give it a try here!
    ○ Note: Since it is still experimental, things may change!
    ● Extension support in Snap! so autograders can be used there, too?
    ● There are many remaining opportunities around autograding:
    ○ Can we grade more than just custom blocks?
    ■ Distributed applications?
    ■ Visual feedback?
    ■ Interactive applications?
    ○ Can we give feedback for partial solutions?
    ○ Should we add support for submitting solutions to platforms like Coursera?

    View Slide

  6. Questions/Comments? Reach out at
    [email protected]
    Try it out at https://dev.netsblox.org
    Thank you!

    View Slide