NetsBlox: Bringing the Full Power of the Internet to Snap!

NetsBlox is an extension of Snap! that introduces two new concepts for developing distributed applications: RPCs and message passing. These two concepts open up a plethora of possibilities accessible to beginners. RPCs can be used to access existing web services like Google Maps, Twitter, ThingSpeak, and The MovieDB or real-world datasets including climate, eclipse, and COVID data. Message passing enables students to build engaging social applications such as chat applications, shared whiteboards, multiplayer games, and many more.

This environment empowers even novice programmers to get hands-on experience with many advanced topics such as distributed computing, machine learning, cybersecurity, and the Internet of Things. Using NetsBlox, students have built their own mesh networks, encrypted chat applications, and programs for breaking encrypted messages. They have designed programs to analyze tweets from Twitter and even British periodicals from the 1800s. The very same abstractions also enable NetsBlox programs to connect to WiFi enabled devices such as educational robots or smartphones. The NetsBlox program runs in the browser and controls these devices remotely enabling students to create non-trivial distributed applications. In the course of a summer camp, students have even been able to not only program controllers to drive their own robots using their phone but hack and defend these robots from cyberattacks.

In this talk we will present an overview of NetsBlox as well as some of the newest features including:

Accessing your phone’s sensors from your project and placing interactive GUI elements on the screen. This enables applications like 1) an exercise tracker showing your route on a map and displaying the distance covered and your speed or 2) turning your phone into a remote sprite controller.
A Virtual Reality environment where students get virtual robots that they can control with their NetsBlox programs. They can collaborate on tasks in this shared world or compete against each other.
Create and deploy your own Alexa skill while writing it entirely in NetsBlox.
Create a suite of auto-graded assignments within NetsBlox itself.
Make your own data available to any other user of NetsBlox anywhere in the world instantly.

Slides for the keynote at SnapCon 2021. More information available here: https://www.snapcon.org/conferences/2021/program/proposals/225

Transcript

1. NETSBLOX: BRINGING THE FULL POWER OF THE INTERNET TO SNAP! Brian Broll and Akos Ledeczi Vanderbilt University
2. WHAT DO THESE HAVE IN COMMON?
3. DO BLOCK-BASED ENVIRONMENTS HAVE HIGH CEILINGS? YES, BUT NOTICE THE WALLS…
4. LET’S TEAR DOWN THE WALLS
5. CRACKS IN THE WALL • http get to access web resources • Access to hardware devices • Cloud variables • Machine learning libraries • Collaboration support • etc There is no uniform support to accessing resources on the internet and communication across projects…
6. WIDENING THE WALLS: LIBRARIES AND EXTENSIONS
7. THREE NEW BLOCKS
8. TWO NEW (OLD) ABSTRACTIONS • Remote Procedure Call (RPC) • Similar to custom blocks, but the code runs on the server • Self-documenting • Similar RPCs are grouped into Services • Much more than simply forwarding to web APIs • Message Passing • Similar to events • They can carry data • They can go to other projects running anywhere on the internet • They are typed: name and list of named fields • Flexible addressing
9. RPC DEMOS Earthquake Visualization Cast of Any Movie COVID-19 Data
10. MAKE YOUR OWN SERVICE • The "ServiceCreation" service provides RPCs for creating your own NetsBlox service from a CSV or JSON file! • RPCs are inferred from the data automatically but can also be explicitly defined using anonymous functions • This enables the creation of local, relevant data services on NetsBlox! • A tutorial can be found at https://editor.netsblox.org/?action =present&Username=brian&ProjectN ame=ServiceCreationTutorial (check out all the sprites!)
11. MESSAGE PASSING DEMOS 2-User Shared Whiteboard
12. NETSBLOX ARCHITECTURE
13. YOUR PHONE AS A SENSOR • PhoneIoT, an iOS and Android app, provides access to all phone sensors for NetsBlox • Get value (RPC) or turn on streaming (messages) • NetsBlox programs can also place GUI widgets on the phone • GUI events are NetsBlox messages • Can be used to teach IoT and distributed computation • Not a mobile app development environment! • Check out this video: https://tinyurl.com/phoneiot Exercise Tracker (phone display) Plot 3-axis acceleration (stage) Devin Jean
14. ROBOTICS WITH A TWIST • RPCs and messages can be used to control WiFi-enabled robots • Code runs in the browser (no wires, no downloads) • Supports remote robotics out of the box • Wireless communication can be intercepted by other students motivating cybersecurity • Has been used to teach robotics and cybersecurity • Encryption, secure key exchange, denial of service attacks, replay attacks, etc. • Multiple summer camps https://stemforall2019.videohall.com/presentations/1343
15. VIRTUAL ROBOTICS • To enable remote learning, we have created a Unity-based 3D virtual robotics environment. • Same interface, same code as physical robots. • No hardware cost, no maintenance. • Students can remotely access shared space to collaborate or compete solving challenge problems • Can support a wide range of robots and environments. • Extends beyond robotics. • Accessible via VR headsets. • 3 Summer Camps in June 2021 • Final project: drive your robot with your phone https://tinyurl.com/bhkz2ed9 Clear the table Traffic light control Gordon Stein
16. ROBOT CONTROL WITH PHONE PhoneIoT on Mobile Device NetsBlox Client on Computer/Browser RoboScape Online Unity VR Environment in the Cloud NetsBlox Server
17. ALEXA SKILL CREATION • Voice assistants are ubiquitous and can now be created entirely within NetsBlox! • Users can define custom intents leveraging the many NetsBlox services such as robotics, message passing, and cloud variables!
18. AUTOGRADERS • Autograders can be defined entirely in NetsBlox for NetsBlox and shared with students as a single URL • This facilitates the use of NetsBlox for structured activities such as in independent study, MOOCs, and for bell ringers • Creating autograders in NetsBlox makes it easy to share, reuse, and remix autograded activities • A curated set of exercises makes it simple to compose your own autograded activities! https://tinyurl.com/netsblox-exercises
19. CONCLUSIONS • Opening the internet creates a rich set of possibilities: • Access to online data and services • Distributed programs: social apps, multi player games, etc. • Support for a wide variety of devices including robots, phones and voice assistants • Real-time collaboration, remote learning • User extendibility • Students can create projects that interests them and are relevant to their lives → Engagement • A small set of powerful abstractions supported by a handful of simple blocks keeps the learning curve gentle: • Adding support for phones or robots requried no changes on the client and no new blocks! Supported by
20. CONTRIBUTORS • Graduate students: • Gordon Stein, Devin Jean, Hamid Zare, Ben Yett • Vanderbilt Faculty and Research Staff: • Peter Volgyesi, Miklos Maroti, Janos Sallai, Corey Brady, Cliff Anderson, Ole Molvig, Chris Vanags, Gautam Biswas • Collaborators: • Shuchi Grover, Tiffany Barnes, Veronica Catete, Marnie Hill, Dan Garcia, Jens Monig • Countless undergraduate students, summer interns, high school teachers and students Supported by