ConCodeIt! A Comparison of Concurrency Interfaces in Block- based Visual Robot Programming The 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), 2020 Michael Jae-Yoon
 Chung Maya Cakmak Elana Hummel Sai Harshita
 Neti Mino Natura Anthony Lu 1

Source: Mashable, 10/07/2017 Source: The Independent, 7/16/2020 2

CustomPrograms,
 Huang et al. HRI 2016 CoBlox, Weintrop et al. CHI 2018 Wizard Easy Programming (Credit: ABB) Cozmo Code Lab (Source: THE VERGE) 3

What is Concurrency? gesture gaze Hello… speech screen 4

Expressing Concurrency “Start sound AND shimmy in PARALLEL” “Go to ‘Lobby 1’ WHILE waiting for a screen touch event” 5

Research Question “How should we design a concurrency interface for block-based visual programming for interactive robots?”
 6

Robot API • Robot actions as function calls • Designed to achieve use cases Block-Based Visual Robot Programming + Blockly • An open source visual programming interface like Scratch • Wraps a procedural 
 scripting language like Python Adopted from CustomPrograms, Huang et al. HRI 2016 7

8

Robot APIs: Actions Name say(string text) gesture(string expression) displayText(string text, number duration) displayButtons(string[] choices, number duration) sleep(number duration) Name startSaying(string text) startGesture(string expression) startDisplayText(string text, number duration) startDisplayButtons(string[] choices, number duration) startSleep(number duration) Blocking Non-Blocking 9

Robot APIs: Events Event Name Value Type speechDetected string buttonPressed string Event Name Value Type sayDone boolean gestureDone boolean displayTextDone boolean displayButtonDone boolean sleepDone boolean Externally
 Triggered Action
 Triggered // register a callback when(
 “speechDetected”,
 function() {...}); 
 // block until event waitFor(“speechDetected”); 
 // read current state value getLastEventValue(
 “speechDetected"); 10

Three Block-based Concurrency Interfaces Async • asynchronous procedure calls • imperative
 paradigm • e.g., socket programming Callback • callback
 • event-driven paradigm • e.g., UI programming Waitfor • promise-based event synchronization • imperative 
 paradigm • e.g., async/await pattern 11

Three Block-based Concurrency Interfaces Async • asynchronous procedure calls • imperative
 paradigm • e.g., socket programming Callback • callback
 • event-driven paradigm • e.g., UI programming Waitfor • promise-based event synchronization • imperative 
 paradigm • e.g., async/await pattern 12

Three Block-based Concurrency Interfaces Async • asynchronous procedure calls • imperative
 paradigm • e.g., socket programming Callback • callback
 • event-driven paradigm • e.g., UI programming Waitfor • promise-based event synchronization • imperative 
 paradigm • e.g., async/await pattern 13

Three Block-based Concurrency Interfaces Async • asynchronous procedure calls • imperative
 paradigm • e.g., socket programming Callback • callback
 • event-driven paradigm • e.g., UI programming Waitfor • promise-based event synchronization • imperative 
 paradigm • e.g., async/await pattern 14

Example Robot Behavior Step 1: The robot should 
 say “Hello there, my name is Meebo. Goodbye now!” 
 and wait for the human to say something.
 Step 2: When the robot finished speaking 
 or the human says something,
 the robot should display “On standby.” 15

Async Example Robot Behavior
 
 Step 1: The robot should 
 say “Hello there, my name is Meebo. Goodbye now!” 
 and wait for the human to say something.
 
 Step 2: When the robot finished speaking 
 or the human says something,
 the robot should display “On standby.” 16

Callback Example Robot Behavior
 
 Step 1: The robot should 
 say “Hello there, my name is Meebo. Goodbye now!” 
 and wait for the human to say something.
 
 Step 2: When the robot finished speaking 
 or the human says something,
 the robot should display “On standby.” 17

WaitFor Example Robot Behavior
 
 Step 1: The robot should 
 say “Hello there, my name is Meebo. Goodbye now!” 
 and wait for the human to say something.
 
 Step 2: When the robot finished speaking 
 or the human says something,
 the robot should display “On standby.” 18

Systematic Analysis • Goal: to compare conciseness • Procedure: expert
 implemented
 6 unit behaviors rep.
 concur. patterns • Measure: 
 conciseness via
 6 block counts 0 5 10 15 20 25 30 Average Number of Blocks async callback waitfor 0 1 2 3 Average Number of Functions async callback waitfor 0 1 2 3 Average Number of Variables async callback waitfor 0 1 2 3 Average Number of Branches async callback waitfor 0 1 2 3 Average Number of Loops async callback waitfor 0 1 2 3 Average Number of Conditions async callback waitfor 6 Block Count Measures Across 6 Unit Behaviors most concise 19

User Study • Between-Group design • Study duration: max. 90 min • Participants (N = 21, 14 F) • with some programming experiences • took intro CS, no robotics • Conducted Online “hello” “Bye” Online User Study Web UI 20

Task Description • Step 1: On start, the robot should
 display text “Press or say ‘start’ to begin interaction”
 and display a button with “start” text. • Step 2: If the user presses the button or says “start,” 
 then the robot should introduce itself by 
 saying “Hello, my name is Meebo”
 and make a “happy” gesture. • Step 3: … • Step 4: … 21

Concurrency Patterns Perform two actions in parallel say(“Hello I'm robot”) gesture(WAVE) next function Wait for all functions to return time Perform an action while monitoring an event goto(ROOM145) detect(BUTTON_PRESSED) next function Wait for one function to return time Monitor two events in parallel detect(BUTTON_PRESSED) next function Wait for one function to return time detect(SPEECH_COMMAND) 22

Results 0 20 40 60 80 100 120 Average Number of Blocks async callback waitfor -2 0 2 4 6 8 10 12 async callback waitfor -2 0 2 4 6 8 10 12 Average Number of Branches async callback waitfor -2 0 2 4 6 8 10 12 async callback waitfor counted # of blocks 0 5 10 15 Average Score async callback waitfor scored using a rubric most concise most correct 23

Results 0 20 40 60 80 100 120 Average Number of Blocks async callback waitfor -2 0 2 4 6 8 10 12 async callback waitfor -2 0 2 4 6 8 10 12 Average Number of Branches async callback waitfor -2 0 2 4 6 8 10 12 async callback waitfor counted # of blocks 0 5 10 15 Average Score async callback waitfor scored using a rubric why? 24

Example “Callback” User Program A loop for checking a change introduce change 25

Summary 1. Robot API designed with concurrency in mind 2. Three representative interfaces: • asynch, callback, waitfor 3. Comparative evaluation • systematic analysis • user study https://github.com/mjyc/concodeit {mjyc,mcakmak}@cs.washington.edu 26