Justin Cheng [email protected] 1 A Comparison of Macro- and Microtasks Jaime Teevan [email protected] Shamsi Iqbal [email protected] Michael Bernstein [email protected] Break it Down 

A Comparison of Macro- and Microtasks 

A Comparison of Macro- and Microtasks Picking the best photos for an album 

A Comparison of Macro- and Microtasks Large 

A Comparison of Macro- and Microtasks Monolithic 

A Comparison of Macro- and Microtasks Overwhelming 

A Comparison of Macro- and Microtasks Pairwise comparisons of photos 

A Comparison of Macro- and Microtasks Smaller 

A Comparison of Macro- and Microtasks Done in seconds 

A Comparison of Macro- and Microtasks Done independently 

A Comparison of Macro- and Microtasks Transcribing a speech Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit. 

A Comparison of Macro- and Microtasks Transcribing individual sentences Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit.Lorem ipsum dolor sit amet, consectetur adipiscing elit. Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet. Transcribing a speech 

Are macrotasks or microtasks better? 

How do macrotasks and microtasks differ? 

People already break tasks down. 

Tasks are perceived in segments. Zacks, J. M., Tversky, B., and Iyer, G. Perceiving,remembering, and communicating structure in events. In Exp. Psychol. Gen. 2001. Wickens, C. D. Multiple resources and performance prediction. In Theor. Issues Ergon. 2002. = 

Many tasks are done in short bursts. Gonzalez, V. M., and Mark, G. Constant, constant,multi-tasking craziness: managing multiple working spheres. In CHI 2004. 

They are constantly interrupted. Gonzalez, V. M., and Mark, G. Constant, constant,multi-tasking craziness: managing multiple working spheres. In CHI 2004. Hey, quick question… Have a minute? 

Microtasking is prevalent in crowdsourcing. 

Microtasking can help us be productive in spare moments of time. Vaish, R., et al. Twitch crowdsourcing: crowd contributions in short bursts of time. In CHI 2014. 

How do macrotasks and microtasks differ in terms of…? 

How do macrotasks and microtasks differ in terms of time taken? 

How do macrotasks and microtasks differ in terms of work quality? 

How do macrotasks and microtasks differ in terms of interruptibility? 

How do macrotasks and microtasks differ in terms of worker preference? 

× Task Format (Macrotask/Microtask) Presence of Interruptions (Interruptions/No Interruptions) Arithmetic Sorting Transcription 3 Tasks 110 Participants on Clickworker (+MTurk) 

Arithmetic Macrotask and Microtask 

Sorting Macrotask and Microtask 

Sorting Macrotask and Microtask 

Transcription Macrotask and Microtask 

How do macrotasks and microtasks differ in terms of time taken? 

Microtasks generally take longer Task Time (s) 0 50 100 150 200 Arithmetic Sorting Transcription Macrotask Microtask p<0.001 

How do macrotasks and microtasks differ in terms of work quality? 

Microtasks result in fewer errors Error Rate 0 0.1 0.2 0.3 0.4 Arithmetic Sorting Transcription Macrotask Microtask p<0.001 

take more time fewer errors take less time more errors Macrotasks Microtasks … … 

How do macrotasks and microtasks differ in terms of interruptibility? 

Macrotasks more susceptible to interruptions Task Time w/o Interruptions 0 20 40 60 80 100 120 140 Uninterrupted Interrupted Macrotask Microtask p<0.10 

Microtasks less susceptible to interruptions Task Time w/o Interruptions 0 20 40 60 80 100 120 140 Uninterrupted Interrupted Macrotask Microtask p<0.10 

How do macrotasks and microtasks differ in terms of worker preference? 

77% of workers preferred microtasks to macrotasks. 

take more time fewer errors take less time more errors unaffected by interruptions preferred more slowed by interruptions preferred less Macrotasks Microtasks 

Information workers can benefit from microtasking. (Can we write research papers on our phones?) 

Information workers can benefit from microtasking. (Can turkers benefit from macrotasking?) 

44 Break it Down A Comparison of Macro- and Microtasks Justin Cheng [email protected] Jaime Teevan [email protected] Shamsi Iqbal [email protected] Michael Bernstein [email protected]