Crowdsourcing Performance Evaluations of User Interfaces

Can computer users be trusted to paricipate in user interface studies from the comfort of their home? Can user interface researchers give up control over their subjects' environment? In this project we study whether we can use Amazon Mechanical Turk to conduct user interface studies reliably. To do so, we replicated three previously known performance experiments, the "Bubble Cursor," the "Split Menus," and the "Split Interface," both in our lab and on Mechanical Turk. We compared the lab with the online population in terms of performance metrics such as speed, accuracy, and consistency. The results show that the Mechanical Turk participants perform just as well as the lab participants.