Wednesday, March 31, 2010

Getting High Quality Results on MTurk

In a previous post, an anonymous commenter left a pretty interesting comment, which I think is worth repeating:

On the subject of higher pay not rendering better work, there are some AMT requesters who pay well, and in the instructions for the task, put in bold print a note that says something along the lines of:

"We pay very well for these tasks, because we expect perfect, error-free work. You don't have to rush through these tasks to make money doing them, so slow down and take your time. Poor quality work WILL be rejected and you WILL be blocked from working for us in the future if your work is poor."

My gut feeling, as a task worker at AMT, is that this is an effective method for increasing the quality of the work (high pay, plus a message reminding the worker to take their time because they don't HAVE to rush through these tasks to make money as an incentive, plus the motivating threat of having work rejected and getting blocked as a disincentive to rushing) but it would be neat to see it officially demonstrated.

Anyone tried this or willing to try? I would be very interesting in hearing the results.