Getting High Quality Results on MTurk
In a previous post, an anonymous commenter left a pretty interesting comment, which I think is worth repeating:
On the subject of higher pay not rendering better work, there are some AMT requesters who pay well, and in the instructions for the task, put in bold print a note that says something along the lines of:
"We pay very well for these tasks, because we expect perfect, error-free work. You don't have to rush through these tasks to make money doing them, so slow down and take your time. Poor quality work WILL be rejected and you WILL be blocked from working for us in the future if your work is poor."
My gut feeling, as a task worker at AMT, is that this is an effective method for increasing the quality of the work (high pay, plus a message reminding the worker to take their time because they don't HAVE to rush through these tasks to make money as an incentive, plus the motivating threat of having work rejected and getting blocked as a disincentive to rushing) but it would be neat to see it officially demonstrated.
Anyone tried this or willing to try? I would be very interesting in hearing the results.
As an Mturk requestor and occasional worker, I've found that I get the best results by putting a lot of work into creating tasks that are clearly explained with well designed forms. As an occasional worker, I'm shocked by the poor quality of assignments. Small things such as making the iframe large enough to avoid iframe scrolling would make a big time difference for people working for pennies per task. Efficient tasks that are priced right get done and done right.
ReplyDelete