Tuesday, February 3, 2009

Duration of Tasks on Mechanical Turk

OK, time for some more statistics about Mechanical Turk. I was checking how long it takes for different tasks posted on Mechanical Turk to be completed. So, I checked the tasks posted over the last month, and I checked for how long these tasks have been available.

Plotting duration of tasks in hours, against the total number of HITs that took that that many hours to be completed. What was the resulting distribution? The usual suspect! A power law! Here is the plot:

Here is also a dynamic visualization, using the Google Visualuation API (so that it gets updated dynamically, as I gather more data):

(Since most readers do not supports IFRAMEs, those of you reading the post using a reader will need to visit the blog post to see the scatterplot.)

Interestingly enough, I observed the same behavior for individual HITs as well: while many of them are being completed in a short period of time, some of them tend to take a very significant period of time. Not sure what exactly causes the long tails in this process but seems to be something worth investigating.