Sunday, February 1, 2009

Monitoring the Dynamics of Mechanical Turk

Every time that I post a task on Mechanical Turk, I have the same experience: Turkers start working very fast on the posted HITs, and it seems that the task will be completed in a few hours. Then, as time progresses, the rate slows down.

So, I have been wondering why this happens? Did someone else post many HITs, burying my own? Does it make sense to post my HITs at some specific day or time, when Turkers work a lot and there are only a few competing HITs?

To answer this question, I started collecting data from Mechanical Turk, so that I can examine the dynamics of the system. As a first outcome of this effort, I built a small dashboard that shows the number of posted projects, the number of available HITs (a project may have many HITs), and the total amount of rewards that is available on MTurk.

As a side note, it was a nice opportunity for me to actually write some code. I built the dashboard using the Google Visualization API (pretty cool!). Now I am learning about exporting data sets using the Google Data Source API's, which will allow for easy embedding of the generated charts and will allow third parties to get direct access to my data.

Since this is the very first version of the dashboard, I welcome any comments. What else you would like to see? Is there something that you do not like? Let me know in the comments!