Tuesday, September 30, 2008

Sarah Palin and Markov Models

How good are n-gram Markov models for language modeling?

Apparently pretty good for modeling the responses of Sarah Palin during her last couple of interviews! Check them out:

http://interviewpalin.com/

http://palinspeak.com/

Friday, September 12, 2008

How Much Turking Pays?

After reporting the results about "why Turkers Turk," I received a set of questions about further things that people would like to know about the Turkers. One of the most common questions was about the compensation of Turkers: "How much do they make by Turking?"

Well, there is no question about Mechanical Turk, that Mechanical Turk cannot answer, so here we go. I posted the very same question on MTurk, asking people about their average compensation per week. Without further ado, here are the results:

A small number of people make more than $100 per week, about 20% make more than$20 per week, and the majority get less than \$20. So indeed it seems unlikely that people work on MTurk for a living.

It is much more likely that people actually enjoy what they are doing, and getting some cash is a nice side-effect. Furthermore, it is work that can be done even while working, and doing the tasks on MTurk helps other people. My own gut feeling is that the research about the motivations that drive people to contribute to open source projects can also be applied here to explain why Turkers turk.

(Info: The current survey paid 5 cents per HIT, and received responses for 200 Turkers. I will keep running the survey to collect 1,000 responses and will report if I see any significant changes. But so far the results seem remarkably stable.)

Thursday, September 11, 2008

Why People Participate on Mechanical Turk, Now Tabulated

A few months back, I decided to ask Turkers about their motivation for participating on Mechanical Turk. I found their responses quite fascinating, so I decided to list them in their raw format, without any further tabulation and processing.

However, as the time passed, I realized that I wanted to have the results in a more summarized and accessible format. Therefore, I bit the bullet and organize the results. Of course, I had no time for such a big task. So, what to do? First, I hired two coders using RentACoder.com, to read and identify the main reasons listed in the responses. The two coders agreed on 9 broad categories:

A. To Kill Time
B. Fruitful way to spend free time (Instead of watching TV, Not to waste time, Rather than playing video games/online games, Sense of purpose when watching TV, Something to do during downtime in work)
C. Income purposes (Gas, Bills, Make money, Credit card, Groceries, School, Help family)
E. For entertainment, for fun, interesting, addiction
F. Challenge, self-competition
G. Unemployed, no regular job, s part-time job
H. To sharpen/ To keep mind sharp
I. Learn English

Then, I simply listed the responses on Mechanical Turk, and asked (new) Turkers to identify the category (or categories) for each response. Here are the percentages for each category (note that one response can be classified into multiple categories):

and the actual percentages:

 A. 20.50% B. 14.00% C. 49.00% D. 34.00% E. 42.00% F. 5.50% G. 3.50% H. 3.50% I. 4.00%

So, we can see that many Turkers complete such tasks to get some extra cash and pay for gas (may be we should wish for high oil prices :-) but there is a significant fraction that does it for fun, because they consider Turking interesting, and sometimes even addicting!

I still consider the responses themselves more interesting than the tabulated version, so go and take a look yourself!