Tuesday, March 4, 2008

Course Evaluations and Prediction Markets: The Results

A couple of weeks back, I described my attempt to use prediction markets to predict my final course evaluation. The final result of the market was:

$49.95 (closed)
$36.86 (closed)
$4.60 (closed)
$4.34 (closed)
$4.23 (closed)

Taking the weighted average of these predictions, the markets shows a predicted outcome of:

$\begin{align}49.95 \cdot 6.25 & + \\ 36.86 \cdot 6.75 & + \\ 4.60 \cdot 5.25 & + \\ 4.34 \cdot 5.75 & + \\ 4.23 \cdot 3.0 & = 6.227 \end{align}$

And what was the final course evaluation? Did the market work? Well, the final course evaluation was a 6.212 with 35 student ratings. A relative error of 0.002 or 0.2%. I cannot think that I could get a more accurate prediction!

Interestingly enough, by observing the market, I can see that very few people actually picked the 6.0-6.5 range. Most of the players bought contracts of the 6.5-7.0 range. These contracts played their role of counterbalancing the few players that bought contracts in the 1.0-5.0 and in the 5.0-6.0 ranges. Therefore, while most of the action was in contracts that did not predict the correct range, this activity was crucial for the market to balance and give the correct prediction.