A public service announcement, to advertise CrowdScale (http://www.crowdscale.org/) a cool workshop at HCOMP 2013 that focuses on challenges that people face when applying crowdsourcing at scale.
A couple of interesting twists from the classic workshop recipe;
- The workshop invites submission of short (2-page) position papers which identify and motivate key problems or potential approaches for crowdsourcing at scale, even if there aren’t satisfactory solutions proposed. (Deadline: October 4)
- Second, there is a shared task challenge, which also carries a cool $1500 reward for the winner.
The CfP follows:
Crowdsourcing at a large scale raises a variety of open challenges:
- How do we programmatically measure, incentivize and improve the quality of work across thousands of workers answering millions of questions daily?
- As the volume, diversity and complexity of crowdsourcing tasks increase, how do we scale the hiring, training and evaluation of workers?
- How do we design effective elastic marketplaces for more skilled work?
To advance the research and practice in crowdsourcing at scale, our workshop invites position papers tackling such issues of scale. In addition, we are organizing a shared task challenge regarding how to best aggregate crowd labels on large crowdsourcing datasets released by Google and CrowdFlower.
- How do we adapt models for long-term, sustained contributions rather than ephemeral participation of workers?We believe tackling such problems will be key to taking crowdsourcing to the next level – from its uptake by early adopters today, to its future as how the world’s work gets done.
Twitter: #crowdscale • @CrowdAtScale
Organizers
- Tatiana Josephy (@tatianajosephy), CrowdFlower
- Matthew Lease (@mattlease), University of Texas at Austin
- Praveen Paritosh (@heuristicity), Google