High Incentives, Sorting on Skills--or Just a “Taste” for Competition? Field Experimental Evidence from an Algorithm Design Contest Kevin J. Boudreau, London Business School & Harvard University, [email protected] Karim R. Lakhani, Harvard Business School, [email protected] Abstract: In the past decade, the dual role of incentive regimes—to not just shape incentives but also to induce worker sorting—has been an area of growing study. This paper contributes field evidence to this stream of work. We present evidence from a 10-day field experiment in which elite software developers competed for cash prizes in a competitive contest regime to solve a challenging computational algorithmic problem of practical significance. We present an experimental and analytical approach that allows us to precisely ascribe variation in problem-solving performance to sorting versus incentive effects. In particular, we highlight a distinction between sorting on raw problem-solving skill (whereby more highly-skilled workers prefer the competitive contest regime) and sorting on the basis or regime “taste”, above and beyond skill. We find that the effect of sorting on regime taste is an order of magnitude larger than skill-based sorting, and of roughly the same magnitude as the effect of varying the formal cash incentives by $1000 in his context. We show that worker’s tastes reflect distinct behavioral orientations (tastes, attitudes, etc.) of workers, rather than unobserved or idiosyncratic skills and human capital in the classical sense. Those sorted on regime taste made different choices in effort and hours worked, roughly doubling performance on account of these choices. Thus, thus sorted workers not only have different skill endowments, but also appear to be responding to different “utility functions”, and the latter effect was the dominant one. Rather than shaping the elasticity of the labor supply curve (i.e., responsiveness to formal cash incentives), regime preference affected workers uniformly, without interacting with formal cash incentives. Keywords: Sorting, Incentives, Problem-Solving, Innovation Contests, Field Experiment, On-line Platform Acknowledgements: We are grateful to members of NASA’s Space Life Sciences Directorate, including Jeff Davis, Elizabeth Richard, Jennifer Fogarty and Bara Reyna, for assistance in identifying an appropriate computational engineering challenge. Jason Crusan, from NASA’s Space Operations Directorate also provided significant help with the project. The TopCoder team, including Jack Hughes, Rob Hughes, Mike Lydon, Ira Heffan, Jessie Ford and Lars Backstrom, provided invaluable assistance in carrying out all aspects of the experiment. This research particularly benefitted from thoughtful comments by Kenneth Arrow, Pierre Azoulay, Iain Cockburn, Peter Coles, Daniel Elfenbein, Silke Januszewski Forbes, Shane Greenstein, Nicola Lacetera, Anja Lambrecht, Mara Lederman, Joshua Lerner, Muriel Niederle, Gary Pisano, Al Roth, Sandra Slaughter, Scott Stern, Catherine Tucker, Eric von Hippel, Heidi Williams and D. J. Wu. Eric Lonstein provided outstanding project management and research assistance. All errors are our own. Kevin Boudreau would like to acknowledge the financial support of the LBS M-Lab for sponsoring prizes. Karim Lakhani would like to acknowledge the generous support of the HBS Division of Research and Faculty Development. A Google Faculty Research Grant supported both authors.High Incentives, Sorting on Skills--or Just a “Taste” for Competition? 2 I. INTRODUCTION Might a “taste for competition” in itself, above and beyond skills and incentives, impact effort and performance in a tournament setting? Economic theory, combined with field and laboratory experimental results have demonstrated that “high-powered,” performance-contingent incentive regimes, like tournaments and variable pay, result in higher output by workers (Field Data: Lazear 2000; Franceschelli, Galiani and Gulmez 2010; Shi 2010; Bandiera, Guiso, Prat, and Sadun 2010; Experiments: Cadsby, Song and Tappon 2007; Ericksson, Teyssier and Villeval 2009; Dohmen & Falk 2011). The main drivers of increased productivity are both the presence of high-powered incentives and the non-random sorting and self-selection of higher skilled individuals (e.g.: Lazear 2000; Cadsby, Song and Tappon 2007; Eriksson & Villeval 2008; Dohmen & Falk 2011). The existence of a high-powered incentive scheme is by itself motivating to workers eliciting more effort (XXXX). And equally importantly, the higher output observed is also due to the fact that higher-powered incentives attract those with superior relevant skills, as these workers can more reasonably expect to be get higher compensation in such payment regimes (XXXX). However, emerging experimental evidence also indicates that the basis for sorting into contests, tournaments and other variable-pay incentives schemes is multi-dimensional and not just dependent upon skills. Scholars have sought to explain the worker’s decision to enter (or not) performance-contingent incentive regimes by investigating the role of various explanations including; overconfidence (Niederle & Vesturlund 2007; Larkin & Leider forthcoming), risk aversion (Dohmen & Falk 2011), high self-regard (Dohmen & Falk 2011), feedback aversion, and more generally a preference, or a taste for competition (Niederle & Vesturlund 2007). With the emerging consensus being that both skills and a whole host of behavioral factors drive the decision of an individual to join and participate in a work context that offers some sort of a variable incentive regime. While the entry decision has been well investigated, we seek to build on this nascent literature by investigating how sorting and self-selection drives performance and effort inHigh Incentives, Sorting on Skills--or Just a “Taste” for Competition? 3 variable-pay settings. Specifically, we present results from a field experiment which features an assignment procedure and experimental design which allows for separation of the performance effects of