J Neurophysiol 101 1901 1912 2009 First published December 24 2008 doi 10 1152 jn 90974 2008 Near Optimal Combination of Sensory and Motor Uncertainty in Time During a Naturalistic Perception Action Task A Aldo Faisal and Daniel M Wolpert Department of Engineering University of Cambridge Cambridge United Kingdom Submitted 28 August 2008 accepted in final form 8 December 2008 Real world behavior requires the brain to combine a stream of sensory information and motor actions over time This problem is complicated given that sensory inputs and motor outputs are subjected to noise and more generally uncertainty Faisal et al 2008 In the sensory domain it has previously been shown that human subjects have knowledge about the uncertainty in their sensory modalities and can combine these modalities in a statistically optimal fashion to reduce overall sensory uncertainty van Beers et al 1996 Ernst and Banks 2002 Hillis et al 2004 Jacobs 1999 Knill 2003 Sober and Sabes 2005 van Beers et al 1999 However these studies examine only synchronous presentation of stimuli and therefore ignore the role of time in acquiring sensory information In the motor domain goal directed movements seem to be conducted in such a way as to reduce motor variability Harris and Wolpert 1998 and minimize the task relevant parts of movement uncertainty Todorov and Jordan 2002 Thus in action and perception tasks subjects behave in a way to minimize the negative consequences of uncertainty Battaglia and Schrater 2007 Unlike previous studies most natural situations involve asynchronous and possibly overlapping episodes of sensory information acquisition and motor action from reaching to an object that you have previously looked at to using your side view mirror while driving Thus successful behavior requires a combination of sensation and action across time Here we examine how subjects choose to allocate time to perception and action To do this we used a simple virtual reality experiment catching a falling ball with a paddle Fig 1A We enforced a trade off between perception and action phases by making the ball invisible once movement is initiated such that no additional sensory information can be acquired about the ball s trajectory and landing position Therefore subjects can tradeoff their sensory and movement uncertainties by choosing the amount of time they allocate to perception and the amount of time remaining for action before the ball touches the ground that is by making the decision when to switch from perception to action Our approach is to measure independently the time dependence of sensory variability and motor variability and to predict their combined effect on the ball catching task Note that the use here of the term variability encompasses many sources For example motor variability here the endpoint variability of the position of the paddle is constituted by signal dependent motor noise Harris and Wolpert 1998 motor planning variability Churchland et al 2006 van Beers et al 2004 noise in nerve fibers of the CNS and PNS Faisal and Laughlin 2007 and other sources Faisal et al 2008 We investigate whether subjects have knowledge of the time dependence of their uncertainty in both perception and action by examining whether they choose the optimal switching time so as to minimize the overall variability of the task and maximize their chances of catching the ball We can consider an ideal actor whose aim is to maximize the probability of catching the ball by minimizing the distance between the paddle and ball at touchdown Both sensory and motor variability contribute to the overall variability of where the paddle is placed relative to the ball see Fig 1B for illustration The longer the actor perceives the lower the sensory variability about where the ball will land Fig 1B green curve and axes but the higher the motor variability because the remaining time for movement decreases Fig 1B blue curve and axes An ideal actor should therefore choose a switching time that minimizes the combined effect of sensory and motor variability Fig 1B black line To predict the optimal switching time for each subject we independently quantify in two separate experiments the time dependence of the sensory and motor variability by their variance We assume that the two sources of sensory and motor variability are independent and therefore the combined task variability 2C is the sum of the time dependent sensory variability variance Address for reprint requests and other correspondence A A Faisal Dept of Engineering Univ of Cambridge Trumpington St CB2 1PZ Cambridge UK E mail aaf23 cam ac uk The costs of publication of this article were defrayed in part by the payment of page charges The article must therefore be hereby marked advertisement in accordance with 18 U S C Section 1734 solely to indicate this fact INTRODUCTION www jn org 0022 3077 09 8 00 Copyright 2009 The American Physiological Society 1901 Downloaded from jn physiology org on March 26 2009 Faisal AA Wolpert DM Near optimal combination of sensory and motor uncertainty in time during a naturalistic perception action task J Neurophysiol 101 1901 1912 2009 First published December 24 2008 doi 10 1152 jn 90974 2008 Most behavioral tasks have time constraints for successful completion such as catching a ball in flight Many of these tasks require trading off the time allocated to perception and action especially when only one of the two is possible at any time In general the longer we perceive the smaller the uncertainty in perceptual estimates However a longer perception phase leaves less time for action which results in less precise movements Here we examine subjects catching a virtual ball Critically as soon as subjects began to move the ball became invisible We study how subjects trade off sensory and movement uncertainty by deciding when to initiate their actions We formulate this task in a probabilistic framework and show that subjects decisions when to start moving are statistically near optimal given their individual sensory and motor uncertainties Moreover we accurately predict individual subject s task performance Thus we show that subjects in a natural task are quantitatively aware of how sensory and motor variability depend on time and act so as to minimize overall task variability 1902 A A FAISAL AND D M WOLPERT 2S and motor variability 2M Therefore if the total task time is T and subjects switch from perception to action at time t this means they have time t for
View Full Document
Unlocking...