Abstract—Basic neural prosthetic control of a computer cursor has been recently demonstrated by Hochberg et al. [1] using the BrainGate system (Cyberkinetics Neurotechnology Systems, Inc.). While these results demonstrate the feasibility of intracortically-driven prostheses for humans with paralysis, a practical cursor-based computer interface requires more precise cursor control and the ability to “click” on areas of interest. Here we present the first practical point and click device that decodes both continuous states (e.g. cursor kinematics) and discrete states (e.g. click states) from a single neural population in human motor cortex. We describe a probabilistic multi-state decoder and the necessary training paradigms that enable point and click cursor control by a human with tetraplegia using an implanted microelectrode array. We present results from multiple recording sessions and quantify the point and click performance. I. INTRODUCTION EURAL interface systems (NISs) based upon an intracortical sensor aim to restore lost function to paralyzed humans by sensing movement-related activity of neurons, decoding this activity into control signals and using these signals to control external devices or the person’s own limbs. Initial results from a human NIS [1] demonstrated that neural spiking activity could be detected from the motor Manuscript received February 16, 2007. This work is partially supported by NIH-NINDS R01 NS 50867-01 as part of the NSF/NIH Collaborative Research in Computational Neuroscience Program and by the Office of Naval Research (award N0014-04-1-082). We also thank the European Neurobotics Program FP6-IST-001917 and Rehabilitation Research and Development Service, Department of Veterans Affairs. The pilot clinical trial from which these data are derived is sponsored by Cyberkinetics Neurotechnology Systems, Inc. S.-P. Kim and M. J. Black are with the Dept. of Computer Science, Brown Univ., Providence, RI 02912 USA (phone: 401-863-7600; fax: 401-863-7657; e-mail: ({spkim,black}@ cs.brown.edu). J. D. Simeral is with the Center for Restorative and Regenerative Medicine, Rehabilitation R&D Service, Dept. of Veterans Affairs Medical Center, Providence RI; and the Dept. of Neuroscience, Brown Univ., Providence, RI 02912 USA (e-mail: [email protected]). L. R. Hochberg is with the Center for Restorative and Regenerative Medicine, Rehabilitation R&D Service, Dept. of Veterans Affairs Medical Center, Providence RI; Dept. of Neuroscience, Brown Univ., Providence, RI 02912 USA; and Massachusetts General Hospital, Harvard Medical School, Boston, MA (e-mail: [email protected]). J. P. Donoghue is with the Dept. of Neuroscience, Brown Univ., Providence, RI 02912 USA and Cyberkinetics Neurotechnology Systems, Inc., Foxborough, MA (e-mail: [email protected]). G. M. Friehs is with the Dept. of Clinical Neuroscience, Brown Univ., RI 02912 USA (e-mail: [email protected]). Conflict of Interest: LH: Clinical trial support, Cyberkinetics Neurotechnology Systems, Inc. (CKI); GF: consultant, stock options, CKI; JD: Chief Scientific Officer, compensation, stock holdings, director, CKI. cortex of humans with long-term paralysis, decoded, and used for voluntary control of prosthetic devices including robotic arms and computer cursors. Despite this initial success, the quality of neural cursor control in these initial demonstrations was below the level of cursor use typically achieved by able-bodied humans using standard pointing devices. This human NIS [1] used a linear regression method to directly decode cursor position from a history of neural firing rates. Studies in able-bodied monkeys, however, have demonstrated motor cortical neurons code for velocity [2] and that improved cursor control could be obtained by decoding velocity using a Kalman filter [3]. Recent work by our group, in collaboration with Cyberkinetics Neurotechnology Systems, Inc. (Foxborough, MA), has similarly improved the quality of human cursor control by decoding cursor velocity from motor cortical activity using a Kalman filter [4]. The results from multiple recording sessions showed more stable and accurate cursor control for reaching designated targets. Beyond precise cursor positioning, practical applications of computer control typically assume the ability to click on targets of interest (e.g. select menu items on a computer screen). For an NIS, this point and click capability requires the simultaneous decoding of both continuous (cursor motion) and discrete (clicking) states in real time from a population of motor cortical neurons. Multiple recording devices might provide separate signals for continuous movement and click states but in our work a single recording array was used. Multi-state decoding then requires the extraction of both continuous and discrete states from a single neural population. Additionally this neural population is currently small (on the order of tens of cells) and hence much smaller than the number of neurons engaged in the actual performance of such a task. Preliminary studies in non-human primates have shown that it was possible to decode both continuous and discrete states from the same population of motor cortical cells. Darmanjian et al. [5] decoded movement/rest states using hidden Markov models (HMMs) and hand position using multiple linear filters from multiple motor cortical areas of a monkey. Wood et al. [6] developed a Bayesian method to decode from MI firing activity both a discrete state, representing whether a monkey was performing a task or not, and the continuous kinematics of the monkey’s arm movements. In this method, a discrete state was decoded using a linear discriminant analysis (LDA) classifier [7] and embedded into a particle filtering algorithm [8] for decoding Multi-state decoding of point-and-click control signals from motor cortical activity in a human with tetraplegia Sung-Phil Kim, John D. Simeral, Leigh R. Hochberg, John P. Donoghue, Gerhard M. Friehs, and Michael J. Black Ncontinuous kinematics. To decode discrete and continuous states from human motor cortical neural activity, we present a multi-state decoder based on the model of Wood et al. [6] but modified to use a Kalman filter decoder for real time performance. We also present the associated training paradigms sufficient for training the multi-state decoding algorithm. We report here results of using the multi-state decoder for point and click