Beyond mind-reading: multi-voxel pattern analysis of fMRI dataIntroductionThe benefits of MVPAMore sensitive detection of cognitive statesRelating brain activity to behavior on a trial-by-trial basisCharacterizing the structure of the neural codeMVPA methodsChoosing a classifierMVPA case studies: going beyond mind-readingDecoding the neural representation of visual object categoriesDecoding the neural representation of line orientationConclusionsReferencesTICS-500; No of Pages 7Beyond mind-reading: multi-voxelpattern analysis of fMRI dataKenneth A. Norman1, Sean M. Polyn2, Greg J. Detre1and James V. Haxby11Department of Psychology, Princeton University, Green Hall, Washington Road, Princeton, NJ 08540, USA2Department of Psychology, University of Pennsylvania, 3401 Walnut Street, Philadelphia, PA 19104, USAA key challenge for cognitive neuroscience isdetermining how mental representations map ontopatterns of neural activity. Recently, researchers havestarted to address this question by applyingsophisticated pattern-classification algorithms todistributed (multi-voxel) patterns of functional MRI data,with the goal of decoding the information that isrepresented in the subject’s brain at a particular pointin time. This multi-voxel pattern analysis (MVPA)approach has led to several impressive feats of mindreading. More importantly, MVPA methods constitute auseful new tool for advancing our understandingof neural information processing. We review howresearchers are using MVPA methods to characterizeneural coding and information processing in domainsranging from visual perception to memory search.IntroductionThe most fundamental questions in cognitive neurosciencedeal with the issue of representation: what information isrepresented in different brain structures; how is thatinformation represented; and how is that informationtransformed at different stages of processing? FunctionalMRI (fMRI) constitutes a powerful tool for addressingthese questions: While a subject performs a cognitive task,we can obtain estimates of local blood flow (a proxy for localneural processing) from tens of thousands of distinct neu-roanatomical locations, within a matter of seconds. How-ever, the large size of these datasets (up to severalgigabytes) and the high levels of noise inherent in fMRIdata pose a challenge to researchers interested in miningthese datasets for information about cognitive processes.Traditionally, fMRI analysis methods have focused oncharacterizing the relationship between cognitive vari-ables and individual brain voxels (volumetric pixels). Thisapproach has been tremendously productive. However,there are limits on what can be learned about cognitivestates by examining voxels in isolation. The goal of thisarticle is to describe a different approach to fMRI analysis,where — instead of focusing on individual voxels —researchers use powerful pattern-classification algo-rithms, applied to multi-voxel patterns of activity, todecode the information that is represented in that patternof activity. We call this approach multi-voxel pattern ana-lysis (MVPA).The idea of applying multivariate methods to fMRI data(i.e. analyzing more than one voxel at once) is not new. Forexample, several researchers have used multivariatemethods to characterize functional relationships betweenbrain regions (e.g. [1–5]). A major development in the lastfew years is the realization that fMRI data analysis can beconstrued, at a high level, as a pattern-classification pro-blem (i.e. how we can recognize a pattern of brain activityas being associated with one cognitive state versusanother). As such, all of the techniques that have beendeveloped for pattern classification and data mining inother domains (e.g. handwriting recognition) can be pro-ductively applied to fMRI data analysis. This realizationhas led to a dramatic increase in the number of researchersusing pattern-classification techniques to analyze fMRIdata. This trend in the fMRI literature is part of a broadertrend towards the application of pattern-classificationmethods in neuroscience (for applications to EEG data,see [6–11]; for applications to neural recording data fromanimal studies, see [12–14]).The first part of the article provides an overview of themain benefits of the MVPA approach, as well as a listing ofsome of the feats of ‘mind reading’ that have been accom-plished with MVPA. The next part provides a moredetailed overview of the methods that make this mindreading possible. The third part of the article discussessome case studies in how researchers can go beyond mindreading (for its own sake), and use MVPA to addressmeaningful questions about how information is repre-sented and processed in the brain.The benefits of MVPAMore sensitive detection of cognitive statesGiven the goal of detecting the presence of a particularmental representation in the brain, the primary advantageof MVPA methods over individual-voxel-based methods isincreased sensitivity. Conventional fMRI analysis meth-ods try to find voxels that show a statistically significantresponse to the experimental conditions. To increase sen-sitivity to a particular condition, these methods spatiallyaverage across voxels that respond significantly to thatcondition. Although this approach reduces noise, it alsoreduces signal in two important ways: First, voxels withweaker (i.e. non-significant) responses to a particular con-dition might carry some information about the presence/absence of that condition. Second, spatial averaging blursout fine-grained spatial patterns that might discriminatebetween experimental conditions [15].ReviewTRENDS in Cognitive Sciences Vol.xxx No.xPlease cite this article as: Kenneth A. Norman et al., Beyond mind-reading: multi-voxel pattern analysis of fMRI data, TRENDS in Cognitive Sciences (2006), doi:10.1016/j.tics.2006.07.005.Corresponding author: Norman, K.A. ([email protected]).Available online xxxxxx.www.sciencedirect.com 1364-6613/$ – see front matter ß 2006 Elsevier Ltd. All rights reserved. doi:10.1016/j.tics.2006.07.005Like conventional methods, the MVPA approach alsoseeks to boost sensitivity by looking at the contributions ofmultiple voxels. However, to avoid the signal-loss issuesmentioned above, MVPA does not routinely involve spatialaveraging of voxel responses. Instead, the MVPA approachuses pattern-classification techniques to extract the signalthat is present in the pattern of response across multiplevoxels, even if