Predicting Human Brain Activity Associated with the Meanings of Nouns Tom M Mitchell et al Science 320 1191 2008 DOI 10 1126 science 1152876 The following resources related to this article are available online at www sciencemag org this information is current as of May 30 2008 Supporting Online Material can be found at http www sciencemag org cgi content full 320 5880 1191 DC1 This article cites 31 articles 13 of which can be accessed for free http www sciencemag org cgi content full 320 5880 1191 otherarticles This article appears in the following subject collections Psychology http www sciencemag org cgi collection psychology Information about obtaining reprints of this article or about obtaining permission to reproduce this article in whole or in part can be found at http www sciencemag org about permissions dtl Science print ISSN 0036 8075 online ISSN 1095 9203 is published weekly except the last week in December by the American Association for the Advancement of Science 1200 New York Avenue NW Washington DC 20005 Copyright 2008 by the American Association for the Advancement of Science all rights reserved The title Science is a registered trademark of AAAS Downloaded from www sciencemag org on May 30 2008 Updated information and services including high resolution figures can be found in the online version of this article at http www sciencemag org cgi content full 320 5880 1191 Predicting Human Brain Activity Associated with the Meanings of Nouns Tom M Mitchell 1 Svetlana V Shinkareva 2 Andrew Carlson 1 Kai Min Chang 3 4 Vicente L Malave 5 Robert A Mason 3 Marcel Adam Just3 The question of how the human brain represents conceptual knowledge has been debated in many scientific fields Brain imaging studies have shown that different spatial patterns of neural activation are associated with thinking about different semantic categories of pictures and words for example tools buildings and animals We present a computational model that predicts the functional magnetic resonance imaging fMRI neural activation associated with words for which fMRI data are not yet available This model is trained with a combination of data from a trillion word text corpus and observed fMRI data associated with viewing several dozen concrete nouns Once trained the model predicts fMRI activation for thousands of other concrete nouns in the text corpus with highly significant accuracies over the 60 nouns for which we currently have fMRI data he question of how the human brain represents and organizes conceptual knowledge has been studied by many scientific communities Neuroscientists using brain imaging studies 1 9 have shown that distinct spatial patterns of fMRI activity are associated with viewing pictures of certain semantic categories including tools buildings and animals Linguists have characterized different semantic roles associated with individual verbs as well as the types of nouns that can fill those semantic roles e g VerbNet 10 and WordNet 11 12 Computational linguists have analyzed the statistics of very large text corpora and have demonstrated that a word s meaning is captured to some extent by the distribution of words and phrases with which it commonly co occurs 13 17 Psychologists have studied word meaning through feature norming studies 18 in which participants are asked to list the features they associate with various words revealing a consistent set of core features across individuals and suggesting a possible grouping of features by sensory motor modalities Researchers studying semantic effects of brain damage have found deficits that are specific to given semantic categories such as animals 19 21 This variety of experimental results has led to competing theories of how the brain encodes meanings of words and knowledge of objects including theories that meanings are encoded in sensorymotor cortical areas 22 23 and theories that they are instead organized by semantic categories such as living and nonliving objects 18 24 Although these competing theories sometimes lead to differ T ent predictions e g of which naming disabilities will co occur in brain damaged patients they are primarily descriptive theories that make no attempt to predict the specific brain activation that will be produced when a human subject reads a particular word or views a drawing of a particular object We present a computational model that makes directly testable predictions of the fMRI activity associated with thinking about arbitrary concrete nouns including many nouns for which no fMRI data are currently available The theory underlying this computational model is that the neural basis of the semantic representation of concrete nouns is related to the distributional properties of those words in a broadly based corpus of the language We describe experiments training competing computational models based on different assumptions regarding the underlying features that are used in the brain for encoding of meaning of concrete objects We present experimental evidence showing that the best To whom correspondence should be addressed E mail Tom Mitchell cs cmu edu n yv cvi fi w i 1 1 where fi w is the value of the ith intermediate semantic feature for word w n is the number of semantic features in the model and cvi is a learned scalar parameter that specifies the degree to which the ith intermediate semantic feature activates voxel v This equation can be interpreted as predicting the full fMRI image across all voxels for stimulus word w as a weighted sum of images one per semantic feature fi These semantic feature images defined by the learned cvi constitute a basis set of component images that model the brain activation associated with different semantic components of the input stimulus words Predictive model stimulus word celery predicted activity for celery Intermediate semantic features extracted from trillion word text corpus 1 Machine Learning Department School of Computer Science Carnegie Mellon University Pittsburgh PA 15213 USA 2 Department of Psychology University of South Carolina Columbia SC 29208 USA 3Center for Cognitive Brain Imaging Carnegie Mellon University Pittsburgh PA 15213 USA 4Language Technologies Institute School of Computer Science Carnegie Mellon University Pittsburgh PA 15213 USA 5Cognitive Science Department University of California San Diego La Jolla CA 92093 USA of these models predicts fMRI neural activity well enough that it can successfully match words it has not yet encountered to