The interaction of top-down and bottom-up statistics in the resolution of syntactic category ambiguityIntroductionExperiment 1MethodParticipantsMaterialsProcedureResultsComprehension question performanceReading timesDiscussionExperiment 2MethodParticipantsMaterialsProcedureResultsComprehension question performanceReading timesDiscussionExperiment 3MethodParticipantsMaterialsProcedureResultsDiscussionGeneral discussionItems for Experiment 1Experiment 1 residual (and raw) reading times per word, in millisecondsItems for Experiment 2Experiment 2 residual (and raw) reading times per word, in millisecondsItems for Experiment 3Residual (and raw) reading times per word (in milliseconds) for the determiner conditions in Experiment 3ReferencesThe interaction of top–down and bottom–up statisticsin the resolution of syntactic category ambiguityqEdward Gibson*Department of Brain and Cognitive Sciences, MIT, Cambridge, MA 02139, USAReceived 13 June 2005; revision received 18 November 2005Available online 3 February 2006AbstractThis paper investigates how people resolve syntactic category ambiguities when comprehending sentences. It is pro-posed that people combine: (a) context-dependent syntactic expectations (top–down statistical information) and (b)context-independent lexical-category frequencies of words (bottom–up statistical information) in order to resolve ambi-guities in the lexical categories of words. Three self-paced reading experiments were conducted involving the ambiguousword ‘‘that’’ in different syntactic environments in order to test these and other hypotheses. The data support the top–down/bottom–up approach in which the relative frequencies of lexical entries for a word are tabulated independent ofcontext. Data from other experiments from the literature are discussed with respect to the model proposed here. 2005 Elsevier Inc. All rights reserved.Keywords: Sentence comprehension; Synactic category disambiguation; Statistical language processing; FrequencyIntroductionMost English content words and many function wordsare ambiguous among multiple lexical entries, includingdifferent senses and syntactic categories. For example,most verbs are ambiguous as nouns (‘‘desert,’’ ‘‘train,’’‘‘fire,’’ ‘‘light,’’ etc.), and many past-tense verbs areambiguous as past-participle verbs (‘‘walked,’’ ‘‘kicked,’’etc.). MacDonald (1993, 1994) demonstrated that peopleare sensitive to a variety of statistical properties of the dif-ferent lexical entries for a word in resolving these ambigu-ities (cf. Corley & Crocker, 2000; Frazier & Rayner, 1987;MacDonald, Pearlmutter, & Seidenberg, 1994). In aninfluential paper, Tabor, Juliano, and Tanenhaus (1997)(cf. Juliano & Tanenhaus, 1994) provided evidence thatpeople are sensitive to the syntactic context in resolvinglexical category ambiguities. In Tabor et al.’s first experi-ment, the determiner/complementizer ambiguity of theword ‘‘that’’ was resolved in two environments: (1)0749-596X/$ - see front matter  2005 Elsevier Inc. All rights reserved.doi:10.1016/j.jml.2005.12.005qI thank the following people for helpful discussions onpresentations of this work: Timothy Desmet, the audience atthe 12th CUNY Human Sentence Processing Conference,Evelina Fedorenko, Dan Grodner, Neal Pearlmutter, CarsonSchutze, Whitney Tabor, John Trueswell, and Tessa Warren. Iespecially thank San Tunstall for her help in constructing thematerials for these experiments, for help in running theexperiments, and for discussing earlier versions of this work. Ialso thank Doug Rohde, Florian Wolf, Charlene Chuang, andRina Patel for their help in conducting all the corpus searches inthis paper. Funding for this work was provided by a Grantprovided by the National Science Foundation, award numberSBR-9729037. Additional funding for the project was providedby the JST/MIT joint international research project ‘‘Mind/Articulation.’’*Fax: +1 617 258 8654.E-mail address: [email protected] of Memory and Language 54 (2006) 363–388www.elsevier.com/locate/jmlJournal ofMemory andLanguagesentence-initially or (2) post-verbally (cf. Grodner, Gib-son, & Tunstall, 2002, for related results for the main-verb/past-participle ambiguity):(1) a. That cheap hotel was clean and comfortable toour surprise.b. That cheap hotels were clean and comfortablesurprised us.(2) a. The lawyer insisted that cheap hotel was cleanand comfortable.b. The lawyer insisted that cheap hotels were cleanand comfortable.In sentence-initial contexts like (1), there was a pref-erence to resolve the ambiguity in favor of the determin-er interpretation (1a), as evidenced by slower readingtimes for the continuation ‘‘hotels were clean...’’ in(1b), compared to the continuation ‘‘hotel was clean ...’’in (1a). In contrast, in post-verbal contexts like (2), therewas a preference to resolve the ambiguity in favor of thecomplementizer interpretation (2b), as evidenced byslower reading times for the continuation ‘‘hotel wasclean...’’ in (2a), compared to the continuation ‘‘hotelswere clean ...’’ in (2b).As discussed by Tabor et al., this result was not pre-dicted by most existing theories of syntactic ambiguityresolution at the time (e.g., Frazier, 1987; Gibson,1991). Specifically, none of those theories predicted dif-ferent resolution preferences of the determiner/comple-mentizer ambiguity across the two environments, incontrast to what was observed. In order to account fortheir results, Tabor et al. provided a computationalmodel consisting of two parts: (1) a simple recurrent net-work like that of Elman (1991) to simulate learning ofthe relevant syntactic knowledge, and (2) an attractor-based dynamical system to simulate preferences andprocessing times in sentence comprehension. First, thenetwork was trained on a naturalistic corpus consistingof simplified sentence forms like the sentences in (1)and (2), whose structural frequencies matched the fre-quencies observed in the Brown corpus (Kucera & Fran-cis, 1967). The weights on the network were then fixed sothat no more learning would take place. The attractorsin the second component of the model, the dynamicalsystem, were then calculated based on the resultant hid-den units of the network. Target sentences were then fedto the network, and the values for the hidden units’ acti-vations after each word was processed were recorded. Inorder to simulate