6.863J Natural Language ProcessingLecture 9: Writing grammars; feature-based grammarsRobert C. Berwick6.863J/9.611J Lecture 9 Sp03The Menu Bar• Administrivia:• Schedule alert: Lab 3 out; due next Weds.• Lab time today, tomorrow• Please read notes3.pdf!!englishgrammar.pdf (on web)•Agenda:• Building grammars – basics to complex• Limits of context-free grammars: the trouble with tribbles• Foundation for the laboratory6.863J/9.611J Lecture 9 Sp03Grammars for natural languages• Where do the rules come from?• Roughly: read them off of parse trees…• A “rule-based”, construction-based point of view• Take ‘surface’ phrase patterns (mostly)• But we still want to map to an underlying‘logical’ form• How do we start out?6.863J/9.611J Lecture 9 Sp03Reading rules from parse trees…S→NP VPVP→V NPNP→Det NNP→ N*Can’t we get a computer to do this?6.863J/9.611J Lecture 9 Sp03Key elements – part 1• Establish basic phrase types: S, VP, NP, PP, …• Where do these come from???6.863J/9.611J Lecture 9 Sp03Whatkindsof phrases are there?• Noun phrases, verb phrases, adjectival phrases (“green with envy”), adverbial phrases (“quickly up the hill”), prepositional phrases (“off the wall”), etc.• In general: groundedon lexical items• Shows us the constraintson context-freerules for natural grammars• Example:6.863J/9.611J Lecture 9 Sp03Phrase types are constrained by lexicalprojectionVerb Phrase → Verb Noun Phrase “is-a” (“kick the ball”)Prepositional Phrase →Preposition Noun Phrase(“on the table”)Adjective Phrase → Adjective Prep. Phrase(“green with envy”)Etc. … what is the pattern?6.863J/9.611J Lecture 9 Sp03Function-argument relationXP →X arguments, where X= Noun, Verb, Preposition, Adjective (all lexical categories in the language)Like function-argument structure(so-called “Xbar theory”)Constrains what grammar rules cannotbe:Verb Phrase →Noun Noun Phraseor evenVerb Phrase →Noun Phrase Verb Noun Phrase6.863J/9.611J Lecture 9 Sp03English is function-argumentformfunctionatargsgreensoldthe stocka bargain pricewith envytheover-priced stock6.863J/9.611J Lecture 9 Sp03Other languages are the mirror-inverse: arg-functionatgreensoldthe stocka bargain pricewith envytheover-priced stockThis is like Japanese6.863J/9.611J Lecture 9 Sp03Key elements – part 2• Establishverb subcategories• What are these?• Different verbs take different # arguments• 0, 1, 2 arguments (‘complements’)• Poirot thought; Poirot thought the gun; Poirotthought the gun was the cause.• Some verbs take certain sentence complements:•I know who John saw/? I think who John sawpropositional types:• Embedded questions: I wonder whether…• Embedded proposition:I think that John saw Mary6.863J/9.611J Lecture 9 Sp03Key elements• Subtlety to this• Believe, know, think, wonder,…• ? I believe why John likes ice-cream• I know why John likes ice-cream• I believe that John likes ice-cream• I believe (that) John likes ice-cream• # args, type: Verb subcategories• How many subcategories are there?• What is the structure?6.863J/9.611J Lecture 9 Sp03Idea for phrases• They are based on ‘projections’ of words (lexical items) – imagine features ‘percolating’ upknow [V +proposition]XP [ ]V +proposition6.863J/9.611J Lecture 9 Sp03Heads of phrasesknow [V +proposition]V +proposition6.863J/9.611J Lecture 9 Sp03The parse structure for ‘embedded’ sentencesI believe (that) John likes ice-creamSNP VPIVbelievethat J. likes ice-cream6.863J/9.611J Lecture 9 Sp03New phrase type: S-barNP VPIVbelieveSSbarthat J. likes ice-cream6.863J/9.611J Lecture 9 Sp03SbarVPVbelieveSbarthatComp SJ. likes ice-cream6.863J/9.611J Lecture 9 Sp03SbarVPVbelieveSbarComp SJ. likes ice-creamε6.863J/9.611J Lecture 9 Sp03In fact, true for all sentences…Comp SεSJ. likes ice-creamSbarJohn likes ice-creamWhy?6.863J/9.611J Lecture 9 Sp03What rules will we need?• (U do it..)6.863J/9.611J Lecture 9 Sp03Verb types - continued• What about:Clinton admires honesty/Honesty admires ClintonHow do we encode these in a CFG?Should we encode them?• Colorless green ideas sleep furiously• Revolutionary new ideas appear infrequently6.863J/9.611J Lecture 9 Sp03Features6.863J/9.611J Lecture 9 Sp03The trouble with tribblesmorphology of a single word: Verb[head=thrill, tense=present, num=sing, person=3,…] → thrillsprojection of features up to a bigger phrase VP[head=α, tense=β, num=γ…] → V[head=α, tense=β, num=γ…] NPprovided α is in the set TRANSITIVE-VERBSagreement between sister phrases:S[head=α, tense=β] → NP[num=γ,…] VP[head=α, tense=β, num=γ…]provided α is in the set TRANSITIVE-VERBS6.863J/9.611J Lecture 9 Sp033 Common Ways to Use FeaturesNPVerbVPNPSA roller coaster thrills every teenagerVerb[head=thrill, tense=present, num=sing, person=3,…] → thrillsVP[head=α, tense=β, num=γ…] → V[head=α, tense=β, num=γ…] NPS[head=α, tense=β] → NP[num=γ,…] VP[head=α, tense=β, num=γ…]num=singnum=sing(comprehensionperspective)num=singthrills6.863J/9.611J Lecture 9 Sp03CFG Solution• Encode constraints into the non-terminals• Noun/verb agreementSÆ SgSS Æ PlSSgS Æ SgNP SgVPSgNP Æ SgDet SgNom• Verb subcategories:IntransVP Æ IntransVTransVP Æ TransV NP6.863J/9.611J Lecture 9 Sp03Problems with this – how much info?6.863J/9.611J Lecture 9 Sp03Agreement gets complex…POSSUBPOSGENDERNUMBERCASEPOSSGPOSSNPERSONTENSEDCOMPNEGVOICEVAR–Czech: AGFS3----1A---- Lots of features(tense, number, person, gaps, vowels, commas,wh, etc., etc....) Sorry, that’s just how language is … You know too much to write it down easily!Hegonehas6.863J/9.611J Lecture 9 Sp03Other sentence types• Questions:• Will John eat ice-cream?• Did John eat ice-cream?• How do we encode this?6.863J/9.611J Lecture 9 Sp03`Empty’ elements or categories• Where surface phrase is displaced from its canonical syntactic position• Examples:• The ice-cream was eaten vs.• John ate the ice-cream• What did John eat?• What did Bill say that that John thought the cat ate?• For What x, did Bill say… the cat ate x• Bush is too stubborn to talk to• Bush is too stubborn [x to talk to Bush]• Bush is too stubborn to talk to the Pope• Bush is too stubborn [Bush to talk to the Pope]6.863J/9.611J Lecture 9 Sp03More interesting clause types• Apparently “long distance” effects: ‘displacement’ of phrases
View Full Document
Unlocking...