6.863J Natural Language ProcessingLecture 8: Going nonlinear - Marxist analysisInstructor: Robert C. [email protected]/9.611J SP04 Lecture 8The Menu Bar• Administrivia:• Lab 2a/2b due FridayAgenda:Going nonlinear: beyond finite-state machines• Marxist analysis – simple & post-modern• What: hierarchical representations; constituents, representation• How: constituent or ‘context-free’ parsing (next time – how to do it fast)• Why: to extract ‘meaning’6.863J/9.611J SP04 Lecture 8Motivation• What, How, and Why• What: word chunksbehave as units, like words or endings (morphemes), like ing• How: we have to recover these from input• Why: chunks used to discover meaning• Parsing: mapping from stringsto structured representation6.863J/9.611J SP04 Lecture 8Why parsing?• A (context-free) grammar tells us what (syntactic)structure(s) we can assign to a string • It doesn't tell us is how we should go about assigning a string a structure6.863J/9.611J SP04 Lecture 8Applications of parsing Grammar checking (Microsoft) Indexing for information retrieval (Woods 72-1997)... washing a car with a hose ... vehicle maintenance Information extraction (Keyser, Chomsky ’62 to Hobbs 1996)NY TimesarchiveDatabasequery6.863J/9.611J SP04 Lecture 8Why: Q&A systems (lab 4)(top-level)Shall I clear the database? (y or n) y>John saw Mary in the parkOK.>Where did John see MaryIN THE PARK.>John gave Fido to MaryOK.>Who gave John FidoI DON'T KNOW>Who gave Mary FidoJOHN>John saw FidoOK.>Who did John seeFIDO AND MARY6.863J/9.611J SP04 Lecture 8Language & hierarchical structure• Claim: Most, perhaps all properties in syntax are defined over hierarchical structure • One needs to parse to see subtle distinctions6.863J/9.611J SP04 Lecture 8More examples: Marxist analysis• This morning, I shot an elephant in my pajamas• “How he got into my pajamas, I’ll never know” (G. Marx)6.863J/9.611J SP04 Lecture 8Examples (courtesy Dave Barry)• National Park Service:• Avoid the traffic by using a shuttle bus and view the elk rut with a park ranger• PA Patriot News:•“Smoking organ causes stir at nursing home”• Where do these come from??• Visiting relatives can be dangerous/smoking organs can be dangerous6.863J/9.611J SP04 Lecture 8Why: linguistic properties defined over hierarchical structure• What are the linguistic properties we need?• Subject-of, object-of – to get predicate structure• Scope• Structural ambiguity (hence multiple meaning)• All these from syntax6.863J/9.611J SP04 Lecture 8Predication depends on configuration• Subject-of: Bill-kill• Object-of: kill-BillSentenceNoun phraseVerb phraseVerbkillVerb phraseVerbkillNoun phraseBill6.863J/9.611J SP04 Lecture 8Configurational properties More sophisticated configurational propertySara likes her. (her ≠ Sara)Sara thinks that someone likes her. (her = or ≠ Sara)Sara dislikes anyone’s criticism of her. (her = Sara or her ≠ Sara)Who did John see? → For which x, x a person, likes(Bill, x)Distinction here is based on hierarchical structure = scopein natural language6.863J/9.611J SP04 Lecture 8Why: express ‘long distance’ relationships via adjacency• The guy that we know in Somerville likes ice-cream• Who did the guy who lives in Somerville see __?SNP+sg VP+sgSThe guythat we know in SomervilleVNPlikesice-cream6.863J/9.611J SP04 Lecture 8Why: recover meaning from structureJohn ate ice-cream → ate(John, ice-cream)-This must be done from structure -Actually want something like λxλy ate(x,y)How?6.863J/9.611J SP04 Lecture 8Structure mustbe recovered whodid SVVPxSNP‘gap’ orempty elementsee6.863J/9.611J SP04 Lecture 8But now we have a more complex Marxist analysis• I shot an elephant in my pajamas• This is hierarchicallyambiguous – not just linear! (each possible hierarchical structure corresponds to a distinctmeaning)• A case of structural ambiguity6.863J/9.611J SP04 Lecture 8What is the structure that matters?Turns out to be SCOPE for natural languages!S6.863J/9.611J SP04 Lecture 8The language for hierarchical structure• What are the basic elements• How are they put together?6.863J/9.611J SP04 Lecture 8The elements1. What: hierarchical representations (anything with recursion) using phrasesAKA “constituents”2. Constituents are equivalence classes of words 3. How: context-free parsing (plus…)4. Why: (meaning)6.863J/9.611J SP04 Lecture 8Recursive Transition Networks to context-free grammars (CFGs) and back: 1-1 correspondenceSentence:NP VPNP:VP:S→NP VPNameDet NounVerb NPVP→Verb NPNP→NameNP→Det Noun+ terminal expansionrules6.863J/9.611J SP04 Lecture 8Added information• FSA represents pure linearrelation: what can precedeor (follow) what• CFG/RTN adds a single new predicate: dominate• Claim: The dominance and precedence relations amongst the words exhaustively describe its syntacticstructure• When we parse, we are recovering these predicates6.863J/9.611J SP04 Lecture 8Dominance & precedence definecontext-free grammars completely• Definition of context-free grammar (CFG)• Definition of derives : determines hierarchy• We’ll get to that soon…but first, from linear machines to hierarchical ones…6.863J/9.611J SP04 Lecture 8The deepest lesson• Claim: allapparently nonadjacent relationships in languge can be reduced to adjacent ones via projection to a new level of representation• (In one sense, vacuous; in another, deep)• Example: Subject-Verb agreement (agreement generally)• Example: so-calledwh-movement6.863J/9.611J SP04 Lecture 8OK: start with finite-state machines• Marxist analysis, step 1• Then historical revisionism…6.863J/9.611J SP04 Lecture 8Marxist analysis: simple version• Suppose just linear relations to recover• Still can be ambiguity – multiple paths• Consider:Fruit flies like a banana6.863J/9.611J SP04 Lecture 8Parsing for fsa’s: keep track of what ‘next state’ we could be in at each stepNB: ambiguity = > 1 path through network= > 1 sequence of states (‘parses’)= > 1 ‘syntactic rep’ = >1 ‘meaning’fruit flies like a bananafruitfruitfliesflies012345likelikeεabanana6.863J/9.611J SP04 Lecture 8Methods for parsing• How do we handle ambiguity? • Methods:1. Backtrack2. Convert to deterministic machine (ndfsa →dfsa): offlinecompilation3. Pursue all paths in parallel: onlinecomputation (“state
View Full Document
Unlocking...