6.863J Natural Language ProcessingLecture 9: Going nonlinear - Marxist analysisInstructor: Robert C. [email protected]/9.611J SP04 Lecture 9The Menu Bar• Administrivia:• Lab 2a/2b due MONDAY; 3a out MondayAgenda:Going nonlinear: beyond finite-state machines• Parsing strategies: parsing as search; top-down; bottom-up methods• Parsing strategies: chart parsing as all-purpose search data structure – algorithm & time complexity; CKY and Earley algorithm• Preview of Lab 36.863J/9.611J SP04 Lecture 9Three senses of rules• generation (production): S → NP VP• parsing (comprehension): S ← NP VP• verification (checking): S = NP VP• CFGs are declarative –tell us what the well-formed structures & strings are• Parsers are procedural –tell us howto compute the structure(s) for a given string 6.863J/9.611J SP04 Lecture 9Where are we at? Marxist analysis: simple version for linear parsing• Suppose just linear relations to recover• Need complete description of parse state = [state name, ‘you are here’ index]=[dotted rule, i] = (for example):[q→t • q’, 1]Fruit flies like a bananatq q’6.863J/9.611J SP04 Lecture 9Parsing = building State sets via operation of ‘scanning’ from word to wordS2S1S0:{[q0→ •fruit q1, 0, 0]}fruit{[q0→ fruit • q1,0, 1],[q0→ fruit • q0, 0, 1]}SCANflies{[q1→ flies • q2, 1, 2],[q0→ flies • q1, 0, 2]{[q1→ flies • q3 , 2, 2}SCAN6.863J/9.611J SP04 Lecture 9State set parsing = compute machine state after i words• Given grammar G, input string w=w1w2…wnNote: we mark interword positions 0w1w2…wn• Initialize: write down what can be in “start state set” S0• Loop: for each wordwi, computeSifromSi-1• Final: see if final state is in last state setSn6.863J/9.611J SP04 Lecture 9What information do we need for nonlinear (hierarchical) parsing?• All that we need in linear case (state name AKA dotted rule, where we are in input) plus• One more additional piece of information• Who dominates me? (Who called me?)• This is necessary for hierarchical description• This plus precedence (as we saw) is also sufficient 6.863J/9.611J SP04 Lecture 9Picture: what we need0 1 2 3 4The ice-cream with jimmies0 1 2 The ice-cream2 3 4with jimmiesNoun Phrase (NP)Prepositional Phrase (PP)Starts (called from)position 2Ends at position 4 =PP[0, 4]= NP[0,4]6.863J/9.611J SP04 Lecture 9Another way to view itNPStartPhrase(“predict”Or “push”)Complete phrase(“pop”)NNguyScanDTtheScan6.863J/9.611J SP04 Lecture 9Three operations suffice• One to scan (needed for linear relation) plus:• One to push for new phrases • One to complete or pop new phrases• These 3 show up under different stage names, but always there as ‘abstract’ ops for context-free parsing6.863J/9.611J SP04 Lecture 9So the extra info we need• In addition to the phrase type• The start and stop position (spanning) of a phrase• The start position tells us ‘who called’ us (= caller address, also return address)• This is all we need to describe hierarchical info• If we update the State set algorithm with this, we can extend it to parse context-free grammars (we will simulate a nondeterministic machine on-the-fly, as before. Q: why can’t we convert it offline?)6.863J/9.611J SP04 Lecture 9Example0 I 1 shot 2 an 3 elephant 4 in 5 my 6 pajamas 7 #S(entence)[0, 7]; NP[0, 1]; Verb Phrase VP[1, 7]; NP[2, 3]; PP[4,7]; NP[5, 6]What else?6.863J/9.611J SP04 Lecture 9Parsing as search• “All” we need to do is find the right elements S[0,7], etc. – these are ‘points’ in a search space of possibilities• How?• Q: What is the size of the search space?• Q: well, consider the # of possible elts of the form X[i,j]• Question 2: How do we search from point to point? 6.863J/9.611J SP04 Lecture 9What does the search look like?6.863J/9.611J SP04 Lecture 9Parsing as a Search Problem (II)• Search space: The set of phrasal extents• PhraseType[start:end]• E.g.: NP[0:2]• Goal: • Find a set of paths through the search space…That don’t overlap…And that connect S[0:n] to each word.• Size of search space: |G|n2(G=grammar; n=words)• Time to search the space: ?• If we look at each phrasal extent once, Gn2• otherwise, it might be more (exponential)6.863J/9.611J SP04 Lecture 9How should we explore the ‘phrase space’ most efficiently?• Depth first search = top-down parsing• Parallel - Breadth-first search• Bottom-up parsing• ?Best first (we’ll get to it later)• Let’s take a quick look6.863J/9.611J SP04 Lecture 9Top-Down Parsing• Two basic operations:1. Expand LHS of rule into RHS elements2. Match against against input• When good?• When bad?• When does it do useless work?• What is its complexity?6.863J/9.611J SP04 Lecture 9Top Down Parsing Issues• "Left-recursive" rules can cause infinite loops• NP → NP and NP• Explores trees that are inconsistent with the input• Redundant parsing of phrases."I saw the dog in the tall building behind the hill."(the dog was in the building)"I saw the dog in the tall building behind the hill."(I was in the building)6.863J/9.611J SP04 Lecture 9Bottom-up parsing• Two basic operations1. Shift words onto stack2. Reduce stack elts and replace with LHS of rule6.863J/9.611J SP04 Lecture 9Parsing Issues: Solutions• Re-use the sub-parses we've already computed• Combine top-down and bottom-up approaches• Get the "best of both worlds"• We need some common representationfor the information from top-down and bottom-up approaches.• Use heuristics to decide when to use bottom-up or top-down approaches.6.863J/9.611J SP04 Lecture 9Use can use a chartto record hypotheses about possible syntactic constituents. A chart contains a set of edgesEach edge represents a possible phrase.Edges provide a common representation for parse information.NPDet NI saw the man on the hillChart{SentenceEdgePhraseTypePhraseChildrenChart Parsing6.863J/9.611J SP04 Lecture 9General method: Chart Parsing• Note: parses sharecommon constituents• Build chart = graph data structure for storing partial & complete parses (AKA well-formed substring table)• Graph:• Vertices: used to delimit subsequences of the input• Edges (active, inactive)• Active = denote incompletely parsed (or found) phrase• Inactive = completely found phrase• Labels = name of phrase• Note:
View Full Document
Unlocking...