Natural Language Processing Lecture 17 10 29 2013 Jim Martin Today Finish Statistical CFG Parsing Dependency parsing Dependency trees Basic transition based parsing Machine learning 01 13 19 Speech and Language Processing Jurafsky and Martin 2 Simple Probability Model A derivation tree consists of the bag of grammar rules that are in the tree The probability of a tree is the product of the probabilities of the rules in the derivation 01 13 19 Speech and Language Processing Jurafsky and Martin 3 Improved Approaches There are two approaches to overcoming these shortcomings 1 Rewrite the grammar to better capture the dependencies among rules 2 Integrate lexical dependencies into the model 1 And come up with the independence assumptions needed to make it work 01 13 19 Speech and Language Processing Jurafsky and Martin 4 Solution 2 Lexicalized Grammars Lexicalize the grammars with heads Compute the rule probabilities on these lexicalized rules Run Prob CKY as before 01 13 19 Speech and Language Processing Jurafsky and Martin 5 Dumped Example 01 13 19 Speech and Language Processing Jurafsky and Martin 6 Declare Independence When stuck exploit independence and collect the statistics you can There are a large number of ways to do this Let s consider one generative story given a rule we ll 1 Generate the head 2 Generate the stuff to the left of the head 3 Generate the stuff to the right of the head 01 13 19 Speech and Language Processing Jurafsky and Martin 7 Example That is the rule probability for is estimated as 01 13 19 Speech and Language Processing Jurafsky and Martin 8 Dependency Parse ROOT I booked a morning flight booked I booked flight flight a flight morning Tree Constraints Words can only have one head One incoming arc Every word has to have a head Result is a tree There s a path from the root to each word There s only one path from the root to any word These are the formal constraints on dependency trees For any given sentence there will be lots of such trees Most of which are non sense 01 13 19 Speech and Language Processing Jurafsky and Martin 10 Dependency Grammar The linguistic constraints underlying correct trees are usually called a dependency grammar Which may or may not correspond to an explicit formal generative grammar of the kind we ve been using The parsing technique discussed today doesn t use an explicitly represented grammar 01 13 19 Speech and Language Processing Jurafsky and Martin 11 Transition Based Parsing Transition based parsing is a greedy word by word approach to parsing A single dependency tree is built up an arc at a time as we move left to right through a sentence No backtracking A classifiers is used to make decisions as we move through the sentence 01 13 19 Speech and Language Processing Jurafsky and Martin 12 Dependency Parse I booked a morning flight Transition Based Parsing We can again view this as a search space through a set of states for a state that contains what we want In the standard notation a state consists of three elements A stack representing partially processed words A list containing the remaining words to be processed A set containing the relations discovered so far 01 13 19 Speech and Language Processing Jurafsky and Martin 14 States So the start state looks like root sentence A valid final state looks like root R Where R is the set of relations that we ve discovered The represents the fact that all the words in the sentence are accounted for 01 13 19 Speech and Language Processing Jurafsky and Martin 15 Example Here s our example Start root I booked a morning flight End root booked I booked flight flight a flight morning 01 13 19 Speech and Language Processing Jurafsky and Martin 16 Parsing The parsing problem is how to get from the start state to the final state To begin we ll define a set of three basic operators that take a state and produce a new state Left Right Shift 01 13 19 Speech and Language Processing Jurafsky and Martin 17 Shift Shift takes the next word to be processed and pushes it onto the stack and removes it from the list So a shift for our example at the start looks like this root I booked a morning flight root I booked a morning flight 01 13 19 Speech and Language Processing Jurafsky and Martin 18 Left The Left operator 1 Adds relation a b to the set of relations where a is the first word on the word list b is the word at the top of the stack 2 Pops the stack So for our current state root I booked a morning flight root booked a morning flight booked I 01 13 19 Speech and Language Processing Jurafsky and Martin 19 Right The Right operator 1 Adds b a to the set of relations Where b and a are the same as before a is the first work in the remainder list and b is the top of the stack 2 Removes the first word from the remainder list 3 Pops the stack and places the popped item back at the front of the remaining word list 01 13 19 Speech and Language Processing Jurafsky and Martin 20 Example Operation New State root I booked a morning flight 01 13 19 Speech and Language Processing Jurafsky and Martin 21 Example Operation New State root I booked a morning flight Shift 01 13 19 Speech and Language Processing Jurafsky and Martin 22 Example Operation New State root I booked a morning flight Shift 01 13 19 root I booked a morning flight Speech and Language Processing Jurafsky and Martin 23 Example Operation New State root I booked a morning flight Shift root I booked a morning flight Left 01 13 19 Speech and Language Processing Jurafsky and Martin 24 Example Operation New State root I booked a morning flight Shift root I booked a morning flight Left root booked a morning flight booked I 01 13 19 Speech and Language Processing Jurafsky and Martin 25 Example Operation New State root I booked a morning flight Shift root I booked a morning flight Left root booked a morning flight booked I Shift root booked a morning flight booked I 01 13 19 Speech and Language Processing Jurafsky and Martin 26 Example Operation New State root I booked a morning flight Shift root I booked a morning flight Left root booked a morning flight booked I Shift root booked a morning flight booked I Shift root booked a morning flight booked I 01 13 19 Speech and Language Processing Jurafsky and Martin 27 Example Operation New State root I booked a morning flight Shift root I booked a morning flight Left root booked a morning flight booked I Shift root booked a morning flight booked I Shift root booked a morning flight booked I Shift root booked a
View Full Document
Unlocking...