View Full Document

# Conditional Random Fields

View Full Document
View Full Document

2 views

Unformatted text preview:

Conditional Random Fields William W Cohen CALD Announcements Upcoming assignments Today Sha Pereira Lafferty et al Mon 2 23 Klein Manning Toutanova et al Wed 2 25 no writeup due Mon 3 1 no writeup due Wed 3 3 project proposal due personnel 1 2 page Spring break week no class Review motivation for CMM s Ideally we would like to use many arbitrary overlapping features of words identity of word ends in ski is capitalized is part of a noun phrase is in a list of city names is under node X in WordNet is in bold font is indented is in hyperlink anchor S t 1 St S t 1 is Wisniewski part of noun phrase ends in ski O t 1 Ot O t 1 Motivation for CMMs identity of word ends in ski is capitalized is part of a noun phrase is in a list of city names is under node X in WordNet is in bold font is indented is in hyperlink anchor S t 1 St S t 1 is Wisniewski part of noun phrase ends in ski O t 1 Ot O t 1 Idea replace generative model in HMM with a maxent model where state depends on observations and previous state Pr st xt st 1 Implications of the model Does this do what we want Q does Y i 1 depend on X i 1 a nodes is conditionally independent of its non descendents given its parents Label Bias Problem Consider this MEMM P 1 and 2 ro P 2 1 and ro P 1 ro P 2 1 and o P 1 r P 1 and 2 ri P 2 1 and ri P 1 ri P 2 1 and i P 1 r Since P 2 1 and x 1 for all x P 1 and 2 ro P 1 and 2 ri In the training data label value 2 is the only label value observed after label value 1 Therefore P 2 1 1 so P 2 1 and x 1 for all x However we expect P 1 and 2 ri to be greater than P 1 and 2 ro Per state normalization does not allow the required expectation Label Bias Problem Consider this MEMM and enough training data to perfectly model it Pr 0123 rib 1 Pr 0453 rob 1 Pr 0123 rob Pr 1 0 r Z1 Pr 2 1 o Z2 Pr 3 2 b Z3 0 5 1 1 Pr 0453 rib Pr 4 0 r Z1 Pr 5 4 i Z2 Pr 3 5 b Z3 0 5 1 1 How important is label bias Could be avoided in this case by changing structure Our models are always wrong is this wrongness a problem

Unlocking...