Syntactic Attributes3 views of a context-free ruleExamplesPowerPoint Presentation3 Common Ways to Use AttributesSlide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Log-Linear Models of Rule ProbabilitiesSlide 16Post-ProcessingSimpler Grammar + Post-ProcessingSlide 19What Do These Enhancements Give You? And What Do They Cost?Analogy: What Does Dyna Give You?Slide 22Part of the English Tense SystemTenses by Post-Processing: “Affix-hopping” (Chomsky)Slide 25Slide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Slide 38Slide 39Slide 40Slide 41Slide 42Slide 43Gaps (“deep” grammar!)GapsSlide 46Slide 47Slide 48Slide 49Slide 50Slide 51600.465 - Intro to NLP - J. Eisner 1Syntactic AttributesMorphology, heads, gaps, etc.Note: The properties of nonterminal symbols are often called “features.”However, we will use the alternative name “attributes.” (We’ll use “features” to refer only to the features that getweights in a machine learning model, e.g., a log-linear model.)600.465 - Intro to NLP - J. Eisner 23 views of a context-free rulegeneration (production): S  NP VPparsing (comprehension): S  NP VPverification (checking): S = NP VPToday you should keep the third, declarative perspective in mind.Each phrase has an interface (S) saying where it can goan implementation (NP VP) saying what’s in itTo let the parts of the tree coordinate more closely with one another, enrich the interfaces: S[attributes…] = NP[attributes…] VP[attributes…]600.465 - Intro to NLP - J. Eisner 3ExamplesNPVerbVPNPSA roller coaster thrills every teenagerVerb  thrillsVP Verb NPS  NP VP600.465 - Intro to NLP - J. Eisner 4morphology of a single word: Verb[head=thrill, tense=present, num=sing, person=3,…]  thrillsprojection of attributes up to a bigger phrase VP[head=, tense=, num=…]  V[head=, tense=, num=…] NPprovided  is in the set TRANSITIVE-VERBSagreement between sister phrases:S[head=, tense=]  NP[num=,…] VP[head=, tense=, num=…] 3 Common Ways to Use Attributes600.465 - Intro to NLP - J. Eisner 53 Common Ways to Use AttributesNPVerbVPNPSA roller coaster thrills every teenagerVerb[head=thrill, tense=present, num=sing, person=3,…]  thrillsVP[head=, tense=, num=…]  V[head=, tense=, num=…] NPS[head=, tense=]  NP[num=,…] VP[head=, tense=, num=…] num=singnum=singnum=sing(generationperspective)thrills600.465 - Intro to NLP - J. Eisner 6NPVerbVPNPSA roller coaster thrills every teenagerVerb[head=thrill, tense=present, num=sing, person=3,…]  thrillsVP[head=, tense=, num=…]  V[head=, tense=, num=…] NPS[head=, tense=]  NP[num=,…] VP[head=, tense=, num=…] num=singnum=sing(comprehensionperspective)num=singthrills3 Common Ways to Use AttributesDetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSNDetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSN[num=1][num=1][num=1][num=1][num=1]NP[n=1]  Det N[n=1]N[n=1]  N[n=1] VPN[n=1]  planVP[n=1]  V[n=1] VPV[n=1]  hasS NP[n=1] VP[n=1]DetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSN[num=1][num=1][num=1][num=1][num=1]NP[n=]  Det N[n=]N[n=]  N[n=] VPN[n=1]  planVP[n=]  V[n=] VPV[n=1]  hasS NP[n=] VP[n=]DetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSNNP[h=]  Det N[h=]N[h=]  N[h=] VPN[h=plan]  plan[head=plan][head=plan][head=plan]DetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSNNP[h=]  Det N[h=]N[h=]  N[h=] VPN[h=plan]  plan[head=plan][head=plan][head=plan][head=swallow] [head=Wanda][head=Otto][head=swallow][head=swallow][head=thrill][head=thrill][head=thrill][head=thrill][head=thrill]DetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSNNP[h=]  Det N[h=]N[h=]  N[h=] VPN[h=plan]  plan[head=plan][head=plan][head=plan][head=swallow] [head=Wanda][head=Otto][head=swallow][head=swallow][head=thrill][head=thrill][head=thrill][head=thrill][head=thrill]Morphology (e.g.,word endings)N[h=plan,n=1]  planN[h=plan,n=2+]  plansN[h=thrill,tense=prog]  thrillingN[h=thrill,tense=past]  thrilledN[h=go,tense=past]  went Why use heads?DetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSNNP[h=]  Det N[h=]N[h=]  N[h=] VPN[h=plan]  plan[head=plan][head=plan][head=plan][head=swallow] [head=Wanda][head=Otto][head=swallow][head=swallow][head=thrill][head=thrill][head=thrill][head=thrill][head=thrill]Subcategorization (i.e., transitive vs. intransitive)When is VP  V NP ok?VP[h=]  V[h=] NP restrict to   TRANSITIVE_VERBSWhen is N  N VP ok?N[h=]  N[h=] VP restrict to   {plan, plot, hope,…}  Why use heads?DetTheNplantoVPVPVswallowNPWandaVhasVbeenVthrillingNPOttoNPVPVPVPSNNP[h=]  Det N[h=]N[h=]  N[h=] VPN[h=plan]  plan[head=plan][head=plan][head=plan][head=swallow] [head=Wanda][head=Otto][head=swallow][head=swallow][head=thrill][head=thrill][head=thrill][head=thrill][head=thrill]Selectional restrictionsVP[h=]  V[h=] NPI.e., VP[h=]  V[h=] NP[h= ]Don’t fill template in all ways: VP[h=thrill]  V[h=thrill] NP[h=Otto] *VP[h=thrill]  V[h=thrill] NP[h=plan] Why use heads?leave out, or low probLog-Linear Models of Rule ProbabilitiesWhat is the probability of this rule? S[head=thrill, tense=pres, animate=no…]  NP[head=plan, num=1, …]VP[head=thrill, tense=pres, num=1, …]We have many related rules. p(NP[…] VP[…] | S[…]) = (1/Z) exp k k  fk(S[…]  NP[…] VP[…])We are choosing among all rules that expand S[…].If a rule has positively-weighted features, they raise its probability. Negatively-weighted features lower it.Which features fire will depend on the attributes!Log-Linear Models of Rule ProbabilitiesS[head=thrill, tense=pres, …]  NP[head=plan, num=1, animate=no, …]VP[head=thrill, tense=pres, num=1, …]Some features that might fire on this …The raw rule without attributes is S  NP VP.Is that good? Does this feature have positive weight?The NP and the VP agree in number. Is that good?The head of the NP is “plan.” Is that good?The verb “thrill” will get a