CMSC 723 Computational Linguistics I Session 4 Part of Speech Tagging Jimmy Lin The iSchool University of Maryland Wednesday September 23 2009 Source Calvin and Hobbs Today s Agenda What are parts of speech POS What at iss POS OS tagging tagg g Methods for automatic POS tagging z z Rule based POS tagging Rule based Transformation based learning for POS tagging Along the way z z Evaluation Supervised machine learning Parts of Speech Equivalence class of linguistic entities z Categories or types of words Study dates back to the ancient Greeks z z z Dionysius Thrax of Alexandria c 100 BC 8 parts of speech noun verb pronoun preposition adverb conjunction participle article Remarkably enduring list 4 How do we define POS By meaning z z z By the syntactic environment z z What occurs nearby What does it act as By what morphological processes affect it z Verbs are actions Adjectives are properties Nouns are things What affixes does it take Combination of the above Parts of Speech Open class z z Impossible to completely enumerate New words continuously being invented borrowed etc Closed class z z z Closed fixed membership Reasonably easy to enumerate Generally short function words that structure sentences Open Class POS Four major open classes in English z z z z Nouns Verbs Adjectives Adverbs All languages have nouns and verbs but may not have the other two Nouns Open class z Semantics z z Generally words for people places things But not always bandwidth energy Syntactic environment z z New inventions all the time muggle webinar Occurring with determiners Pluralizable possessivizable Other characteristics z Mass vs count nouns Verbs Open class z Semantics z Generally denote actions processes etc Syntactic environment z z New inventions all the time google tweet Intransitive transitive ditransitive Alternations Other characteristics z z z Main vs vs auxiliary verbs Gerunds verbs behaving like nouns Participles verbs behaving like adjectives Adjectives and Adverbs Adjectives z Generally modify nouns e g tall girl Adverbs z z z A semantic and formal potpourri Sometimes modify verbs e g sang beautifully Sometimes modify adjectives e g extremely hot Closed Class POS Prepositions z z z In English occurring before noun phrases Specifying some type of relation spatial temporal Examples on the shelf before noon Particles z z Resembles a preposition but used with a verb phrasal verbs Examples find out turn over go on Particle vs Prepositions He came by the office in a hurry He came by his fortune honestly by preposition by particle We ran up the phone bill We ran up the small hill up particle up preposition He lived down the block He never lived down the nicknames down preposition down particle More Closed Class POS Determiners z z Establish reference for a noun Examples a an the articles that this many such Pronouns z z z Refer to person or entities he she it Possessive pronouns his her its Wh pronouns what who Closed Class POS Conjunctions Coordinating conjunctions z z Join two elements of equal status Examples cats and dogs salad or soup Subordinating conjunctions z z z Join two elements of unequal status Examples We ll leave after you finish eating While I was waiting in line I saw my friend Complementizers are a special case I think that you should finish your assignment Lest you think it s an Anglo centric world It s It s time to visit The Linguistic Twilight Zone Digression The Linguistic Twilight Zone Perhaps not so strange Perhaps strange Turkish uygarla t ramad klar m zdanm s n zcas na l t d kl d uygar la t r ama d k lar m z dan m s n z cas na behaving as if you are among those whom we could not cause to become civilized Chinese No verb adjective distinction beautiful to be beautiful Digression The Linguistic Twilight Zone Tzeltal Mayan language spoken in Chiapas Only 3000 root forms in the vocabulary The verb EAT has eight variations General TUN Bananas and soft stuff LO LO Beans and crunchy stuff K UX Tortillas and bread WE M t and Meat d Chilies Chili TI Sugarcane TZ U q UCH Liquids Digression The Linguistic Twilight Zone Riau Indonesian Malay No Articles No Tense Marking 3rd person pronouns neutral to both gender and number N ffeatures No t distinguishing di ti i hi verbs b ffrom nouns Digression The Linguistic Twilight Zone Riau Indonesian Malay Ayam y chicken Makan eat The chicken is eating The chicken ate The chicken will eat The chicken is being eaten Where the chicken is eating g How the chicken is eating Somebody is eating the chicken The chicken that is eating B k tto regularly Back l l scheduled h d l d p g programming g POS Tagging What s the task Process of assigning part of speech tags to words But ut what at tags are a e we e go going g to ass assign g z z z Important issues to remember z z Coarse grained noun verb adjective adverb Fine grained proper common noun Even finer grained proper common noun animate Choice of tags encodes certain distinctions non distinctions distinctions non distinctions Tagsets will differ across languages For English g Penn Treebank is the most common tagset g Penn Treebank Tagset 45 Tags Penn Treebank Tagset Choices Example z The DT grand JJ jury NN commmented VBD on IN a DT number NN of IN other JJ topics NNS Distinctions and non distinctions z z Prepositions and subordinating conjunctions are tagged IN Although IN I PRP Except the preposition complementizer to is tagged TO Don tt think this is correct Doesn t Don Doesn t make sense Often must suspend linguistic intuition and defer to the annotation guidelines Why do POS tagging One of the most basic NLP tasks z Useful for higher level analysis z z Nicely illustrates principles of statistical NLP Needed for syntactic analysis Needed for semantic analysis Sample applications that require POS tagging z z z Machine translation Information extraction Lots more Why is it hard Not only a lexical problem z Remember ambiguity Better modeled as sequence labeling problem z Need to take into account context Try your hand at tagging The back door On myy bac O back Win the voters back Promised to back the bill Try your hand at tagging I thought that you That at day was as nice ce You can go that far Why is it hard Part of Speech Tagging How do you do it automatically How o well e does itt work o This first evaluation It s all about the benjamins Evolution of the Evaluation Evaluation by argument Evaluation a uat o by inspection spect o o of e examples a p es Evaluation by demonstration Evaluation by