CMSC 723 Computational Linguistics I Session 10 Semantic Distance Jimmy Lin The iSchool University of Maryland Wednesday November 4 2009 Material drawn from slides by Saif Mohammad and Bonnie Dorr Progression of the Course Words z z Structure z z Finite state morphology Part of speech tagging TBL HMM CFGs parsing CKY Earley N gram language models Meaning Today s Agenda Lexical semantic relations WordNet o d et Computational approaches to word similarity Lexical Semantic Relations What s meaning Let s start at the word level How o do you de define e tthe e meaning ea g o of a word od Look it up in the dictionary Well that really doesn t Well doesn t help help Approaches to meaning Truth conditional Semantic Se a t c network et o Word Senses Word sense distinct meaning of a word Same Sa e word o d d different e e t se senses ses z Homonyms homonymy unrelated senses identical orthographic form is coincidental Example E l fi financial i l iinstitution tit ti vs side id off river i ffor b bank k z Polysemes polysemy related but distinct senses Example financial institution vs sperm bank z Metonyms M t metonymy t stand t d in i ttechnically h i ll a sub case b off polysemy Examples author for works or author building for organization capital city it for f governmentt Different word same sense z Synonyms synonymy Just to confuse you Homophones same pronunciation different orthography different meaning z Examples would wood to too two Homographs distinct senses same orthographic form different pronunciation z Examples bass fish vs bass instrument Relationship Between Senses IS A relationships z From specific to general up hypernym hypernymy Example bird is a hypernym of robin z From general to specific down hyponym hyponymy Example p robin is a hyponym yp y of bird Part Whole relationships z z wheel is a meronym of car meronymy car is a holonym of wheel holonymy WordNet Tour Material drawn from slides by Christiane Fellbaum What is WordNet A large lexical database developed and maintained at Princeton University Includes most English nouns verbs adjectives adverbs Electronic format makes it amenable to automatic manipulation used in many NLP applications WordNets generically refers to similar resources in other languages WordNet History Research in artificial intelligence z z z How do humans store and access knowledge about concept Hypothesis concepts are interconnected via meaningful relations Useful for reasoning The WordNet project started in 1986 z z Can most all of the words in a language be represented as a semantic network where words are interlinked by meaning If so the result would be a large semantic network Synonymy in WordNet WordNet is organized in terms of synsets z Unordered set of roughly synonymous words or multi word phrases Each synset expresses a distinct meaning concept WordNet Example Noun pipe p p tobacco p pipe p a tube with a small bowl at one end used for smoking tobacco pipe pipage piping a long tube made of metal or plastic that is used to carry water or oil or gas etc pipe tube a hollow cylindrical shape pipe pipe a tubular wind instrument organ pipe pipe pipework the flues and stops on a pipe organ Verb shriek shrill pipe up pipe utter a shrill cry pipe transport by pipeline pipe oil water and gas into the desert pipe play on a pipe pipe pipe a tune tune pipe trim with piping pipe the skirt Observations about sense granularity The Net Part of WordNet conveyance transport hyperonym vehicle bumper hyperonym motor vehicle automotive vehicle meronym car door hyperonym yp y meronym car auto automobile machine motorcar meronym hyperonym doorlock meronym car window car mirror hyperonym cruiser squad car patrol car police car prowl car hinge flexible joint cab taxi hack taxicab armrest WordNet Size Part of speech Word form Synsets Noun 117 798 82 115 Verb 11 529 13 767 Adjective 21 479 18 156 Adverb 4 481 4 481 3 621 3 621 Total 155 287 117 659 http wordnet princeton edu MeSH Medical Subject Headings another example of a theasuri z http www nlm nih gov mesh MBrowser html Thesauri ontologies taxonomies etc Word Similarity Intuition of Semantic Similarity Semantically close z z z z z z z z bank money b k apple fruit tree forest bank river pen paper p p p run walk mistake error car wheel Semantically distant z z z z z z z z doctor beer d t b painting January money river apple penguin nurse fruit pen river clown tramway car algebra 19 Why Meaning z World knowledge z The two concepts are close in terms of their meaning The two concepts have similar properties often occur together or occur in similar contexts Psychology z We often think of the two concepts together 20 Two Types of Relations Synonymy two words are roughly interchangeable Semantic similarity distance somehow related z Sometimes explicit lexical semantic relationship Sometimes relationship often often not 21 Validity of Semantic Similarity Is semantic distance a valid linguistic phenomenon Experiment pe e t Rubenstein ube ste a and d Goode Goodenough oug 1965 965 z z Compiled a list of word pairs Subjects asked to judge semantic distance from 0 to 4 for each of the word pairs Results z z Rank correlation between subjects is 0 0 9 9 People are consistent 22 Why do this Task automatically compute semantic similarity between words Theoretically useful for many applications z z z z Detecting paraphrases i e automatic essay grading plagiarism detection Information retrieval Machine translation Solution in search of a problem Types of Evaluations Intrinsic z z Internal to the task itself With respect to some pre defined criteria Extrinsic z Impact on end to end task Analogy with cooking 24 Evaluation Correlation with Humans Ask automatic method to rank word pairs in order of semantic distance Compare this ranking with human created ranking Measure correlation 25 Evaluation Word Choice Problems Identify that alternative which is closest in meaning to g the target accidental wheedle ferment inadvertent abominate imprison incarcerate writhe meander inhibit 26 Evaluation Malapropisms Jack withdrew Jac t d e money o ey from o tthe e ATM next e t to tthe e ba band d band is unrelated to all of the other words in its context 27 Evaluation Malapropisms Jack withdrew Jac t d e money o ey from o tthe e ATM next e t to tthe e ba bank Wait you mean bank 28 Evaluation Malapropisms Actually semantic distance is a poor technique What s at s a simple s p e better bette so solution ut o Even still task can be used for a fair