Slide 1Reminder: lemma and wordformLemmas have sensesHomonymyHomonymy causes problems for NLP applicationsPolysemySlide 7How do we know when a word has more than one sense?SynonymsSynonymsSynonymy is a relation between senses rather than wordsAntonymsHyponymy and HypernymyHyponymy more formallyHyponyms and InstancesSlide 16Slide 17Applications of Thesauri and OntologiesWordNet 3.0Senses of “bass” in WordnetHow is “sense” defined in WordNet?WordNet Hypernym Hierarchy for “bass”WordNet Noun RelationsWordNet 3.0Slide 25The MeSH HierarchyUses of the MeSH OntologySlide 28Slide 29Word SimilarityWhy word similarityWord similarity and word relatednessTwo classes of similarity algorithmsPath based similarityRefinements to path-based similaritySlide 36Problem with basic path-based similarityInformation content similarity metricsInformation content similarityInformation content similarityInformation content: definitionsUsing information content for similarity: the Resnik methodDekang Lin methodDekang Lin similarity theoremLin similarity functionThe (extended) Lesk AlgorithmSummary: thesaurus-based similarityLibraries for computing thesaurus-based similarityEvaluating similaritySlide 50Slide 51Problems with thesaurus-based meaningDistributional models of meaningIntuition of distributional word similarityReminder: Term-document matrixReminder: Term-document matrixThe words in a term-document matrixThe words in a term-document matrixThe Term-Context matrixSample contexts: 20 words (Brown corpus)Term-context matrix for word similarityShould we use raw counts?Pointwise Mutual InformationComputing PPMI on a term-context matrixSlide 65Slide 66Weighing PMISlide 68Slide 69Slide 70Slide 71Using syntax to define a word’s contextCo-occurrence vectors based on syntactic dependenciesPMI applied to dependency relationsReminder: cosine for computing similarityCosine as a similarity metricSlide 77Other possible similarity measuresEvaluating similarity (the same as for thesaurus-based)Slide 80Word Meaning and SimilarityWord Senses and Word RelationsDan JurafskyReminder: lemma and wordform•A lemma or citation form•Same stem, part of speech, rough semantics•A wordform•The “inflected” word as it appears in textWordform Lemmabanks banksung singduermes dormirDan JurafskyLemmas have senses•One lemma “bank” can have many meanings:•…a bank can hold the investments in a custodial account…•“…as agriculture burgeons on the east bank the river will shrink even more”•Sense (or word sense)•A discrete representation of an aspect of a word’s meaning.•The lemma bank here has two senses12Sense 1:Sense 2:Dan JurafskyHomonymyHomonyms: words that share a form but have unrelated, distinct meanings:•bank1: financial institution, bank2: sloping land•bat1: club for hitting a ball, bat2: nocturnal flying mammal1. Homographs (bank/bank, bat/bat)2. Homophones:1. Write and right2. Piece and peaceDan JurafskyHomonymy causes problems for NLP applications•Information retrieval•“bat care”•Machine Translation•bat: murciélago (animal) or bate (for baseball)•Text-to-Speech•bass (stringed instrument) vs. bass (fish)Dan JurafskyPolysemy•1. The bank was constructed in 1875 out of local red brick.•2. I withdrew the money from the bank •Are those the same sense?•Sense 2: “A financial institution”•Sense 1: “The building belonging to a financial institution”•A polysemous word has related meanings•Most non-rare words have multiple meaningsDan Jurafsky•Lots of types of polysemy are systematic•School, university, hospital•All can mean the institution or the building.•A systematic relationship:•Building Organization•Other such kinds of systematic polysemy: Author (Jane Austen wrote Emma) Works of Author (I love Jane Austen)Tree (Plums have beautiful blossoms) Fruit (I ate a preserved plum)Metonymy or Systematic Polysemy: A systematic relationship between sensesDan JurafskyHow do we know when a word has more than one sense?•The “zeugma” test: Two senses of serve?•Which flights serve breakfast?•Does Lufthansa serve Philadelphia?•?Does Lufthansa serve breakfast and San Jose?•Since this conjunction sounds weird, •we say that these are two different senses of “serve”Dan JurafskySynonyms•Word that have the same meaning in some or all contexts.•filbert / hazelnut•couch / sofa•big / large•automobile / car•vomit / throw up•Water / H20•Two lexemes are synonyms •if they can be substituted for each other in all situations•If so they have the same propositional meaningDan JurafskySynonyms•But there are few (or no) examples of perfect synonymy.•Even if many aspects of meaning are identical•Still may not preserve the acceptability based on notions of politeness, slang, register, genre, etc.•Example:•Water/H20•Big/large•Brave/courageousDan JurafskySynonymy is a relation between senses rather than words•Consider the words big and large•Are they synonyms?•How big is that plane?•Would I be flying on a large or small plane?•How about here:•Miss Nelson became a kind of big sister to Benjamin.•?Miss Nelson became a kind of large sister to Benjamin.•Why?•big has a sense that means being older, or grown up•large lacks this senseDan JurafskyAntonyms•Senses that are opposites with respect to one feature of meaning•Otherwise, they are very similar!dark/light short/long fast/slow rise/fallhot/cold up/down in/out•More formally: antonyms can•define a binary opposition or be at opposite ends of a scale• long/short, fast/slow•Be reversives:• rise/fall, up/downDan JurafskyHyponymy and Hypernymy•One sense is a hyponym of another if the first sense is more specific, denoting a subclass of the other•car is a hyponym of vehicle•mango is a hyponym of fruit•Conversely hypernym/superordinate (“hyper is super”)•vehicle is a hypernym of car•fruit is a hypernym of mangoSuperordinate/hyper vehiclefruit furnitureSubordinate/hyponymcar mangochairDan JurafskyHyponymy more formally•Extensional:•The class denoted by the superordinate extensionally includes the class denoted by the hyponym•Entailment:•A sense A is a hyponym of sense B if being an A entails being a B•Hyponymy is usually transitive •(A hypo B and B hypo C entails A hypo C)•Another name: the IS-A hierarchy•A IS-A B (or A ISA B)•B subsumes ADan