CS6322: CS6322: Information Retrieval Information Retrieval Sanda HarabagiuSanda HarabagiuLecture 14: Query expansion Lecture 14: Query expansion and Relevance Feedbackand Relevance FeedbackCS6322: Information RetrievalCS6322: Information RetrievalThis lecture Improving results For high recall. E.g., searching for aircraft doesn’t match with plane; nor thermodynamic with heat Options for improving results… Global methods Query expansion Thesauri Automatic thesaurus generation Local methods Relevance feedback Pseudo relevance feedbackCS6322: Information RetrievalCS6322: Information RetrievalRelevance Feedback Relevance feedback: user feedback on relevance of docs in initial set of results User issues a (short, simple) query The user marks some results as relevant or non-relevant. The system computes a better representation of the information need based on feedback. Relevance feedback can go through one or more iterations. Idea: it may be difficult to formulate a good query when you don’t know the collection well, so iterateSec. 9.1CS6322: Information RetrievalCS6322: Information RetrievalRelevance feedback We will use ad hoc retrieval to refer to regular retrieval without relevance feedback. We now look at four examples of relevance feedback that highlight different aspects.Sec. 9.1CS6322: Information RetrievalCS6322: Information RetrievalSimilar pagesCS6322: Information RetrievalCS6322: Information RetrievalRelevance Feedback: Example Image search engine http://nayana.ece.ucsb.edu/imsearch/imsearch.htmlSec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalResults for Initial QuerySec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalRelevance FeedbackSec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalResults after Relevance FeedbackSec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalAd hoc results for query caninesource: Fernando DiazCS6322: Information RetrievalCS6322: Information RetrievalAd hoc results for query caninesource: Fernando DiazCS6322: Information RetrievalCS6322: Information RetrievalUser feedback: Select what is relevantsource: Fernando DiazCS6322: Information RetrievalCS6322: Information RetrievalResults after relevance feedbacksource: Fernando DiazCS6322: Information RetrievalCS6322: Information RetrievalInitial query/results Initial query: New space satellite applications1. 0.539, 08/13/91, NASA Hasn’t Scrapped Imaging Spectrometer2. 0.533, 07/09/91, NASA Scratches Environment Gear From Satellite Plan3. 0.528, 04/04/90, Science Panel Backs NASA Satellite Plan, But Urges Launches of Smaller Probes4. 0.526, 09/09/91, A NASA Satellite Project Accomplishes Incredible Feat: Staying Within Budget5. 0.525, 07/24/90, Scientist Who Exposed Global Warming Proposes Satellites for Climate Research6. 0.524, 08/22/90, Report Provides Support for the Critics Of Using Big Satellites to Study Climate7. 0.516, 04/13/87, Arianespace Receives Satellite Launch Pact From Telesat Canada8. 0.509, 12/02/87, Telecommunications Tale of Two Companies User then marks relevant documents with “+”.+++Sec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalExpanded query after relevance feedback 2.074 new 15.106 space 30.816 satellite 5.660 application 5.991 nasa 5.196 eos 4.196 launch 3.972 aster 3.516 instrument 3.446 arianespace 3.004 bundespost 2.806 ss 2.790 rocket 2.053 scientist 2.003 broadcast 1.172 earth 0.836 oil 0.646 measureSec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalResults for expanded query1. 0.513, 07/09/91, NASA Scratches Environment Gear From Satellite Plan2. 0.500, 08/13/91, NASA Hasn’t Scrapped Imaging Spectrometer3. 0.493, 08/07/89, When the Pentagon Launches a Secret Satellite, Space Sleuths Do Some Spy Work of Their Own4. 0.493, 07/31/89, NASA Uses ‘Warm’ Superconductors For Fast Circuit5. 0.492, 12/02/87, Telecommunications Tale of Two Companies6. 0.491, 07/09/91, Soviets May Adapt Parts of SS-20 Missile For Commercial Use7. 0.490, 07/12/88, Gaping Gap: Pentagon Lags in Race To Match the Soviets In Rocket Launchers8. 0.490, 06/14/90, Rescue of Satellite By Space Agency To Cost $90 Million218Sec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalKey concept: Centroid The centroid is the center of mass of a set of points Recall that we represent documents as points in a high-dimensional space Definition: Centroidwhere C is a set of documents.∑∈=CddCCrr||1)(µSec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalRocchio Algorithm The Rocchio algorithm uses the vector space model to pick a relevance fed-back query Rocchio seeks the query qoptthat maximizes Tries to separate docs marked relevant and non-relevant Problem: we don’t know the truly relevant docs))](,cos())(,[cos(maxargnrrqoptCqCqqµµrrrrrr−=∑∑∉∈−=rjrjCdjnrCdjroptdCdCqrrrrr11Sec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalThe Theoretically Best Query xxxxoooOptimal queryx non-relevant documentso relevant documentsoooxxxxxxxxxxxx∆xxSec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalRocchio 1971 Algorithm (SMART) Used in practice: Dr = set of known relevant doc vectors Dnr= set of known irrelevant doc vectors Different from Cr and Cnr qm= modified query vector; q0= original query vector; α,β,γ: weights (hand-chosen or set empirically) New query moves toward relevant documents and away from irrelevant documents∑∑∈∈−+=nrjrjDdjnrDdjrmdDdDqqrrrrrr110γβα!Sec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalSubtleties to note Tradeoff α vs. β/γ : If we have a lot of judged documents, we want a higher β/γ. Some weights in query vector can go negative Negative term weights are ignored (set to 0)Sec. 9.1.1∑∑∈∈−+=nrjrjDdjnrDdjrmdDdDqqrrrrrr110γβαCS6322: Information RetrievalCS6322: Information RetrievalRelevance feedback on initial query xxxxoooRevised queryx known non-relevant documentso known relevant documentsoooxxxxxxxxxxxx∆xxInitial query∆Sec. 9.1.1CS6322: Information RetrievalCS6322: Information RetrievalRelevance Feedback in vector spaces We can modify the query based on relevance feedback and apply standard vector space model. Use only the docs that were marked. Relevance
View Full Document
Unlocking...