Recommender SystemsAgendaWhat are they and Why are theyTypes of RSTypes of RS – Content based RSSlide 6Types of RS – Collaborative RSSlide 8Other Variations of RSSlide 10Slide 11Slide 12Popular RS techniques in E-CommerceImplicit Feedback in RSRelevance to information architectureSome general considerations in RSSlide 17Possible Improvement in RSSlide 19Slide 20Slide 21Slide 22Slide 23Slide 24ExerciseSlide 26Recommender Recommender SystemsSystemsRecommender Recommender SystemsSystemsAalap KohojkarAalap KohojkarYang LiuYang LiuZhan ShiZhan ShiMarch 31, 2008Recommender SystemsAgenda•What are recommender systems•Why are they useful•What are different types of them•Relation with information architecture•Limitations and possible improvements•Relation with Social Networking•Class Exercise!•Q&ARecommender SystemsWhat are they and Why are they•RS – problem of information filtering•RS – problem of machine learning•Enhance user experience–Assist users in finding information–Reduce search and navigation time•Increase productivity •Increase credibility•Mutually beneficial propositionRecommender SystemsTypes of RSThree broad types:1. Content based RS2. Collaborative RS3. Hybrid RSRecommender SystemsTypes of RS – Content based RSContent based RS highlights–Recommend items similar to those users preferred in the past–User profiling is the key–Items/content usually denoted by keywords–Matching “user preferences” with “item characteristics” … works for textual information–Vector Space Model widely usedRecommender SystemsTypes of RS – Content based RSContent based RS - Limitations–Not all content is well represented by keywords, e.g. images –Items represented by same set of features are indistinguishable–Overspecialization: unrated items not shown–Users with thousands of purchases is a problem–New user: No history available–Shouldn’t show items that are too different, or too similarRecommender SystemsTypes of RS – Collaborative RSCollaborative RS highlights–Use other users recommendations (ratings) to judge item’s utility–Key is to find users/user groups whose interests match with the current user–Vector Space model widely used (directions of vectors are user specified ratings)–More users, more ratings: better results–Can account for items dissimilar to the ones seen in the past too–Example: Movielens.orgRecommender SystemsTypes of RS – Collaborative RSCollaborative RS - Limitations–Different users might use different scales. Possible solution: weighted ratings, i.e. deviations from average rating –Finding similar users/user groups isn’t very easy–New user: No preferences available–New item: No ratings available–Demographic filtering is required–Multi-criteria ratings is requiredRecommender SystemsOther Variations of RSCluster Models–Create clusters or groups–Put a customer into a category–Classification simplifies the task of user matching–More scalability and performance–Lesser accuracy than normal collaborative filtering methodRecommender SystemsOther Variations of RSItem to item collaboration (one that Amazon.com uses)–Compute similarity between item pairs–Combine the similar items into recommendation list–Vector corresponds to an item, and directions correspond to customers who have purchased them–“Similar items” table built offline–Example: Amazon.com ExampleRecommender SystemsOther Variations of RSAlgorithm for Amazon’s item to item collaborative filteringFor each item in product catalog, I1For each customer C who purchased I1For each item I2 purchased by customer CRecord that a customer purchased I1and I2For each item I2Compute the similarity between I1 and I2Similarity between two items depends on number of customers who bought them bothRecommender SystemsOther Variations of RSKnowledge based RS–Use knowledge of users and items–Conversational Interaction used to establish current user preferences–i.e. “more like this”, “less like that”, “none of those” … –No user profiles maintained, preferences drawn through manual interaction–Query by example … tweaking the source example to fetch resultsRecommender SystemsPopular RS techniques in E-Commerce•Browsing•Similar Item/s•Email•Text Comments•Average Rating•Top-N results•Ordered search resultsRecommender SystemsImplicit Feedback in RSObservable behavior for implicit feedbackRecommender SystemsRelevance to information architecture•Increase findability•Reduce searching efforts •Improve organizational systems•Enhance browsing •Provide more useful “local navigation” options•“Targeted Advertising” a much better substitute to common advertisements that are often irrelevantRecommender SystemsSome general considerations in RSDifficult to Set Up–Lot of development required for setup–Moving to RS takes time, energy and long-term commitmentThey could be wrong–RS not just a technical challenge, but also a social challenge–Amazon took some heat when it started cross-promoting its new Clothing site by recommending clean underwear to people who were shopping for DVDMaintenanceRecommender SystemsSome general considerations in RS•Context is important in “user X items” space•Similarity is a non-uniform concept, is highly contextual and task-oriented•Users sometimes need motivation to rate itemsRecommender SystemsPossible Improvement in RSBetter understanding of users and items–Social network (social RS)1. User level–Highlighting interests, hobbies, and keywords people have in common2. Item level–link the keywords to eCommerce (by RS algorithms)Recommender SystemsPossible Improvement in RSSystem transparency–Help users understand how the RS works–Example: http://www.pandora.com/Amazon.comResult:–Generate trust–Convince usersRecommender SystemsPossible Improvement in RSMultidimensionality of Recommendations–Take into consideration the contextual informationExamples: MovieTravelRecommender SystemsPossible Improvement in RSRandomnessRecommender SystemsPossible Improvement in RSOther–GiftAmazon–Privacy (CF methods)One-way hash: easily computed one direction, impossible in the other–Malicious use (recommendation spam)Probabilistic techniques to determine the honesty of a score (unusual pattern)Recommender SystemsPossible Improvement in RSCommon business models adapted:–Charge recipient of recommendations–Provide incentives for