CS 224N Final ProjectBoost up! Sentiment Categorization with MachineLearning TechniquesAndr´es Cassinelli, Chih-Wei ChenJune 5, 2009AbstractWe address the problem of categorizing documents by overall sentiment into twoclasses (i.e. positive or negative) and into multiple classes (e.g. one to five stars).We apply machine learning techniques to categorize a data set of movie reviews. Inparticular, we use a boosting algorithm. For determining the polarity of a review, wefound that the algorithm has an interpretation similar to previous work in sentimentanalysis, yet it achieves better accuracy in a more efficient way. Similar results canbe seen when we apply the techniques to the multi-class categorization task. Withoutexplicitly using the relationships of different labels during training, our classifier candiscover the sentime nt affinity between categories.1 IntroductionGiven the vast amount of unstructured information available online today, th ere is much tobe gained from the development of automated systems that can effectively organize and clas-sify this data, so that it can be leveraged by human users in a meaningful way. While it canbe useful to categorize this kind of information according to its subject matter, classifyingit according to the authors’ opinions, or sentiment, can also provide researchers, businessleaders, and policy-makers with valuable information ranging from rates of customer satis-faction to public opinion trends. Sentiment analysis has drawn great interest in recent yearsbecause of the surge of subjective content (blog posts, movie and restaurant reviews, etc.)being created and shared by Internet users, and the scope of new applications enabled byunderstanding the sentiments embedded in that content. For example, extracting the senti-ment of a review can help provide succinct summaries to readers, and can be very useful inautomatically generating recommendations for users.Sentiment classification can also help determine the perspective of different sources ofinformation, and yet another possible application would be the processing of answers toopinion questions. Specifically within the field of reviews, the numerical ratings that come1with many of them enable us to categorize them into finer-grained scales than just positive ornegative categories. This richer information makes it possible to rank items or quantitativelycompare opinions of several reviewers, thus allowing more nuanced analyses to be carriedout.2 Background and Related WorkWhile m ovie reviews may not b e the domain where the application of sentiment analysishas the greatest potential to generate value, monetary or otherwise, it is an interesting andchallenging testing ground for different sentiment classification approaches. After evaluatingthe performance of a simple unsupervised learning algorithm in the binary sentiment analysisof reviews across four different domains, Turney (2002) found movie reviews to be the mostdifficult. It was hypothesized that this was due to the tendency of reviewers to rate theindividual elements of a movie differently from the movie as a whole within the same review,and addressing this issue remained a matter of future work. However, as Pang et al. (2002)suggest, the machine learning methods and features used when classifying movie reviews donot have to be specific to that domain. If a significant level of accuracy can be achieved inthis difficult domain through a generalizable approach, then the benefits may easily transferto other areas where sentiment classification can be applied.For binary sentiment classification, Turney proposed counting the positive and negativeterms and expressions in a review to determine its polarity. This idea was then augmentedby Kenn edy and Inkpen (2006), who also took contextual valence shifters, such as negationwords, intensifiers, and diminishers, into account, and managed to improve the accuracy ofthe system. In this case, positive and negative terms were identified by querying a collectionof dictionaries. In the same paper, a second method that uses Support Vector Machines(SVMs) was also proposed, and it was shown that this machine learning algorithm performssignificantly better than the term-counting method with valence shifters.As far as machine learning methods for sentiment analysis are concerned, Pang et al.(2002) and Pang and Lee (2004) have compared the performance of various classifiers whendetermining the sentiment of a document, and also found that SVMs were generally the bestapproach. Unigrams, bigrams, part-of-speech (POS) tags and term positions were consideredas features, and using unigrams alone yielded the best results. Thus, SVMs have repeatedlyemerged as remarkably effective tools for performing sentiment analysis, even when usingvery simple features in the classifier.3 Problem Statement and Data SetWe propose to further improve the aforementioned systems by applying machine learningtechniques to learn the weights for each term, since these terms may not be equal in theirstrengths. For the binary classification task (i.e. determining whether reviews are positive ornegative), we will use a data set of classified movie reviews prepared by Pang and Lee (2004).2The data set contains 1,000 positive and 1,000 negative reviews collected from Internet MovieDatabase (IMDb) archive rec.arts.movies.reviews.For multi-class sentiment analysis, Pang and Lee (2005) provided another data set withnormalized subjective ratings on a numerical rating scale. They applied a meta-algorithmbased on a metric labeling formulation of the problem. The ratings were quantized to threeclasses and four classes, respectively, for different experiments.4 MethodOur aim is to solve the sentiment classification task for movie reviews using machine learn-ing techniques. Under a machine learning framework, we have a dataset of m instances((x1, y1), . . . , (xm, ym)), where xiis the feature vector extracted from the ith data instance,and yiis the label for that particular data instance. The feature can be, for example, aboolean value indicating the presence of the unigram word “great”; the nu mber of appear-ances of the bigram “fully awesome” in the document; or any other convoluted value thatcaptures the characteristics of the data instance. We will explain how features are selectedfor our work. The labels in our task are “positive” or “negative” for binary classification,and a nu mber of stars for estimating the
View Full Document
Unlocking...