Unsupervised sentiment classification of English movie reviewsusing automatic selection of positive and negative sentiment itemsJohn Rothfels Julie TibshiraniJune 2, 2010AbstractWe consider the problem of classifyingdocuments not by topic, but by overall sen-timent. Previous approaches to sentimentclassification have favored domain-specific,supervised machine learning (Naive Bayes,maximum entropy classification, and supportvector machines). Inherent in these method-ologies is the need for annotated trainingdata. Building on previous work, we ex-amine an unsupervised system of iterativelyextracting positive and negative sentimentitems which can be used to classify docu-ments. Our method is completely unsuper-vised and only requires linguistic insight intothe semantic orientation of sentiment.1 IntroductionHumans love categorization. Robert Sapolsky, a pro-fessor of biology at Stanford University who studieshuman behavior, explains that categorizing or “buck-eting” (no intentional homage to hashing, alas) infor-mation enables the human brain to process informa-tion in a more meaningful and natural way. As anexample, humans may not be able to tell the differ-ence between two beams of light at a wavelengths of510nm and 511nm, but it is easy and natural to calleach “green”. The green “bucket” thus encompassesa range of wavelengths which through a single wordwe may call the “same”, more or less.Today, very large amounts of information are pub-licly available online. As part of an effort to give or-ganization to this sprawling mess of data, researchershave studied the problem of text categorization, i.e.assigning a label to text on the basis of topical sub-ject. More recent studies have focused on the sub-ject of sentiment classification, i.e. assigning a labelto text on the basis of the overall sentiment of theauthor (positive or negative, for example). The ap-plication of such research is immediate. Often whatpeople care about when reading a review, for exam-ple, is not what the review says exactly but whether itis holistically positive or negative. This type of classi-fication can be used not only in the consumer sector,but also in business sector for product recommenda-tion or inflammatory message filtering. In general,we see that sentiment classification allows us to ex-tract something quantitative out of a vast amountsof qualitative information.Pang, Lee, and Vaithyanathan [7] concluded thatsentiment classification is inherently more difficultthan text categorization, finding as one example thata common phenomenon in classifying the sentiment offilms was a “thwarted expectations” narrative, wherethe author sets up a deliberate contrast to earlier dis-cussion: “This film should be brilliant. It soundslike a great plot, the actors are first grade, and thesupporting cast is good as well, and Stallone is at-tempting to deliver a good performance. However,it cant hold up”. More generally, we acknowledgethat the subjective nature of sentiment makes classi-fication particularly difficult (even human annotatorscan have difficulty detecting the sentiment of certaintext).On the spectrum of sentiment, we make one nec-essary but grossly simplifying assumption: that senti-ment is binary (i.e positive or negative). The assump-tion, while deliberate, results in a serious problem,namely that the difference between sentiment “buck-ets”, to use Sapolsky’s term, is magnified. Two doc-uments differing only slightly in sentiment (one everso slightly positive and one ever so slightly negative)will appear markedly dissimilar by our classificationscheme, in much the same way that an artificial wave-length boundary separates blue from green while twobeams of light on either side of this boundary lookidentical. Thus at some level, sentiment classifica-1tion is an all-too-artificial construct. These issuesacknowledged, we press on.2 Related Work2.1 Supervised MethodsSupervised approaches to sentiment classificationhave attracted quite a bit of recent attention. Pang,Lee, and Vaithyanathan [7] compared multiple su-pervised machine learning algorithms (Naive Bayes,maximum entropy classifiers, support vector ma-chines) for the task of sentiment classification ofmovie reviews. They experimented with a wide va-riety of features and obtained accuracy as high as82.9% when classifying movie reviews. Neverthe-less, they were not able to achieve accuracies on sen-timent classification comparable to those reportedfor standard topic-based categorization. Many typi-cal misclassifications were easily recognizable by hu-mans. Experimentation with feature selection hasmanifested in other research: Dave, Lawrence, andPennock [2] experimented with the use of linguistic,statistical, and n-gram features and measures for fea-ture selection and weighting. Pang and Lee [5] useda graph-based technique to identify and analyze onlysubjective parts of texts. Yu and Hatzivassiloglou [9]use semantically oriented words for identification ofpolarity at the sentence level.In all supervised approaches, reasonably high ac-curacy can be obtained subject only to the require-ment that test data be similar to training data. Tomove a supervised sentiment classifier to another do-main would require collecting annotated data in thenew domain and retraining the classifier. This de-pendency on annotated training data is one majorshortcoming of all supervised methods.2.2 Unsupervised MethodsUnsupervised approaches to sentiment classificationcan solve the problem of domain dependency and re-duce the need for annotated training data. Turney [8]uses two arbitrary seed words (poor and excellent) tocalculate the semantic orientation of phrases, wherethe orientation of a phrase is defined as the differ-ence of its association with each of the seed words(as measured by pointwise mutual information). Thesentiment of a document is calculated as the aver-age semantic orientation of all such phrases. Thisapproach was able to achieve 66% accuracy for themovie review domain at the document level.Zagibalov and Carroll [10] describe a method ofautomatic seed word selection for unsupervised sen-timent classification of product reviews in Chinese.The method only requires information about com-monly occurring negations and adverbials in order toiteratively find sentiment bearing items. The resultsobtained are close to those of supervised classifiersand sometimes better, up to an F1score of 92%. Wediscuss their strategy in detail in the
View Full Document
Unlocking...