OverviewIdentifying summary sentencesTheir approachResultsNew dataNatural language summarizationWordNet-based sentiment lexiconText classifierAspect extractionSummarizationA few thoughts on extensionsTextual sentiment summarizationChris PottsLinguist 287 / CS 424P: Extracting Social Meaning and Sentiment, Fall 2010Oct 51 Overview• Visualize summarization is covered in the slideshow ‘Visual sentiment summarization’.• This handout looks briefly at two approaches to textual summarization:– The supervised approach of Beineke et al. (2003).– The largely unsupervised approach of Blair-Goldensohn et al. (2008) (whose lexiconconstruction method is covered in the ‘Sentiment lexicons’ handout).2 Identifying summary sentencesBeineke et al. (2003) seek to extract summary sentences from reviews, using movie reviews fromRottenTomatoes.com. The situation they face is very much like the (daunting) one in tab. 2.2.1 Their approachi. Sentence-level classification: The model predicts, for each sentence S in each review T,whether or not S is the summary sentence for T.ii. Multinomial word features: A number of words are over-represented in summary texts.These tend to be words having to do with emotion and with the general domain (movies).This suggests that word-level features will be useful. Beineke et al. (2003) retain the top1,000 most frequent words for features.iii. Binary location features: Position of the sentence, position of the containing paragraph,position within the containing paragraph.2.2 ResultsOther location variables are used to indicate whether asentence occurs in the final paragraph, whether it is the firstsentence in a paragraph, and whether it is the last sentencein its paragraph.Using these features, we fit statistical models to estimate(3)Our chosen summary sentence for document is the onethat maximizes the above quantity. We fit these models bytwo different methods: Naive Bayes and regularized logisticregression.Naive BayesThe multinomial Naive Bayes model on a dictionaryis afamiliar option for text classification, e.g. (Gale, Church, &Yarowski 1992), (McCallum & Nigam 1998). When thereare additional features, the Naive Bayes model has also anatural extension: We simply assume that each additionalfeature is independent of all the others, conditional upon.In this case, we invert Bayes’ Law by observing:(4)Regularized Logistic RegressionGiven feature vectors and , a linear logistic regressionmodel takes the form:(5)Most often, this model is fit by maximizing the condi-tional likelihood of the parameters for the training giventhe feature values. However, this is not desirable when thenumber of features is too large. In order to prevent over-fitting, a regularization parameter is introduced. Then wehave a modified maximization problem.(6)Here we penalize the coefficients that are associated withtype features but not the ones associated with location fea-tures. This is because type features are only rarely active,whereas location features are frequently active, so their co-efficients are easier to estimate.Regularized logistic regression has been used in other textclassification problems, as in (Zhang & Yang 2003). Forfurther information on regularized model fitting, see for in-stance (Hastie, Tibshirani, & Friedman 2001).ResultsModels were fit using 25 randomly chosen sets of 2000training documents each. Figure 4 shows their success rateat identifying the correct sentence in test documents. Whenthe desired quotation spans multiple sentences, a predictionthat chooses any of them is deemed correct.Method Features Pct. Correct Std. ErrorRandom none 6.3% –Logist. Reg. loc. 14.5% 0.3%Naive Bayes loc.; type 23.1% 0.5%Logist. Reg. loc.; type 25.8% 0.6%Figure 4: Prediction match rateA complication in viewing these results is the fact thatsome review documents contain multiple statements of theiroverall opinion. For instance, the following sentence is pre-dicted as a sentiment summary: “Mulholland Drive is raptand beautiful and absorbing, but apart from a few scenes ...it lacks the revelatory charge that Blue Velvet had 15 yearsago.” Although this does not match the Rotten Tomatoesquotation, it is otherwise an excellent choice.The above example suggests that other approaches can beuseful in evaluating automatically-produced sentiment sum-maries. This is one of many topics for further study in sen-timent summarization. Although Rotten Tomatoes is an ex-cellent source of supervised data in the movie domain, thesummarization problem will differ according to conte xt. Insome cases, we will want methods that do not require largeamounts of domain-specific supervised data. Here we havetreated the problem as one of text classification, but manyapproaches are possible.ReferencesGale, W. A.; Church, K. W.; and Yarowski, D. 1992. Amethod for disambiguating word senses in a large corpus.Computers and the Humanities 26:415–439.Hastie, T.; Tibshirani, R.; and Friedman, J. 2001. TheElements of Statistical Learning: Data Mining, Inference,and Prediction. Springer-Verlag.McCallum, A., and Nigam, K. 1998. A compari-son of learning models for naive bayes text classification.In Working Notes of the 1998 AAAI/ICML Workshop onLearning for Text Categorization.Pang, B.; Lee, L.; and Vaithyanathan, S. 2002. Thumbsup? sentiment classification using machine learning tech-niques. In Proceedings of the 2002 Conference on Empiri-cal Methods in Natural Language Processing (EMNLP).Porter. 1980. An algorithm for suffix stripping. Program14 (3):130–137.Zhang, J., and Yang, Y. 2003. ”robustness of regular-ized lienar classification methods in text categorization”.In Proceedings of the 26th Annual International ACM SI-GIR Conference (SIGIR 2003).Table 1: Beineke et al.’s (2003) results. It should be kept in mind that the chosen summarysentence could easily be just one of many that would serve as an adequate summary. This is boundto be a source of lowered effectiveness scores unless we hand-annotate the data with the full setof intuitively good summary sentences.LING287/CS424P, Stanford (Potts) Sentiment summarizationHelpful: 6 of 7 people found the following review helpfulRating: 5.0 out of 5 starsSummary: If you haven’t already, introduce your child to Max.Review: With "Where The Wild Things Are," Sendak set a new era of Children’s story books - those that appeal toadults and children alike. At the time of its first publishing, "Where The Wild Things Are" brought ground-breaking