Predicting Box Office Success: Do Critical Reviews Really Matter?By: Alec KennedyIntroduction:Information economics looks at the importance of information in economic decision-making. Consumers that hold more information, it theorizes, will make decisions that yield higher expected payoffs.So when a consumer decides to go see a movie, he or she should want to get as much information about the movie beforehand as possible. So where can they get this information? Well, movie trailers offer much information but only for the ones that spend enough money to have the trailer widely distributed. And these trailers only reach the people who actually go to the theater or watch television enough to catch them. Some movies start their advertising campaign a year in advance in order to guarantee that their movie is well known by the time of its release. But, how much can these trailers tell the consumer about the quality of the movie? They can’t really. This is where the movie critics come in.In order to obtain the best information about the quality of a movie before a movie is actually released, consumers must rely on movie critics as their guide. Movie critics act as advisors to consumers telling them which movies will be worth their money. Their reviews can tell their readers, before they decide to see a movie or not, how funny, entertaining, well-acted, and gripping a variety of movies are. Readers can then take this information and use it to decide whether to spend their money on a movie or a more worthwhile alternative.Looking at this situation from the movie industry’s perspective, if there is a link between critical reviews and getting people out to see a movie, this could help with distribution decision-making. If a movie does well in test screenings or if they anticipate good reviews from the critics then they can decide to release it on opening weekend in more theaters in hopes of bringing in more revenue. Even though it may be hard to predict the reviews of critics, some movies often come with “Oscar Buzz” where they are expected, very early, to gain very good critical reviews. Also, some movies give advance screenings to critics to test their reactions to it. Movies that gain glowing reviews from these advance screenings (such as 2008’s “The Curious Case of Benjamin Button” which has already been pegged as a sure-thing Best Picture nomination when it has not even been released) may want to spend the extra money to release it in more theaters if they believe that these positive reviews can lead to a more profitable movie.My Sources for the Data:I must first acknowledge Timothy King’s article “Does film criticism affect box office earnings? Evidence from movies released in the U.S. in 2003.” Most of the ideas presented in this study are based on the ideas from this article.For this study I took data from 220 movies released in 2007. I looked at domestic box office grosses, opening weekend theaters, production costs (when given), and domestic weekly box office grosses (for 22 selected movies). All of these were collected from boxofficemojo.com. Critical reviews scores were taken from metacritic.com.Metacritic is a website that aggregates critical reviews for music albums, games, books, and movies. What they do is they look at reviews from various sources (listed on their site) and assign a score to them (from a 100 point scale). The conversions are given on their website, but for example, if a film receives 3.5 stars out of 4 stars it would receive 88 points out of 100. Metacritic also adds greater weights to scores given by certain reviewers who generally write more detailed reviews. This adds greater weight to a reviewer from The New Yorker as opposed to TV Guide. This adds a bit of subjectivity to their scores, but when plotted against scores givenby the less subjective rottentomatoes.com we see that they are highly correlated suggesting that this subjectivity does not have much effect on distorting metacritic scores. A more detailed explanation can be found in King’s article.Metacritic also gives Internet users a chance to weigh in their opinion on the movies as well (“The User Score”). Users are given the chance to rate a movie on a 10-point scale. These scores are then averaged and placed on the page next to the averaged critical score. I chose the 220 movies from 2007 by taking only the movies that had at least 10 reviews in both the critical and user categories in order to make sure these averages weren’t too biased. Although, it seems, that this exclusion could have an effect on some of my later results.User and Critical Scores:If we are trying to look at whether critical reviews are giving valuable information to moviegoers we might first want to take a look to whether they typically agree on the quality of movies. Using R we can plot the user scores vs. the critical scores:We see from the plot that there is an obvious positive trend suggesting that higher critical reviews generally get higher user reviews. When calculating the correlation we find that r = 0.7, which suggests that there is a very strong relationship between these two scores. But when we look at the average difference between the two scores we see that users tend to be more generous (Average Differences (Users-Critics) = 8.13) than critics in their evaluations of movies. When applying a t-test we find a p-value of 1.190e-06. This small value suggests that these two scores do come from different distributions and that users are more generous than the critics. This difference does not seem to be large enough to say that the critics views and the user views differ that much, this is because 8 points corresponds to less than half of a star when looking at four star reviews. We cannot conclude from this plot that critics and the general population totally agree on the quality of a film. Users on metacritic are self-selecting. We have to think about the type of people who would go onto metacritic and rate the movies themselves. It seems reasonable to believe that these users would include the people who are huge fans of the movies that they are rating (as well as huge “haters” of the film). Also it seems reasonable to believe that the people who have heard about metacritic and are an active part of its program are part of a younger 20 40 60 8020 40 60 80Us vs. The CriticsCriticsUsdemographic. But the fact that there is such a high correlation is still very interesting,