The Effects of Review Aggregator Sites on Film Marketing
Introduction
In recent years, a trend has emerged in film marketing, particularly in how films are advertised on television, which emphasizes a film’s performance on the popular review aggregator site Rotten Tomatoes, in order to promote a film’s quality hopefully attracting more theatrical viewers. Though Rotten Tomatoes has gained widespread popularity as a source for movie reviews among audiences, this usage begs the question as to whether this type of advertising is actually warranted. Furthermore, given the slew of other viable review aggregator sites, such as IMDB and Metacritic, it also seems worth understanding whether or not any of these competing sites could serve as a more successful motivator for driving audiences to theaters. In this analysis, I will explore both of these questions, not as a means to predict the financial success of a film based on its reviews, but as an exploratory means to identify which types of reviews offer the most success to studios and viewers alike.
Data
The data used in this analysis has been scraped from Box Office Mojo for variables specifically relating to the film’s attributes (such as any variables corresponding to genre, rating, or monetary values) while all ratings data have been sourced from their aggregator websites. Films were only included in the dataset if they had a wide release (600 theaters or more) and if this release took place on or after January 1st, 2000. In total, 3200 films were included. All monetary values have been adjusted to the 2020 US dollar and Total Gross serves the place of Domestic Gross as the motivation for this analysis mainly concerned marketing tactics within the U.S.
Analaysis
As a preliminary exploration of our data, we use a localized regression on the ratings variables for each individual film and their total domestic gross in order to better understand the relationship between each review aggregator site and a film’s success at the box office. We would expect an aggregator site with a stronger relationship to box office returns to have a higher average total gross for positive reviews and lower total gross for negative reviews when compared to similar reviews in other rating systems.
*Reviews adjusted to account for the total number of reviews as well as differences in scale.
As expected, all review aggregators maintain a positive relationship with total gross to some extent, i.e. as the score of each aggregator increases so too does its average total gross. However, of all review aggregators, Rotten Tomatoes demonstrates the strongest relationship with total gross in both audience and critics metrics as it demonstrates the strongest rates of increase and decrease at both extremes of possible reviews. Interestingly, though IMDB possesses no particularly extreme decrease in total gross for negatively reviewed films, positively rated films are correlated with the highest increase in total gross in relation to all other rating systems. As Metacritic reviews appear to be the least affected by changes to varying score values, the remainder of this analysis will focus primarily on Rotten Tomatoes and IMDB.
As we can see from the graph above, even small increases in a film’s score on Rotten Tomatoes can lead to large increases in its domestic gross, increasing exponentitally after the 75% range, or after a film is determined to be “Certified Fresh.” Therefore, it would seem appropriate that, when marketing films, trailers should include a films Rotten Tomatoes Critic score if it earns this ranking. Interestingly, high IMDB and Rotten Tomatoes Audience scores appear to lead to a higher total gross than the Rotten Tomatoes Crtitic score, raising the question as to whether or not one of these competing systems may be more beneficial to a film’s financial success.
When we compare the summary statistics for the Domestic Box Office Gross pertainigng to the 300 top rated films (the top 10%) for each review aggregator site , we can see that consisently, films with higher IMDB scores gross more money than films with high Rotten Tomatoes scores, either by audiences or by critics. Across all aspects we see the top IMDB films typically gross the most money, followed by the top Rotten Tomatoes Audience Films and lastly the Top Critic Films, each differing by about $10,000,000. Thus, it would appear that a highly rated film on IMDB, and even highly rated by Rotten Tomatoes Audiences, is more valuable than a critically successful film.
Conversely, if we look at the top 10% of films ordered by their Total Gross, we can see a slighlty different relationship.
Of the 300 top grossing movies from the past 20 years, the majority of these films have garnered reviews establishing them within the top 75th percentile of Rotten Tomatoes Critic reviews and IMDB reviews. Though highly rated Rotten Tomatoes Critic Films and Audience Films have switched positions, IMDB ratings still appear to be the most important indicator of success. In tandem with the previous table, it would seem that not only do highly rated IMDB films tend to be more successful than their counterparts, but they also seem to be some of the most successful films in general.
There could be a variety of possible factors as to why IMDB is correlated with higher increases in domestic gross for highly rated movies. One potential reason involves how the films are scored on IMDB vs Rotten Tomatoes. While IMDB utilizes are normal star system that calculates a movie’s total rating using a weighted average of all reviews, Rotten Tomatoes converts star ratings submitted by both audiences and critics to a single binary value of a like or dislike which it then uses to calculate a total percentage of users who liked the film. So, for example, two films on IMDB were to have an average rating of 6 and 9, they could potentially obtain the same score on Rotten Tomatoes if they had the same proportion of reviewds that were a 6 or higher. Since IMDB can then be seen as a more accurate measurement of the quality of a film, it would make sense that high quality films would have the highest gross.
Another possible reason as to why IMDB serves as a better indicator of a film’s success could be the number of users IMDB has over Rotten Tomatoes.
*38 films omitted from calculation as they maintained an abnormally high number of Rotten Tomatoes reviews.
Looking at the relationship between a film’s score and the number of reviews it obtains on each aggregator site, we can see that a highly rated film on IMDB will obtain significantly more reviews than either Rotten Tomatoes or Metacritic after the film surpasses the 75% range. Interestingly, this benchmark also corresponds to the point on our graph of Total Gross and Aggregator Score where IMDB films become more successful than Rotten Tomatoes, supporting the notion that there exists some relatiopnship between the number of reviews IMDB receives and its relationship with a film’s success.
Not only does the number of reviewers appear to have some bearing between IMDB and its higher increase in Total Gross, but also its number of users. According data pertaining to the number of times IMDB was googled compared to Rotten Tomatoes, we see that IMDB has nearly 5x as much traffic. While this can be partially attributed to the larger number of features on IMDB, such as trivia about a particular film, it is likely that IMDB is a more popular destination for reviews as well. According to Wikipedia, IMDB started 10 years prior to Rotten Tomatoes so it would not be unreasonable to assume IMDB has ammassed a much larger user base in this time.
Sourced from Google Trends
Conclusion
Including a film’s Rotten Tomatoes score into marketing appears to be a viable way to increase a film’s visibility and ultimate box office revenue. Althoughm as IMDB has a much wider user base and is correlated with higher increases in domestic gross over all other review aggregator sites, it would appear that a film could significantly benefit from including this score in marketing if it were sufficiently high.
From a plot of comparing the total grosses of individual films against their aggregator score, we see the adjusted value where IMDB overtakes Rotten Tomatoes is 75, or in IMDB’s raw form, a score of 7.02. Any film film with a score above 7.02 should be advertised as such, either along side or in lieu of its Rotten Tomatoes Percent. Still, only about half of the top rated IMDB films overlap with the top rated films on Rotten Tomatoes indicating these scores ultimately serve different purposes, one user based and the other critic based, suggesting additional benefit if both were to be used when appropriate.