Proceedings - Österreichische Gesellschaft für Artificial Intelligence

develop a metric for positivity of terms, and examine
their relative distributions. This is followed
by an examination of the relation between ratings
and texts in the two data sets. We show that
the hypothesis is strongly confirmed in all three
of its variants. Finally, we observe that these results
could have far-reaching implications for the
interpretation of recommender systems and user
ratings, the use of which has exploded in recent
years.
45000
40000
35000
30000
25000
20000
15000
10000
5000
0
1 2 3 4 5 6 7 8 9 10
2 Data
The Danish data was downloaded from the Danish
movie website scope.dk and contains rated
user reviews from 829 films and has a total size of
1,624,049 words. The U.S. data was downloaded
from The Internet Movie Database (imdb.com)
and contains rated user reviews from 678 films
and has a total size of 34,599,486 words.
A search function on www.imdb.com was used
to create a list of films and matching IMDb ID
tags for films produced in the years 1920-2011.
678 films on the list had a match in the Scope data
on title and production year . The IMDb ID tags
was used to find the page containing data for each
of the films and all reviews which had a correlated
rating were downloaded for those 678 films. The
U.S. IMDb reviews are rated on a scale of 1 to
10, while the Danish Scope reviews are rated on a
scale of 1 to 6.
3 Ratings
Figure 1 gives the number of reviews in each category
for IMDb.
For IMDb, the top category of 10 has by far the
most reviews. For the most part the number of
reviews decreases from category 10, with a modest
increase in the number of reviews for the lowest
category, 1. This distribution makes intuitive
sense – it’s not surprising that people would be
most motivated to write reviews of films they are
most enthusiastic about, and, to a lesser extent,
also be motivated in cases where they have strong
negative feelings. This has been noted in the literature:
(Wu and Huberman, 2010) point out that
the so-called “brag and moan” view of ratings is
fairly typical (as also mentioned by (Hu et al.,
2006; Dellarocas and Narayan, 2006)). The tendency
of the top category to be the most frequent
Figure 1: IMDb reviews per category
is also mentioned on the yelp.com site, where the
top category of 5 is the most frequent: “The numbers
don’t lie: people love to talk about the things
they love!” (FAQ, 2012).
5000
4500
4000
3500
3000
2500
2000
1500
1000
500
0
1 2 3 4 5 6
Figure 2: Scope reviews per category
There is a very different distribution in the Danish
Scope data, as shown in Figure 2. Here, category
4 (out of 6) is the most frequent. This supports
the general prediction that highly positive
evaluations are over-represented in the U.S. data
compared to the Danish data.
4 Text
We turn now to a second version of our hypothesis:
that highly positive terms are overrepresented
in the U.S. data. We consider highly
positive terms to be those that tend to occur in the
most positive category and tend not to occur in the
other categories.
320
Proceedings of KONVENS 2012 (PATHOS 2012 workshop), Vienna, September 21, 2012