NUI Galway – UL Alliance First Annual ENGINEERING AND - ARAN ...

Improving Twitter Search by Removing Spam and Ranking Results
Steven Connolly & Colm O’Riordan,
CIRG, Information Technology, NUIG,
s.connolly13@nuigalway.ie, colm.oriordan@nuigalway.ie
Abstract
With the ability to post real-time status updates, Twitter
is an excellent source of first hand news. Twitter search
is widely used for retrieving real-time information.
However, with such high spam content on Twitter,
search results often return multiple spam Tweets. With
this in mind we aim to identify and remove spam and to
rank remaining Tweets promoting high Tweet content
and novelty in the answer set. We analyse Twitter data
to identify a set of features to detect spam. We aim to
improve the quality of a search result set by ranking
Tweets based on a number of identified heuristics.
Introduction
Twitter is a real-time information network. Since its
launch in July 2006, Twitter has acquired 175 million
registered users posting 95 million tweets and
performing 900,000 search queries per day. Twitter
posts or Tweets are short text message a maximum of
140 characters in length.
Twitter Spam
Spam is the electronic transmission of messages, in
large volume, to people who do not choose to receive
them. Spam accounts have multiple behaviors including,
posting harmful links to malware and phishing web
sites, repeatedly posting duplicate Tweets, posting
misleading links or following and un-following accounts
to draw attention [1]. Twitter provides limited spam
removal. Spam accounts can be reported to Twitter.
These accounts are reviewed and investigated for abuse
and any account showing spamming behavior is
permanently suspended. However, this method will not
prevent spam Tweets appearing in search results and
spam is getting worse as Twitter becomes more
prevalent [2].
Approach
Our research focuses on correctly identifying spam
Tweets. Our approach involves gathering a data set of
Twitter records. A sample of 11,000 records was
gathered based on various keywords. The Twitter
Streaming API was used to gather the data. Each record
consisted of information related to the Tweet along with
user account detail, necessary to our research.
Heuristics
Based on literature and our own preliminary
analysis, we identified a set of heuristics which we use
for spam detection and data ranking. Some of these
include, Follower Count, Account Age, Hash Tag Count
77
URL Count which refers to the no. of URLs present in a
Tweet, Status Count referring to the no. of Tweets
posted from an account, List Count which refers to the
no. of lists/groups an account belongs to, Duplicate
Tweets referring to duplicate records present in a record
set etc.
Following analysis of these heuristics we were able
to identify the techniques proving most successful for
spam detection. Some of our results can be seen below:
Results Spam % Spam
0 Followers 93 65 65%
Acc Age <
1day
145 55 63%
Hash Tags >
8
23 22 95%
Initial analysis showed that individual heuristics can be
used to detect spam. Further analysis showed that
combination of particular heuristics detected spam more
accurately e.g. an account created within 24 hours with
40 friends returns only spam Tweets.
Analysis confirmed that one of the main
characteristics of Twitter spam is repetition. Spam
accounts repeatedly post the same Tweet in an effort to
be seen. We explain the use of the Levenshtein distance
to measure the similarity between Tweets and removed
very similar Tweets. The results show that 20% of our
dataset comprise of duplicate or near duplicate Tweets.
Future Work
To date we have been concerned with identifying
heuristics to give high precision. We are now working
towards measuring recall of these heuristics
Following this we aim to develop a ranking
algorithm which will give a score to the remaining
Tweets and accurately rank them as a measure of time
and query relevance.
References
[1] Ryan Kelly, Aug 2009, Twitter Study, Pear
Analytics.
[2] Danny Sullivan, Jun 2009, Twitter’s Real Time
Spam Problem.