NUI Galway – UL Alliance First Annual ENGINEERING AND - ARAN ...

Reconstruction of Threads in Internet Forums
Erik Aumayr, Jeffrey Chan and Conor Hayes
Digital Enterprise Research Institute, NUI Galway, Ireland
{erik.aumayr, jkc.chan, conor.hayes}@deri.org
Abstract
Online discussion boards, or Internet forums, are a
significant part of the Internet. People use Internet
forums to post questions, provide advice and
participate in discussions. These online conversations
are represented as threads, and the conversation trees
within these threads are important in understanding the
behaviour of online users. Unfortunately, the reply
structures of these threads are generally not publicly
accessible or not maintained. Hence, we introduce an
efficient and simple approach to reconstruct the reply
structure in threaded conversations. We contrast its
accuracy against an existing and a baseline algorithm.
1. Introduction
Internet forums are an important part of the web for
questions to be asked and answered and for public
discussions on all types of topics. In forums, conversations
are represented as a sequence of posts, or threads,
where the posts are replies to one or more earlier posts.
Links exist between posts if one is the direct reply to
another. However, the reply structure of threads is not
always available. For instance, the structure is not
maintained by the provider, or lost. We propose a new
method to reconstruct the reply structure of posts in
forums. It uses a set of simple features and a decision
tree classifier to reconstruct the reply structure of
threads. We evaluate the accuracy of the algorithm
against an existing and a heuristic baseline approach.
2. Methodology
Definitions A post in a thread provides us with the
following, basic information: creation date, name of
author, quoting: name of the quoted author and
content. The creation date of posts establishes a
chronological order. From that ordering we can
compute the distance of one post to another. Distance
means how far away is a post to its reply. If there is no
other post between a post and its reply, then they have a
post distance of 1. If there is another post in between,
then the distance is 2, and so forth.
Note that the data we use stores the reply interaction
in the way that each post can only reply to one other
post at once. Although a user can reply to several posts
at once, and our approach is able to return more than
one reply candidate, we limit replies to one target post
in our evaluation.
Baseline approaches In our data, we found that
79.7% of the replies have a post distance of 1, i.e. they
follow directly the post they refer to. Hence, our first
baseline approach is to link each post to its immediate
predecessor, called “1-Distance Linking”.
80
Wang et al. 2008 [1] introduced a thread
reconstruction that relies on content similarity and post
distance. That serves as our second baseline approach.
Features Based on the information a pair of posts
provides, we extract the following features for our
classification task: reply distance, posting-time
difference, author quoted and cosine similarity. The
cosine similarity compares the contents of two posts and
returns a similarity score from 0 to 1, where 0 means not
similar and 1 means exactly equal.
Classifier As a classifier, we investigate the widely
used C4.5 decision tree algorithm. It handles huge
amount of data very efficiently due to its relative
simplicity which is important for our task to present a
fast and efficient way of reconstructing threads.
3. Evaluation
For the evaluation we use a subset of our Boards.ie
dataset. Namely 13,100 threads, consisting of 133,200
posts in total.
In order to compare classification results of the
approaches, we use the measurements precision, recall
and F-score where F-score is the harmonic mean of
precision and recall. For training the classifier, we
applied a 10 fold cross validation to minimise bias.
Table 1 shows the comparison between our
classification algorithm “ThreadRecon” and the two
baseline approaches.
Wang et al. 2008 1-Distance Linking ThreadRecon
44.40% 79.70% 85.70%
Table 1: F-score comparison between ThreadRecon and
baseline approaches
An extended version of this work will be published in
“Reconstruction of Threaded Conversations in Online
Discussion Forums”, International Conference on Weblogs
and Social Media 2011
8. References
[1] Wang, Joshi, Cohen and Rosé. 2008. Recovering implicit
thread structure in newsgroup style conversations. In
Proceedings of the 2nd International Conference on Weblogs
and Social Media (ICWSM II), 152–160