NUI Galway – UL Alliance First Annual ENGINEERING AND - ARAN ...

Detecting Topics and Events in Video Content via
Synchronized Twitter Streams
Smitashree Choudhury & John G.Breslin
DERI, National University of Ireland, Galway, Ireland
2 School of Engineering and Informatics, National University of Ireland, Galway, Ireland
{smitashree.choudhury, john.breslin }@deri.org
Abstract
This work is an attempt to annotate event videos from
contextual sources, in this case from user tweets. The
problem involves two subtasks, 1) Identifying
interesting entities and events 2) aligning the detected
entities to the recorded video. Combination of linguistic
processing statistical data and domain knowledge helps
to get high quality result for a time stamped concept
based annotation of various event videos. To evaluate
our approach we studied four live events in two
different domains.
1. Introduction
Being able to automatically annotate videos on the
social web is a complex problem to solve. It usually
requires combinations of expensive content-processing
algorithms, speech recognition techniques or expert
manual annotations, but not scalable. To address the
above problem, we explored a unique yet extremely
lightweight approach by leveraging user-generated
tweet streams to annotate event videos. Our proposed
approach involves the detection of interesting topics
using various combinations of statistical and natural
language processing techniques, and the
synchronization of topics to a video timeline using
simple heuristics. A similar approach has been reported
in [1] for detecting audience sentiment during US
presidential debate.
2. Methodology
2.1. Data processing
The experiment started with data collection from 4
different events using various related keywords and
hashtags. Cleaning of data started with pre-processing,
segregating hashtags, usernames and identifying
relevant tweets from the non-relevant.
2.2. Feature Selection
Features selection includes statistical features such
as Twitter volume, unique users etc, linguistic features
such as name variations, hashtags and some domain
knowledge such as event name and participants
(conference) and player’s name (sports) were used.
2.3. Entity and Topic Detection
The entity and topic detection performed using
multiple approaches including burst detection, tf-idf
measure and feature based classification of tweets.
96
Figure 1: Interface for the time based annotation of the
WWDC keynote video.
3. Result
The experiment was evaluated using IR measures
such as recall, precision and F-measure against the
manually-annotated ground truth.
Figure 2: Results for topic and event detection from the
tweets.
A simple evaluation of the automatic synchronization
was performed against the same users labeled as ground
truth. The objective of the evaluation is to see whether
the heuristics adopted are sufficient to localize the
topics.
Figure 3: Results for twitter to timeline alignment
4. References
[1] Shamma, D.A., Kennedy, L. and Churchill, E. “Tweet the
Debates”, ACM Multimedia Workshop on Social Media
(WSM), (2009).