Handwritten Word Spotting in Old Manuscript Images using Shape ...

Handwriting word-spotting is the pattern classification task which consists in detecting words
in handwriting images document. In this dissertation, we are concerned on the detection of several
words into our documents.
In documents where all pages are written by the same author (or few authors), the images of
multiple instances of the same word are likely to look similar. Word-spotting [20] treats a collection
of documents as a collection of words. Then, the first step consists in segmenting the document
into word images, and then, pair wise “distances” between word images are calculated , which are
used to cluster all words with similar features. Ideally, each cluster contains all the samples of the
same word.
There are two types of word-spotting approaches, depending on how the input is specified:
query-by-string and query-by-example. In query-by-string, character models have been trained
in advance and in time of execution the character models are combined to form words and the
probability of each word is evaluated; in query-by-example the input is an image of the word to
search, and the output is a set of the most representative images of the query word.
Problem statement
This work addresses the problem of handwritten word spotting in historical manuscripts. While historical
approaches are based on contextual methods like Hidden Markov Model (HMM) or Dynamic
Time Warping (DTW), using the sequential information of graphemes in a word. We propose a
holistic approach using shape matching techniques. We propose two approaches. The first one uses
a pixel-based descriptor tolerant to distortions. The second one is inspired in Loci characteristic
and allows to aggregate pseudo-structural information in the descriptor. Handwritten collection of
documents, that we will explained with more details in following sections, are used in this work.
Objectives
As started above, this work wins to develop shape descriptors for handwritten word spotting, in
particular the objectives are:
• To investigate different shape descriptors that allow to describe handwritten words with
invariance of variations in writer, acquisition conditions, etc. We aim to focus in pixel-based
descriptors and structural ones.
• Based on the above descriptors, define clustering criteria allowing to build indexation structures
for word spotting purposes.
• Define an experimental framework. Construct a ground truth from a collection of a real
application (Barcelona marriage records).
Outline of the approach
In our work we have used query-by-example. It consists in matching an input image with one or
multiple query images to determine the distance that might indicate a correspondence.
A spotting architecture consists of four tasks. First, a pre-processing step is done. Second, a
fast rejection with the words segmented is done. Third, a normalization step is done. And fourth,
3