RR_03_02

Poorly Structured Handwritten Documents Segmentation using Continuous
Probabilistic Feature Grammars
Abstract
This work deals with poorly structured handwritten documents
segmentation such as pages of handwritten notes produced with
pen-based interfaces. We propose to use a formalism, based on
Probabilistic Feature Grammars. that exhibit some interesting
features. It allows handling ambiguities and to taking into
account contextual information such as spatial relations between
objects in the page.
KEYWORDS: Segmentation. Poorly structured documents.
Note-taking, Probabilistic Feature Grammars.
1. Introduction
A number of new mobile terminals have appeared in the
last few years, e-books, note-taking devices, portfolios.
More recently, new terminals have been commercialized
that are a mix between c lassical wearable computers and
digital tablets. These objects, being most often mobile
devices, have come with pen-based interfaces, allowing
the user to write, store, edit, manage handwritten
documents, and in a limited way to recognize these.
Documents produced using pen-based interfaces may be
very hetreogeneous both in thei r structure (e.g.
handwritten notes, maps etc) and in their content that may
include text, drawings, images, equations etc. Such
documents cannot be efficiently used in their rough form
of electronic ink; one needs a higher level representation
both for their structure (identi fication of lines,
drawings, ... ) and for their content (e.g. partial recognition
of textual regions).
Up to now, segmentation of written documents (e.g.
pages) has bee n most often studied for off-line
documents. Lots of works has been done in this context
and various methods have been developed for e.g.
newspapers and table segmentation, see in [2,6,9, 10, II]
for popular techniques. Besides, the case of on-line
docurrents has been investigated very recently [5, 12].
However, existing segmentation techniques tuned for offline
documents are not well adapted to on-line documents.
Actually, on -line document segmentation and off-line
document segmentation do not deal with the same kind of
documents. Off-l ine document segmentation deals with
documents with a complex structure (e.g. newspaper) but
T. Artieres
LlP6, Universite Paris 6,
8 rue du Capitaine Scott, 75015, France
Thierrv.Artieres@lip6.[r
Tel: (33fl-44-27-72-20 Fax: (33)-1-44-27-70-00
5
with an homogeneous local structure (e.g. purely
horizontal and parallel text lines, uniform inter-line distance
etc). At the opposite, on-line handwritten documents
produced with pen-based interfaces (using a mobile
device) are often much simpler in their structure
(handwritten notes in a single col umn form ... ) but also
much more heterogeneous. In a same page, lines may be
differently slanted; characters size may vary in a same line
etc. The main difficulty lies then in the inherent ambiguity
and heterogeneous of such documents.
As a consequence methods developed for off-l ine
documents segmentation relying on global features of the
documents (uniform slant of the lines, ... ) are adapted to
well structured and homogeneous documents. In this
study, we investigate the development of generic
segmentation tools for much less structured documents as
those typically acquired through pen-based ilterfaces
with note-taki ng devices. This is a prospecti ve work and,
up to now, no strict evaluation has been conducted.
The paper is organized as follows. In section §2, we
discuss the choice of a generic model to express
handwritten documents structure and present Probabislitic
Feature Grammars (PFGs), which is the formalism we built
on. In section §3, we detail how we have extended and
adapted PFGs to deal with 2D handwritten documents. In
§4, we discuss the use of such models for on-line
documents segme ntation.
2. Document structure model
2.1. Dealing with on-line documents
A few previous works have been published on the
segmentation and recognition of poorly structured
handwritten documents [5,9, 12]. Most often, techniques
operate in two steps. First, the page is segmented into text
and non text; then text areas are segmented into lines, lines
into words etc. This is done using bottoITHlp or top-down
approaches (for example with histogram projection
techniques to detect lines and then to detect words in
lines). [n a second step, segmented elements (i.e. word) are
recognized using an isolated word recognition engine.