RR_03_02

Ana Costa e Silva *
Banco de Portugal, Portugal
acsilva@bportugal.pt
Selection of Table Areas for Information Extraction
Abstract
Information contained in companies' financial
statements is valuable to support a variety of decisions. In
such documents, much of the relevant information is
contained in tables and is extracted mainly by hand. We
propose a method that accomplishes a preliminary step of
the task of automatically extracting information from tables
in documents: selecting the lines which are likely to belong
to the tables containing the information to be extracted.
Our method has been developed by empirically analysing a
set of Portuguese companies' financial statements, using
statistical and data mlntng techniques. Empirical
evaluation indicates that more than 99% of the table lines
are selected after discarding at least 50% of them. The
method can cope with the complexity of styles used in
assembling information on paper and adapt its
performance accordingly, thus maximizing its results.
1. Introduction
Companies worldwide have the legal obligation of
producing and publishing, on at least a yearly basis,
reports accounting for their activities. These reports,
called financial statements, contain tables with
information that supports the decisions of a variety of
economic agents, for whom it is often necessary to
combine information from several documents. However,
such aggregate analysis requires the time consuming
activity of capturing the data manually to a database.
Although it is expected that in the future these
statements will be published in a predefined structured
format such as XBRL (eXtensible Business Reporting
Language, [9]), in many countries it will most likely take
a long time before they are fully adopted.
To automatically extract information from the tables
contained in financial statements, or in any document,
five basic tasks have to be fulfilled [2]:
Location: "differentiating tables from other elements
such as body text, heading, titles, bibliographies, line
drawings and bitmap graphics", [7] .
.
The opinions expressed in this article are the responsibility of the
authors and not necessarily coincide with the Banco de Portugal 's.
Alfpio Jorge
Universidade do Porto,
F aculdade de Economia do
Porto, LIACC, Portugal
arnjorge@liacc.up.pt
15
LUIS Torgo
Universidade do Porto,
Faculdade de Economia do
Porto, LIACC, Portugal
ltorgo@liacc.up.pt
Segmentation: delimiting the table's physical
structures -its columns and lines, its simple and spanning
cells (those spreading over more than one column/ line).
Functional analysis: identifying the function each
physical unit plays; there are two basic functions:
containing data and describing data, any titles and
footnotes should be identified as such.
Structural analysis: detecting the relationships
between the cells, identifying the attribute-value threads
which have to be read conjointly.
Interpretation: going one step beyond simply
identifying relationships to knowing what they mean and
deciding upon them. More than knowing the cells
containing strings "Total assets" - "5000" - "Real" have to
be read conjointly, interpreting means being able to affirm
"Total assets are 5000 PTE".
The task we approach in this paper is a preliminary
step of this process: given a text document, we want to
automatically select groups of lines that are likely to
belong to tables. This step reduces the search effort in the
subsequent tasks. To serve as training examples, we
downloaded from the Web 19 financial reports published
by 13 Portuguese companies in a period ranging from
1997 to 2000. On the whole there were 87,843 lines, of
which 70,584 are not table lines and 9,685 are part of the
tables containing the information we wish to extract. We
then proceeded to convert the original files to plain text
files using the pdftotxt linux utility, and imported them to
a database table. Each line in the report became a distinct
record.
2. Selecting target areas
Current software to locate tables in documents is
generally based on detecting the alignment of certain
characters, either words, e.g. [3], non-space characters,
e.g. [6], or white spaces, e.g. [5], being vertical alignment
often the key feature searched for. Other methods, such
as [4], examine the contents of each line in the document,
character by character, to detect the presence of groups of
contiguous spaces, count them, among other
characteristics, and present the counts to a decision tree to
determine whether or not the line belongs to a table .