PDFlib Text Extraction Toolkit (TET) Manual

Table 10.11 Suboptions for the contentanalysis option of TET_open_page( ) and TET_process_page( )
option
bidi
bidilevel
dehyphenate
dropcapsize
dropcapratio
ideographic
includeboxorder
description
(Keyword; will be ignored for granularity=glyph; has an effect only if right-to-left characters are present
on the page) Control the inverse Bidi algorithm which reorders right-to-left and left-to-right text in a
chunk (default: logical):
visual
logical
Keep RTL and LTR characters in a chunk in visual order, i.e. do not apply the inverse Bidi
algorithm
Apply the inverse Bidi algorithm to bring the characters in a chunk in logical order.
(Keyword) Specify the page’s base level (i.e. the main direction of text progression) for the inverse Bidi algorithm
(default: auto):
auto Determine the main direction of text progression heuristically based on the content.
ltr Assume left-to-right as main direction of text progression (e.g. Latin documents)
rtl Assume right-to-left as main direction of text progression (e.g. Hebrew or Arabic documents)
(Boolean) If true, hyphenated words will be identified and the text fragments surrounding the hyphen
will be combined. The hyphen itself will be treated according to the keephyphens option. Default: true
(Float) The minimum size at which large glyphs will be recognized as a drop cap. Drop caps are large
characters at the beginning of a zone that are enlarged to »drop« down several lines. They will be
merged with the remainder of the zone and form part of the first word in the zone. Default: 35
(Float) The minimum ratio of the font size of drop caps and neighboring text. Large characters will be recognized
as drop caps if their size exceeds dropcapsize and the font size quotient exceeds dropcapratio.
In other words, this is the number of text lines spanned by drop caps. Default: 4 (drop caps spanning
three lines are very common, but additional line spacing must be taken into account)
(Keyword) Control word boundary detection for ideographic characters. It is recommended to set this option
to keep although the default is split for compatibility reasons (default: split):
keep
split
Ideographic characters generally don’t constitute a word boundary. Punctuation and the
transition between ideographic and non-ideographic characters still constitute a word
boundary. For granularity=word ideographic comma U+3001 and ideographic full stop
U+3002 also constitute word boundaries. For granularity=page no line separator will be
inserted at the end of a line.
Ideographic characters always constitute a word boundary.
(Integer) When multiple include boxes have been supplied (see option includebox), this option controls
how the order of boxes affects the Wordfinder (default: 0):
0 Ignore include box ordering when analyzing the page contents. The result will be the same as
if all the text outside the include boxes was deleted. This is useful for eliminating unwanted
text (e.g. headers and footers) while not affecting the Wordfinder in any way.
1 Take include box ordering into account when creating words and zones, but not for zone
ordering. A word will never belong to more than one box. The resulting zones will be sorted in
logical order. In case of overlapping boxes the text will belong to the box which is earlier in
the list. Other than that, the ordering of include boxes in the option list doesn’t matter. This
setting is useful for extracting text from forms, extracting text from tables, or when include
boxes overlap for complicated layouts.
2 Consider include box ordering for all operations. The contents of each include box will be
treated independently from other boxes, and the resulting text will be concatenated
according to the order of the include boxes. This is useful for extracting text from forms in a
particular ordering, or extracting article columns in a magazine layout in a predefined order.
In these cases advance knowledge about the page layout is required in order to specify the
include boxes in appropriate order.
172 Chapter 10: TET Library API Reference