PDFlib Text Extraction Toolkit (TET) Manual

Compatibility decomposition. Characters which are compatibility equivalent represent
the same abstract character, but may differ in appearance or behavior. Examples
include isolated forms of Arabic characters (e.g.
) vs. context-specific shaped forms
U+FEB2
U+FEB4
(e.g. , , ). Compatibility equivalent characters differ in formatting. Removing
this formatting information implies loss of information, but may simplify processing
for certain types of applications (e.g. searching). Compatibility decompositions
remove the formatting information.
In the Unicode code charts compatibility mappings are marked with the symbol
ALMOST EQUAL TO
, followed by the decomposition name (or »tag«) in angle
brackets, e.g. . If no tag name is provided, is assumed. The tag names
are identical to the option names in Table 7.5. As can be seen in some of the examples,
the result of a decomposition may convert a single character to a sequence of multiple
characters.
Note While all entries in Table 7.5 describe compatibility decompositions, the »compat« tag includes
only »other« compatibility decompositions, i.e. those without a specific name.
Note Keep in mind that PDF documents may already map glyphs to the decomposed sequence instead
of the non-decomposed Unicode value. In this situation the decompose option will not affect
the output.
Decomposition examples. Decompositions in TET can be controlled with the document
option decompose. A decomposition can be restricted to operate only on some, but
not all Unicode characters. The subset on which a decomposition operates is called its
domain. Table 7.5 lists the suboptions for all Unicode decompositions along with examples.
The following examples for the decompose option must be supplied in the option list
for TET_open_document( ). The decomposition names in the decompose option list are
taken from Table 7.5.
Disable all decompositions:
decompose={none}
U+FEB3
U+00C4 U+2248
Preserve wide (double-byte or zenkaku) and hankaku (narrow) characters:
decompose={wide=_none narrow=_none}
Map all canonical equivalents to their counterparts:
decompose={canonical=_all}
The following option list enables the circle decomposition, but disables all other decompositions:
decompose={none circle=_all}
U+0633
7.3 Unicode Postprocessing 101