PDFlib TET PDF IFilter 4.0 Manual

2.4.2 Unicode Decomposition
Canonical decomposition. Characters or character sequences which are canonically
equivalent represent the same abstract character and should therefore always have the
same appearance and behavior. Common examples include precomposed characters
U+00C4
(e.g. ) vs. combining sequences (e.g. ): both representations are canonically
equivalent. Switching from one representation to the other does not remove information.
Canonical decompositions replace one representation with another which is
considered the canonical representation.
In the Unicode code charts 1 (but not the character tables) canonical mappings are
marked with the symbol IDENTICAL TO
. The decomposition name is
implicitly assumed. Table 2.4 contains several examples.
The following document option maps all canonical equivalents to their equivalent
counterparts:
decompose={canonical=_all}
Table 2.4 Canonical decomposition: suboption for the decompose option (canonically equivalent characters are
marked with the symbol IDENTICAL TO in the Unicode code charts)
U+00C4 U+2261
decomposition
name description
before
decomposition
canonical 1 Canonical decomposition
U+00C0
U+F9F4
U+2126
U+3070
U+FB2F
U+0041 U+0308
U+00C4 U+2261
after decomposition
1. By default this decomposition is not applied to all characters in order to preserve certain characters; see the TET manual for
details.
U+0041 U+0300
U+6797
U+03A9
U+2126 U+306F U+2126 U+306F U+3099
U+05D0 U+05B8
Compatibility decomposition. Characters which are compatibility equivalent represent
the same abstract character, but may differ in appearance or behavior. Examples
include isolated forms of Arabic characters (e.g.
) vs. context-specific shaped forms
U+FEB2
U+FEB4 U+FEB3
(e.g. , , ). Compatibility equivalent characters differ in formatting. Removing
this formatting information implies loss of information, but may simplify processing
for certain types of applications (e.g. searching). Compatibility decompositions
remove the formatting information.
U+0633
1. See www.unicode.org/Public/5.2.0/charts/
30 Chapter 2: Indexing PDF Contents