PDFlib Text Extraction Toolkit (TET) Manual

6.3 Chinese, Japanese, and Korean Text
6.3.1 CJK Encodings and CMaps
TET supports Chinese, Japanese, and Korean (CJK) text, and converts horizontal and vertical
CJK text in arbitrary legacy encodings (CMaps) to Unicode. TET supports all of Adobe’s
CJK character collections:
> Simplified Chinese: Adobe-GB1-5
> Traditional Chinese: Adobe-CNS1-5
> Japanese: Adobe-Japan1-6
> Korean: Adobe-Korea1-2
The PDF CMaps in turn cover all of the CJK character encodings which are in use today,
such as Shift-JIS, EUC, Big-5, KSC, and many others. CJK font names encoded with localespecific
encodings (e.g. Japanese font names encoded in Shift-JIS) are normalized to Unicode.
Note In order to extract CJK text which is encoded with legacy encodings you must configure access
to the CMap files which are shipped with TET according to Section 0.1, »Installing the Software«,
page 7.
6.3.2 Word Boundaries for CJK Text
Word boundary detection for CJK text can be controlled with the ideographic page option:
> With ideographic=split ideographic characters always constitute a word boundary, i.e.
single ideographs are returned in granularity=word. While ideographic CJK characters
are considered as word boundaries, Katakana characters are not treated as word
boundaries.
> With ideographic=keep ideographic characters generally don’t constitute a word
boundary. Punctuation and the transition between ideographic and non-ideographic
characters still constitute a word boundary. For granularity=word ideographic comma
U+3001 and ideographic full stop U+3002 also constitute word boundaries. For
granularity=page no line separator is inserted at the end of a line.
For compatibility reasons the default value is ideographic=split, but it is strongly recommended
to use ideographic=keep to improve text extraction for CJK text.
6.3.3 Vertical Writing Mode
TET supports both horizontal and vertical writing modes, and performs all metrics calculations
as appropriate for the respective writing mode. Keep the following in mind
when dealing with text in vertical writing mode:
> The glyph reference point in vertical writing mode is at the top center of the glyph
box. The text position will advance downwards as determined by the font size and
character spacing, regardless of the glyph width (see Figure 6.3).
> The angle alpha is 0˚ for standard vertical text. In other words, fonts with vertical
writing mode and alpha=0° progress downwards, i.e. in direction -90˚.
> Because of the differences noted above client code must take the writing mode into
account by using the following pCOS code (note that not all text which appears vertically
actually uses a font with vertical writing mode):
6.3 Chinese, Japanese, and Korean Text 79