The Annoyance Filter.pdf - Fourmilab

§96 ANNOYANCE-FILTER UNICODE INTERPRETER CLASS 79
96. Unicode Interpreter class.
This interpreter class parses Unicode characters into a form which can be comprehended by the parser.
〈 Class definitions 10 〉 +≡
class Unicode MBCSinterpreter : public MBCSinterpreter {
public:
Unicode MBCSinterpreter( )
{
setPrefixSuffix ("␣", "␣");
}
virtual string name (void)
{
return "Unicode";
}
string getNextDecodedChar (void);
};
97. Our getNextDecodedChar implementation attempts to represent the Unicode characters in a fashion
which will best enable the parser to classify them. Characters in the first 256 code positions, which
are identical to ISO-8859 are output as ISO characters. Other codes are represented as “UCS−nnnn”
where nnnn is the Unicode code value in hexadecimal. Codes representing iedographs are output separated
by spaces while codes for alphanumeric characters are not space-separated.
〈 Class implementations 11 〉 +≡
string Unicode MBCSinterpreter ::getNextDecodedChar (void)
{
assert(dp ≠ Λ);
}
int dc = dp ⃗ getNextDecodedChar ( );
if (dc < 0) {
return ""; /∗ End of input stream ∗/
}
if (dc ≤ # FF) {
string r(1, static cast〈char〉(dc)); /∗ ASCII character ∗/
}
return r;
ostringstream os ;
os .setf (ios ::uppercase );
if (((dc ≥ # 3200) ∧ (dc < # D800)) ∨ ((dc ≥ # F900) ∧ (dc < # FAFF))) {
os ≪ prefix ≪ "UCS−" ≪ hex ≪ dc ≪ dec ≪ suffix ; /∗ Ideographic–space on both sides ∗/
}
else {
os ≪ "UCS−" ≪ hex ≪ dc ≪ dec; /∗ Alphabetic—don’t space around characters ∗/
}
return os .str ( );