The Fortress Language Specification - CiteSeerX

If a character (or any other syntactic entity) x precedes another character y in the program, we say that x is to the left
of y and that y is to the right of x, regardless of how they may appear in a typical rendered display of the program.
Thus, it is always meaningful to speak, for example, of the left-hand and right-hand operands of a binary operator, or
the left-hand side of an assignment expression.
5.4 ASCII Conversion
To facilitate interaction with legacy tools and particularly to aid in program entry, Fortress specifies an ASCII encoding
for programs. For every valid Fortress program, there is an equivalent program that contains only ASCII characters. 3
To support this encoding, a Fortress program undergoes ASCII conversion, which produces an equivalent Fortress
program. ASCII conversion is idempotent: converting a program that resulted from conversion results in the same
program. Unless otherwise specified, all constraints and properties of Fortress programs stipulated in this specification
apply to the programs after they have been converted. This section gives a high-level overview of ASCII conversion.
ASCII conversion consists of two steps. The first step consists of “pasting” words across line breaks, so that long
identifiers and numerals can be split across lines. Identifiers may be very long in ASCII because many Unicode
characters are encoded with long sequences of ASCII characters (the actual conversion to Unicode characters is done
in the next step). Roughly speaking, in this step, two consecutive lines are pasted together if the first ends with
an ampersand that is immediately preceded by a word character, and the second begins with an ampersand that is
immediately followed by a word character.
The second step replaces certain restricted words, and sequences of operator and special characters, with single Unicode
characters. Roughly speaking, if a restricted word is either the official Unicode 5.0 name with underscores in
place of spaces and hyphens, or a specified alternative name, of some character that is not a printable ASCII character,
then the restricted word is replaced by that character. In some cases, even a fragment of a restricted name may be
replaced by a single character (most commonly a Greek letter). Some multicharacter sequences of ASCII operator and
special characters are also replaced by non-ASCII operator or special characters; we call such a sequence ASCII shorthand.
However, this replacement is not generally done within string literals, which instead provide escape sequences
to get non-ASCII characters (see Section 5.10).
Precise descriptions of both these steps are given in Appendix E, including the rules for replacing fragments of restricted
words and the specification of alternative names for non-operator characters. Alternative names for operator
characters are given in Appendix F.
5.5 Input Elements and Scanning
After ASCII conversion, a Fortress program is broken up into input elements by a process called scanning. 4 That is,
scanning transforms a Fortress program from a sequence of Unicode characters to a sequence of input elements. The
characters that comprise an input element always appear contiguously in the input sequence. Every input element is
a whitespace element (comments are whitespace elements) or a token. Every token is a reserved word, a literal, an
identifier, an operator token, or a special token. There are five kinds of literals: boolean literals, character literals,
string literals, the void literal, and numerals (i.e., numeric literals).
Conceptually, we can think of scanning as follows: First, the comments, character literals and string literals are
identified. Then the remaining characters are divided into words (i.e., contiguous sequences of word characters:
letters, digits, connecting punctuation, primes and apostrophes), whitespace characters, and other characters. In some
cases, words separated by a single ‘ . ’ or whitespace character (and no other characters) are joined to form a single
3 See Appendix E for the precise notion of equivalence guaranteed by ASCII conversion.
4 Fortress has a facility for defining new syntax, discussed in Chapter 36. However, except for that chapter, this specification generally ignores
this facility, and describes the Fortress language only for programs that use the standard Fortress syntax.
45