The SRA Symposium - College of Medicine

Papers
of vowels per word, some on the number of consonants per word. Some count a sentence whenever
a period, question mark or exclamation point is found, some whenever a colon or semi-colon
is found.
Hochhauser (1997) analyzed one consent form using six software programs, not including Microsoft
Word. As shown in Table #2, they did not all agree.
Table #2: Comparison of six readability software programs
Six readability
programs
Number of
sentences
Number of
syllables
Words per
sentence
Flesch
Reading Ease
Flesch-
Kincaid
Range 30-35 1,201-1,313 22-25 37-45 12.1-15.5
Average 31.9 1,279 22.9 40.5 14.0
Using the same readability formula but in different programs found the Flesch-Kincaid varying
from 12.1 to 15.5—a difference of 3.4 grades! Such software differences show that readability grade
levels will vary considerably for the same formula depending on which software program is chosen
to calculate reading grade level. That’s a major problem for consent form readability research.
Writing computer programs to count sentences, syllables and words is more difficult that it appears.
In an early attempt to computerize readability formulas, Fang (1968) developed a computer
program for the Flesch Reading Ease Score, but noted that “The basis of the syllable counter is
that one vowel equals one syllable. However, there are many exceptions, and many exceptions to
the exceptions. Two years later, Coke and Rothkopf (1970) stated that “A word may be defined
as any set of alphanumeric characters delimited by blanks and punctuation marks, while a sentence
can consist of all words occurring between two periods.” But consent forms usually include
headings or subheadings that would become part of the next sentence, as well as many periods
(such as M.D., R.N., e.g., etc.), colons, semicolons and hyphenated technical terms that make exact
counts difficult. Counting syllables is even harder. Coke and Rothkopf tested three methods for
estimating syllable counts in readability formulas: 1) total number of vowels, including “y”, 2) total
number of consonants, and 3) total number of letters, and concluded that the number of vowels
per word had the best correlation to the researcher’s syllable count by hand.
Since readability software programs do not disclose the methods used to count sentences, words
or syllables, it’s impossible to know if computerized formulas are consistent with the scores that
would be obtained if calculated by hand. These computational choices explain the puzzling findings
by Mailloux, Johnson, Fisher, et al, (1995) who found inexplicable differences in their comparison
of four readability software programs: “It is interesting to note that, despite the fact that
the Flesch-Kincaid formula and Gunning Fog Index formulas were reported to be identical across
software programs the Flesch Reading Ease Formulas were nearly identical, the software programs
provided different grade results. This finding is difficult to explain because if the formula were
truly identical, no discrepancy should be found” (p. 224). Unfortunately, they did not compare the
computerized readability grade levels with a level calculated by hand, so they could not state which
software readability was most or least accurate; that comparison study has yet to be done.
But such discrepancies should be expected if different programmers choose different ways to
count sentences, words, and syllables. The discrepancies are not with the formulas, but with the
programmers who convert the formulas into software code. The discrepancies found by Mailloux,
et al, might also be due to their methodology of scanning the documents into a computer and analyzing
them via readability formulas. Plus, they did not state if they “cleaned” the file (to remove
extra periods) before running the readability formulas.
118 2005 Symposium Proceedings Book