The SRA Symposium - College of Medicine

Papers
followed by 39 side effect bullet points. That’s hardly an improvement.
Misusing the formula. Many IRBs recommend using the Flesch-Kincaid readability formula in Microsoft
Word. Unfortunately, the authors who make this recommendation apparently do not know
that Word’s version of the formula is flawed, since it does not report scores above grade 12, even
though the formula itself score up to grade 17 (Hochhauser, 2003c). Some IRB websites even misuse
readability formulas to analyze one or two sentences from consent forms to show high grade
levels with original language and lower grade levels with revised language. But readability formulas
were not designed for one or two sentences. Because most readability formulas require 200-300
words for analysis, using readability formulas for one or two sentences is completely inappropriate,
meaningless, and probably indefensible in a lawsuit.
Cleaning the file: Consent form files must be “cleaned” before running them through a readability
formula. Cleaning a file refers to removing extra periods in abbreviation (e.g., i.e., M.D., etc.)
because readability formulas count a sentence every time they find a period. Readability formulas
were designed for narrative text, so consent form writers should remove all titles, headings, subheadings,
bullet points, charts, tables, or anything else that is not a complete sentence. Otherwise
the formula will give inaccurate reading grade level estimates. Some IRB web sites show researchers
how to use the readability statistics function with Microsoft Word. But since these recommendations
don’t describe how to clean the consent file, simply clicking on “show readability statistics”
will give an inaccurate reading grade level, in addition to the reporting limit of grade 12. If you
were deposed in a readability related lawsuit, could you justify the way in which your researchers
or IRB calculated reading grade levels?
Consent form reader problems:
Adolescent vs adult readers. Because readability formulas were developed to select school books,
most formulas are based on reading skills of children and adolescent—not adults throughout the
lifespan. Clinical trials often include subjects who range in age from 18-80, or from 18 to no upper
limit, or somewhere during childhood and adolescence. Research has not shown that readability
formulas based on children and adolescents—the Dale Chall is based on words known to 4th
graders-- is relevant for all adults.
Eighth-grade students probably have very different reading skills than 40-year old adults who have
an eighth grade education. The common assumption that a consent form written at an 8th grade
level can be understood by anyone with eight years of education is false; there’s no consent form
comprehension research to support that assumption. Anyone who believes that a consent form at
a 6th- 8th grade level can actually be understood by 8th graders should give it to groups of 6th-
8th graders and see how well they understand it. If they don’t understand it very well—which they
probably won’t because they haven’t yet developed abstract thinking skills or a research vocabulary—what
does that mean for the continued recommendation of an 8th grade consent form reading
level?
Text comprehension problems: Reliance on readability formulas alone implies that all patients are
the same. Writing comprehensible consent forms requires writers to know patients’ educational
attainment and intelligence, reading ability, their experiences with clinical research and ethical
concepts, their motivation for reading the consent form (voluntary or involuntary), how much attention
they’ll likely pay to the consent form, etc. Neither readability proponents (such as Klare) or
opponents (such as Redish and Shriver) recommend “writing to the formula,” because there is no
evidence that writing shorter sentences with smaller words does much to improve comprehension.
Because no one in reading research recommends “writing to the formula,” recommendations from
federal agencies or IRBs that consent forms be rewritten to the formula until they reach the magical
6th-8th grade reading level are just wrong. Because readability formulas measure only what
120 2005 Symposium Proceedings Book