July/August 2004 VOICE FOR THE DEFENSE 1
July/August 2004 VOICE FOR THE DEFENSE 1
July/August 2004 VOICE FOR THE DEFENSE 1
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
the testees compared and changed many of their answers. This<br />
invalidated whatever validity the test results might otherwise<br />
have had.<br />
Caution on Test Scoring<br />
You must also determine how the results of the tests were<br />
scored: while tests such as the MMPI, MCMI and PIC can be and<br />
often are scored by computer, some psychologists will interpret<br />
the results themselves. This is called “idio-syncratic” grading.<br />
There is no information to show the reliability or validity of<br />
this type of scoring.<br />
The computer-generated report for the Personality Inventory<br />
for Children starts off with a paragraph stating, “This<br />
PIC interpretation is based on the systematic analysis of data<br />
obtained in the evaluation of behaviorally disturbed children<br />
and adolescents. This report consists of a series of hypotheses<br />
that may serve to guide further investigation.” 34<br />
In fact, one of psychology’s “dirty little secrets” is the fact that<br />
many of the tests administered are “scored” by computer (e.g.<br />
Exner Rorschach, MMPI, MCMI, PIC, Behavior Assessment<br />
for Children). Many mental health professionals will argue<br />
computer scoring increases the reliability or validity of the tests<br />
and the conclusions drawn by the mental health professional.<br />
All too often, the “hypotheses” are lifted, verbatim, from the<br />
computer printouts and inserted into the psychological evaluations.<br />
However, the psychologists will usually not produce<br />
the computer printouts and, even if they are produced, you<br />
will have no access to the computer programs themselves. The<br />
psychologists will have no idea as to the validity of the program<br />
or of the data that went into formulating the program and its<br />
hypotheses. 35<br />
In the context of a criminal case, the ability to challenge the<br />
programs that score the tests, the data underlying those programs<br />
and the resultant hypotheses, presents serious Confrontation<br />
Clause questions. The author would urge defense counsel<br />
to try to obtain the questions, the answers, the scoring templates<br />
(if scored by the mental health professional), the underlying<br />
programs, the computer printouts, etc., from the mental health<br />
professional before the Rule 702 challenge is heard. 36 If possible,<br />
subpoena the mental health professional’s entire file. Do<br />
not fall for the gambit of “raw data.” Make sure the computer<br />
scores/printouts are included in what you get. Also, try to get<br />
the mental health professional to bring the manuals on the tests<br />
that were administered. 37<br />
Caution on Tests and Questionnaires<br />
The way that a test is constructed can severely influence<br />
the validity and reliability of the results obtained. A classic<br />
example of this is “PMS” — Premenstrual Syndrome. The<br />
original questionnaire, given only to women, was called the<br />
“Menstrual Distress Questionnaire.” This is an improper way<br />
to label the questionnaire, because it tells the test subjects what<br />
the tester is looking for and therefore skews the results obtained.<br />
A “Menstrual Joy” Questionnaire got quite different results.<br />
And when the word “menstruation” was removed completely<br />
and a “Mood Symptom Checklist” was administered to women<br />
and men, there weren’t any significant sex differences! It seems<br />
that men get grumpy from time to time and have headaches<br />
too. But if a test is biased, the results will be incomplete and<br />
misleading. 38<br />
Another example of this problem is the statement that a test<br />
is designed for an eighth grade reading level. That statement<br />
assumes the eighth grade is uniform throughout the country<br />
or that reading skills are uniform within any given class. It<br />
does not take a vivid imagination to be able to see that average<br />
reading skills for an eighth grader attending a private school<br />
in Highland Park, Dallas, is going to be substantially different<br />
from those of illegal immigrants attending public school in<br />
Laredo. 39 Or that their comprehension of the English language<br />
is going to be substantially different. But, if you don’t challenge<br />
this, the judge and jury will assume that your client fits into the<br />
“ideal” test-taker.<br />
Caution on Lack of Control Groups<br />
Medicine recognizes the placebo effect. This is why all<br />
FDA-approved medicines must undergo double-blind studies,<br />
where the patients (and those who administer the drug) do not<br />
know who is getting the real medicine or a placebo, to see if<br />
the medicine is actually more efficacious than a placebo. The<br />
persons who receive the placebo are the control group.<br />
A control group is used so that one can reduce or eliminate<br />
alternate explanations for results seen instead of relying on<br />
guesswork. This is the essential notion behind the use of control<br />
groups. One adds a group that gets no exposure to the variable<br />
of interest, or no treatment at all, but in all other ways possible<br />
keeps the group similar to those receiving treatment. In this<br />
way, one can determine not only which treatment is more effective,<br />
but whether the treatment is effective at all or compares<br />
favorably to no treatment.<br />
The control group for psychological tests should be normal<br />
people. But what is Normal, besides a town in Illinois? Psychologists<br />
tend to think of “normal” as a very narrow range<br />
of conduct when it actually encompasses a very wide range of<br />
conduct. For example, is it normal for a young, adult man, in<br />
below-freezing weather, to strip his clothes off from the waist<br />
up, paint his upper body, arms and face in bright hues, consume<br />
copious amounts of alcohol and then sit for hours, making loud<br />
noises. No? Well, have you ever taken a look at the young men<br />
in the stands at Lambeau Field when the Green Bay Packers<br />
are in the playoffs? Considered normal for cheeseheads. Or is<br />
it normal to stand on the railing of a bridge, hundreds of feet<br />
above a river, and dive head-first toward the bottom? Suicide?<br />
No, base jumping or bungee jumping. Considered normal by<br />
<strong>July</strong>/<strong>August</strong> <strong>2004</strong> <strong>VOICE</strong> <strong>FOR</strong> <strong>THE</strong> <strong>DEFENSE</strong> 27