Training Scientists to Make the Right Moves - Howard Hughes ...

Training Scientists to Make the Right Moves
Tips for Developing an Effective Evaluation Questionnaire
The evaluation questions should be linked to the goals and objectives of
the training event. Consider using a mixture of short answer, multiple
choice, and open-ended comment boxes in the questionnaire (see appendix
4 for an example of a mixed-format form used for the 2005 BWF-HHMI
Course in Scientific Management). Evaluation pros offer some additional
advice on designing the questionnaire:
Don’t ask the question if you won’t do something with the
data. Stick to questions that will help you determine whether you
met your objectives. Avoid fishing expeditions—asking questions
about things over which you have no control serves no purpose.
Steer clear of questions about intangibles. An example of such a
question is “How would you rate the university’s commitment to
training?” Evaluation is all about meeting your objectives; direct your
questions to those specific aims.
Avoid ambiguous wording. An example is “Did this lecture (workshop,
session, etc.) meet your expectations?” If you don’t know what
the respondents’ expectations were before the training, you can’t evaluate
their responses.
Ask questions that get at the root of purported benefit. For
example, you might ask participants to explain why they think they
changed a particular behavior entirely because of a specific training
session—why they could not or would not have made the change
otherwise.
Keep the questions short and to the point. Remember that no one
really likes to fill out surveys, and if time and finances are tight, minimize
the number of open-ended text boxes that elicit qualitative data.
Be aware of question bias. Avoid framing the question to lead to
the answer you want. For example, by asking “How do you think this
workshop will improve your skills?” respondents are being prompted
to talk only about positives. A better way to phrase the question
might be “Please describe which of your skills have been improved
by taking this workshop.” The answer may require more work to analyze,
but you will get a better idea of how respondents view the training
and it may help them crystallize their own opinions about how
well their time was spent.
Even if you have obtained a suitable survey from a colleague or other
source, it is a good idea to step back and consider whether it might benefit
from tailoring to the specifics of your situation. Remember that whether or
not a question is poorly framed depends on the objective of the question
(see figure 9.2 for examples).
84