Survey Research Types of Surveys Surveys are classified according ...

Survey Research
Types of Surveys
Surveys are classified according to their focus and scope (census and sample surveys) or according to the
time frame for data collection (longitudinal and cross-sectional surveys).
A survey that covers the entire population of interest is referred to as a census. In research, however the
population is used to refer to the entire group of individuals to whom the findings of a study apply. The
researcher defines the specific population of interest.
Classifying surveys on the basis of their scope and their focus gives four categories:
1) A census of tangibles: one seeks information about a small population, such as a single school, and
when the variables are concrete, there is little challenge in finding the required answers.
2) A census of intangibles: one seeks information about constructs that aren’t directly observable but
must be inferred from indirect measures. Such constructs may include pupil achievement or
aspirations, teacher morale, parents’ attitudes toward school, or the achievement testing program
carried out by most schools.
3) A sample survey of tangibles: one seeks information about large groups. Sampling techniques are
used and the information collected from the sample is used to make inferences about the population
as a whole.
4) A sample survey of intangibles: one seeks information about constructs that aren’t directly observable
but must be inferred from responses made by the subjects to questionnaires or interviews. For
example, how someone is going to vote is an intangible, but what is marked on a ballot is tangible.
Surveys Classified According to the Time Dimension
1) Longitudinal surveys: gather information at different points in time in order to study changes over
extended periods of time.
2) Panel study: the same subjects are surveyed at different times over an extended period. Because the
same subjects are studied over time, researchers can see the changes in the individuals’ behavior and
investigate the reasons for the changes.
3) Trend study: different people from the same population are surveyed at different times.
4) Cohort study: a specific population whose members changed over the duration of the study is
followed over a length of time.
5) Cross-sectional surveys: study a cross section (sample) of a population at a single point in time.
The Survey Technique
The survey permits one to gather information from a large sample of people relatively quickly and
inexpensively. There are five basic steps involved in survey research:
Ma660 Survey Research Notes Page 1

1) Planning: Survey research begins with a question that the researcher believes can be answered most
appropriately by means of the survey method. The researcher needs to decide on the data-gathering
technique that will be used.
2) Sampling: The researcher must make decisions about the sampling procedure that will be used and
the size of the sample to survey. If one is to generalize the sample findings to the population, it is
essential that the sample selected be representative of that population.
3) Constructing the instrument: A major task in survey research is the construction of the instrument
that will be used to gather the data from the sample.
4) Conducting the survey: Once the data-gathering instrument is prepared, it must be field-tested to
determine if it will be provide the desired data. Also included in this step would be training of the
users of the instrument, interviewing subjects or distributing questionnaires to them, and verifying the
accuracy of the data gathered.
5) Processing the data: The last step includes coding the data, statistical analysis, interpreting the
results, and reporting the findings.
Data Gathering Techniques
There are two basic ways in which data are gathered in a survey research: interviews and questionnaires.
Each of these has two options, thus providing four different approaches to collecting data:
1) Personal interview: the interviewer reads the questions to the respondent in a face-to-face setting
and records the answers.
Advantages of personal interviews:
a. Flexibility: The interviewer has the opportunity to observe the subject and the total situation
in which he or she is responding. Questions can be repeated or their meanings explained in
case they are not understood by the respondent(s). The interview can also press for additional
information when a response seems incomplete or not entirely relevant.
b. Greater Response Rate: Response rate refers to the proportion of the selected sample that
agrees to be interviewed or returns a completed questionnaire. Personal contact increases the
likelihood that the individual will participate and will provide the desired information.
Furthermore, the interviewer is able to obtain an answer to all or most of the questions.
c. Interviewer Control of Question Order: The interviewer has control over the order with which
questions are considered. In some cases, it is very important that respondents not know the
nature of later questions because their responses to these questions might influence earlier
responses. This problem is eliminated in an interview, where the subject does not know what
questions are coming up and cannot go back and change answers previously given. For
individuals who cannot read and understand a written questionnaire, interviews provide the
only possible information-gathering technique.
Disadvantages of personal interviews:
a. Cost: Personal interviews are more costly than other survey methods. The selection and
training of the interviewers and their travel to the interview site make the procedure costly. It
Ma660 Survey Research Notes Page 2

takes a great deal of time to contact potential respondents, set up appointments, and actually
conduct the interview.
b. Possibility of Interviewer Bias: Interviewer bias occurs when the interviewer’s own feelings
and attitudes or the interview’s gender, race, and other characteristics influence the way in
which the questions are asked or interpreted. The interviewer may verbally or nonverbally
encourage or reward “correct” responses that fit his or her expectations.
c. Social Desirability Bias: Social desirability bias occurs when the respondents want to please
the interviewer by giving socially acceptable responses that they would not necessarily give
on an anonymous questionnaire.
2) Telephone interview: The telephone interview has become more popular and compares favorably
with face-to-face interviewing.
Advantages of telephone interviews:
a. Lower Cost and Faster Completion with Relatively High Response Rates: Telephone
interviews can be conducted over a relatively short time span with persons scattered over a large
geographic area. The phone permits the survey to reach people who would not open their doors to
an interviewer, but who would not open their doors to an interviewer, but who might be willing to
talk on the telephone.
b. Respondent Anonymity: Respondents have a greater feeling of anonymity—hence there may be
less interviewer bias and less social desirability bias than is found with personal interviews.
c. Computer Usage: Computers can be used in administering the interview and in coding the
responses. Wearing earphones, the interviewer can sit at a computer while it randomly selects a
telephone number and dials. When the respondent answers, the interviewer reads the questions
that appear on screen and types the answers directly into the computer. This saves the researcher
time usually spent in coding, getting the data organized, and entering it into the computer for
analysis.
Disadvantages of telephone interviews:
a. Less Opportunity for Establishing Rapport with the Respondent: It takes a great deal of skill
to carry out a telephone interview so that valid results are obtained. It is often difficult to
overcome the suspicions of the surprised respondents, especially when personal or sensitive
questions are asked. An advance letter that informs the potential respondents of the approaching
call is sometimes used to deal with this problem, but the letter can induce another problem. The
recipient has time to think about responses or to prepare a refusal to participate when the call
comes.
b. Survey Exclusion: Households without telephones and those with unlisted numbers are
automatically excluded from the survey, which may bias results. There is, however, a technique
known as random digit dialing that solves the problem of unlisted numbers (although it does not
help to reach the households without a telephone).
Ma660 Survey Research Notes Page 3

3) Mailed questionnaire: Often much of the same information can be obtained by means of a
questionnaire that is mailed to each individual in the sample, with a request that it be completed and
returned at a given date. Because the questionnaire is mailed, it is possible to include a larger number
of subjects as well as subjects in more diverse locations than is practical with the interview.
Advantages of mailed questionnaires:
a. Confidentiality or Anonymity: A mailed questionnaire guarantees confidentiality or anonymity,
thus perhaps eliciting more truthful responses than would be obtained with a personal interview.
b. No Interview Bias: The mailed questionnaire eliminates the problem of interviewer bias.
Disadvantages of mailed questionnaires:
a. Possibility of Question Misinterpretation: It is difficult to formulate a series of questions whose
meanings are crystal-clear to every reader. The investigator may know exactly what is meant by a
question, but the respondent may interpret question differently.
b. Low Return Rate: It is easy for the individual who receives a questionnaire to lay it aside and
simply forget to complete and return it. A low response rate limits the generalizability of the
results of the questionnaire study. Factors that have been found to influence the rate of returns for
a mailed questionnaire are:
1) the length of the questionnaire
2) the cover letter
3) the sponsorship of the questionnaire
4) the attractiveness of the questionnaire
5) the ease of completing it and mailing it back
6) the interest aroused by the content
7) the use of a monetary incentive
8) the follow-up procedures
4) Directly-administered questionnaire: This questionnaire is administered to a group of people at a
certain place for a specific purpose. Examples include surveying the freshmen or their parents
attending summer orientation at a university.
Advantages of directly-administered questionnaire:
a. High Response Rate: The response typically reaches 100 percent.
b. Low Cost
c. Researcher Availability: Researcher is present to provide assistance or answer questions.
Disadvantages of directly-administered questionnaire:
a. Researcher is restricted in terms of where and when the questionnaire can be administered.
b. Because the sample is usually quite specific, the findings are generalizable only to the population
that the sample represents.
Ma660 Survey Research Notes Page 4

Constructing the Instrument
Types of Questions: Because survey data consist of peoples’ responses to questions, it is very important to
start with good questions. Two basic types of questions are used in survey instruments:
1) Closed-ended Questions: One uses closed-ended questions when all of the possible, relevant
responses to a question can be specified and the number of possible responses is limited.
Advantages:
a. Responses are easier to tabulate.
b. Responses can be coded directly on scannable sheets that can be “read” and the data put into a
computer for analysis.
c. Respondents can be answered more easily and quickly.
d. Ensures that all subjects will have the same frame of reference in responding and may also make
it easier for subjects to respond to questions dealing with topics of a sensitive or private nature.
Disadvantages:
a. Take more time to construct.
b. Do not provide more insight into whether respondents really have information or any clearly
formulated opinions about an issue.
c. Easier for the uninformed respondent to choose one of the suggested answers than to admit to
lack of knowledge on an issue.
d. Respondents who have the knowledge or who have well-informed opinions on the issue may
dislike being restricted to simple response categories that do not permit them to qualify their
answers.
2) Open-ended questions are used when there are a great number of possible answers or when the
researcher is not able to predict all the possible answers.
Advantages:
a. Permit a free response rather than restricting the respondent to a choice from among stated
alternatives.
b. Individuals are free to respond from their own frame of reference, thus providing a wide range of
responses.
c. Easier to construct
Disadvantages:
a. Tedious analysis and time-consuming as the researcher must read and interpret each response,
then develop a coding system that will make possible a quantitative analysis of the responses.
b. Some responses may be unclear, and the researcher is unsure how to classify or code the
response.
Ma660 Survey Research Notes Page 5

Writing Survey Questions
Before beginning to write a structured set of survey questions, it can be helpful to have focus groups discuss
the questions in a nonstructured form. A moderator keeps the discussions focused on a preset agenda and
asks questions to clarify comments. Focus group discussions help the researcher understand how people talk
about the survey issues, which is helpful in choosing vocabulary and in phrasing questions. Focus groups
often can suggest issues, concerns, or points of view about the topic that the researcher has not considered.
Characteristics of a Good Questionnaire:
1) It deals with a significant topic, one the respondent will recognize as important enough to warrant
spending one’s time on. The significance should be clearly and carefully stated on the questionnaire
or in the letter that accompanies it.
2) It only seeks information that cannot be obtained from other sources such as school reports or census
data.
3) It is as short as possible and only long enough to get the essential data. Long questionnaires
frequently find their way into the wastebasket.
4) It is attractive in appearance, neatly arranged, and clearly duplicated or printed.
5) Directions are clear and complete. Important terms are defined. Each question deals with a single idea
and is worded as simply and clearly as possible.
6) The questions are objective with no leading suggestions as to the responses desired.
7) Questions are presented in good psychological order, proceeding from general to more specific
responses. The order helps respondents to organize their own thinking so that their answers are
logical and objective.
8) It is easy to tabulate and interpret.
Preparing the Cover Letter
Researchers may find it useful to mail an introductory letter to potential respondents in advance of the
questionnaire itself. This procedure alerts the subject to the study so that he/she is not overwhelmed by the
questionnaire package. In any case, a cover letter addressed to the respondent by name and title must
accompany the questionnaire. The cover letter should be as brief as possible. One page is the maximum
recommended length. Enclose the letter in an envelope along with the questionnaire. Always include a selfaddressed,
stamped return envelope for the respondent’s use. This is indispensable for a good return rate.
Token monetary incentive increases response rate; however, offering a payment isn’t always possible
because a token amount can greatly increase the cost of the survey if the sample is large.
The cover letter introduces the potential respondents to the questionnaire and “sells” them on responding.
The cover letter should include the following elements:
1) The Purpose of the Study: The first paragraph should explain the purpose of the study and its
potential usefulness. It will be helpful to relate the importance of the study to a reference group with
which the individuals may identify.
2) A Request for Cooperation: The letter should explain why the potential respondent was included in
the sample and should make an appeal for the respondent’s cooperation. Respondents should be made
to feel that they can make an important contribution to the study.
Ma660 Survey Research Notes Page 6

3) The Protection Provided the Respondent: The letter must not only assure the respondents that their
responses will be confidential but must also explain how that confidentiality will be maintained.
Researchers may use an identification coding system.
4) Sponsorship of the Study: The signature on the letter is important in influencing the return of the
questionnaire. If the study is part of a doctoral dissertation, it would be helpful if a person well
known to the respondents, such as the head of a university department or the dean of the school, signs
or countersigns the letter. Such a signature is likely to be more effective than that of an unknown
graduate student.
5) Promise of Results: An offer may be made to share the findings of the study with the respondents if
they are interested. They should be told how to make the request for the results known to the
researcher.
6) Appreciation: An expression of appreciation for their assistance and cooperation with the study
should be included.
7) Recent Data on the Letter: The cover letter should be dated near the day of mailing. A potential
respondent will not be impressed by a letter dated several weeks before receipt.
8) Request for Immediate Return: Urge immediate return of the questionnaire. A questionnaire that
fails to receive attention within a week is not likely ever to be returned.
In order to reach the maximum percentage of returns in a mailed questionnaire survey, planned follow-up
mailings are essential. First follow-up having both a letter and a replacement questionnaire should occur in
about 7-10 days after the initial mailing; second follow-up in about 3 weeks, and third follow-up in about 6
to 7 weeks. It is suggested that the researcher include in the third follow-up a postcard on which subjects
could indicate that they do not wish to participate in the survey and will not be returning the questionnaire.
Such a procedure permits definite identification of nonrespondents.
Ma660 Survey Research Notes Page 7