Conducting a Participatory Situation Analysis of.pdf - Global HIV ...

that school year completed decreases as length of time
with a non-parent increases. It is often useful to conduct
a cross-tabulation with the following variables:
• Child’s relationship to head of household (non-parent,
father, or mother) in relation to emotional wellbeing
issues
• Child’s age and gender in relation to emotional
issues (e.g., having bad dreams, feeling sad most of
the time)
• Child’s home in relation to average distance to nearest
available school
• Child’s home in relation to average time to reach
school
Tests of statistical significance
The Advisory Team will want to know if there is a significant
or “real” difference among certain variables, or if
the difference could be the result of random measurement
error. Determining the significance involves calculating
inferential statistics or probabilities of an event
occurring. For example, does living with a non-parent
increase the likelihood that an orphan will drop out of
school? Another area of inquiry might be whether
orphans in non-family care experience more, less, or the
same psychological distress as orphans in foster care. If
the population sample is large enough, various statistical
tests can be performed to determine if there are significant
differences among and between certain variables.
Two commonly used test statistics for the data collected
using the surveys in this Guide include the chi-square
and the t-test. When applied within a given confidence
level, the chi-square test assumes that there is no difference
among variables and that if a difference is found
then it is not due to chance; there is evidence of a significant
finding. The t-test can determine if, for example, a
5% difference between two variables is significant given
the parameters of the confidence level. T-tests are used
when comparing the means of a continuous variable
between groups (usually two groups), while chi-square
tests are most applicable to categorical variables.
The Situation Analysis Advisory Team need not be proficient
in statistical analysis. A basic understanding of
why a particular test statistic is used and the strengths
and limitations of that test statistic builds confidence in
one’s ability to defend and take action on the findings.
Also, the Advisory Team can question the results by asking,
for example, whether a 10% difference among two
items or variables is relevant. The Advisory Team should
Statistical Confidence
Because the data collection procedure asks questions
of only a sample of the total number of possible
respondents, the variable calculated from the replies
of respondents may not reflect the true proportion
of members of the sub-population that fall into a specific
category (e.g., those who are OVC or those who
care for OVC). It is, therefore, useful to calculate confidence
intervals with respect to the variable.A confidence
interval is the range in which one can be
assured, reasonably certain, or confident that the proportion
of people responding to a given question
accurately reflects the situation. In fact, it is important
to remember that the true values of proportions are
never known. Rather, using statistical theory, confidence
intervals are constructed that give a range
within which it is assumed the true value lies.
Typically, the level of confidence used to calculate this
range is 90%. For example, if a survey shows that 30%
of orphans attended school in the past six months,
and the 95% confidence interval has been calculated
to be between 26% and 34%, one can state, with a
95% degree of confidence, that the true value lies
between these values.This helps to understand the
accuracy and precision of the estimates.As the sample
size becomes larger, the confidence intervals
become narrower, and one can be more confident
about the precision of the estimates.Technically, a
95% confidence interval means that if the study is
repeated 100 times using the same exact procedures,
including true random sampling, and the 95% confidence
interval is calculated 100 times, then 95 times
out of 100 the true value will lie within the confidence
interval.
not make hurried conclusions that are not supported by
the situation analysis findings. There are many ways to
misrepresent or distort the data. The boundaries
between legitimately representing data and distorting it
to promote or advocate an issue must be understood.
Critical thinking as well as dialogue with people with
varying viewpoints about the situation analysis can
mitigate this risk.
Another area of concern is overemphasis on statistical
significance at the expense of practical or substantive
significance. A large sample size will have significance
among or between some variables, but the items found
64
Guidelines and Tools