Instructions for Analyzing Data from CAHPS Surveys

CAHPS ® Surveys and Instructions
across items (as described in the section on variance estimation) for analyses of multiitem
composites.
2
We use the method of moments to estimate the between-entity variance τδ
of the
variance. Because ε and δ are independent,
∑ ∑ ∑
∑
E ( Sˆ
− S)
= E δ + E
2 2 2
i i i i i i
ε
2 2
( n 1) τδ
2 Si
/ ( mi
1)
i
= − + −
where
S
=
∑i
∑
( m −1)
Sˆ
i
i
( m −1)
i
i
estimates S
0
. The CAHPS macro output contains (in the
variable VP) the value Vˆ ˆ
macro, i
= Si / mi, the squared standard error as opposed to the
sample variance. Hence, the between-entity component of the variance of the variance
is estimated by
(
ˆ
∑i
i ∑i
i i )
τ = ( S −S) −2 S / ( m −1) / ( n−1)
2 2 2
δ
and the square of the coefficient of variation is given by
CV
2 2 2
τ δ
/ S
= .
2
The square of the coefficient of variation of the chi-square distribution is CV = 2/ A ,
where A is the degrees of freedom of the distribution (which can be thought of as the
2
inverse of a prior weight). Therefore, it makes sense that we use A = 2/ CV as the
weight of the pooled variance across the entities in the expression for the usual precisionweighted
estimator of the posterior mean of the variance of an individual entity’s ratings.
Substituting into (1), we obtain
Sˆ
smoothed, i
=
AS + ( m 1) ˆ
i
− Si
A+ ( m −1)
i
.
We express this in terms of sampling variances (using the relationship Vˆ ˆ
i
= Si / m ) to
i
obtain:
ˆ
ˆ ˆ
AS / mi + ( mi −1)
Vmacro,
i
Vsmoothed,
i
= Ssmoothed,
i
/ mi
=
.
A+ ( m −1)
This ensures that Vˆ smoothed, i
≈ S / mi
when m
i
is small (implying little information about
the variance) and Vˆ
ˆ
smoothed, i
≈ V macro, i
when mi
→∞ (large amount of information for
i
Instructions for Analyzing Data from CAHPS Surveys
Document No. 2015
Updated 4/2/12
Page 60