Aggregate versus Disaggregate Data in Measuring School Quality

esults validate this hypothesis. However, aggregate data are more likely to suffer from errors in
measurement than disaggregate data. As stated in the introduction, this is due to student mobility,
and in general, the fact that averages are not taken over the same group of students.
Table 3 shows the results for samples with mean school size of 20 and a variance of 250,
which implies that 70% of schools will have sizes between 10 and 50 students. This is done to
consider the case when policy makers require evaluations at the grade rather than school level.
Results are as before; the aggregate estimator is better than OLS and only slightly worse than the
disaggregate estimator.
2
As school size increases, the variation in averaged residuals due to students ( σ e / n )
becomes insignificant and the averages come closer to their true means. This implies that
aggregation becomes less of a concern for estimating school effects and heteroskedasticity is
almost insignificant. The problem with small or large schools being consistently rewarded almost
disappears. In fact, table 4 shows results for a mean school size of 300 and variance of 100,000.
Differences among ranking measures have narrowed for all estimators, and OLS, the only
estimator that does not rely on estimating variance components performs at its best.
4. Conclusions
Researchers argue that value-added multilevel models provide the most accurate
measures of school quality. But most states continue to use aggregate data (usually not in a value
added framework) to rank and reward schools. Research criticizing aggregate models, by
comparing them with disaggregate models, have used ordinary least squares rather than
maximum likelihood estimators. This article shows that the criticisms of aggregate models have
been overstated.
17