Journal of Research in Innovative Teaching - National University

assessment ratings for graduates versus undergraduates were very similar (also see Table
2). Comparisons among the college/schools revealed very little (see Table 3), except that
the College of Letters and Science (COLS) did the best job of balancing low GPAs with
high assessment ratings. Finally, full-time and associate faculty (compared to adjuncts)
did better at keeping the GPA low and the assessment ratings high (see Table 4).
Validity Analysis
One way to assess validity is through factor analysis. Factor analysis is a statistical
technique that shows whether items tend to fall into meaningful clusters. Ideally, if the
assessment instrument were valid, we would expect the items listed for Student
Assessment of Learning (SAL) to form one unique factor, items under Assessment of
Teaching (AT) to form another unique factor, and items under Assessment of Course
Content (ACC) to form a third unique factor. (Assessment of Web-Based Technology
could not be included because only the online students received these items.) The set of
assessment questions did not yield a three-factor solution. Instead, two distinct factors
were uncovered. In general, the items listed under AT formed one factor, and the items
under SAL and ACC formed the second factor. Although this factor analysis was not
completely consistent with the expected factor structure, the results give some credence
to the validity of the assessment because the factors suggest that the instrument measures
teaching ability and some combination of learning and course content.
Another way to examine validity is to see if the assessment instrument is related
to other variables as expected. In the comparative analysis the ratings were related to the
type of class and faculty rank in ways that we would expect a valid instrument to behave
(e.g., online classes get lower rating, full-time and associate faculty get higher ratings).
Another variable of interest is grades. Although most instructors believe that
assessment ratings are related to grades, the empirical data have not always supported this
presumed relationship. However, most studies have only looked at linear relationships
and have not considered the possibility that the relationship might be curvilinear. Indeed,
when we look at the function that relates GPA to the question that asks for an overall
assessment of the teacher (“Overall, the instructor was an effective teacher”), we see a
non-linear trend (see Figure 1). The traditional Pearson correlation coefficient was not
significant (r = .007, p = .242) and the linear trend was not significant [F (1, 30396) =
1.37, p = .242]. The curvilinear (quadratic) trend, however, was statistically significant [F
(1, 30395) = 44.25, p < .000]. This finding and the positive nature of the relationship
shows that student satisfaction was positively related to grades in that students who earn
higher grades tend to give their instructors higher evaluations. As noted below, this
finding can be interpreted in at least two ways. One way to view these results is to argue
that student ratings are associated with student performance and thus supports the validity
of the assessment instrument.
178