Conceptual Framework and Overview of Psychometric Properties

or to compare their students= performancewith data from other institutions on amutually-determined basis for purposes ofbenchmarking and institutional improvement.Validity, Reliability, and Credibilityof Self-Report DataAs with all surveys, the NSSE relies onself-reports. Using self-reports from studentsto assess the quality of undergraduateeducation is common practice. Someoutcomes of interest cannot be measured byachievement tests, such as attitudes and valuesor gains in social and practical competence.For many indicators of educational practice,such as how students use their time, studentreports are often the only meaningful sourceof data.The validity and credibility of self-reportshave been examined extensively (Baird, 1976;Berdie, 1971; Pace, 1985; Pike, 1995;Pohlmann & Beggs, 1974; Turner & Martin,1984). The accuracy of self-reports can beaffected by two general problems. The mostimportant factor (Wentland & Smith, 1993) isthe inability of respondents to provideaccurate information in response to aquestion. The second factor is unwillingnesson the part of respondents to provide whatthey know to be truthful information (Aaker,Kumar, & Day, 1998). In the former instance,students simply may not have enoughexperience with the institution to render aprecise judgment or they may not understandthe question. The second problem representsthe possibility that students intentionallyreport inaccurate information about theiractivities or backgrounds. Research showsthat people generally tend to respondaccurately when questions are about their pastbehavior with the exception of items thatexplore sensitive areas or put them in anawkward, potentially embarrassing position(Bradburn & Sudman, 1988).The validity of self-reported time use has alsobeen examined (Gershuny & Robinson, 1988).Estimates of time usage tend to be lessaccurate than diary entries. However, thisthreat to validity can be amelioratedsomewhat by asking respondents aboutrelatively recent activities (preferably sixmonths or less), providing a frame ofreference or landmark to use, such as theperiod of time to be considered (Converse &Presser, 1989). Such landmarks aid memoryrecall and reduce distortion by telescoping,the tendency for respondents to rememberevents as happening more recently than theyactually did (Singleton, Straits, & Straits,1993). Requesting multiple time estimatesalso makes it possible to control for outliers,those whose combined estimates of time areeither so high that the total number of hoursreported exceeds the number available for theset of activities or those that are unreasonablylow.Student self-reports are also subject to thehalo effect, the possibility that students mayslightly inflate certain aspects of theirbehavior or performance, such as grades, theamount that they gain from attending college,and the level of effort they put forth in certainactivities. To the extent this Ahalo effect@exists, it appears to be relatively constantacross different types of students and schools(Pike, 1999). This means that while theabsolute value of what students report maydiffer somewhat from what they actually do,the effect is consistent across schools andstudents so that the halo effect does not appearto advantage or disadvantage one institutionor student group compared with another.With this in mind, self-reports are likely to bevalid under five general conditions (Bradburn& Sudman, 1988; Brandt, 1958; Converse &Presser, 1989; DeNisi & Shaw, 1977;Hansford & Hattie, 1982; Laing, Swayer, &Noble 1989; Lowman & Williams, 1987;Pace, 1985; Pike, 1995). They are: (1) whenFramework & Psychometric PropertiesPage 3 of 26