Conceptual Framework and Overview of Psychometric Properties

Reading, Writing, and Other EducationalProgram CharacteristicsSome additional items address other importantaspects of how students spend their time andwhat the institution asks them to do, whichdirectly and indirectly affect theirengagement. The results discussed in thissection are not presented in a table but areavailable from the NSSE website. The fiveitems about the extent to which the institutionemphasizes different kinds of mental activitiesrepresent some of the skills in Bloom=s(1956) taxonomy of educational objectives.The standardized alpha for these items is .70when the lowest order mental function item,memorization, is included. However, thealpha jumps to .80 after deleting thememorization item. This set of items is amongthe best predictors of self-reported gains,suggesting that the items are reliablyestimating the degree to which the institutionis challenging students to perform higherorder intellectual tasks.Patterns of correlations among these items areconsistent with what one would expect. Forexample, the item related to the number ofhours spent preparing for class is positivelyrelated to several questions surroundingacademic rigor such as the number of assignedcourse readings (.25), coursework emphasison analyzing ideas and theories (.16) andsynthesizing information and experiences(.16), the number of mid-sized (5-19 pages)written papers (.15), and the challengingnature of exams (.21). Likewise, the numberof assigned readings is predictably related tothe number of small (.24) and mid-sized (.29)papers written. Interestingly, the quality ofacademic advising is positively correlatedwith the four higher order mental activities,analyzing (.15), synthesizing (.17), evaluating(.15), and applying (.17), and is alsopositively related to the challenging nature ofexaminations (.20).The set of educational program experiences(e.g., internships, study abroad, communityservice, working with a faculty member on aresearch project) have an alpha of .52.Working on a research project with a facultymember is positively related to independentstudy (.27), culminating senior experiences(.25), and writing papers of 20 pages or more(.15). Also, students who had taken foreignlanguage coursework were more likely tostudy abroad (.24). It=s worth mentioning thatthe national College Student ExperiencesQuestionnaire database shows that theproportion of students saying they haveworked on research with a faculty member hasactually increased since the late 1980s,suggesting that collaboration on research maybe increasingly viewed and used as adesirable, pedagogically effective strategy(Kuh & Siegel, 2000; Kuh, Vesper, Connolly,& Pace, 1997).Finally, the time usage items split into twosets of activities, three that are positivelycorrelated with other aspects of engagementand educational and personal gains (academicpreparation, extracurricular activities, work oncampus) and three items that are either notcorrelated or are negatively associated withengagement (socializing, work off campus,caring for dependents). Less than 1% of fulltimestudents reported a total of more than100 hours across all six time allocationcategories. Three quarters of all studentsreported spending an average of between 35and 80 hours a week engaged in theseactivities plus attending class. Assuming thatfull-time students are in class about 15 hoursper week and sleep another 55 hours or so aweek, the range of 105 to 150 hours taken upin all these activities out of a168-hour week appears reasonable.A few of these items have out-of-range butexplainable skewness and kurtosis indicators.They include the number of hours spentworking on campus (72% work five or fewerhours per week), the number of papers of 20Framework & Psychometric PropertiesPage 9 of 26