Grauerholz 2003 Assessing critical thinking sociology.pdf - Oncourse

EXPLORING CRITICAL SOCIOLOGICAL THINKING*Much has been written about enhancing students' critical thinking abilities,but very little empirical research on this important learning outcome existswithin the sociological literature. Indeed, there is little consensus among sociologists(and non-sociologists) about what critical thinking is. In this paper wereview ways in which sociologists have conceptualized and measured criticalthinking and introduce a new concept--critical sociological thinking--that em-bodies the type of higher-level thinking many sociologists want to help studentsattain. Critical sociological thinking refers to the ability to logicaly andreasonably evaluate an argument or problem while maintaining an awarenessof and sensitivity to social forces and contexts. Further, we develop a scalethat can be used to measure critical sociological thinking and demonstrate itsusefulness in the empirical analysis of student writing. Implications for futureresearch and teaching are discussed.LIZ GRAUERHOLZPurdue UniversitySHARON BOUMA-HOLTROPPurdue UniversityONE OF THE MOST IMPORTANT and commonlearning goals in sociology is enhancingstudents' critical thinking abilities(Goldsmid and Wilson 1980), yet empiricalresearch and theoretical clarity on this importantlearning outcome is sorely lacking(Baker 1981; Geertsen 2003a). Withoutsuch research, we remain at a loss to determinewhether certain teaching techniqueseffectively enhance such learning, whethersome are more effective than others, or iftheir effectiveness varies by student, group,or institution.In this paper, we explore ways in whichcritical thinking has been used theoreticallyand empirically primarily within the sociologicalliterature and attempt to clarify itsmeanings. Building upon the concepts ofreferential thinking, as introduced by Geert-"*The authors would like to thank Reed Geertsen,Janet Wilmoth, John Bean and the anonymousreviewers for their insightful comments onearlier drafts of this paper. Please address allcorrespondence to Liz Grauerholz, Departmentof Sociology and Anthropology, Purdue University,West Lafayette, IN 47907;e-mail: grauer@sri. soc.purdue.edu.Editor's note: The reviewers were, inalphabetical order, Reed Geertsen, PriscillaReinertsen, and Norma Shepelak.sen (2003a), and discipline-specific criticalthinking, as discussed by McPeck (1985),we introduce the concept of critical sociologicalthinking to describe the type of criticalthinking of particular interest to sociologists-criticalthinking with a sensitivity toand awareness of social and cultural contexts.Data from a study of student writingare presented to demonstrate ways in whichcritical sociological thinking can be measuredempirically and used to assess studentlearning.CONCEPTUALIZINGCRITICAL THINKINGCritical thinking seems to be much likegood art: we know it when we see it, wehave some sense of how we might encourageor even teach it, but we are not surehow to assess or measure it. Certainly, theconcept has been overused and impreciselydefined. Baker's (1981:359) claim morethan 20 years ago that "critical thinking isone of those buzz words which seems tomean just about anything to anybody" stillrings true today. Among those who do attemptdefinitions, there is considerable variability(Geertsen 2003a). For instance, criticalthinking has been defined as:Teaching Sociology, Vol. 31, 2003 (October:485-496) 485

486 TEACHING SOCIOLOGYlearning names for the world's objects anddeveloping problem-solving skills. (Goldsmidand Wilson 1980:78)...the ability to create logical arguments basedon the "sociological imagination." (Green andKlug 1990:462)...the process of reasonably deciding what todo and/or believe. (Dorn, cited in Green andKlug 1990:465)the development of "deep professional competenceand sophisticated ethical judgment."(Nelson 1999:175)...identification of ambiguity and its role inreasoning,...discovery of assumptions andvalue conflicts,... evaluation of evidence,...assessmentof logic,...examination ofmetaphorical allusions,...identification of significantomitted information, ...generating alternativeinferences, and...development of areasoned judgment. (Browne and Litwin1987:384)...active critical and creative inquiry".(Shepelak, Curry-Jackson, and Moore1992:19)Others simply bypass the question of definingcritical thinking altogether while claimingto have found ways to enhance "it"(e.g. Zeller 1988).Several scholars have attempted to synthesizethe literature on critical thinking. Ruminskiand Hanks (1995:5) suggest thatdespite disagreement over how criticalthinking should be defined, most expertsagree that "critical thinking includes skillsin applying, analyzing, and evaluating information."In his review of the literatureon critical thinking, Baker (1981:328) suggeststhat critical thinking is usually conceptualizedin two ways: as a "problem-solvingprocess" and as a process of "deductivelogic and argument analysis." Similarly,Dorn (cited in Green and Klug 1990) suggeststhat critical thinking involves microlevelintellectual abilities and skills, such asthe ability to clarify issues and identifyvalue assumptions; macro-level dispositions,such as a predisposition to ask forevidence or to be creative; and macro-levelvalues, such as a commitment to fairness.Geertsen (2003a) reviews various ways inwhich critical thinking has been conceptualizeddating back to John Dewey. Similar toDewey, he distinguishes between criticalthinking, which involves more confirmatoryprocesses such as corroborating evidenceand narrowing perspectives, and reflectivethinking, which involves a broadening ofperspectives. Geertsen (2003a) suggests thatcritical thinking and reflective thinking, intheir many forms, represent higher-levelthinking: "I believe the appropriate umbrellaterm for all types of extraordinarythinking is higher-level thinking" (p. 8).Geertsen (2003a) further identifies sixdimensions of higher-level thinking, each ofwhich involves both critical and reflectivethinking. For example, one dimension ofhigher-level thinking is strategic thinking(thinking that is oriented toward application),which includes decision-making(critical thinking) and problem solving(reflective thinking).Although educators are likely to welcomeall forms of higher-level thinking in theirstudents, Geertsen (2003a) suggests thatcertain types of higher-level thinking aremore important to different disciplines.Medical professionals, for example, need tobe proficient in strategic thinking; that is,decision making and problem solving:Doctors are expected to understand and useestablished procedures for diagnosing andtreating unambiguous diseases; if they do not,a malpractice suit is likely to follow. In otherwords, they must make correct decisionswithin the parameters of established medicalknowledge. However, problem solving comesinto play when doctors encounter symptomsthat do not respond well to existing treatments.This ambiguity calls for a consideration of abroaderange of possibilities (Geertsen 2003a:11).Although many scholars argue that criticalthinking involves abilities and skills that canbe applied to any problem or subject matter(e.g., Paul 1993; Wade and Tavris 1993),

488 TEACHING SOCIOLOGYcal thinking is discipline-specific criticalthinking.EMPIRICAL STUDIESOF CRITICAL THINKINGDespite the ambiguity and overuse of theterm critical thinking (as seen in the previoussection), there has been much moretheoretical work on this topic than empiricalinvestigation. In Paul Baker's (1981:326)insightful article on critical thinking, heasks: "If critical thinking is so important,why have so few sociologists designed testingmethods which can demonstrate theachievement of such educational aims?"Nearly two decades later, sociologists continueto recognize the value of teachingcritical thinking and many have proposedteaching strategies to encourage students'growth in this area, but very few have empiricallytested critical thinking. Obviously,developing reliable and valid measures presentsa daunting task, in part because thereis so little empirical research from which todraw and also because such highly complexlearning is not easily tapped by standardmeasurement techniques. As Browne andLitwin (1987:390) argue: "Critical thinkingis a process, not a body of knowledge likevocabulary that can be mastered. "Baker (1981) and Norris and Ennis (1989)provide extensive reviews of the literatureon critical thinking, including informationabout standardized exams to assess criticalthinking. For instance, Norris and Ennis(1989) offer a useful overview of commerciallyavailable tests as well as guidelinesfor constructing and using both multiplechoiceand open-ended tests of criticalthinking that could be adapted to coursematerial. However, most assessment toolshave been developed by non-sociologistsand do not relate specifically to sociologicalcontent. Also, Baker (1981) and Norris andEnnis (1989) do not offer any results ofempirical studies using these assessmenttools.A few sociologists have attempted tomeasure critical thinking. Logan (1976)developed a test using 20 items, each ofwhich was an example of uncritical thinking(e.g., "Americans should not allow theirforeign policy to continue to be dictated byan egomaniac who would yell 'Yahoo' inthe Taj Mahal just to hear the sound of hisown voice"). He first asked students to respondto statements in any way (to determinestudents' inclination to think critically,even when not prompted) and next to identifyproblems inherent in the statements (totest students' ability to think critically whenasked to do so).Shepelak et al. (1992) developed a questionnairethat taps students' use of and appreciationfor critical thinking, relying uponstudents' perceptions of change in their ownlevel of critical thinking ability. In addition,instructors assessed the level of change theyperceived in students' level of critical thinkingthrough "subjective impressions of theirachievements in class (e.g., papers andpresentations)" (Shepelak, et al. 1992:25).Similarly, Stoecker et al. (1993:336) used"in-depth quantitative and qualitativeevaluations" to assess critical thinking aswell as students' own assessment of their"ability to critically analyze an argument"(Stoecker et al. 1993:337).Such measures, however, often lack empiricalclarity and sensitivity to the sociologicalimagination. There have been a fewexceptions. For instance, Green and Klug's(1990) research on the effects of studentdebate on critical thinking incorporates adistinctly sociological approach. Facultygraders who evaluated students' essays weregiven the following guidelines for evaluatingthe essays:Are sociological concepts used to illuminateand analyze the issue? Does the writer indicatean awareness of historical, cultural, and socialstructural (e.g., class, gender, age, race) contexts?How adequate is the logic of the argument?Some common logical goals to look for:* uses evidence selectively, or uses out ofdate evidence, or uses examples of dubiouspertinence.* generalizes from personal experience or

CRITICAL SOCIOLOGICAL THINKING 489uses ad hominem arguments."* fails to define key terms or uses circulardefinitions."* insensitive to weaknesses and contradictionsin own logic and/or evidence."* appeals to numbers or prevailing opinion(e.g., "we all know that..."; "since mostpeople think that...it follows that..."-thebandwagon rationale)."* not sensitive to questions of reliabilityand validity of evidence."* oversimplifies or misrepresents opponents'arguments-the straw person fallacy."* value assumptions left unstated and unrecognized."* cites the opinions of persons whose expertiseis dubious or undocumented.(Green and Klug 1990:470)These criteria were provided as guidelinesrather than indicators. In Green and Klug'sstudy, graders simply applied a letter gradeto the essay (e.g., "B").Another empirical study that attempted tomeasure students' sociological perspectivewas that by Bengston and Hazzard (1990).Students were asked to answer 20 questionsabout crime, family, stratification and religionthat reflected general sociological conceptsand perspectives (e.g., "The idea ofsocial class involves more than income andwealth. It also involves more subjectivenotions of power and prestige") (Bengstonand Hazzard 1990:44).In the current study, we developed empiricalmeasures of critical sociologicalthinking, relying heavily upon Green andKlug's (1990) criteria as well as Geertsen's(2003a) concept of referential thinking. Wedevelop and present a scale that may beuseful to researchers who are interested inmeasuring students' critical sociologicalthinking. We also suggest ways the scalecan be used by instructors to assess students'learning and improve their teaching.METHODSThe StudyThe data used in this study were obtainedfrom undergraduate students enrolled infour sections of a Sociology of Marriageand Family course that were taught overthree semesters. The sections were taughtby the same instructor, used identical outlinesand readings, had similar structures (acombination of lecture and discussion), andthe same number and type of exams. Onlythe type and amount of writing differed.All students across the four sections weregiven the same final exam question. Thedata used for this study come from thesefinal exams. The exam was designed to assessstudents' ability to think sociologicallyand to demonstrate critical thinking. Theessay question was as follows:One of the major principles of social psychologyis that social forces impact individualbehavior, emotions, values and attitudes andthat individuals can and do impact social institutions.Throughouthis course, we've exploredvarious ways in which this has occurredwith respect to marriage and family relationshipsin American society.Choose a relationship/family dynamic discussedin class (e.g., power, division of labor,intimacy, mate selection, communication,parenting, divorce) and discuss how this dynamiccan be understood from a social psychologicalapproach. In other words, explain howthis dynamic has been shaped by various socialforces (broadly defined) and also how individuals(or groups) have reshaped this dynamicto produce social change. For example,if you explore the dynamic of power, youwould want to discuss social factors that contributeto power dynamics in contemporarymarriages (e.g., race, class, gender norms,resources) and also how some couples havesuccessfully negotiated nontraditional powerrelationships.Your essay should include the following:1. A clear statement and description of whatdynamic you are investigating.2. A clear and concise discussion of how thisdynamic is shaped by social forces.Throughouthis discussion, be certain toincorporate points from lecture and readingto substantiate your claims, be clear aboutwhat social forces you are analyzing, and bethorough in discussing relevant forces.

490 TEACHING SOCIOLOGY3. A clear and concise discussion of the extentto which and how individuals or groupshave managed to alter this relationship dynamic.You should provide evidence fromreadings or lecture to substantiate yourclaims.The explicit instructions given to studentsencourage them to demonstrate their abilityto think critically rather than their inclinationto do so (Logan 1976). In other words,we are testing whether students are able toproduce evidence of critical sociologicalthinking in their writing when asked to doso. It should also be noted that students receivedthis exam both during the first weekof the semester and at the end of the term.Throughout the semester, the instructorrepeatedly urged students to consider thesequestions in relationship to specific topicscovered.All 207 students enrolled in the fourclasses took the final exam. Of these, 81percent agreed to have their exams used inthis study. In all, there were a total of 167cases analyzed, about 40 from each section.Eighty percent of the respondents werewomen, and 79 percent were Liberal Artsmajors.MeasuresBuilding upon the work of Green and Klug(1990) and Geertsen (2003a), we developedeleven items that tapped aspects of criticalsociological thinking. Five items reflectconceptualized thinking (Geertsen 2003a)and involve the ability to narrow down andanalyze an issue by using examples, analyzetwo sides of an issue, and so on (see items1-5 below). Four items concern contextualizedthinking and tap into thinking generallyassociated with the sociological imagination(see items 6-9 below). In addition to specificindicators, we used holistic measuresof critical thinking (iteml0 below) and sociologicalimagination (item 11). Each itemwas rated on a 1-5 scale, with 5 indicatingstronger skills. The scale items are as follows:1. Uses examples (5= sufficient number, usedappropriately; 1= none and/or used inappropriately)2. Examples are pertinent/relevant to argument(5 =meaningful/pertinent;1 = unnecessary/meaningless)3. Reasoning (5=clear, complete; 1= vague,inadequate)4. Bias (5= never appeals to prejudice, prevailingopinion, feelings; 1l=often appeals toprejudice, prevailing opinion, feelings)5. Mentions two sides of an issue (5= often;presents alternative perspectives or arguments;1l=never; presents only one side ofthe issue)6. Indicates an awareness of social structuralcontexts (e.g., class, gender, age, race,religion, sex orientation) (5=clear, meaningfulmention of social structural contexts;1 =no mention)7. Indicates an awareness of historical contexts(5= appropriate mention of historical contexts;1 = ignores historical contexts)8. Indicates an awareness of cross-culturalcontexts (5 =appropriate mention/descriptionof cross-cultural contexts;1 = ignores cross-cultural contexts)9. Uses sociological concepts to illuminate andanalyze the issue (5= several appropriateconcepts used; 1l=none used)10.Demonstrates critical thinking (5=highlevel; 1l=very little)11. Demonstrates sociological imagination(5=clear awareness of private/publiconnection;1 =completely unaware)The exams were coded by two advancedgraduate students who were familiar withthe study and who had taught the course inthe past. About 20 percent of the examswere coded by both individuals in order totest inter-coder reliability. The findingssuggested that exact matches in codes averagedonly about 46 percent. However, 85percent of the time, coders were only onepoint off from each other (e.g., 4 versus 5on a five-point scale).AnalysisWe begin by presenting means, standarddeviations, and bivariate correlations for

CRITICAL SOCIOLOGICAL THINKING 491each of the items tested. Next, we performedfactor analysis of all items to determinewhether the items loaded separately oron two or more factors. This analysis canhelp us determine empirically whether theseitems are part of the same construct or differentones. We utilized a principal componentanalysis using varimax with Kaisernormalization for rotation. We also presentresults of reliability analysis to illustrateadditional properties of the scale.FINDINGSTable 1 presents the means and standarddeviations for each of the items used tomeasure critical sociological thinking. Thescores ranged from a high of 4.2 (examplesused are pertinent/relevant to argument) to alow of 1.8 (indicates awareness of crossculturalcontexts). The high mean scores formost items suggest that students tended toperform fairly well (a mean of 4.0 would becomparable to a "B").The bivariate correlations, shown in Table2, illustrate that many items are closelyrelated to each other. Indeed, when the factoranalysis was conducted, we found thatmost of the indicators loaded on a singlefactor (see Table 3). Further, the factoranalysis suggests that these measures werenot factorially pure. Two items--"indicatesan awareness of historical contexts" and"indicates an awareness of cross-culturalcontexts"-loaded on a separate factor. Athird item, "appeals to prejudices, prevailingopinion, feelings or personal experience"(bias), produced weak loadings onboth factors.In the final analysis, we omitted the threeitems that did not load on the first factor.The result was an eight-item scale consistingof: 1) uses examples, 2) examples arepertinent/relevant to argument, 3) reasoning,4) considers two sides of an issue, 5)indicates an awareness of social structural(e.g., class, gender, age, race, religion, sexorientation) contexts, 6) uses sociologicalconcepts to illuminate and analyze the issue,7) demonstrates critical thinking, and 8)Table 1. Means and standard deviations forcritical sociolo-ical thinking items.StandardMean DeviationUses examples 4.05 1.00Examples pertinent 4.20 1.04Reasoning 3.73 1.06Bias 4.00 1.05Two-sides 3.83 .93Aware social structure 3.99 1.00Aware historical contexts 3.60 1.49Aware cross-cultural 1.80 1.30Use sociological concepts 4.08 1.04Critical thinking (holistic) 3.76 .99Sociological imagination(holistic) 3.87 1.04demonstrates sociological imagination. Asseen in Table 4, these items are highly correlatedand the alpha coefficient for thisscale was very high (.95). Thus, the itemswere summed and divided by 8, so that thetheoretical range was 1-5. The mean scorefor the scale was 3.94 with a standard deviationof .873.DISCUSSIONDevelopment of students' critical thinkingabilities is considered to be a core learninggoal in sociology. For decades scholarshave explored creative strategies to helpstudents achieve this goal. Unfortunately,despite this attention, the concept of criticalthinking has remained unclear. Further, fewsociologists have attempted to test empiricallywhether such strategies are effective inenhancing critical thinking. In fact, fewmeasures of critical thinking in sociologyhave been developed to facilitate this typeof discovery.In this study, we considered some of thevarious ways sociologists have defined criticalthinking. We propose the concept of"critical sociological thinking" to refer tothe ability to evaluate, reason, and question

492 TEACHING SOCIOLOGYocdoorOCD l 00* *,l?) * *0 C 00"E0 o 00Soo r-CC). •,o t-.- N ?OS• • 0 0 0o~ - o r0 (a's(r -OI.0= **: * *0000.,- 0 0 00-vll? * * * **sm. o1U? * * * * *00?• .-. * * * * ***~ C' 0~0 O O 0-- -O •N: N ru "• .0No~ O (rs 0 * * cN *S. . ** *?- .•. N~, 0~ 00 N o0 O N0'-o o o .-*U2SO Lfs - iN ~ O N~N 00* *6• O *. ** * *.*0* . *. * *-o +'3 o • ?a • oS•.,.•.(00,- r"_U, U, .`! crso 0 *: I U, 0 *sr0, • • •• • ..•'UII I IU,,0"-- ? ?0 .-o~ CU(• 0.~O 1. o cO cOc , ~ 0U, 0O o * -dl~~U I I I I 0 Ir C)_______________a _ _ __ _ _ _0.0• •-. - •

CRITICAL SOCIOLOGICAL THINKING 493Table 3. Results of factor analyses using allcritical sociological thinking items.Compo- Component1 nent 2Uses examples .821 -.005Examples pertinent .797 .150Reasoning .904 .004Bias .456 .308Two-sides .836 -.007Aware social structure .859 .008Aware historical contexts .218 .740Aware cross-cultural .168 -.703Use sociological concepts .766 .008Critical thinking (holistic) .938 .004Sociological imagination(holistic) .941 .007Table 4. Reliability analyses for combined"critical sociologicai thinking" scale.Corrected Item-Total CorrelationUses examples .774Examples pertinent .740Reasoning .867Two-sides .783Aware social structure .825Use sociological concepts .708Critical thinking (holistic) .912Sociological imagination(holistic) .926Scale mean = 3.94Standard deviation = .873Alpha coefficient = .9502McPeck's (1990) understanding of criticalthinking as discipline-specific and isgrounded in Geertsen's (2003a) notion ofcritical judging and his two-sided conceptionof referential thinking, which encompassesboth critical and reflective thinking.Thus, we argue that while there may besome basic critical thinking skills and abilitiesthat are applicable to any subject matteror problem (e.g., avoid emotional reasoning),critical thinking within sociology islikely to reflect different skills than criticalthinking in another discipline. Criticalthinking within sociology, or critical sociologicalthinking, requires sociologicalknowledge and awareness. This knowledgeconsists not only of concepts but, more importantly,employs the sociological imagination.In this type of thinking, students perceiveand understand that their individuallife choices, circumstances, and troubles areshaped by larger social forces such as race,gender, social class and social institutions.We also developed empirical measures ofcritical sociological thinking that we hopewill promote greater exploration into howand under what conditions students learn.Most of the measures developed for thisstudy demonstrate high construct and convergentvalidity. That is, eight of the elevenitems tested were highly correlated andloaded on a single factor. Thus, we proposean eight-item Likert scale, consisting ofitems generally associated with referentialthinking (Geertsen 2003a). Several itemstap the students' ability to bring reasonableevidence to bear on an argument (e.g., usesexamples that are meaningful and pertinentto the argument) while others tap students'ability to engage their sociological imaginations(e.g., indicates awareness of socialstructural contexts).The three items tested in this study thatdid not perform well in the factor analysisincluded "awareness of historical contexts,""awareness of cross-cultural contexts," and"bias" (appeals to prejudice, prevailingideas and information while demonstrating opinion, feelings). We suspect that the veryawareness of broader social and cultural weak correlations between cross-culturalcontexts. This concept is consistent with awareness and other items results from the

CRITICAL SOCIOLOGICAL THINKING 495and in class participation. As an instructionaltool, the scale could be used as ascoring rubric for essay exams, debates, orother student work. Secondly, because thescale presented here reflects some of thekey components of critical sociologicalthinking, it can be used to guide curriculumdesign by identifying key learning goals thatshould be addresssed. Third, when usingthis scale to assess students' work, we canidentify gaps in our teaching. For instance,it became clear to the instructor of thecourses used in this study that greater attentionneeded to be paid to cross-cultural factorsafter realizing so few students offeredsuch evidence in their final exams. We believestudents must be taught how to thinkcritically and sociologically in order to demonstratesuch skills on an outcome measure(Logan 1976); thus, when students performpoorly on one or more of the items discussedhere, it may be an indication of areasthat need to be more carefully addressed inour teaching.In conclusion, this study represents aninitial attempt to clarify what critical thinkingin sociology involves and to create empiricalmeasures of critical sociologicalthinking. Our efforts discussed here representa concerted effort but hardly a definitiveone. More research is needed to helpimprove our understanding and measurementof this important learning outcome.REFERENCESBaker, Paul J. 1981. "Learning Sociology andAssessing Critical Thinking." Teaching Sociology8:325-63.Bean, John. 2003. Personal communication.February 27.. 1996. Engaging Ideas. San Francisco,CA: Jossey-Bass.Bengston, William F. and John W. Hazzard.1990. "The Assimilation of Sociology intoCommon Sense: Some Implications for Teaching."Teaching Sociology 18:39-45.Bidwell, Lee D. Millar. 1995. "Helping StudentsDevelop a Sociological Imagination ThroughInnovative Writing Assignments." TeachingSociology 23:401-6.Browne, M. Neil and James L. Litwin. 1987."Critical Thinking in the Sociology Classroom:Facilitating Movement From VagueObjective to Explicit Achievement." TeachingSociology 15:384-91.Geertsen, Reed. 2003a. "Rethinking Thinkingabout Higher-Level Thinking." Teaching Sociology31:1-19..2003b. Personal communication. April 7.Goldsmid, Charles A. and Everett K. Wilson.1980. Passing on Sociology: The Teaching ofa Discipline. Washington, DC: ASA TeachingResource Center.Green, Charles S. III and Hadley G. Klug. 1990."Teaching Critical Thinking and Writingthrough Debates: An Experimental Evaluation."Teaching Sociology 18:462-71.Logan, Charles H. 1976. "Do SociologistsTeach Students to Think More Critically?"Teaching Sociology 4:29-48.McPeck, John E. 1985. "Critical Thinking andthe 'Trivial Pursuit' Theory of Knowledge."Teaching Philosophy 8:295-308.. 1990. Teaching Critical Thinking: Dialogueand Dialectic. New York: Routledge.Nelson, Craig E. 1999. "On the Persistence ofUnicorns: The Trade-Off between Content andCritical Thinking Revisited." Pp. 168-84 inThe Social Worlds of Higher Education: Handbookfor Teaching in a New Century, edited byBernice A Pescosolido and Ronald Aminzade.Thousand Oaks, CA: Pine Forge Press.Norris, Stephen P. and Robert H. Ennis. 1989.Evaluating Critical Thinking. Pacific Grove,CA: Midwest Publications.Paul, Richard. 1993. Critical Thinking: WhatEvery Person Needs to Survive in a RapidlyChanging World, edited by Jane Willsen andA.J.A. Binker. Santa Rosa, CA: Foundationfor Critical Thinking.Ruminski, Henry J. and William E. Hanks.1995. "Critical Thinking Lacks Definition andUniform Evaluation Criteria." Journalism andMass Communication Educator. 50:4-11.Shepelak, Norma J., Anita Curry-Jackson, andVernon L. Moore. 1992. "Critical Thinking inIntroductory Sociology Classes: A Program ofImplementation and Evaluation." TeachingSociology 20:18-27.Stoecker, Randy, Mary Schmidbauer, JoanMullin, and Michelle Young. 1993."Integrating Writing and the Teaching Assistantto Enhance Critical Pedagogy." TeachingSociology 21:332-40.Thompson, Becky and Sangeeta Tyagi. 1993."Multicultural Education and the SociologicalImagination." Teaching Sociology 21:192-96.

496 TEACHING SOCIOLOGYWade, Carole and Carol Tavris. 1993. Criticaland Creative Thinking: The Case of Love andWar. New York: HarperCollins.Zeller, Richard A. 1988. "On Teaching aboutDiscrimination." Teaching Sociology 16:61-66.Liz Grauerholz is an associate professor of sociologyat Purdue University. Her research and teachinginterests center around gender, family, and power. Sheis currently conducting research on the effects of studentwriting on learning and problems facing first-yearcollege students.Sharon Bouma-Holtrop is a graduate student insociology at Purdue University and adjunct professorof sociology of Huntington College. Her researchfocuses on teaching, gender, children, and power.