Grauerholz 2003 Assessing critical thinking sociology.pdf - Oncourse

More documents

Recommendations

Info

CRITICAL SOCIOLOGICAL THINKING 489uses ad hominem arguments."* fails to define key terms or uses circulardefinitions."* insensitive to weaknesses and contradictionsin own logic and/or evidence."* appeals to numbers or prevailing opinion(e.g., "we all know that..."; "since mostpeople think that...it follows that..."-thebandwagon rationale)."* not sensitive to questions of reliabilityand validity of evidence."* oversimplifies or misrepresents opponents'arguments-the straw person fallacy."* value assumptions left unstated and unrecognized."* cites the opinions of persons whose expertiseis dubious or undocumented.(Green and Klug 1990:470)These criteria were provided as guidelinesrather than indicators. In Green and Klug'sstudy, graders simply applied a letter gradeto the essay (e.g., "B").Another empirical study that attempted tomeasure students' sociological perspectivewas that by Bengston and Hazzard (1990).Students were asked to answer 20 questionsabout crime, family, stratification and religionthat reflected general sociological conceptsand perspectives (e.g., "The idea ofsocial class involves more than income andwealth. It also involves more subjectivenotions of power and prestige") (Bengstonand Hazzard 1990:44).In the current study, we developed empiricalmeasures of critical sociologicalthinking, relying heavily upon Green andKlug's (1990) criteria as well as Geertsen's(2003a) concept of referential thinking. Wedevelop and present a scale that may beuseful to researchers who are interested inmeasuring students' critical sociologicalthinking. We also suggest ways the scalecan be used by instructors to assess students'learning and improve their teaching.METHODSThe StudyThe data used in this study were obtainedfrom undergraduate students enrolled infour sections of a Sociology of Marriageand Family course that were taught overthree semesters. The sections were taughtby the same instructor, used identical outlinesand readings, had similar structures (acombination of lecture and discussion), andthe same number and type of exams. Onlythe type and amount of writing differed.All students across the four sections weregiven the same final exam question. Thedata used for this study come from thesefinal exams. The exam was designed to assessstudents' ability to think sociologicallyand to demonstrate critical thinking. Theessay question was as follows:One of the major principles of social psychologyis that social forces impact individualbehavior, emotions, values and attitudes andthat individuals can and do impact social institutions.Throughouthis course, we've exploredvarious ways in which this has occurredwith respect to marriage and family relationshipsin American society.Choose a relationship/family dynamic discussedin class (e.g., power, division of labor,intimacy, mate selection, communication,parenting, divorce) and discuss how this dynamiccan be understood from a social psychologicalapproach. In other words, explain howthis dynamic has been shaped by various socialforces (broadly defined) and also how individuals(or groups) have reshaped this dynamicto produce social change. For example,if you explore the dynamic of power, youwould want to discuss social factors that contributeto power dynamics in contemporarymarriages (e.g., race, class, gender norms,resources) and also how some couples havesuccessfully negotiated nontraditional powerrelationships.Your essay should include the following:1. A clear statement and description of whatdynamic you are investigating.2. A clear and concise discussion of how thisdynamic is shaped by social forces.Throughouthis discussion, be certain toincorporate points from lecture and readingto substantiate your claims, be clear aboutwhat social forces you are analyzing, and bethorough in discussing relevant forces.
490 TEACHING SOCIOLOGY3. A clear and concise discussion of the extentto which and how individuals or groupshave managed to alter this relationship dynamic.You should provide evidence fromreadings or lecture to substantiate yourclaims.The explicit instructions given to studentsencourage them to demonstrate their abilityto think critically rather than their inclinationto do so (Logan 1976). In other words,we are testing whether students are able toproduce evidence of critical sociologicalthinking in their writing when asked to doso. It should also be noted that students receivedthis exam both during the first weekof the semester and at the end of the term.Throughout the semester, the instructorrepeatedly urged students to consider thesequestions in relationship to specific topicscovered.All 207 students enrolled in the fourclasses took the final exam. Of these, 81percent agreed to have their exams used inthis study. In all, there were a total of 167cases analyzed, about 40 from each section.Eighty percent of the respondents werewomen, and 79 percent were Liberal Artsmajors.MeasuresBuilding upon the work of Green and Klug(1990) and Geertsen (2003a), we developedeleven items that tapped aspects of criticalsociological thinking. Five items reflectconceptualized thinking (Geertsen 2003a)and involve the ability to narrow down andanalyze an issue by using examples, analyzetwo sides of an issue, and so on (see items1-5 below). Four items concern contextualizedthinking and tap into thinking generallyassociated with the sociological imagination(see items 6-9 below). In addition to specificindicators, we used holistic measuresof critical thinking (iteml0 below) and sociologicalimagination (item 11). Each itemwas rated on a 1-5 scale, with 5 indicatingstronger skills. The scale items are as follows:1. Uses examples (5= sufficient number, usedappropriately; 1= none and/or used inappropriately)2. Examples are pertinent/relevant to argument(5 =meaningful/pertinent;1 = unnecessary/meaningless)3. Reasoning (5=clear, complete; 1= vague,inadequate)4. Bias (5= never appeals to prejudice, prevailingopinion, feelings; 1l=often appeals toprejudice, prevailing opinion, feelings)5. Mentions two sides of an issue (5= often;presents alternative perspectives or arguments;1l=never; presents only one side ofthe issue)6. Indicates an awareness of social structuralcontexts (e.g., class, gender, age, race,religion, sex orientation) (5=clear, meaningfulmention of social structural contexts;1 =no mention)7. Indicates an awareness of historical contexts(5= appropriate mention of historical contexts;1 = ignores historical contexts)8. Indicates an awareness of cross-culturalcontexts (5 =appropriate mention/descriptionof cross-cultural contexts;1 = ignores cross-cultural contexts)9. Uses sociological concepts to illuminate andanalyze the issue (5= several appropriateconcepts used; 1l=none used)10.Demonstrates critical thinking (5=highlevel; 1l=very little)11. Demonstrates sociological imagination(5=clear awareness of private/publiconnection;1 =completely unaware)The exams were coded by two advancedgraduate students who were familiar withthe study and who had taught the course inthe past. About 20 percent of the examswere coded by both individuals in order totest inter-coder reliability. The findingssuggested that exact matches in codes averagedonly about 46 percent. However, 85percent of the time, coders were only onepoint off from each other (e.g., 4 versus 5on a five-point scale).AnalysisWe begin by presenting means, standarddeviations, and bivariate correlations for
Page 2 and 3: EXPLORING CRITICAL SOCIOLOGICAL THI
Page 5: 488 TEACHING SOCIOLOGYcal thinking
Page 9 and 10: 492 TEACHING SOCIOLOGYocdoorOCD l 0
Page 12 and 13: CRITICAL SOCIOLOGICAL THINKING 495a

Grauerholz 2003 Assessing critical thinking sociology.pdf - Oncourse

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?