P P i t Edit d b PowerPoint Edited by: Dr. Tarek Chebbi Florida ...

PowerPoint P Edited d by:Dr. Tarek ChebbiFlorida International UniversityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

How to Design and Evaluate Researchin EducationIndex of ChaptersCh. 1: Nature of ResearchCh. 2: Research ProblemCh. 3: Variables & HypothesisCh. 4: Ethics & ResearchCh. 5: Literature ReviewCh. 6: SamplingCh. 7: InstrumentationCh. 8: Validity & ReliabilityCh. 9: Internal ValidityCh. 10: Descriptive StatisticsCh. 11: Inferential StatisticsCh. 12: Statistics PerspectiveCh. 13: Experimental ResearchCh. 14: Single-Subject ResearchCh. 15: Correlational ResearchCh. 16: Causal-ComparativeComparativeCh. 17: Survey ResearchCh. 18: Qualitative ResearchCh. 19: Observation & InterviewCh. 20: Content AnalysisCh. 21: Ethnographic ResearchCh. 22: Historical ResearchCh. 23: Action ResearchCh. 24: Research ReportsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Key IdeasReasons research is importantProblems with research todayResearch definedThe Research ProcessEthical Considerations in ResearchSkills needed for researchMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Research?The researcher asks a questionThe researcher collects dataThe researcher indicates how the dataanswered the question(In other words, research is a process for providing answers to questions… a way ofknowing,… an approach to creating new and useful knowledge. (Grosof and Sardy, 2001),or it is an approach by which we attempt to find out, systematically, with the support ofdemonstrable facts and standards, how the world and its component works (Babbie, 2002).McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Importance of ResearchReason 2: Research Helps Improve Practice• Educators gain new ideas• Educators gain new insight into methods• Educators gain new insight into studentsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Lines of ResearchResearchLinesWhat lines ofResearch helpYou?What doesthe Research Say?(Literature review)What do youWant to do?(the research Question)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Importance of ResearchReason 3: Research Informs Policy Debates(eg. How to establish rates for Portmore toll road)• Research helps people weigh differentperspectives on issues• Research helps people make informeddecisions regarding policyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Importance of ResearchReason 4: Research Builds Research Skills,such as those below:• Organizational skills• Analytical skills• Writing skills• Presentation skillsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Problems With Research TodayContradictory or vague findingsQuestionable dataUnclear intent of the studyLack of full disclosure of the datacollectedUnclear research problemMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research Spiral:Identify the Research ProblemSpecify a problemJustify a problemSuggest a need to study it foraudiencesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research Spiral:Reviewing i the LiteraturetLocate the Resources• Books• Journals• Electronic ResourcesSelect Resources• Determine the relevant resources for the topic• Organize the resources by developing a “LiteratureMap”Summarize the resources in a Literature ReviewMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research Spiral:Specify a Purpose for the ResearchIdentify the purpose statementt tNarrow the purpose statement:• Quantitative: Research Questions and or hypothesis• Qualitative: Central Phenomenon and sub-questionsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research Spiral: Collecting DataDetermine the data collection methodSelect the individuals to study (Sample – thesample must represent the larger population)Design data collection instruments andoutline proceduresObtain permissionsGather informationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research Spiral:Analyzing and Interpreting ti DataBreaking down the dataRepresenting the dataExplaining the dataMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research Spiral:Reporting and Evaluating ResearchDetermine the audience for the reportStructure the reportWrite the report sensitively andaccuratelyEnsure that the report and findings arepresented in an accepted (universal)standard format.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Skills Needed for ResearchCuriosity to solve puzzlesPatience and determinationUse of the library and technologyWriting and editingMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Research StudiesIntervention studiesAssociational studiesDescriptive studiesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Intervention studiesare used to determine if changes inone variable cause correspondingchanges in an outcome variable. Theresearcher intervenes in theenvironment nment in order to gain thecontrol necessary for verifying causeand effect relationships. Experimentalresearch falls under the heading of anintervention study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Associational studiesare used to investigate relationships. Thesestudies are conducted by measuring variableswithout ih purposefully fll intervening i in the environmentother than to take measurements. Relationshipsfound among variables can suggest cause andeffect, but cannot verify that the changes in onevariable are responsible for changes in othervariable. Correlational l and Causal-ComparativeC research fit under the general heading ofassociational studies.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Descriptive studiesare used to describe thecharacteristics of individuals, places,events, or anything else that can bedescribed. Survey research,qualitative research, content-analysisresearch, and historical research allusually fit best under the generalheading of descriptive studies.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Descriptive studiesis a type of research where the researchermanipulates some conditions (e.g., method ofinstruction) to see if this will effect an outcome(e.g., test scores). Experimental research fallsin the general category of intervention researchbecause the researcher intervenes in theenvironment that he or she is studying. Anexperiment is the most conclusive way todetermine cause and effect relationships. Thatis because the researcher has the most completecontrol in this type of study than in any othertype of study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Correlational researchis a type of research where researchersmeasure two or more variables (e.g., self-esteem and motivation) to see if they arerelated. These measurements are takenwithout otherwise intervening in theenvironment. Although lh h the outcomes ofcorrelational research might suggest thatchanges in one variable causes changes inother variables, this cannot be verifiedwithout an experiment. Correlational researchfalls in the general category of associationalresearch.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Causal-comparative ati researchMcGraw-Hillis conducted td to compare two or more groups that talready differ on some variable of interest(e.g., public school students and private schoolstudents). Such research is called "causal"because the researcher looks for otherdifferences caused by the group difference(e.g., school achievement) or for differencesthat might cause the group difference (e.g.,income of parents). This research does notverify cause and effect as an experiment would,but suggests possible cause and effectrelationships. Causal-comparative research fallsin the general category of associationalresearch.© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Survey research:is conducted by asking the samequestions to a large number ofindividuals. The purpose of surveyresearch is to determine thecharacteristics, i including attitudesand opinions, of a defined group ofindividuals (e.g., parents in a schooldistrict). Survey research fits underthe general heading of descriptiveresearch.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Historical ical research:is conducted to study the past inorder to reconstruct as accurately aspossible what happened during thattime and to explain why it did.Historical research fits loosely underthe general heading of descriptiveresearch.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Content analysis:-research is conducted by studyingdocuments and materials. The purposeof this research is to unobtrusivelystudy the characteristics ofindividuals. Content analysis researchfits under the general heading ofdescriptive research.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Qualitative research:-is a type of in-depth study of the behaviors andinteractions of a defined group of individuals.Qualitative i researchers keep track of manyvariables and use many different types ofmeasuring techniques in order to collect a richset of data that describes these individuals andtheir interactions as completely as possible.Qualitative ti research fits loosely l under thegeneral heading of descriptive research.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

EpeimentalReseachExperimental ResearchMost conclusive of all scientific methods.The researcher establishes treatments andstudies the effects, which can lead to clearinterpretations.• The Experimental Group• The Control Group• The independent variable: What is being tested• The dependent variable: What is the outcome(i.e., score)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Correlational ResearchExamines a relationship among two or morevariables; looks for cause and effect.Can help make more intelligent predictions.This approach requires no manipulation orintervention, except to administer theinstrument(s).Used when you want to look for anddescribe relationships that may existnaturally.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Causal-Comparative Compa ati ResearchDetermines the cause for, orconsequences of, differences betweengroups of people.Interpretations are limited due to thefact that the investigator can not sayconclusively whether a particular factoris a cause for or a result of a behavior.Differences may occur, but theinvestigator will not be able to say forsure what caused the difference.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Survey ResearchCollects data to study specificcharacteristics of a group.Variety of survey techniques exist, suchas:• Close-ended questions• Open-ended questionsThere are difficulties with surveyresearch. These include,(but not limitedto):• Ensuring that questions are clear and not misleading• Getting participants to answer questions honestly• Getting acceptable return/participation rate.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ethnographic ResearchA form of Qualitative ResearchEmphasizes documenting or portraying theeveryday experiences of individuals byobserving and interviewing them in anaturalistic setting.Data can include descriptions, audiotapes,video footage, flowcharts showingrelationships, etc.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Historical ical ResearchAnother form of Qualitative Research.Some aspect of the past is studied.Data are collected and evaluatedobjectively in order to establish whethercauses, effects, or trends of a past eventmay explain present or future events oroccurrences.The major problem with this research isthe question of using an event or timesequence as a true outcome.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Action ResearchMcGraw-HillDiffers from previous methods intwo ways:1) Generalizations to other persons,settings, or situations is of minimalimportance• researchers focus on getting informationthat will enable them to change conditionsin a particular r situation ti (i.e., identifyinginmethods to improve special ed services at aschool)2) Subjects become active members ofthe research process by collectingdata etc© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Basics of EducationalResearchChapter TwoMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Key ConceptsWhat is a research problem and why is itimportant?Can and should the “problem” beresearched?How does the “problem” differ forquantitative and qualitative research?What are the five elements that comprisethe “statement of the problem” section?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is a ResearchProblem?• Researcher must discover and define a specific topic\problemfrom a broad and general problem area.• This specific topic is the Research Problem• A research problem is an educational issue or concern that aninvestigator presents and justifies in a research study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Characteristics i of Research ProblemIt should ask about a relationship between two ormore variables.It should be clearly/unambiguously l l statedt (precise and concise)It can be stated as a question or as a statement,indicating the primary objective of the study.It should be testable by empirical methods -should be possible to collect data to answer thequestions(s) asked.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Why is the ResearchProblem Important?t?Establishes the importance of topicCreates reader interestFocuses reader’s attention on how study willadd to literatureMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Where is the Research ProblemLocated?Look in the opening paragraphs, and askyourself:• What was the issue or problem that the researcherwanted to address?• What is the concern being addressed “behind” thisstudy?• Why was the study undertaken in the first place?• Why is this study important?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

How Does It Differ from OtherParts of Research?A research problem is an issue or problem in thestudyA research topic is the broad subject matter beingaddressed in a study.A purpose is the major intent or objective of thestudy.Research questions are questions the researcherwould like answered or addressed in the study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Differences Among Topic, Problem,Purpose and QuestionsGeneralTopicDistance LearningSpecificResearchProblemPurposeStatementResearchQuestionLack of students in distanceclassesTo study why students do notattend distance education classes ata community college.Does the use of web site technologyin the classroom deter studentsfrom enrolling in a distanceeducation class?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Can and Should the Problem BeResearched?Can you study the problem?• Do you have access to the research site?• Do you have the time, resources and skills tocarry out the research?Should you study the problem?• Does it advance knowledge?• Does it contribute to practice?• Is there a need which requires a resolution?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

How Does the Research Problem Differ forQ i i d Q li i R h?Quantitative and Qualitative Research?Use quantitative if yourresearch problemrequires you toMeasure VariablesAssess the impact of thesevariables on an outcomeTest theories or broadexplanationsApply results to a large number ofpeopleUse qualitative if your researchproblem requires you toLearn about the views of thepeople you plan to studyAssess a process over timeGenerate theories based onparticipant perspectivesObtain detailed informationabout a few people or researchsites.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Variables and HypothesesChapter ThreeMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is a Variable?(A Characteristic or Attribute)That can beandMeasured(Can be assessedon an instrumentand recorded onan instrument)Varies(Different valuesor scores forDifferent people)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Examples of variables and non-variablesVariables• Leadership style• Organizationalcontrol• AutismDifficult butpossiblymeasurablevariables• Socialization• Imagination• Intuition• DiscriminationAlmost impossibleto measurevariables‣Subconsciousthoughts‣World poverty‣StereotypesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Categorical and Continuousmeasures of variablesA categorical measure is a value of a variable iblassigned by the researcher into a small numberof categories. (e.g. Gender). (These variables do not vary indegree, amount or quantity).A continuous measure is the value of a variableassigned by the researcher to a point along acontinuum of scores, from low to high. (e.g.Age). (These variables vary in degree, amount or quantity).McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Variables and ConstructsA Variable is an attribute or characteristicstated in a specific or applied wayA Construct is an attribute or characteristicexpressed in an abstract, general way.ConstructVariableStudent AchievementGrade Point AverageMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

A good research question isStated clearly and concisely;Expresses the relationship between independent anddependent variables;Defines the variables in measurable terms.A good research question must be Specific, Measurable, Achievable,Realistic and have a statedTimeframe for completion of research. Therefore, a good research question must beS.M.A.R.TFor example: Do participants (P) who receive treatment (X) dobetter on measurable variable (Y) than those participantswho do not get the treatment X ( or get some othertreatment (specify)).?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Family of Variables in QuantitativeStudiesProbableCause(X)(Z)Effect(Y)IndependentVariables•Treatment•MeasuredInterveningDependentVariablesVariablesControlVariablesModeratingVariablesConfoundingVariablesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Families of VariablesDependent Variables: An attribute orcharacteristic influenced by the independentvariable. Dependent variables are sometimescalled:Outcome VariablesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Families of VariablesIndependent Variable: An attribute orcharacteristic that influences or effects anoutcome or dependent variable. Independentvariables are sometimes called:• Treatment Variable• Measured Variable• Control Variable• Moderating Variable• Experimental Variable• Manipulated Variable• Extraneous VariablesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Intervening VariablesIntervening Variables (Mediating Variables):An attribute or characteristic that “standsbetween” the dependent and independentvariables.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of an Intervening VariableStep 1ExampleIndependentVariableConvenient office hoursfor studentsDependentVariableStudent seeks helpfrom facultyStep 2ExampleIndependentIndependentVariableVariableConvenient office hoursfor studentsInterveningVariableStudent becomes willingto take risksStep 3Independentd InterveningVariableVariableDependentVariableExampleConvenient office hoursfor studentsStudent becomes willingto take risksStudent seeks helpfrom facultyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Families of VariablesConfounding Variables (Spurious Variables):Attributes or characteristics that theresearcher cannot directly measure becausetheir effects cannot be easily separated fromthe other variables, even though they mayinfluence the relationship between theindependent and the dependent variable.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Theories as Bridges BetweenIndependent d and Dependent VariablesIndependentVariablesDependentVariablesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is a Hypothesis?A hypothesis is a prediction regarding the possibleoutcome of a studyAdvantages of stating hypotheses include:• Forces us to think more deeply and specifically about the possibleoutcomes of the study• Enables us to make specific predictions based on prior evidence ortheoretical argument• Helps to clarify if we are investigating a relationshipDisadvantages of stating hypotheses include:• May lead to a bias on the Chapter of the researcher• In some studies, it would be presumptuous to predict what findingswould be• Focusing on the hypothesis could prevent the researcher from seeingother phenomena that might be important to the studyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Hypotheses from a Single ResearchQuestionMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research HypothesesThree types:• Null hypothesis (specifies that there is no relationship in thepopulation)• Directional alternative hypothesis (indicates the specificdirection that a researcher expects to emerge in a relationship)• Non-directional alternative hypothesis. (there is nospecific prediction about what direction the outcome of a study will take)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Directional vs. Non-directional HypothesesA directional hypothesis is one in which h the researcher indicates thespecific direction that he or she expects will emerge in a relationshipin the study.• The direction is based on what the researcher has found from:• Literature• Personal experiences• Experience from othersA non-directional hypothesis is when there is no specific predictionabout what direction the outcome of a study will take.• Sometimes it is difficult to make specific predictions upon a study• A study may state that it would point to non-specific directions vs. distinct possibilitiesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ethics and ResearchChapter FourMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ethical Considerations in ResearchRespect the rights of the participantsHonor the requests and restrictions of the researchsiteReport the research fully and honestly (if you donot honestly share the research information, it maylead to bias in analysis and conclusion).Ensure that the participants are NOT subjected toany harm (whether physical or psychological)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Considered Ethical Regarding Research?Ethics refers to the Question of “Right” and “Wrong”.Is it “Okay” or “Right” to conduct a particular study.“Will any physical or psychological harm result fromthis particular research?”By practicing ethical procedures, the researcher isdoing “what is right”.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Statement of Ethical PrinciplesEthical behavior is defined ed as conforming o tothe standards of conduct of a givenprofession or group (Webster’s).An agreement among researchers dictateswhat is considered right and wrong.The Statement of Ethical Principles suggeststhree very important issues:• The protection of participants from harm• Ensuring confidentiality of the research data• The question of deceptions of subjectsThe Statement of Ethical Principles can be found on pp. 93-95McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Protecting Participants from HarmConsidered the most important ethical consideration ofall the principlesIt is the responsibility of all researchers to ensure thatparticipants are protected from physical or psychologicalharm, discomfort, or danger due to research proceduresAny study that could cause harm should not beconsideredAnother responsibility of the researcher is to obtaininformed consent from each participantThree important questions to ask about harm in a study:• Could people be harmed?• If so, could the study be changed to determine the sameresults?• Are the results important enough to cause possible harm?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ensuring ConfidentialityOnce data are collected, researchers should make surethat no one else has access to the data.Names of subjects should be removed from all dataforms, if possible.The researcher should have no knowledge of particulardata linked to a subject.All subjects should be assured that any data collectedwill be held in confidence,All participants have the right to withdraw from thestudy, if necessary.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Consent FormCONSENT TO SERVE AS A SUBJECT IN RESEARCHI consent to serve as a subject in the research investigation entitled: ………………………………….The nature and general purpose p of the research procedure and the knownrisks involved have been explained to me by ____________________________The investigator is authorized to proceed on the understanding that I mayterminate my service as a subject at any time I so desire.I understand the known risks are: __________________________________________________________________________________________________________________________________________________________________I understand also that it is not possible to identify all potential risks in an experimental procedure,and I believe that reasonable safeguards have been taken to minimize both the known and thepotentially unknown risks.Witness _________________________ Signed _________________________(subject)Date__________________________To be retained by the principal investigatorMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Should Subjects Be Deceived?Many studies cannot be carried out unless some deception takesplace.Deception refers to intentionally misinforming the subjects tosome or all aspects of the research topic.It is often difficult to find situations where observations orobtainment of data can occur at certain specified times.The possibility of harm could be created from these actions, sothe following guidelines have been developed:• Whenever possible, conduct the study using methods that donot require deception• If alternative methods cannot be devised, the researcher mustdetermine if deception is justified by the value obtained• If participants are deceived, the researcher must ensure thatthe participants are provided with sufficient explanation assoon as possibleMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research With ChildrenStudies using children as participants present some special issues forresearchers.The young are more vulnerable, have fewer legal rights, and maynot understand the language of informed consent.The following guidelines below need to be considered:• Informed consent of the parents or guardians is required regarding theuse of minors as subjects• Researchers do not present themselves as diagnosticians or counselorsin reporting results to parents, nor do they report information given bychildren in confidence• Children may never be coerced into participation i in any study• Any form of remuneration for the child’s services does not affect theapplication of these ethical principlesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Another Example of a Consent FormAUTHORIZATION FOR A MINOR TO SERVE AS A SUBJECT IN RESEARCHI authorize the service of ____________________________________ as a subjectin the research investigation entitled:_________________________________________________________________________The nature and general purpose of the research procedure and the knownrisks involved have been explained to me. I understand that __________________(name of minor)will be given a preservice explanation of the research and that he/she may declineto serve. Further, I understand that he/she may terminate his/her service in thisresearch at any time he/she so desires.I understand the known risks are: ____________________________________________________________________________________________________________________________________________________________________I understand also that it is not possible to identify all potential risks in an experimental procedure, and I believe thatreasonable safeguards have beentaken to minimize both the known and the potential but unknown risks.I agree further to indemnify and hold harmless S.F. State University and itsagents and employees from any and all liability, actions, or causes of actions that may accrue to the subject minor as a result ofhis/her activities for which this consent is granted.Witness _________________________ Signed ______________________ (parent or guardian) Date ______To be retained by the principal investigatorMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Regulation of ResearchResearch involving the use of human subjects isdifficult to conduct.Before any research commences, the research must beapproved by the (IRB) Institutional Review Board at theinstitution.The review takes place regardless of the number ofresearchers or who is financing the project.The IRB is in charge of weighing the risks of subjects,checking informed consents, and the treatment ofsubjects.Once the research has been approved, the researchermust obtain a group of subjects who are willing toparticipate in the study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Review of LiteratureChapter FiveMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Literature Review is:A systematic study, survey andinterpretation of the material publishedon a particular topic ( Babbie, 2002).This involves the review prior work donein the particular area of interest. Thisbody of previous work is referred to asthe literature.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Literature Review involves asummary of:Conceptual - Theoretical grounding (basictheoretical knowledge) for the study - howothers have defined the problem and relatedconcepts.Research - what relationships have beenidentified between variables in previousstudies – findings and conclusions of priorstudies.Only relevant, reputable sources should becited - no opinionated, unsubstantiatedsourcesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Value of a Literature t ReviewThe literature review is helpful in twoways:• Helps researchers present the ideas of others interestedin a particular research question• Allows them to see the results of similar studiesperformed or questions answeredThis is usually required for theses and dissertations.Researchers need to know what has taken place in otherwork related to their study, and Literature Review helps toprovide this knowledge.Researchers also need to evaluate this work to determine itsrelevance to their research question.Literature Review helps to enhance and add to what othershave already done.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of SourcesResearchers need to be familiar with three basictypes of sources:• General references: most are either indexes or abstractslisted by the author, title, or place of publication• Primary sources: publications found in journals publishedmonthly or quarterly• Secondary sources: textbooks where authors describe theworks of othersThe two methods used to perform a literature revieware:• Manually: traditional paper approach• Electronically: use of a computerMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps Involved in a Literature SearchDefine the research problem as precisely as possibleLook at relevant secondary sourcesSelect one or two appropriate general reference worksFormulate search terms pertinent to the problemSearch the general references for relevant primarysourcesObtain and read relevant primary sources; note andsummarize key points in the sourcesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Doing a Computer Search• Many online computer terminals are linked to one ormore information retrieval systems (i.e., ERIC).• Steps involved in a computer search are as follows:• Define the problem as precisely as possible• Decide on the extent of the search (number ofreferences)• Decide on the database (i.e., ERIC)• Select descriptors (words that tell the computer whatto search for), a.k.a Boolean operators• Conduct the search by entering them into the computer• Obtain a printout of desired referencesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Researching the World Wide WebA vast reservoir of information on all topicsUsing the WWW is an essential element whenperforming a literature review, despite the use ofERIC and other databases of choiceUse of the following is essential when searchingon the WWW:• Use of a Web Browser:• Use of search engines (programs that allow extensivesearches throughout the WWW by indexing them viathe description typed)• Use of directories (websites linked by similarcategorical terms)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Advantages of Searching theWorld Wide WebCurrency – Information is current and up-to-dateAccess to a wide variety of materialsVaried formats – To meet various needsImmediacy- Information is immediately availableMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Disadvantages of Searching theWorld Wide WebDisorganizationValidity of information sometimes not knownOutdated web sitesLack of empirical or peer-reviewed studiesPossible hidden costsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

USING YOUR LIBRARYTIME EFFICIENTLYIdentify the key search concept and termsbefore going to the libraryCheck catalogue to determine what is availableCompile a tentative bibliographyCommence search systematically – Journalsthen books then official reports and so on.Make use of Indexes, Abstracts, and OtherGeneral References.Use electronic search toolsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

USING YOUR LIBRARY TIMEEFFICIENTLY cont…Select citations that are relevant to yourtopicRecord all basic information as you readeach sourceOrganize and edit materialsIdentify strategies for obtaining additionalsources not available in libraryMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Literature Review ReportA literature review report consists of:• Introduction• Body of the review• Summary• Researcher’s conclusion• BibliographyA meta-analysis is a statistical procedure when theresearcher averages the results of a group of selectedstudies to get an overall index of outcome or relationship.A literature review should include a search for relevantmeta-analysis reports, as well as individual studiesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

SamplingChapter SixMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Who Will You Study?Identifying the Unit of AnalysisUnit of analysis is the level (e.g. individual,family, school, school district) at which thedata will be gathered.There may be different units of analysis• One for the dependent variable• One for the independent variableMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Population and SampleA population is a group of individuals that comprisethe same characteristicsA sample is a sub-group of the target populationthat the researcher plans to study for the purpose ofmaking generalizations about the targetpopulation.• Samples are only estimates• The difference between the sample estimate andthe true population is the “sampling error.”McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is a Sample?Sampling is the process of selecting a number ofindividuals from a population, preferably in a waythat the individuals are representative of the largergroup from which they were selected.A sample is any group on which information isobtained.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Populations and SamplesTargetPopulationSamplePopulation-All teachers in high schools in one city-College students in all communitycolleges-Adult educators in all schoolsof educationSample-All high school biology teachers-Students in one communitycollege-Adult educators in 5 schools ofeducation in the MidwestMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Defining the PopulationA population refers to all the members of a particular group.The first task in selecting a sample is to define the populationof interest.In Educational Research, the population of interest is a groupof persons who possess certain characteristics.A target population is the actual population that theresearcher would like to generalize.• Considered d rarely available• The accessible population would be the group that isavailable (realistic choice)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Probability and Non-Probability SamplingProbability sampling(or Random Sampling) is theselection of individuals from the population so that eachindividual will have an equal chance of being selected.(Normally will result in a representative sample of the population)Non-probability (or non- Random) sampling is theselection of participants because they are available,convenient, or represent some characteristic theinvestigator wants to study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Quantitative SamplingQuantitative Sampling StrategiesProbability SamplingNon-Probability SamplingSimple Stratified Multi-StageRandom Sampling ClusterSampling SamplingConvenience SnowballSampling SamplingMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Random Sampling MethodsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Probability SamplesSimple Random: selecting a sample from thepopulation so all in the population have an equaland independent chance of being selectedCluster Random Sampling: the selection of groups orclusters of subjects, rather than individuals. Theselection process is similar to simple random, exceptgroups rather than individuals are selected.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Probability SamplesMulti-Stage Cluster Sampling: a sample chosenin one or two stages because the population isnot easily identified d or is largeStratified sampling: stratifying the populationon a characteristic (e.g. gender) then samplingfrom each stratum, in the same proportion asthey exist in the population.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Proportional StratificationSampling ApproachPopulation(N=9000)BoysN=6000GirlsN=3000.66 of pop. 200.33 of pop 100Sample = 300McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Non-Probability SamplesConvenience Sampling: participants areselected because they are willing andavailable to be studiedSnowball Sampling: the researcher asksparticipants to identify other participants tobecome members of the sample.Systematic: choosing every “nth” individualor site in the population until the desiredsample size is achievedMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Two Main Types of SamplingSampling may be either random or non-randomRandom sampling is a method of selecting subjects froma population by chance, so that biases do not alter thesample.The 4 most common ways of obtaining this type ofsample are:• Simple Random Sampling• Stratified Random Sampling• Cluster Sampling• Two-Stage Random SamplingMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Simple Random SamplingA Simple Random Sampling is a sample selected from a populationin such a manner that all members have an equal chance of beingselectedIf the sample is large, it is the best method to obtain a samplerepresentative of the population from which it has been selectedThe larger the sample, the more likely to be representativeAny differences that occur are the result of chance rather than biason the part of the researcherDisadvantages of this method are: 1) the difficulty of performingthe sampling and, 2) this method does not ensure that subgroupsare present in the sampling in the same proportion as they are in apopulationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Stratified Random SamplingA Stratified Random Sampling is a sample selected so thatcertain characteristics are represented in the sample in thesame proportion as they occur in the populationThe term strata refers to sub-groupsThe advantage of stratified random sampling is that itincreases the likelihood of representation, especially if thesample size is smallIt virtually ensures that any key characteristics of individualsin the population are included in the same proportions in thesample sizeThe disadvantage is that it requires still more effort on thepart of the researcherMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Selecting a Stratified SampleMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Cluster Random SamplingA Cluster Random Sampling is a sample obtained by using groups asthe sampling unit (cluster), rather than individualsThere are instances where it is not possible to select a sample ofindividuals from a populationThis is considered more effective with large numbers of clustersAdvantages include more efficient and easier to implement in schoolsIts disadvantage is that there is a great chance of selecting a samplethat is not representative of the populationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Two-Stage Random SamplingThis method selects groups randomly and thenchooses individuals randomly from these groups.This becomes a combination of a cluster randomsampling with individual random sampling.Considered d less time consuming but allows for agood representation of the groups at random.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Nonrandom Sampling MethodsThere are 3 main types of nonrandom sampling methodsused in Educational ResearchA Systematic Sample is a sample obtained by selecting everynth name in a populationA Convenience Sample is any group of individuals that isconveniently available to be studied• Are not considered d representative ti of the population and should be avoided, d ifpossibleA Purposive Sample is a sample selected because theindividuals have special qualifications of some sort, orbecause of prior evidence of representation• Personal judgment is used for selection purposes• A major disadvantage is that the researcher’s judgment could be in errorMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Nonrandom Sampling MethodMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Sample SizeThe question remains as to what constitutes t an adequatesample size.Samples should be as large as a researcher can obtainwith a reasonable expenditure of time, money andenergy.The recommended minimum number of subjects are asfollows for the following types of studies:• 100 for a Descriptive Study• 50 for a Correlational Study• 30 in each group for Experimental and Causal-Comparative StudyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

External Validity, a.k.a. GeneralizabilityThe whole notion of science is built on generalizing.External Validity refers to the extent that the results of astudy can be generalized from a sample to a population.Population generalizability is the degree to which asample represents the population of interest.• Obtaining a representative sample becomes veryimportantEcological generalizability refers to the extent to whichthe results of a study can be generalized to conditions orsettings other than those that prevailed in the study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Population as Opposed to EcologicalGeneralizingi McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

InstrumentationChapter SevenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What are Data?Data refers to the information researchers obtain onthe subjects of their research.Demographic information or scores from a test areexamples of data collected.The researcher has to determine what kind of datathey need to collect.The device the researcher uses to collect data is calledan instrument.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Key QuestionsThe instruments and procedures used in collecting data iscalled instrumentation.Questions arise regarding the procedures and conditionsunder which the instruments will be administered:• Where will the data be collected?• When will the data be collected?• How often are the data to be collected?• Who is to collect the data?The most highly regarded types of instruments can provideuseless data if administered incorrectly,for example: if theinstrument is administered by someone disliked byrespondents, under noisy, inhospitable conditions, or whensubjects are exhausted.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

HOW TO DESIGN GOODINSTRUMENTSThe focus is on the researcher to ensure that the researchquestions are very precise and clear.The researcher must also ensure that the questions on theinstrument are:- in simple language- clear- test or field test (give instrument(s) to resource persons ora group of people who has similar characteristics to thoseof intended target population, then compareanswers/feedbacks to your original intention, and makeamendment to the instrument accordingly).McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Validity, Reliability, and ObjectivityValidity is an important consideration in the choice of aninstrument to be used in a research investigation• It should measure what it is supposed to measure• Researchers want instruments that will allow them to makewarranted conclusions about the characteristics of thesubjects they studyReliability is another important consideration, sinceresearchers want consistent results from instrumentation• Consistency gives researchers confidence that the resultsactually represent the achievement of the individualsinvolvedObjectivity refers to the absence of subjective judgments• This should be eliminated i with regards to achievement ofthe subjects, even if its difficult to do soMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

UsabilityAn important consideration for any researcher inchoosing or designing an instrument is how easy theinstrument will actually be to use.Some of the questions asked which assess usabilityare:• How long will it take to administer?• Are the directions clear?• How easy is it to score?• Do equivalent forms exist?• Have any problems been reported by others who used it?Getting satisfactory answers can save a researcher alot of time and energy.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ways to Classify InstrumentstWho Provides the Information?• Themselves: Self-report data• Directly or indirectly: from the subjects of the study• From informants (people who are knowledgeableabout the subjects and provide this information)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of fResearcher-completed ltdInstrumentstRating scalesInterview schedulesTally sheetsFlowchartsLikert ScalePerformance checklistsAnecdotal recordsTime-and-motion logsObservation formsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Subject-completed completed InstrumentsQuestionnairesSelf-checklistsAttitude scalesPersonality inventoriesAchievement testsPerformance testsProjective devicesPsychometric devicesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Item FormatsQuestions used in a subject-completed instrument cantake many forms but are classified as either selectionor supply items.Examples of selection items are:• True-false items• Matching items• Multiple choice items• Interpretive exercisesExamples of supply items are:• Short answer items• Essay questionsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Unobtrusive MeasuresMany instruments require the cooperation of the respondentin one way or another.An intrusion into an ongoing activity could be involvedwhich causes a form of negativity within the respondent.To eliminate this, researchers use unobtrusive measures,data collection procedure that involve no intrusion into thenaturally occurring course of events.In most cases, no instrument is used, however, good recordkeeping is necessary.They are valuable as supplements to the use of interviewsand questionnaires, often providing a useful way tocorroborate what more traditional data sources reveal.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of ScoresQuantitative data is reported in the form of scoresScores are reported as either raw or derived scores• Raw score is the initial score obtained• Taken by itself, a raw score is difficult to interpret, since it has little meaning• Derived score are scores that have been taken from raw scores and standardized• They enable researchers to say how well the individual performed compared to otherstaking the same test• Examples include:• Age and Grade-level Equivalents• Percentile Ranks (refers to the percentage of individuals scoring at or below a given rawscore)• Standard scores are mathematically derived scores having comparable meaningon different instrumentsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Examples of Raw Scores and Percentile RanksRaw Cumulative PercentileScore Frequency Frequency Rank95 1 25 10093 1 24 9688 2 23 9285 3 21 8479 1 18 7275 4 17 6870 6 13 5265 2 7 2862 1 5 2058 1 4 1654 2 3 1250 1 1 4N = 25McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Norm-Referenced vs. Criterion-ReferencedInstr InstrumentsAll derived scores give meaning to individual scores bycomparing them to the scores of a group.The group used to determine derived scores is called thenorm group and the instruments that provide such scoresare referred to as norm-referenced instruments.An alternative to the use of achievement or performanceinstruments is to use a criterion-referenced i test.This is based on a specific goal or target (criterion) foreach learner to achieve.The difference between the two tests is that the criterionreferenced tests focus more directly on instruction.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Measurement ScalesThere are four types of measurement scales• Nominal Scales• Ordinal Scales• Interval Scales• Rti Ratio ScalesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Four Types of Measurement ScalesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Four Types of fMeasurement tScalesMeasurementScaleNominalOrdinalIntervalRatioCharacteristicsGroups and labels data only;reports frequencies or percentages.Ranks data; uses numbers only toindicate ranking.Assumes that equal differences betweenscores really mean equal differences inthe variable used.All of the above, plus true zero point.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Nominal ScalesConsidered the simplest form of measurement.Researchers assign numbers to differentcategories.An example would be to assign a “1” to men, a“2” to women.The advantage to assigning numbers to thecategories is to facilitate computer analysis.Involves groups and labels data only.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Odi Ordinal lScalesInvolves the use of numbers to rank /order scores.An example would be the ranking of high to low scoreson an examination.Differences in rankings would not necessarily be thesame with certain scores.Ordinal scales indicate relative standing amongindividuals.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Interval lScalesInvolves the use of numbers to represent equalintervals in different segments on a continuum.Very similar to an ordinal scale with theexception of the equal intervals of points.Assumes that equal differences between scoresreally mean equal differences in the variablemeasured.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Rti Ratio ScalesA ratio scale involves the use of numbers torepresent equal distances from a known “zeropoint”.A scale designed to measure height would be aratio scale since the zero point represents theabsence of height.Ratio scales are almost never encountered inEducational Research.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Validity and ReliabilityChapter EightMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Validity and ReliabilityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

ValidityValidity has been defined as referring to theappropriateness, correctness, meaningfulness, andusefulness of the specific inferences researchers makebased on the data they collect.It is the most important idea to consider whenpreparing or selecting an instrument.Validation is the process of collecting and analyzingevidence to support such inferences.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Evidence of ValidityThere are 3 types of evidence a researcher mightcollect• Content-related evidence of validity• Content and format of the instrument• Criterion-related evidence of validity• Relationship between scores obtained using the instrument andscores obtained• Construct-related evidence of validity• Psychological construct being measured by the instrumentMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Content-related t ltdEid EvidenceA key element is the adequacy of the samplingof the domain it is supposed to represent.The other aspect of content validation is theformat of the instrument.Attempts to obtain evidence that the itemsmeasure what they are supposed to measuretypify the process of content-related evidence.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Cit Criterion-related i ltdEid EvidenceA criterion is a second test presumed to measure the samevariable.There are two forms of criterion-relatedrelated validity:1) Predictive validity: time interval elapses between administeringthe instrument and obtaining criterion scores2) Concurrent validity: instrument data and criterion data aregathered and compared at the same timeA Correlation Coefficient (r) indicates the degree ofrelationship lti that thtexists it between bt the scores of fidiid individualsobtained by two instruments.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Construct-related t ltdEid EvidenceConsidered the broadest of the three categories.There is no single piece of evidence that satisfiesconstruct-related validity.Researchers attempt to collect a variety of typesof evidence, including both content-related andcriterion-related i evidence.The more evidence researchers have fromdifferent sources, the more confident theybecome about the interpretation of theinstrument.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

ReliabilityRf Refers to the consistency it of scores or answersprovided by an instrument.Scores obtained can be considered reliable but notvalid.An instrument should be reliable and valid ,depending on the context in which an instrument isused.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Reliability and ValidityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Errors of fMeasurementBecause errors of measurement are always presentto some degree, variation in test scores arecommon.This is due to:• Differences in motivationi• Energy• Anxiety• Different testing situationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Reliability Coefficienti Expresses a relationship between scores of thesame instrument at two different times or parts ofthe instrument.tThe 3 best known methods are:• Test-retestt t• Equivalent forms method• Internal consistency methodMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Test-Retest Rt tMthd MethodInvolves administering the same test twice to the same group after acertain time interval has elapsed.A reliability coefficient is calculated to indicate the relationship betweenthe two sets of scores.Reliability coefficients are affected by the lapse of time between theadministrations of the test.An appropriate time interval should be selected.In Educational Research, scores collected over a two-month monthperiodisisconsidered sufficient evidence of test-retest reliability.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Equivalent-Forms MethodTwo different but equivalent (alternate or parallel) formsof an instrument are administered to the same groupduring the same time period.A reliability coefficient is then calculated between thetwo sets of scores.It is possible to combine the test-retest and equivalent-forms methods by giving two different forms of testingwith a time interval between the two administrations.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Internal-Consistency MethodsThere are several internal-consistency methods that requireonly one administration of an instrument.Split-half Procedure: involves scoring two halves of a testseparately for each subject and calculating the correlationcoefficient between the two scores.Kuder-Richardson Approaches: (KR20 and KR21) requires 3pieces of information:• Number of items on the test• The mean• The standard deviation• Considered dthe most tfrequent method dfor determining i internalconsistencyAlpha Coefficient: a general form of the KR20 used tocalculate the reliability of items that are not scored right vs.wrong.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Standard d Error of MeasurementAn index that shows the extent to which ameasurement would vary under changedcircumstances.There are many possible standard errors forscores given.Also known as measurement error, a range ofscores that show the amount of error which canbe expected. (Appendix D)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Scoring AgreementScoring agreement requires a demonstration thatindependent scorers can achieve satisfactory agreement intheir scoring.Instruments that use direct observations are highlyvulnerable to observer differences.What is desired is a correlation of at least .90 amongscorers as an acceptable level of agreement.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Internal ValidityChapter NineMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Internal Validity?Internal Validity is when observed differences on the dependentvariable are directly related to the independent variable, and notdue to some other unintended variable.In other words, any relationship observed between two or morevariables should be unambiguous as to what it means rather thanbeing due to “something else” .The “something else” could be:• Age• Ability• Types of materials usedMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Threats to Internal ValiditySubject CharacteristicsMortalityLocationInstrumentationTestingHistoryMaturationAttitude of subjectsRegressionImplementationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Subject tCharacteristicstiThe selection of people may result in differences, eitherbetween individuals or groups, that are related to thevariables being studied.This refers to a ‘selection bias’ or subject characteristicsithreat.If not controlled, these variables may “explain away”whatever differences are found in the study.There are techniques used to either equalize the differencesor control these variables.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

MortalityIt is common to lose subjects as a studyprogressesThis is known as “mortality threat” .Loss of subjects limits generalizability and canintroduce bias.Mortality is the most difficult threat to control forinternal validity.An attempt to eliminate the problem would be toprovide evidence that the subjects lost weresimilar to those who remained in the study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

LocationThe particular locations where data is collectedmay create different results or explanationsknown as ‘location threat’ .The best way to control for this is to keep thelocation o consistent s for all subjects.s.If this is not possible, the researcher shouldensure that different locations do not favor orjeopardize the hypothesis.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Instrumentationt tiThe way instruments are used may constitute athreat to the internal validity of a study.Some examples are as follows:• Instrument decay• Data Collector Characteristics• Data Collector BiasMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

TestingA“ “testing threat” ”is when efforts are achieved dby thesubjects due to “practice” (i.e., pretest, post-testdesigned study)An interaction also could cause this by taking the testand being more aware of a possible interaction,allowing the subjects to be more responsive towardsthe treatment.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

HistoryA history threat is when an unforeseen eventoccurs during the course of the study.(for eg.Hurricane KATRINA could ldhave affected thephysical and psychological locations of personsinvolved in a research in the USA in 2005).Researchers need to be alert to any possibilities ofinfluences that may occur during the course of thestudy.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Mt MaturationtiChange during an intervention may be due tofactors associated with the passing of time ratherthan the intervention.Students could change over the course of a study.This is known as a Maturation Threat.Maturation is only a threat in studies using pre/postdata for the intervention group or in studies thatspan a number of years.The best way to control for this is to include a wellselectedcomparison group in the study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Attitude of SubjectsThe way subjects view a study and their participationcan be considered a threat to internal validity, a.k.a.the ‘Hawthorne effect’.Subjects may perform better based upon a feeling ofreceiving special attention.The opposite may occur, with subjects receiving notreatment at all, resulting in poor performances.A remedy to this would be to provide both groupswith comparable treatments or to make the treatmenta regular Chapter of the study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

RegressionA regression threat is possible when change isstudied in a group that has extreme low or highperformance in the pre-intervention ti stage.As with the maturation threat, this can becontrolled dby the use of an equivalent control orcomparison group.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

ImplementationtiThe experimental group may be treated in waysthat are unintended, giving them an advantage.This is known as an implementation ti threat. tThis can occur in two ways:1) When different individuals id are assigned toimplement different methods, and theseindividuals differ in ways related to the outcome2) When some individuals have a personal bias infavor of one method over the otherMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

How to Minimize Threats toInternal ValidityThere are four alternatives a researcher can use toreduce threats to internal validity. These are:1) Standardize the conditions under which the study occurs2) Obtain more information on the subjects of the study3) Obtain more information on the details of the study4) Choose an appropriate designMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Descriptive StatisticsChapter TenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Descriptive StatisticsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Statistics ti ti vs. ParametersA parameter is a characteristic of a population.• It is a numerical or graphic way to summarize dataobtained from the populationA statistic is a characteristic of a sample.• It is a numerical or graphic way to summarize dataobtained from a sampleMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of fNumerical DataThere are two fundamental types of numericaldata:1) Categorical data: obtained by determining thefrequency of occurrences in each of severalcategories2) Quantitative data: obtained by determiningplacement on a scale that indicates amount ordegreeMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Techniques for Summarizing Quantitative DataFrequency DistributionsHistograms/Stem and Leaf PlotsDistribution curvesAverages/SpreadVariability/CorrelationsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Frequency PolygonsPlaces data in some sort of order (ascending ordesending).A frequency distribution lists scores from high to low(Table 10.1)This results in a grouped frequency distribution (Table 10.2)Since the information is not very visual, a graphicaldisplay called a frequency polygon can help with this(Figure 10.1)• Frequency polygons can be negatively or positively skewed(Figure 10.2)• They can be useful in comparing two or more groupsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Frequency DistributionMcGraw-HillRaw ScoreFrequency64 263 161 259 256 252 151 238 436 334 531 529 527 525 124 221 217 215 16 23 1n = 50Technically, the table should include all scores, includingthose for which there are zero frequencies. We haveeliminated those to simplify the presentation.© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Grouped Frequency DistributionRaw Score(Intervals of Five)64 263 161 259 256 252 151 238 436 334 531 529 527 525 124 221 217 215 16 23 1Frequencyn = 50McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Frequency PolygonMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Positively Skewed PolygonMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Negatively Skewed PolygonMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Two Frequency Polygons ComparedMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Histograms and Stem-and-Leaf PlotsA histogram is a bar graph used to display yquantitativedata at the interval or ratio level of measurement(Table 10.2)A Stem-leaf Plot (stem plot) looks like a histogram,except instead of bars, it shows values for eachcategory• They are helpful for comparing and contrasting twodistributions (Table 10.1)1)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Histogram of Data in Table 10.2McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Normal CurveThis distribution curve shows a generalized distribution ofscores vs. straight lines (frequency polygon)Distribution of data tends to follow a specific shape calleda normal distribution (see Figure 10.6)This distribution is considered ‘bell shaped’ and allowsthe plotting of the following averages:• Mean• Medium• Mode*These measures of central tendencies enable one to summarize the data in a frequencydistribution with a single numberMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Normal CurveMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of the Mode, Median and Mean in aDistributionRaw ScoreFrequency98 197 191 285 180 577 772 565 364 762 1058 345 233 111 15 1n = 50Mode = 62; median = 64.5; mean = 66.7McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Different Distributions ComparedMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

VariabilityRefers to the extent to which the scores on a quantitativevariable in a distribution are spread out.The range represents the difference between the highestand lowest scores in a distribution.A five number summary reports the lowest, the firstquartile, the median, the third quartile, and highest score.Five number summaries are often portrayed graphically bythe use of box plots.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Box plotsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Standard d DeviationConsidered the most useful index of variability.It is a single number that represents the spread of adistribution.See p. 348 to calculate the mean of the distribution.Table 10.5 will illustrate the calculation of the SD of adistribution.ib tiIf a distribution is normal, then the mean plus or minus 3SD will encompass about 99% of all scores in thedistribution.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Calculation of the Standard Deviation of a DistributionRaw MeanScoreVariance (SD 2 ) =_ _ _(X) (X) X –X (X–X) 285 54 31 96180 54 26 67670 54 16 25660 54 6 3655 54 1 150 54 -4 1645 54 -9 8140 54 -14 19630 54 -24 57625 54 -29 841Standard deviation (SD) ==Σ(X – X)2n364010√= 364Σ(X – X) 2nMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Facts about the Normal Distribution50% of all the observations fall on each side of themean. (Figure 10.11)68% of scores fall within 1 SD of the mean in a normaldistribution.27% of the observations fall between 1 and 2 SD fromthe mean.99.7% of all scores fall within 3 SD of the mean. (Figure10.12)This is often referred to as the 68-95-99.7 ruleMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Fifty Percent of All Scores in a Normal CurveFall on Each Side of the MeanMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Probabilities Under the Normal CurveMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Standard d ScoresStandard scores use a common scale to indicate how anindividual compares to other individuals in a group.The simplest form of a standard score is a Z score.A Zscoreexpresses expresses how far a raw score is from the meanin standard deviation units. (see Figure 10.13)Standard scores provide a better basis for comparingperformance on different measures than do raw scores.A Probability is a percent stated in decimal form and refersto the likelihood of an event occurring.T scores are z scores expressed in a different form(z score x 10 + 50).McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Probability Areas Between the Mean andDifferent Z ScoresMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Examples of Standard ScoresMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

CorrelationResearchers e s seek to determine e e whether e a relationship exists s betweentwo or more quantitative variables.A Scatterplot is a pictorial representation of the relationship betweentwo quantitative variables. (see Figure 10.15)Outliers are scores that deviate or fall considerably outside most ofthe other scores in a distribution or pattern.• They indicate an unusual exception to a general pattern (SeeFigure 10.16)16)Correlation coefficients express the degree of relationship betweentwo sets of scores.• Pearson Product-Moment Correlation Coefficient• EtaMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Relationship Between Family Cohesiveness andSchool Achievement in a Hypothetical Group ofStudentsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Examples of Nonlinear (Curvilinear)RelationshipsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Techniques for SummarizingCt Categorical lDt DataThe Frequency TableBar Graphs and Pie ChartsThe Cross break TableMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Frequency and Percentage of Responses toQuestionnairePercentageResponse Frequency of Total (%)Lecture 15 30Class discussions 10 20Demonstrations 8 16Audiovisualpresentations 6 12Seatwork 5 10Oral reports 4 8Library research 2 4Total 50 100McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Class Assignment10 30 5580 10 3565 80 1540 65 8020 40 7090 2145 7595Find for the set ofNumbers the following:• Mean• Mode• Median• Range• Min.• Max.• SD• PercentileMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Inferential StatisticsChapter ElevenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What are Inferential Statistics?iInferential Statistics refer to certain procedures that allowresearchers to make inferences about a population based ondata obtained from a sample.Obtaining a random sample is desirable since it ensures thatthis sample is representative of a larger population.The better a sample represents a population, the moreresearchers will be able to make inferences.Making inferences about populations p is what InferentialStatistics are all about.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Two Samples from Two Distinct i PopulationsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Sampling ErrorIt is reasonable to assume that each sample willgive you a fairly accurate picture of itspopulation.However, samples are not likely to be identical totheir parent populations.This difference between a sample and itspopulation is known as Sampling Error. (seeFigure 11.2)Furthermore, no two samples will be identical inall their characteristics.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Sampling ErrorMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Distribution ib i of Sample MeansThere are times where large collections of randomsamples do pattern themselves in ways that will allowresearchers to predict accurately some characteristics ofthe population from which the sample was taken.A sampling distribution of means is a frequencydistribution resulting from plotting the means of a verylarge number of samples from the same populationRefer to Figure 11.3McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

A Sampling Distribution of MeansMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Distribution of Sample MeansMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Standard d Error of the MeanThe standard deviation of a sampling distribution ofmeans is called the Standard Error of the Mean (SEM).If you can accurately estimate t the mean and thestandard deviation of the sampling distribution, you candetermine whether it is likely or not that a particularsample mean could be obtained from the population.To estimate the SEM, divide the SD of the sample bythe square root of the sample size minus one.Refer to Figure 11.4McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Confidence IntervalsA Confidence Interval is a region extending bothabove and below a sample statistic within which apopulation p parameter may be said to fall with aspecified probability of being wrong.SEM’s can be used to determine boundaries orlimits, within which the population mean lies.If a confidence interval is 95%, there would be a‘probability’ that 5 out of 100 (population mean)would fall outside the boundaries or limits.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The 95 percent Confidence IntervalMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The 99 percent Confidence IntervalMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Does a Sample Difference Reflect a PopulationDifference?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Distribution of the Difference Between SampleMeansMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Confidence IntervalsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Hypothesis TestingHypothesis testing is a way of determining the probabilitythat an obtained sample statistic will occur, given ahypothetical population parameter.The Research Hypothesis specifies the predicted outcomeof a study.The Null Hypothesis typically specifies that there is norelationship in the population.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Illustration of When a Researcher WouldReject the Null HypothesisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Hypothesis Testing: A ReviewState the research hypothesisState the null hypothesisDetermine ee ethe esample pesa statistics s cspertinent e to the hypothesisessDetermine the probability of obtaining the sample resultsIf the probability is small, reject the null hypothesis andaffirm the research hypothesisIf the probability is large, do not reject the null hypothesisand do not affirm the research hypothesisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Practical vs. Statistical ti ti SignificanceifiThe terms “significance level” or “level ofsignificance” refers to the probability of a samplestatistic occurring as a result of sampling error.Significance levels most commonly used ineducational research are the .05 and .01 levels.Statistical significance and practical significance arenot necessarily the same since a result of statisticalsignificance does not mean that it is practicallysignificant in an educational sense.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

One and Two-tailed tildTestsA one-tailed test is when the researcher obtains apositive difference between the sample mean whichwill support the hypothesis, when using only thepositive tail of the sampling distribution. (Figure11.13)A two-tailed test involves the use of probabilitiesbased on both sides of a sampling distributionbecause the research hhypothesis h is a non-directionali hypothesis.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Significance ifi Area for a One-tailed tildTestMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

One-tailed Test Using a Distribution ofDifferences Between Sample MeansMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Two-tailed Test Using a Distribution ofDifferences Between Sample MeansMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Contingency Coefficient Values for Different-Sized Cross break TablesSize of Table(No. of Cells)Upper limit afor C Calculated2by2 2 .713 by 3 .824 by 4 .875 by 5 .896by6 6 .91aThe upper limit for unequal-sized tables (such as 2 by 3 or 3 by 4) are unknown but can be estimatedfrom the values given. Thus, the upper imit for a 3 by 4 table would approximate .85McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Commonly Used Inferential TechniquesParametricNonparametricQuantitative t-test for independent means Mann-Whitney U testt-test for correlated meansKruskal-Wallis one-wayanalysisofanalysis ofvarianceAnalysis of variance (ANOVA)Sign testAnalysis of covariance (ANCOVA)Friedman two-way analysis of varianceMultivariate analysis of variance (MANOVA)t-test for rCategorical t-test for difference in proportions Chi squareMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Type I vs. Type II ErrorA null hypothesis predicts no relationship.A Type II error results when the researcher failsto reject the null hypothesis that is false.A Type I error results when the researcher rejectsthe null when it is true.Figure 11.16 16 provides an example of Type 1 andType II errors.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Hypothetical Example of Type I and Type II ErrorsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Rejecting the Null HypothesisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Inference TechniquesThere are two basic types of inferencetechniques:1) Parametric: makes assumptions about the nature of thepopulation from which the samples involved in theresearch study were taken2) Non-parametric: makes few assumptions about thenature of the population from which the samples aretk takenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

An Illustration of Power Under an Assumed Population ValueMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Parametric Techniques for Analyzing QuantitativeDataThe t-test is a parametric statistical test used to seewhether a difference between the means of twosamples is significant.There are two forms of t-tests:1) T-test for correlated means2) T-test for independent meansAnalysis of Variance (ANOVA) is used to determineif significant differences exist between two or moregroups.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Parametric Techniques forAnalyzing Quantitative i Data (cont.)Analysis of Covariance (ANCOVA) is a variation of anANOVA used when groups are given a pretest related insome way to the dependent variable and their mean scoreson this pretest are found to differ.Multivariate Analysis of Variance (MANOVA)incorporates two or more dependent variables in the sameanalysis, thus permitting a more powerful test of differencesamong means.T-test for r is used to see whether a correlation coefficientcalculated on sample data is significant.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Non-Parametric Techniques forAnalyzing Quantitative DataThe Mann-Whitney U test is a nonparametric alternative tothe t test used when a researcher wishes to analyze rankeddataThe Kruskal-Wallis one-way analysis of variance is usedwhen you have two or more independent variables tocompareThe Sign test is used when you want to analyze two relatedsamples. Related samples are connected in some wayThe Friedman two-way analysis of variance is used whentwo or more related groups are involvedMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Techniques for MeasuringCt Categorical lDt DataParametric Technique• T-test for Proportions (finding differences inproportions within categories)Non-Parametric Technique• Chi-square test tis used dto analyze data that tarereported in categories• The Contingency Coefficient is a descriptive statisticindicating the degree of relationship that existsbetween two categorical variablesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Power of a Statistical TestPower is the probability that the test will correctlylead to the conclusion that there is a differencewhen, itf fact, a difference exists.Parametric tests are generally, but not always,more powerful lthan non-parametric ti tests.tMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

A Power CurveMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Statistics in PerspectiveChapter TwelveMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Approaches to ResearchEducational Research is performed in one oftwo ways:1) Two or more groups are compared2) Variables within one group are relatedThe data in a study may be eitherquantitative or categorical.Four different combinations of research arepossible. (see Figure 12.1)All groups are made up of individual unitsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Combinations of Data and ApproachesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Comparing Groups: Quantitative DataWhen comparing two or more groups using quantitative data,researchers can compare them through the following:• Frequency polygons• Calculation of averages• Calculations of spreadsRecommendations are made to perform the following:• Prepare a frequency polygon of each group’s score• Use these polygons to decide which measure of central tendency is appropriateto calculateMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Frequency PolygonsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

90% Confidence IntervalMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Comparing Groups:Quantitative Data (cont.)Once the descriptive statistics have been calculated,interpretations must follow.Considerations of the following can assist withinterpreting data more clearly:• Information based on known groups• Calculate the effect size of the groups• Use inferential statistics e.g. t-tests, ANOVA, etc.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Interpretation of Correlation when Testing ResearchHypothesesMagnitude of rInterpretation.00 to .40 Of little practical importance except inunusual circumstances; perhapsof theoretical value. a.41 to .60 Large enough to be of practical as well astheoretical etca use..61 to .80 Very important, but rarely obtained ineducational research..81 to above Possibly an error in calculation; if not, avery sizable relationship.aWhen selecting a very few people from a large group, even correlationsthis small may have a predictive value.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Scatterplots with a Pearson r of .5McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Comparing Groups: Categorical DataWhen using categorical data, groups may be comparedthrough the following methods:• Percentages/Proportions• Frequencies• Crossbreak TablesA limitation of categorical data is that such evaluationsare even harder than with quantitative data.The use of the summary statistic known as contingencycoefficient is recommended but data must be presentedin crossbreak tables.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Recap of RecommendationsUse graphic techniques beforecalculating numerical summaryindicesUse both graphs and summaryindices to interpret results of astudyMake use of external criteria toassess the magnitude of arelationshipUse professional consensus whenevaluating magnitude of effectsizeConsider using inferential statisticsUse tests of statistical significanceonly to evaluate generalizabilityWhen random sampling has notoccurred, treat probabilities asapproximations or crude indicesvs. precise valuesReport confidence intervals ratherthan significance levels if possibleUse both parametric and nonparametrictechniques to analyzedata rather than either one aloneMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Experimental ResearchChapter ThirteenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Uniqueness of Experimental ResearchExperimental Research is unique in two importantrespects:1) Only type of research that attempts to influence a particularvariable2) Best type of research for testing hypotheses about cause-andeffectrelationshipsExperimental Research looks at the followingvariables:• Independent d variable (treatment)t t)• Dependent variable (outcome)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Characteristics of Experimental ResearchThe researcher manipulates the independent variable.They decide the nature and the extent of thetreatment.After the treatment has been administered,researchers observe or measure the groups receivingthe treatments to see if they differ.Experimental research enables researchers to gobeyond description and prediction, and attempt todetermine what caused effects.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

RandomizationRandom assignment is similar but not identical to randomselection.Random assignment means that every individual who isparticipating in the experiment has an equal chance ofbeing assigned to any of the experimental or controlgroups.Random selection means that every member of apopulation has an equal chance of being selected to be amember of the sample.Three things occur with random assignments of subjects:1) It takes place before the experiment begins2) Process of assigning the groups takes place3) Groups should be equivalentMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Control of Extraneous VariablesThe researcher has the ability to control manyaspects of an experiment.It is the responsibility of the researcher to controlfor possible threats to internal validity.This is done by ensuring that t all subjectcharacteristics that might affect the study arecontrolled.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

How to Eliminate Threats due toSubject Characteristics?RandomizationHold certain variables constantBuild the variable into the designMatchingUse subjects as their own controlAnalysis of Covariance (ANCOVA)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Weak Experimental DesignsThe following designs are considered weaksince they do not have built-in controls forthreats to internal validity• The One-Shot Case Study• A single group is exposed to a treatment and its effectsare assessed• The One-Group-Pretest-Posttest Design• Single group is measured both before and after atreatment exposure• The Static-Group Comparison Design• Two intact groups receive two different treatmentsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a One-Shot Case Study DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a One-Group Pretest-Posttest Posttest DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Static-Group Comparison DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

True Experimental DesignsThe essential ingredient of a true experiment israndom assignment of subjects to treatment groupsRandom assignments is a powerful tool forcontrolling threats to internal validity• The Randomized Posttest-only Control GroupDesign• Both groups receiving different treatments• The Randomized Pretest-Posttest Control GroupDesign• Pretest is included in this design• The Randomized Solomon Four-Group Design• Four groups used, with two pre-tested andtwo not pre-testedMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Randomized Posttest-Only Control GroupDesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Randomized Pretest-Posttest Control GroupDesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a Randomized Solomon Four-Group DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Random Assignment withMatchingTo increase the likelihood that groups ofsubjects will be equivalent, pairs of subjectsmay be matched on certain variables.Members of matched groups are thenassigned to experimental or control groups.Matching can be mechanical or statistical.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Mechanical and Statistical i MatchingMechanical matching is a process of pairingtwo persons whose scores on a particularvariable are similar.Statistical ti ti matching does not necessitate t aloss of subjects, nor does it limit thenumber of matching variables.• Each subject is given a “predicted” score on thedependent variable, based on the correlationbetween the dependent variable and the variableon which h the subjects are being matched.• The difference between the predicted and actualscores for each individual is then used to compareexperimental and control groups.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Quasi-Experimental imental DesignsQuasi-Experimental Designs do not includethe use of random assignments but useother techniques to control for threats tointernal validity: ld• The Matching-Only Design• Similar except that no random assignment occurs• Counterbalanced Design• All groups are exposed to all treatments but in a differentorder ode• Time-Series Design• Involves repeated measures over time, both before andafter treatmentMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Results (Means) from a Study Using a Counterbalanced DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Possible Outcome Patterns in a Time-Series i DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Factorial DesignsFactorial Designs extend the number of relationshipsthat may be examined in an experimental study.They are modifications of either the posttest-onlycontrol group or pretest-posttest t tt t control group designswhich permit the investigation of additionalindependent variables.They also allow a researcher to study the interaction ofan independent variable with one or more othervariables (moderator variable).McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Using a Factorial Design to Study Effects of Method and Class Size onAchievementMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Illustration of Interaction and No Interaction in a2 by 2 Factorial DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of a 4 by 2 Factorial DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Controlling Threats to Internal ValiditySubject CharacteristicsMortalityLocationInstrument decayData CollectorCharacteristicsData Collector biasTestingHistoryMaturationAttitudinalRegressionImplementationThe above must be controlled to reduce threats to internal validityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Single-Subject ResearchChapter FourteenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Single-Subject ResearchMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Essential Characteristics ofSingle-Subject S ResearchThere are reasons why single subject researchis selected instead of the study of groups.Instruments can be inappropriate p at times andintense data collection on a few individualscan make more sense.Single-subject designs are adaptations of thebasic time-series design where data iscollected and analyzed for only one subject ata time.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Single-Subject S DesignsSingle-subject designs use line graphs to presenttheir data and to illustrate the effects of aparticular intervention or treatment on anindividual.The first condition is usually the baseline, followedby the intervention (independent variable).Condition lines show if the condition has changedor separated.Data points represent when the data was collectedduring the study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Single-Subject GraphMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Single-SubjectDesignsThe A-B Design• Exposes the same subject, operating under his or her own control, to twoconditions or phases, after establishing a baselineThe A-B-A Design• Called a reverse design, researchers add another baseline period to the A-Bdesign.The A-B-A-B Design• Two baseline periods are combined with two treatment periodsThe B-A-B Design• Used when an individual’s behavior is so severe that a researcher cannotwait for a baseline to be establishedThe A-B-C-B Design• The “C” condition refers to a variation on the intervention in the “B”condition. The intervention is changed during the “C” phase to control forany extra attention the subject may have received during the “B” phaseMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

An A-B BDesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

An A-B-A A DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Illustrations of the Results of a StudyInvolving an A-B-A-B A BDesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

A B-A-B DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

An A-B-C-B B DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Multiple-Baseline l li DesignsThis is considered an alternative to the A-B-A-B design.Multiple-baseline designs are typically used when it is notpossible or ethical to withdraw a treatment and return to thebaseline condition.Researchers collect data on several behaviors compared tofocusing on just one per subject, obtaining a baseline for eachduring the same period of time.The researcher applies the treatment at different times foreach behavior until all of them are undergoing the treatment.If behavior changes in each case only after the treatment hasbeen applied, the treatment is judged to be the cause of thechange.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Multiple-Baseline DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Illustration of a Multiple-Baseline DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

A Multiple-Baseline Design Applied to DifferentSettingsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Variations in Baseline StabilityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Threats to Internal Validity inSingle-Subject S ResearchThe following threats can affect the Internal Validity in Single-Subject StudiesCondition length (how long Return to baseline levelthe baseline and intervention (level should quickly return ifconditions are in effect)the intervention was the causalNumber of variablesfactor)changed when movingfrom one condition toIndependence of behaviorsanother (it is important that (are behaviors that are beingone variable be changed at a measured dependent upon onetime, when moving from onecondition to another)another, or related?)Degree and speed ofNumber of baselines (did anchange (magnitude withextraneous event cause thewhich the data change at the change during the introductiontime the intervention conditiontimes?)is implemented)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Differences in Degree and Speed of ChangeMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Differences in Return to BaselineConditionsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Controlling Threats in a Single-Subject S StudySingle subject designs are most effective in controlling for thefollowing:• Subject characteristics• Mortality• Testing• HistoryThey are less effective with the following:• Location• Data collector characteristics• Maturation• RegressionThey are even weaker with the following:• Collector bias• Attitude• ImplementationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

External Validity andSingle-Subject Subject ResearchSingle-subject studies are weak when it comesto external validity (i.e., generalizability).Treatment on one subject would not beappropriate.As a result, these studies must rely onreplications, across individuals id rather thangroups, if such results are to be found worthyof generalizability.e ab McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Correlational ResearchChapter FifteenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Nature of CorrelationalResearchCorrelational Research is also known asAssociational Research.Relationships among two or more variablesare studied without any attempt to influencethem.Investigates t the possibility of relationshipsbetween two variables.There is no manipulation of variables inCorrelational Research.Correlational studies describe the variable relationship via a correlation coefficientMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Three Sets of Data Showing Different Directionsand Degrees of Correlation (Table 15.1)(A) (B) (C)r = +1.00 r = -1.00 r = 0X Y X Y X Y5 5 5 1 2 14 4 4 2 5 43 3 3 3 3 32 2 2 4 1 51 1 1 5 4 2McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Purpose of CorrelationalResearchCorrelational studies are carried out to explainimportant human behavior or to predict likelyoutcomes (identify relationships amongvariables).If a relationship of sufficient magnitude existsbetween two variables, it becomes possible topredict a score on either variable if a score onthe other variable is known (Prediction Studies).The variable that is used to make the predictionis called the predictor variable.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Purpose of Correlational Research(cont.)The variable about which the prediction is madeis called the criterion variable.Both scatterplots and regression lines are usedin correlational studies to predict a score on acriterion variableA predicted score is never exact. Through aprediction equation (see p. 585), researchersuse a predicted score and an index of predictionerror (standard error of estimate) to conclude ifthe score is likely to be incorrect.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Scatterplot Illustrating aCorrelation of +1.00McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Prediction Using a ScatterplotMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

More Complex CorrelationalTechniquesesMultiple Regression• Technique that enablesresearchers to determine acorrelation between a criterionvariable and the best combinationof two or more predictor variablesCoefficient i of multiple l correlation (R)• Indicates the strength of thecorrelation between thecombination of the predictorvariables and the criterion variableCoefficient of Determination• Indicates the percentage of thevariability among the criterionscores that can be attributed todifferences in the scores on thepredictor variableDiscriminant Function Analysis• Rather than using multipleregression, this technique is usedwhen the criterion value iscategoricalFactor Analysis• Allows the researcher todetermine whether manyvariables can be describedby a few factorsPath Analysis• Used to test the likelihoodof a causal connectionamong three or morevariablesStructural Modeling• Sophisticated method forexploring and possiblyconfirming causationamong several variablesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Scatterplot Illustrating aCorrelation of +1.00McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Prediction Using a ScatterplotMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Path Analysis DiagramamMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Partial CorrelationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Scatterplots Illustrating How a Factor (C)May Not be a Threat toInternal ValidityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Circle Diagrams IllustratingRelationships Among VariablesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Basic Steps in CorrelationalResearchProblem selectionChoosing a sampleSelecting orchoosing properinstrumentsDetermining designand proceduresCollecting andanalyzing dataInterpreting resultsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What Do CorrelationalCoefficients Tell Us?The meaning of a given correlation coefficientdepends on how it is applied.Correlation coefficients below .35 show only a slightrelationship between variables.Correlations between .40 and .60 may havetheoretical and/or practical value depending on thecontext.Only when a correlation of .65 or higher is obtained,can one reasonably assume an accurate prediction.Correlations over .85 indicate a very strongrelationship between the variables correlated.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Threats to Internal Validityin Correlational ResearchSubject characteristicsMortalityLocationInstrument decayTestingHistoryData collectorcharacteristicsData collector biasThe following must be controlled to reduce threats to internal validityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Causal-Comparative ResearchChapter SixteenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Causal-ComparativeResearch?Investigators attempt to determine the cause ofdifferences that already exist between or among groupsof individuals.This is viewed as a form of Associative Research sinceboth describe conditions that already exist (a.k.a. ex postfacto).The group difference variable is either a variable thatcannot be manipulated or one that might have beenmanipulated but for one reason or another, has not been.McGraw-HillStudies in medicine and sociology are causal-comparativecomparativein nature, as are studies of differences between men andwomen.© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Similarities and Differences BetweenCausal-Comparative Comparative andCorrelational ResearchSimilarities• Associative research• Attempt to explainphenomena of interest• Seek to identify variablesthat are worthy of laterexploration throughexperimental research• Neither permits themanipulation of variables• Attempt to explorecausationDifferences• Causal studies comparetwo or more groups ofsubjects• Causal studies involve atleast one categoricalvariable• Causal studies oftencompare averages or usecrossbreak tables insteadof scatterplots andcorrelations coefficientsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Similarities and Differences BetweenCausal-Comparative C and ExperimentalResearchSimilarities• Require at least onecategorical variable• Both compare groupperformances todeterminerelationships• Both compareseparate groups ofsubjectsDifferences• In experimental research,the independent variable ismanipulated• Causal studies are likely toprovide much weakerevidence for causation• In experimental studies,researchers can assignsubjects to treatmentgroups• The researcher has greaterflexibility in formulating thestructure of the design inexperimental researchMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps Involved in Causal-Comparative ati ResearchProblem Formulation• The first step is to identify and define the particular phenomena ofinterest and consider possible causesSample• Selection of the sample of individuals to be studied by carefullyidentifying the characteristics of select groupsInstrumentation• There are no limits on the types of instruments that are used inCausal-comparative studiesDesign• The basic design involves selecting two or more groups that t differ ona particular variable of interest and comparing them on anothervariable(s) without manipulation (see Figure 16.1)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Basic Causal-Comparative C DesignsIndependentDependentGroup variable variable(a) I C O(Group possesses(Measurement)characteristic)II –C O(Group does(Measurement)not possesscharacteristic)(b) I C 1 O(Group possesses(Measurement)characteristic 1)II C 2 O(Group possesses (Measurement)characteristic ti 2)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Examples of the Basic Causal-Comparative DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Threats to Internal Validity inCausal-Comparative Compa ati ResearchSubject Characteristics• The possibility exists that the groups are not equivalenton one or more important variables• One way to control for an extraneous variable is tomatch subjects from the comparison groups on thatvariable• Creating or finding homogeneous subgroups would beanother way to control for an extraneous variable• The third way to control for an extraneous variable is touse the technique of statistical matchingMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Does a Threat to Internal Validity Exist?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Other ThreatsLoss of subjectsLocationInstrumentationHistoryMaturationData collector biasInstrument decayAttitudeRegressionPre-test/treatmentinteraction effectMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Evaluating Threats to Internal Validity inCausal-Comparative StudiesInvolves three sets of steps as shown below:• Step 1: What specific factors are known toaffect the variable on which h groups are beingcompared or may be logically be expected toaffect this variable?• Step 2: What is the likelihood of the comparisongroups differing on each of these factors?• Step 3: Evaluate the threats on the basis ofhow likely they are to have an effect and planto control for them.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Data AnalysisIn a Causal-Comparative Study, the first step is toconstruct frequency polygons.Means and SD are usually calculated if the variablesinvolved are quantitative.The most commonly used inference test is a t-test fordifferences between means.ANCOVAs are useful for these types of studies.Results should always be interpreted with cautionsince they do not prove cause and effect.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Associations BetweenCategorical VariablesThere are no techniques analogous to partialcorrelation or the other techniques that have evolvedfrom correlational research that can be used withcategorical variables.Prediction from crossbreak tables is much lessprecise than from scatterplots.There e are relatively few questions of interest est ineducation that involve two categorical variables.It is common to find researchers who treatquantitative variables conceptually as if they werecategorical, but nothing is gained by this procedureand it should be avoided.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Survey ResearchChapter SeventeenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is a Survey?Researchers are often interested in theopinions of a large group of people abouta particular topic or issue.There are three major characteristics thatmost surveys possess:1) Information is collected from a group of people inorder to describe some aspect of the population2) Information is collected by asking questions of themembers of the selected group3) Information is collected from a sample rather thanfrom every member of the populationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Why Are Surveys Conducted?The purpose of surveys is to describe thecharacteristics of a population.Researchers find out how the members of apopulation distribute themselves on one or morevariables.A selected sample is surveyed and the descriptionof the population is inferred from what is foundout about the sample.In descriptive surveys, researchers are not asconcerned about why the observed distributionexists as with what the distribution is.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of SurveysThere are two types of surveys:1) Cross-sectional survey• Collects information from a sample that has been drawn from apredetermined population• Information is collected at just one point in time, even though itcould take days to gather all the data• A census is when the entire population is surveyed2) Longitudinal survey• Information is collected at different points in time in order tostudy changes over time• Three types are employed:1) Trend study2) Cohort Study3) Panel StudyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Survey Research andCorrelational ResearchTechniques of Correlational Research can be tied inwith Survey Research.Researchers could look at the relationship ofresponses to one question (survey) to another, or ofa score based on one set of questions to another set.The use of calculating correlation coefficients orcontingency tables could be determined andimplemented.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps in Survey ResearchThe following steps are used inconjunction with performing propersurveys in research:• Defining the problem• Identifying the target population (unit ofanalysis)• Choosing the mode of data collection• Direct administration• Mail surveys• Telephone surveys• Personal interviewsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Advantages and Disadvantages of Survey Data-Collection MethodsDirectAdministration Telephone Mail InterviewComparative cost Lowest Same Same HighFacilities needed? Yes No No YesRequire training of questioner? Yes Yes No YesData collection time Shortest Short Longer LongestResponse rate Very high Good Poorest Very highGroup administration possible? Yes No No YesAllow for random sampling? Possibly Yes Yes YesRequire literate sample? Yes No Yes NoPermit follow-up questions? No Yes No YesEncourage response to sensitive topics? Somewhat Somewhat Best WeakStandardization of responses Easy Somewhat Easy HardestMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of an Ideal vs. an Actual Telephone Sample for aSpecific QuestionMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps in Survey Research(cont.)The following steps are used in conjunction withperforming proper surveys in research:• Selecting the sample• Preparing the instrument (questionnaire)• Closed-ended questions• Open-ended questions• Pre-testing the questionnaire• Format• Cover letter• Training interviewersMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of Contingency QuestionsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Non-responseIn almost all surveys, some members of thesample will not respond.Item non-response is due to unclear orquestionable forms of wording.Non-response is a serious problem in manysurveys.A variety of techniques are employed to reducethis problem (e.g., rewards or incentive forcompleting the surveys).McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Evaluating Threats to InternalValidity in Survey ResearchThere are four main threats to internalvalidity in survey research:1) Mortality2) Location3) Instrumentation4) Instrument decayMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Data Analysis in SurveyResearchAfter researchers receive the completedquestionnaires, the task of summarizing theresults remains.The total size of the sample and total percentageof returns should be reported.The percentage of the total sample respondingfor each item should be reported.The percentage of respondents who chose eachalternative for each question should be given.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Nature of QualitativeResearchChapter EighteenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Qualitative Research?Many researchers are more interested in thequality of a particular activity than in how often itoccurs or how it would otherwise be evaluated.Research studies that investigate the quality ofrelationships, p, activities, situations, or materials arefrequently referred to as Qualitative Research.There is a greater emphasis on holistic description.Table 18.1 compares the distinctive features of Qualitative ResearchMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

General Characteristics ofQualitative ResearchBogdan and Biklen (1998) describe fivegeneral features of Qualitative Research:1) The natural setting is a direct source and keyelement2) Collection is in the form of words or pictures3) Researchers are concerned with how thingsoccur4) Construction of a picture while data is beingcollected vs. knowing what to expect5) Special interest in the participants’ thoughtsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Philosophical AssumptionsDifferences between the paradigms of Qualitative vs.Quantitative are expressed in how one approachesan investigation.Quantitative approaches are associated withpositivism, relying on empirical data to explainphenomena.Postmodernism presents a more intensive critique ofresearch.• First, it denies the existence of underlying structures• Second, all natural things are changing constantly over timeMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps in Qualitative ResearchAll qualitative studies have a distinctstarting and ending point. They beginwhen the researcher identifies thephenomenon and ends when a finalconclusion is made.The steps are as follows:• Identification of the phenomenon to be studied (foreshadowedproblems)• Identification of the participants in the study (purposive samplecollecting)• Generation of hypotheses• Data collection (continual observance)• Data analysis• Interpretation/ConclusionsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Approaches to QualitativeResearchThe following are approaches to QualitativeResearch:• Biographical Study• Phenomenological Study• Grounded Theory Study• Case Study• Intrinsic Case Study• Instrumental Case Study• Multiple or Collective Case StudyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Sampling in QualitativeResearchThere are eight types of Purposive Samplings:Typical SampleCritical SampleSnowball SampleOpportunistic SampleHomogeneous Sample Confirming SampleTheoretical SampleMaximal VariationMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Generalization in QualitativeResearchA generalization is usually thought of as astatement or claim that applies to more than oneindividual, group, or situation.The value of a generalization is that it allows us tohave expectations about the future.A limitation of Qualitative Research is that there isseldom justification for generalizing the findings ofa particular study.Due to this problem, replication of qualitativestudies becomes more important than forquantitative studies.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ethics and QualitativeResearchEthical concerns affect qualitativeresearch as do other forms of studies.The following are important to repeat:• Identities of participants should be protected• Participants should always be treated withrespect• Researchers should do their best to ensure thatno physical or psychological harm will come toanyone who participates i t in the studyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Qualitative and QuantitativeResearch CombinedQualitative and Quantitative Research can beused together.Descriptive statistics are sometimes used toprovide quantitative details in a qualitativestudy.There are three mixed-method designs whichcombine the two studies:1) Triangulation Design2) Explanatory Design3) Exploratory DesignMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Observation and InterviewingChapter NineteenMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

ObservationCertain kinds of research questions canbest be answered by observing howpeople act or how things look.There are four different roles withregard to observation:1) Participant Observation2) Non-Participant Observation3) Naturalistic Observation4) SimulationsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Variations in Approaches to ObservationRole of the ObserverFull-participant Partial Onlooker;observation participation observer is an outsiderHow the Observer Is Portrayed to OthersParticipants know Some but not Participants do not knowthat observations are being all of the that observations are beingmade and they know who is participants made or that there ismaking them. know the observer. someone observing them.How the Purpose of the Observation Is Portrayed to OthersThe purpose of the The purpose of the No explanation is False explanations areobservation is fully explained observation is given to any of the given; participants areto all involved. explained to some ofthe participants.participants. deceived about thepurpose of theobservation.Duration of the ObservationsA single observation of limitedMultiple observations; long-termduration (e.g., 30 minutes).duration (e.g., months, even years).Narrow focus: Only a singleelement or characteristic is observed.Focus of the ObservationsBroad focus: Holistic view of the activity orcharacteristic being observed and all ofits elements is sought.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Participant vs. Non-ParticipantObservationIn participant observation studies, theresearcheractually participates as an active member of thegroup in the situation or setting they areobserving.In non-participant observation studies, theresearcher does not participate in an activity orsituation, but observes “from the sidelines”.The most common forms of non-participantobservation studies included naturalisticobservations and simulations.A simulation is an artificially created situation inwhich subjects are asked to act out certain roles.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Observer e EffectThepresenceofanobservercanhaveaobserver can have aconsiderable effect on the behavior of thosebeing observed, and affect the outcome of thestudy.Bernard (2000) suggests to researchers that they“catch a glimpse of people in their naturalbehavior before they see you coming”.Unless a researcher is concealed, it is quite likelythat they will have some form of effect upon theindividuals id being observed.It is for that reason that participants should notbe informed of the study’s purpose p until afterdata has been collected.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Observer e BiasRefers to the possibility that certaincharacteristics or ideas of observers maybias what they “see”.Observer expectations is another relatedconcern: when they know to observecertain characteristics of subjects.Comparing notes or impressions amongother researchers assists in reducing thisthreat.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Coding Observational DataA coding scheme is a set of categories anobserver uses to record a person’s or group’sbehavior.Coding schemes have been used to measureinteractions between parents and adolescentsin a lab setting.An observer still must choose what to observe,even with a fixed coding scheme.These data are coded into categories thatemerge as the analysis proceeds (i.e.,ethnographic research).McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Amidon/Flanders SchemeMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

InterviewingingInterviewing ing is an important way for aresearcher to check the accuracy of theimpressions he or she gained throughobservation.Fetterman (1989) describes interviewing asthe most important data-collection techniquefor qualitative research.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of InterviewsThere are four types of interviews:1) Structured• Verbal questionnaires, formal series of questions designed to elicitspecific answers on the Chapterof respondents• Most useful for obtaining information to test a hypothesis2) Semistructured• Verbal questionnaires, formal series of questions designed to elicitspecific answers on the Chapterof respondents• Best conducted d near the end of a study3) Informal• Less formal than the previous methods• Casual conversations, pursuing the interests of both parties• Considered the most common form of interviewing4) Retrospective• Can be all of the above• Tries to get the respondent to recall and reconstruct from memorysomething from the pastMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Interview QuestionsPatton (1990) has identified six types ofinterview questions:1) Background/demographic questions2) Knowledge questions3) Experience/behavior questions4) Opinion/values questions5) Feelings questions6) Sensory questionsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Interviewing BehaviorA set of expectations exists for all interviews:• Respect the culture ofthe group beingstudied• Respect the individualbeing interviewed• Be naturalal• Develop anappropriate rapportwith the participant• Ask one question at atime• Ask the same questionin different waysduring the interview• Ask the interviewee torepeat an answerwhen in doubt• Vary who controls theflow of communication• Avoid leadingquestions• Don’t interrupttMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Validity and Reliabilityin Qualitative ResearchAn important t check on the validity and reliability of theresearcher’s interpretations in qualitative research is tocompare one informant’s description of something withanother informant’s description of the same thing.Triangulation is a check on reliability/validity by comparingdifferent information on the same topic.Efforts should be made to ensure reliability and validity byincluding:• Proper vocabulary• Recording questions and personal reaction• Describing content and documenting sourcesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Qualitative Research Questions, Strategies, and Data-Collection TechniquesPossible Research Research Examples of Data-Purpose of the Study Questions Strategies Collection TechniquesExploratoryTo investigate a littleunderstoodevent, situation,or circumstanceTo identify or discoverimportant variablesTo generate hypotheses forfurther researchDescriptiveTo document an event, situation,or circumstance of interestExplanatoryTo explain the forces causing anevent, situation, orcircumstanceTo identify plausible causalnetworks shaping an event,situation, or circumstancePredictiveTo predict the outcome of anevent, situation, orcircumstanceTo forecast behaviors or actionsthat might result from an event,situation, or circumstanceWhat is happening in thisschool?What are the importantthemes or patterns in the waysteachers behave in this school?How are these themes or patternslinked together?What are the important behaviors,events, attitudes, processes, and/orstructures occurring in this school?What events, beliefs, attitudes, and/orpolicies are shaping the nature ofthis school?How do these forces interact to shapethis school?What is likely to occur in the future asa result of the policies now in placeat this school?Who will be affected, and in what?Case studyObservationField studyCase studyField studyEthnographyObservationCase studyField studyEthnographyParticipant observationNon-participant observationIn-depth interviewingSelected interviewingParticipant observationNon-participant observationIn-depth interviewingWritten questionnaireContent tanalysisParticipant observationNon-participant observationIn-depth interviewingWi Written questionnaireiContent analysisObservation In-depth interviewingways? Interview Written questionnaireContent analysisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Content AnalysisChapter TwentyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Content AnalysisPartTwentytMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Content Analysis?Content Analysis is a technique that enablesresearchers to study human behavior in anindirect way, through an analysis of ourcommunications.Types of communications are:• Textbooks• Essays• Pictures• SongsA person or group’s conscious and unconsciousbeliefs, attitudes, or values are often revealed intheir communication.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Example of Content AnalysisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Applications of Content Analysis inEducational ResearchDescribe trends inschooling over timeUnderstand organizationalpatternsShow how differentschools handle the samephenomena differentlyInfer attitudes, values,and cultural patterns indifferent countriesCompare the myths thatpeople hold about schoolsGain a sense of howteachers feel about theirworkGain some idea of howschools are perceivedAlso, can be used tosupplement more directmethods of researchMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Categorization in Content AnalysisAll procedures at some point convert thedescriptive information into categories.There are two ways this might be done• The researcher determines the categories before anyanalysis begins. These categories are based onprevious knowledge, theory, and experience.• The researcher becomes very familiar with thedescriptive information collected and allows thecategories to emerge as the analysis continues.See p. 488 for an example of thisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps Involved in Content AnalysisDetermine e objectiveses by obtaininginformation on the following:• Formulate themes for organization• Check other research findings for validation• Obtain information useful in dealing with educationalproblems• Investigate possible relationships to test hypothesisDefine Terms• Clearly define terms before or during the studySpecify the Unit of AnalysisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps Involved in Content Analysis(cont.)Locate Relevant DataDevelop a Rationale• Conceptual link needed d to relate data to the objectivesDevelop a Sampling Plan• Techniques could involve:• Random sample• Stratified sampling• Cluster samplingFormulate Coding Categories (Table 20.1)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps Involved in Content Analysis(cont.)Checking Reliability and Validity• Test-retest method• Check manifest against latent contentAnalyze Data• Counting• Record the base• Use descriptive statistical procedures such asfrequencies and/or percentagesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Sample Tally SheetNumber Number NumberNewspaper of Editorials Subjective of Pro-Abortion of Anti-AbortionID Number Location Circulation Coded Evaluation a Editorials Editorials101 A 3,000,000 29 3 0 1102 B 675,000 21 3 1 1103 C 425,000 33 4 2 0104 D 1,000,000 40 1 0 8105 E 550,000 34 5 7 0aCategories within the subjective evaluation: 1 = very conservative; 2 = somewhat conservative; 3 = middle-of-theroad;4 = moderately liberal; 5 = very liberal.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Advantages of Content AnalysisThe following are considered advantagesof Content Analysis:• Unobtrusive• Useful means of analyzing interview andobservational data• Not limited by time and space to the study ofpresent events• Relatively simple and economical with regardto both time and resources, compared to otherforms of research methodsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Disadvantages of Content AnalysisThe following are considered disadvantagesof Content Analysis:• Usually limited to recorded information• Establishing validity• Question remains as to the true meaning of thecategories themselves• Historical research findings might not beconsidered important today• Temptation to attribute a cause of aphenomenon vs. a reflection of itMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Analysis of the StudyPurposeDefinitionsPrior ResearchHypothesisSampleInstrumentation andData AnalysisInternal ValidityResults andDiscussionMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ethnographic ResearchChapter Twenty-OneMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Ethnographic hi Research?Ethnographic hi Research is considered d the mostcomplex of all research methods.A variety of approaches is used in an attempt toobtain as holistic a picture as possible of aparticular society, group, setting, etc.The emphasis is on documenting the everydayyexperiences of individuals by observing andinterviewing them and relevant others.The key tools are in-depth interviewing andcontinual, ongoing participant observation of asituation.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ethnographic ProceduresesWolcott (1966) stated that ethnographicprocedures require three things:1) Ad detailed ddescription i of the culture-sharing groupbeing studied.2) An analysis of this group in terms of perceivedthemes or perspectives.3) Some interpretation of the group by the researcheras to meanings and generalization about the sociallife of human beings, in general.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Unique Valueof Ethnographic ResearchEthnographic research has a particularstrength that makes it especially appealingto many researchers.It can reveal nuances and subtleties thatother methodologies miss.By going out into the world and observingthings as they occur, we are better able toobtain a more accurate picture.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Ethnographic ConceptsCultureHolistic PerspectiveContextualizationAn Emic PerspectiveThick DescriptionMember CheckingA NonjudgmentalOrientationThese concepts guide the work of ethnographers as they perform field researchMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Topics that Lend Themselves Well toEthnographic ResearchTopics that defy simplei h i l hquantificationeducators play, andTopics that can be bestunderstood in a naturalsettingTopics that involve thestudy of individual orgroup activities overtimeTopics that involve thestudy of the roles thatbehaviors associated withthose rolesTopics that involve thestudy of the activities andbehavior of groups as aunitTopics involving the studyof formal organizations intheir totalityMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Sampling in Ethnographic ResearchEthnographers attempt to observe everything.However, no researcher can observe everythingat once.Samples are small and do not permitgeneralization to a larger population.pTheir goal is the complete understanding of aparticular situation.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Do Ethnographic ResearchersUse Hypotheses?Ethnographers seldom initiate their researchwith precise hypotheses.They attempt to understand an ongoingsituation or set of activities that cannot bepredicted in advance.Ethnographic research relies on bothobservation and interviewing that is continualand sustained over time.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Data Collectionin Ethnographic ResearchThere are two main methods of collectingdata through Ethnographic Research:1) Interviewing is the most important tool2) Participant Observation is crucial to effectivefieldwork, which requires an immersion in theculture• Field notes are used to check the accuracy of anethnographer’s observations• Other forms of writing i used are field jottings, fielddiary, and field logsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Data Analysisin Ethnographic ResearchTriangulation (checkingthe validity by comparingsources of information)Patterns (checkingreliability by revealingconsistencies anddescribing matches)Key Events (a lensthrough which to view aculture)Visual Representations(maps, charts,sociograms)Statistics (use of nonparametrictechniques)Crystallization (wheneverything falls into place)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Triangulation and PoliticsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Advantages and Disadvantagesof Ethnographic ResearchDisadvantages• It is highly dependent onthe researcher’sobservations andinterpretations• There is no way to checkthe validity of theresearcher’s conclusion,since numerical data israrely provided• Observer bias is almostimpossible to eliminate• Generalizations arealmost non-existent sinceonly a single situation isobserved, leavingambiguity in the studyAdvantages• A key strength is that itprovides the researcher witha much morecomprehensive perspectivethan other forms ofresearch• It is also appropriate tobehaviors that are bestunderstood by observingthem within their naturalenvironment (dynamics)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Historical ResearchChapter Twenty-TwoMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Historical ical Research?The systematic collection and evaluation of datato describe, explain, and understand actions orevents that occurred sometime in the past.There is no manipulation or control of variablesas in experimental research.An attempt is made to reconstruct whathappened during a certain period of time ascompletely l and accurately as possible.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Purposesof Historical ical ResearchTo make people aware of what has happenedin the past in order to:• Learn from past failures and successes• Apply them to present-day problems• Make predictions• Test hypotheses concerning relationships ortrends• Understand present educational practices andpolicies more fullyMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps Involvedin Historical ical ResearchDefining the ProblemLocating relevant sources• Documents• Numerical records• Oral statements• RelicsSummarizing information obtained fromhistorical sourcesEvaluation of historical i sources• Internal criticism• External criticismMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What Really Happened?McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Categories of SourcesDocuments are written or printed materials thathave been produced in some form or another.Numerical records can be considered as aseparate type of source in and of themselves oras a subcategory of documents.Oral Statements are stories or other forms of oralexpression that leave a record for futuregenerations.Relics are any objects whose physical or visualcharacteristics can provide some informationabout the past.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Pi Primary vs. Secondary SourcesA primary source is one prepared by anindividual who was a participant in or adirect witness to the event being described.A secondary source is a document preparedby an individual who was not a directwitness to an event, but who obtained adescription of the event from someone else.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Data Analysisin Historical ical ResearchHistorical researchers use the followingmethods to make sense out of large amountsof data:• Theoretical model leading to a content analysis• Use of patterns or themes• Coding system• Quantitative data to validate interpretationsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Advantages and Disadvantagesof Historical ResearchAdvantages• Permits investigation oftopics and questions s thatcan be studied in noother fashion• Can make use of morecategories of evidencethan most other methods(with the exception ofcase studies andethnographic studies)Disadvantages• Cannot control forthreats to internalvalidity• Limitations areimposed due to thecontent analysis• Researchers cannotensurerepresentation of thesampleMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Action ResearchChapter Twenty-ThreeMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is Action Research?Action Research is conducted by one or moreindividuals id or groups for the purpose of solving aproblem or obtaining information in order to informlocal practice.Such studies are seriously limited in generalizability.Action research does not require complete mastery ofthe major types of research earlier discussed.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Basic Assumptions Underlying Action ResearchAssumptionTeachers and other educationprofessionals have the authority tomake decisions.Teachers and other educationprofessionals want to improve theirpractice.Teachers and other educationprofessionals are committed tocontinual professional development.Teachers and other educationprofessionals will and can engage insystematic research.ExampleA team of teachers, after discussions with the schooladministration, decide to meet weekly to revise themathematics curriculum to make it more relevant to lowachievingstudents.A group of teachers decide to observe each other on a weeklybasis and then discuss ways to improve their teaching.The entire staff—administration, teachers, counselors, andclerical staff—of an elementary school go on a retreat to planways to improve the attendance and discipline i policies i for theschool.Following up on the example just listed above, the staff decidesto collect data by reviewing the attendance records of chronicabsentees over the past year, to interview a random sample ofattendees and absentees to determine why they differ, to hold aseries of after-school roundtable sessions between disciplinepronestudents and faculty to identify problems and discussways to resolve issues of contention, and to establish amentoring system in which selected students can serve ascounselors to students needing help with their assigned work.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Assumptions UnderlyingAction ResearchA number of assumptions underlie actionresearch:• Participants have the authority to make decisions• Those involved are seriously committed toimproving their performance• Educators and others involved in schools want toengage in research systematically• Those performing the research will make thenecessary changes and recommendationsMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Types of Action ResearchThere eeaet are two main typeso of acto action research: eseac1) Practical Action Research• Addresses a specific problem• Primary purpose is to improve practice and inform largerissues2) Participatory Action Research• Empower individuals id and groups to improve their lives andbring about a social change• Stakeholders are involved and are active in all processes(collaborative participation)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Levels of ParticipationProvide informationBecome informed ofpurpose of the studyReceive findingsAssist in datacollectionReview findingsParticipate ininterpretationParticipate in designingthe projectParticipate in problemspecificationInitiate studyThere are nine levels, as shown in Figure 23.3.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Levels of Participation in Action ResearchMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Steps in Action ResearchThere are four steps or stages in ActionResearch:1) Identifying the research question2) Gathering the necessary informationi3) Analyzing and interpreting the information4) Developing a plan of actionMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Advantages of Action ResearchIt can be performed by anyone, in any type of schoolor institutionIt can help to improve educational practiceIt can help education and other professionals toimprove their craftIt can help them learn to identify problemssystematicallyIt can build up a small community of researchorientedindividuals at the local levelSimilarities and Differences of Types of Research are shown in Table 23.2.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Similarities and Differences Between Action Research & Formal Quantitative andQualitative ResearchAction ResearchFormal ResearchSystematic inquiry.Goal is to solve problems of local concern.Little formal training required to conduct suchstudies.Intent is to identify and correct problems.Carried out by teacher or other local educationprofessional.Uses primarily teacher-developed instruments.Less rigorous.Usually value-based.Purposive samples selected.Selective opinions of researcher oftenconsidered as data.Generalizability is very limited.McGraw-HillSystematic inquiry.Goal is to develop and test theories and toproduce knowledge generalizable to widepopulation.Considerable training required to conductsuch studies.Intent is to investigate larger issues, of localconcern.Carried out by researcher who is not usuallyinvolved in local situation.Uses primarily professionally-developedinstruments.More rigorous.Frequently value-neutral.Random samples (if possible) preferred.Selective opinions of researcher neverconsidered as data.Generalizability often appropriate.© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Preparing Research Proposalsand ReportsChapter Twenty-FourMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

What is a Research Proposal?It is a written plan of a study.It spells out in detail what the researcher intendsto do.It permits others to learn about the intendedresearch and to offer suggestions for improvingthe study.It also helps the researcher clarify what needs tobe done and to avoid unintentional problems.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

The Major Sections of a Research ProposalThere are four topics addressed in this area:1) The purpose of the study2) The justification for the study3) The research question/hypothesis, includingvariables to be investigated4) The definition of termsSee Figure 24.1, Organization of a Research Report.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Organization of a Research Report (1)Introductory sectionTitle PageTable of ContentsList of FiguresList of TablesMain BodyI. Problem to be investigatedA. Purpose of the study (including assumptions)B. Justification of the studyC. Research question and hypothesesD. Definition of termsE. Brief overview of studyII. Background and review of related literatureA. Theory, if appropriateB. Studies directly relatedC. Studies tangentially relatedIII. ProceduresA. Description of the research designB. Description of the sampleC. Description i of the instruments used(including scoring procedures)McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Organization of a Research Report (2)D. Explanation of the procedures followed (thewhat, when, where, and how of the study)E. Discussion of internal validityF. Discussion of external validityG. Description and justification of the statisticaltechniques or other methods of analysis usedIV. FindingsDescription of findings pertinent to each of theresearch hypotheses or questionsV. Summary and conclusionsA. Brief summary of the research question beinginvestigated, the procedures employed, andthe results obtainedB. Discussion i of the implications of thefindings—their meaning and significanceC. Limitations—unresolved problems andweaknessesD. Suggestions for further researchReferences (Bibliography)AppendixesMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Purpose of the StudyThe purpose states succinctly what the researcherproposes to investigate.This should be a concise statement, providing aframework to which details are added later.Clarification of the field of interest should beconsidered here.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Justification of the StudyThe researcher must make clear why this particularstudy is important to investigate.He or she must present an argument for the study.A good justification should also include any specificimplications that follow if relationships areidentified.dThere could be an implication that current methodsare not good enough, however, this should bemade explicit.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Research Question orHypothesisThe particular question to be investigated shouldbe stated here.Favoring hypotheses will help clarify and becomea strategy point.It should be clearly stated as concisely aspossible.There should be any inference made if ahypothesis is well-stated.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

DefinitionsAll key terms should be defined. edThe researcher’s task is to make the definitionsas clear as possible.Sometimes, terms will have to be modified to fitthe present study.While it is probably bl impossible ibl to eliminate i allambiguity, the clearer the terms used, the fewerthe difficulties that will be encountered with thestudy.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Background and Review ofLiteratureThis may be a lengthy section since it is a partialsummary of previous work related to the focus ofthe study.The researcher should demonstrate a familiaritywith previous research and understand therelevance of the study being planned.A major weakness of many literature reviews isthat they cite references without indicating theirimplications for the planned study.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Poced ProceduresesThe procedures section includes discussion of thefollowing:• Research Design• Sample• Instrumentation• Procedural detail• Internal validity• Data analysisMcGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

Results/FindingsResults of a study can be presented only in a researchreport.Results are usually not found in the proposal section.This section is found near the end of the researchreport and constitutes the description of what kind ofanalyses were performed.The data are revealed by the form of statisticalanalysis that was applied to the data, and anysignificance that was observed.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

DiscussionssionThe discussion section of a report presents theauthor’s interpretation i of what the results imply fortheory and/or practice.Researchers place their results in a broader context.Here, difficulties as well as limitations of the studyare noted, and suggestions for future considerationsare included.Results and Discussion sections should be keptseparate, since the Discussion section goes beyondthe data.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.

General Rules to ConsiderA research report should be written as clearly l andconcisely as possible.Research reports are always written in the past tenseand free from jargon.A style manual (APA manual) should be consulted beforebeginning the report.Once the report is completed, it is a good idea to have athesis formatter/editor check for style and grammar.Computerized programs have made research reportseasier to complete, due to self-correcting gprograms.McGraw-Hill© 2006 The McGraw-Hill Companies, Inc. All rights reserved.