It's Not About the Cut Score: Redesigning Placement Assessment

An ACHIEVING THE DREAM Policy BriefIT’S NOT ABOUTTHE CUT SCORE REDESIGNING PLACEMENT ASSESSMENTPOLICY TO IMPROVE STUDENT SUCCESSBy Michael Lawrence Collins JULY 2008

ACHIEVING THE DREAM: COMMUNITYCOLLEGES COUNT is a national initiative to helpmore community college students succeed (earndegrees, earn certificates, or transfer to other institutionsto continue their studies). The initiative isparticularly concerned about student groups thathave faced the most significant barriers to success,including low-income students and students ofcolor. Achieving the Dream focuses colleges andothers on understanding and making better use ofdata. It acts on multiple fronts, including efforts atcommunity colleges and in research, publicengagement, and public policy. Achieving theDream is funded by Lumina Foundation forEducation and 18 other partner foundations.JOBS FOR THE FUTURE creates strategiesfor educational and economic opportunity. Incommunities across the country, we developpromising education and labor-market models,expand on-the-ground successes, and shape thepolicy environment that enables Americanfamilies and companies to prosper in today’seconomy.www.jff.orgwww.achievingthedream.org

TABLE OF CONTENTSExecutive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iiiUnderstanding the Policy Trade-Offs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Overview of State Placement Assessment Policy Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2Pressures for Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Political and Policy Considerations in Setting Placement Assessment Policy:Three State Rationales for, and Experiences with, Standardizing Placement Cut Scores . . . . . . . . . . 5Virginia: The Process Matters as Much as the Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Connecticut: A Whisper Grows Louder. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8North Carolina: A Process of Continuous Calibration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Policy Recommendations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12Appendix A: Resources: Placement Assessment Policies in Achieving the Dream States . . . . . . . . . 14Appendix B: Methodology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Appendix C: State Policies on Developmental Education in Achieving the Dream States . . . . . . . . 21Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

ABOUT THE AUTHORMichael Collins is a program director in JFF’s YouthTransitions Cluster. Mr. Collins develops and advocatesfor state policies through two national initiatives:Achieving the Dream: Community CollegesCount and the Early College High School Initiative.His work focuses on assisting states to develop andimplement public policies designed to increase thenumbers of low-income students and students ofcolor who successfully transition from high schoolinto college, persist, and earn credentials and degrees.Mr. Collins regularly collaborates with public policydecision makers, state and national intermediaryorganizations, philanthropic organizations, academicresearchers, and policy experts to develop policiesthat support innovative blends of high school andcollege and to design and execute on-the groundpolicy/action agendas for dramatically improving therates at which students and states reach their educationand career/workforce goals.ACKNOWLEDGEMENTSI would like to thank Lumina Foundation forEducation and the Achieving the Dream partnerfoundations for supporting this policy research. Iwould also like to thank the researchers—Eliot Yaffa,John Valant, and Jeff Wetherhold; the interviewees—Monty Sullivan, Paul Susen, and Marc Williams; andthe reviewers—Richard Kazis, Nancy Hoffman,Vickie Choitz, Sue Goldberger, Cheryl Blanco, KarenPaulson, and Bonnie Gordon. A special thanks toLara Couturier, and to the Achieving the Dreamstate lead organizations who patiently helpedus secure placement cut score data and identify stateleveldevelopmental education policies.Before joining JFF, Mr. Collins served as AssistantCommissioner for Participation and Success at theTexas Higher Education Coordinating Board. In thatcapacity, he worked with K-12, higher education, thebusiness community, the Texas legislature, andcommunity-based organizations to increase collegeaccess and success

IT’S NOT ABOUT THE CUT SCOREREDESIGNING PLACEMENT ASSESSMENT POLICYTO IMPROVE STUDENT SUCCESSEXECUTIVE SUMMARYPlacement assessment policy—governing how collegesassess the academic skills of entering students and placethem in courses that are appropriate for their skilllevels—can be an important lever for increasing studentsuccess in community colleges. A coherent placementassessment policy would indicate which students mustbe assessed, specify assessment instruments, set cutscore standards, and articulate the protocols and proceduresto be used uniformly across a state’s communitycollege systems. Well-designed placement assessmentpolicies can help increase student success in a numberof ways: by accurately assessing student skills andplacing students in the courses they need; by ensuringconsistent standards from college to college; and byproviding comparable and timely data on studentoutcomes that states and institutions can use to informtheir practices and policies.Placement assessment policies can also help improvethe college readiness of incoming students—an explicitgoal in many states—by setting clear college-levelperformance expectations, and then communicatingthose expectations so that students arrive ready forcollege-level work.With these benefits in mind, many states are evaluatingtheir placement assessment policies, or lack thereof. Asthey do, they are finding that setting placement assessmentpolicy is not a simple process. This brief describesthe experiences of three states in the national initiativeAchieving the Dream: Community Colleges Count asthey revised their placement assessment policies. Whenthe Virginia Community College System set out to establishcommon cut scores, it unknowingly embarked upona process that quickly made cut scores secondary to standardizingplacement processes and procedures. InConnecticut, questions about cut scores quickly led tomuch larger questions about quality and costs. And theNorth Carolina Community College System finds itselfcontinually adapting and revisiting a process that beganin the 1980s. The stories of Virginia, Connecticut, andNorth Carolina in setting placement assessment policy,as well as the current policies of all Achieving the Dreamstates, suggest lessons for other states.Additional research is needed to understand the impactof common placement standards on success outcomes,and there are many good reasons for embarking on thisanalysis. The brief concludes with recommendations—informed by the experience of Achieving the Dreamand other states—that can help states develop optimalplacement policies from the perspective of improvingcommunity college student outcomes.Achieving the Dream State Placement Assessment PoliciesPlacement Policy Options Yes NoAre students required to take placement tests? AR, CT, FL, HI, MA, NC, OH, OK, SC, TX, VA MI, NM, PA, WADoes the state require specific tests? AR, CT, FL, HI, MA, NC,OH, OK,TX, VA MI, NM, PA, SC, WADoes the state specify which students are exempt? AR, CT, FL, HI, MA, NC, OH, OK, TX, VA MI, NM, PA, SC,WAIs there a common statewide placement cut score? AR, CT, FL, HI, MA, NC, OH, OK, TX, VA MI, NM, PA, SC, WAAre students required to enroll in or completedevelopmental education within a specified timeperiod?AR, FL, OKCT, HI, MA, MI, NC, NM, OH,PA, SC, TX, VA, WAIT’S NOT ABOUT THE CUT SCOREiii

It’s Not About the Cut Score takes an updatedand detailed look at the choices made inAchieving the Dream states regarding placementassessment policies. It provides an insider’s viewof how state placement policy deliberationshave played out on the ground in three states.Interviews with decision makers in Virginia,Connecticut, and North Carolina bring to lifethe trials and tribulations—and triumphs—involved in implementing common placementpolicies. The Resource Section (Appendix A)examines the extent of variation within andbetween states on key policy questions,describes the range of placement cut score policiesin each state, and discusses the implicationsof this variability on student success outcomes.The brief also discusses the lessons learned andrecommendations generated by Achieving theDream state policy teams. One of the mostimportant conclusions is that while settingcommon cut scores might be the initial goal,it is rarely the ultimate focus of the work.Underlying questions about broader placementassessment policy—ranging from how to payfaculty needed to teach developmental courses,to whether students can use calculators whiletaking placement exams—often rise to front andcenter. The state teams also learned that thiswork is never done: it is an iterative process thatrequires vigilant and continual monitoring andrefining. But they feel strongly that they haveembarked upon a necessary process that hassparked critical conversations, pulled back thecovers on unseen data and unquestionedassumptions, provided a rational framework fordecision making, and resulted in useful changes.These lessons and recommendations should beinstructive to states seeking to better understandthe trade-offs between statewide common placementpolicies and more flexible, institution-setplacement policies.OVERVIEW OF STATE PLACEMENT ASSESSMENT POLICY PATTERNSAcross the nation, 27 states have policies thatrequire community colleges to assess enteringstudents’ needs for developmental education,and placement assessment policies appear to bea growing trend (see Table 1). 1 Since JFFpublished Standardization vs. Flexibility in2005, six more states nationwide report havingmandatory placement assessment policies(Prince 2005). Similarly, the number of statesthat require colleges to use a specific placementtest almost doubled between 2002 and 2008,reaching 21 states. Finally, and most dramatically,the number of states requiring standardizedcut scores for placement into developmentaleducation more than tripled during thisperiod.Of the 46 states that provided information in2008:• 27 states (up from 21 states in 2002) reportedthat a state-level policy was in place requiringcommunity colleges to assess students’ needsfor developmental education at the time ofenrollment.• 21 states (up from 11 states in 2002) specifiedone or more approved placement exams forcolleges to use. COMPASS was the mostfrequently approved exam (14 states),followed by ACCUPLACER (11 states) andASSET (7 states).• 19 states (up from 5 states in 2002) requiredcolleges to use standardized cut scores orranges on these exams for placement intodevelopmental education.Policy developments in the Achieving theDream states reflect this national trend towardstandardization. At the start of the initiative,only Florida and Texas had official statewidecommon placement policies. A third state,Virginia, had an unofficial cut score policy thatits colleges generally followed. However, since2005, each of the original Achieving the Dreamstates has reviewed or is in the process ofreviewing its placement assessment policies,with an emphasis—for now, at least—on cutscore policies. New Mexico has collected andposted a cut score data matrix for review. North2 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

PRESSURES FOR CHANGEThere are many compelling reasons for morestates to revise their placement assessment policiesand practices, including the setting or modifyingof cut scores. The pressures for changecome from both inside and outside of communitycollege systems.The powerful interaction between the internaland external pressures has pressed a number ofstates, including many Achieving the Dreamstates, into corralling disparate placement practicesand policies into a coherent placementassessment policy framework. This includesspecifying assessment instruments, cut scorestandards, and protocols and procedures to beused uniformly across the systems.Internal Pressures for ChangeAlarmingly low student success rates: Thesehave compelled many community collegesystems to put all of their policies and practicesunder the microscope, looking for a relationshipbetween placement policies and student success.For example, if cut scores are set too low,students may be placed in courses for whichthey are underprepared. If cut scores are set toohigh, students may spend time and tuitionmoney on developmental education courseswhen they could instead be progressing throughcredit-bearing, college-level courses.Inconsistent entrance standards: Communitycollege systems feel pressure to ensure thatentrance standards are consistent amongcolleges, thereby clarifying student expectationsand eliminating confusion. Standardizing placementscores eliminates the chance that studentsearning the same score on placement exams aredeemed college ready in some of the system’scolleges but are required to enroll in developmentaleducation in others. In states with largenumbers of community colleges, campuseswithin a few miles of each other can havedifferent cut scores, allowing students who testinto developmental education at one college toeffectively game the system by enrolling in asecond college where their score qualifies themas college-ready.Barriers to facilitating transfer: Communitycollege systems wish to better facilitate transferby avoiding the scenario in which students withcollege-ready status transfer to another collegein the same system and are there required toenroll in developmental education underdifferent placement criteria.Inconsistent data: Some community collegesystems are seeking comparable data from allcolleges in order to measure the effectiveness ofinterventions designed to increase studentsuccess. For example, if a system wants toanalyze the graduation rates of students whobegin in developmental education, the standardsfor who enrolls in developmental educationneed to be the same at each college.Unclear course sequences: New researchsuggests that taking courses in the appropriatesequence, and within a reasonable time frame,matters (for example, taking Math 11 soon aftertaking Math 10 generally leads to betteroutcomes). As a result, states may want toencourage community college systems toconsider how to ensure that students enroll inthe right courses at the right time. Historically,the open access mission has made manycommunity colleges reluctant to requirestudents to take developmental education, or totake courses in a certain order or at a certaintime, but some colleges and systems are startingto embrace student success—in addition toaccess—as their mission (Adelman 2006;Bashford & Slater 2008; Prince 2005; Shulock& Moore 2007).External Pressures for ChangeLimited alignment between K-12 and postsecondaryeducation: The greatest external pressuresto develop common placement standardscome from K-12 reform efforts and the nationalpush to align secondary and postsecondarystandards. The disconnect between the skills ofhigh school graduates and the skills needed forcollege and workplace success has compelledmany states to work on aligning their high4 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

Studies and a Task ForceIn January 2007, VCCS began a series ofresearch studies designed to better understandits placement policy’s impact on student success.The initial research, funded by the NationalGovernors Association’s Center for BestPractices, was completed in August 2007. Itfocused on pinpointing the academic weaknessesof recent high school graduates and identifyingcolleges’ strategies to serve students whoenroll needing developmental education. Thenext month, VCCS’s Academic Services andResearch Department followed up the academicweaknesses research with a related longitudinalstudy that tracked success outcomes by placementscores. Both studies raised a host of questionsthat called for further longitudinal study,and both recommended that an internal studybe conducted to reexamine Virginia’s use ofCOMPASS, its cutoff scores, subsequent courseplacements, and student success in thesecourses.To get the colleges’ perspective on the issue,VCCS appointed a placement task force andprepared for a protracted debate on theCOMPASS assessment and cutoff scores.However, there was almost instant consensus—but with a twist. While the taskforce agreed thatdetermining the right assessments and the mostaccurate cut score range was important, a morepressing matter needed to be handled first. AsSullivan notes, a cut score range means littlewhen the placement tests are administeredinconsistently; inconsistent administration willlead to wide variations in student scores.At the beginning of this process, the system’stop priority was gathering empirical evidencefor revising its cut score policy. However, thetask force findings have shifted the approachto broader placement assessment policy.The cut score study itself is somewhat secondaryat this point in the process. It was primary in theearly stages, but it became much more secondaryas we began to think about that we had so manydifferences in the way our colleges managed theprocedures of placement testing that it’s very difficultto compare numbers across colleges whenthere are such huge differences. There is still a lotof work to be done in terms of studying the cutscores, but until we have that solid procedure inplace and everyone understands that we’re operatingfrom the same sheet of music, I think ourlabor in studying cut scores was for naught.The Road AheadThe task force’s early consensus to developcommon procedures for placement, rather thanimmediately grappling with which assessmentsor cutoff scores will be used to place students,has expedited the discussion. Nevertheless, thearduous tasks of building consensus on thespecific assessments that will be used andspecific cutoff scores for math and English lieahead.In addition to carefully navigating the perennial“quality vs. access” debate, which historicallyhas been part of placement policy deliberations,particularly on cut scores, VCCS must also teamup with the K-12 sector. 3 Virginia, in collaborationwith the American Diploma Project, isThe colleges’ placement processes and procedureswere all over the board. Some institutionsrequired students to take the placement test oncampus, while others allowed students to take thetest on line. Some colleges allowed calculators onmath tests; others did not. Some institutionsallowed retests within a short amount of time,while others did not allow retests. Some institutionsrequired an essay to augment the COMPASSassessment; others did not. And some institutionsallowed SAT and ACT scores for placementpurposes; while others did not.IT’S NOT ABOUT THE CUT SCORE 7

aligning its high school exit standards with itscollege entrance standards, with the ultimategoal of creating a high school diploma thatreflects college preparatory-level work and canplay a role in college placement. This alignmentrepresents an enormous step in the right direction,but it also presents new challenges.Sullivan reflects:As we talk about possible revisions to our placementpolicies, there have been discussions amongpeople from Department of Education, the P-16Council, and the American Diploma Project thatwant to explore using the high school diploma forcollege placement purposes. We don’t have aformal position on this at this time, but we’ve saidfrom the beginning that it’s important [that] anyfuture standard go beyond indicating “passing orfailure” and truly indicate readiness for success incollege.VCCS is moving ahead and expects that it willneed to address the specific issue of placementcut scores in the fall of 2008. By then, all of thecolleges will be using the same processes andprocedures to assess students, and the systemwill be able to use data to make “apple-toapple”comparisons of placement, cut scores,and student outcomes. Sullivan expects thatVCCS will have a much better sense of what thecut score data mean as a result of understandinghow the cut scores are derived from campus tocampus. He and his colleagues believe they arebuilding a foundation of evidence that willallow the system to eventually set a placementcut score range with greater accuracy.Sullivan believes the system’s shift in itsapproach to placement assessment policy,particularly on cut scores, will pay big returns inthe end, and that the timing is right for theconversation.In some ways our vice presidents have beenprimed for this conversation. Through participatingin Achieving the Dream, they understand theimportance of common definitions and standardsand the importance of being able to make meaningfulcomparisons across campuses.Connecticut: A Whisper Grows LouderAfter hearing presentations on developmentaleducation, transfer, and the effective use of data atan Achieving the Dream meeting in January 2007,a key staffer from the Connecticut Department ofEducation whispered to a senior staffer at theConnecticut Community College System that theirstate would benefit from developing commonplacement cut scores. In the next legislativesession, the Department of Education facilitatedthe sponsorship and eventual passage of a billmandating the system to identify common cutscores. The following description of placementassessment policy developments is based on aninterview with Paul Susen, Chief Academic Officerwith the Connecticut Community College System.Pressures to StandardizeThe issue of implementing common placementcut scores is not new to Connecticut. In 1986,when the Connecticut Community CollegeSystem (CCCS) was using a precursor of ACCU-PLACER known as the New Jersey Basic SkillsTest, the faculty tried to agree on a common cutscore. That attempt, and many since then, gotmired in the painful political process of settingstandards.In 2007, however, agreeing on common cutscores became a top priority for the CCCS forthree primary reasons. Two years earlier,Connecticut had become an Achieving theDream state and intensified its focus on usingdata to inform policymaking. However, a lackof comparable data across colleges preventedthe CCCS from measuring the effectiveness ofinterventions designed to increase studentsuccess. A key obstacle was the variability ofplacement practices, including testing protocolsand cut score ranges—revealed by a study of theways colleges used ACCUPLACER.Differences in placement cut scores acrosscommunity colleges in the system also strainedthe CCCS’s relationships with both theUniversity of Connecticut and Connecticut StateUniversity System. Without robust indicators ofthe rigor and quality of the community collegetransfer core, assuaging the universities’8 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

concerns about transfer students’ academicreadiness was difficult. It was clear that animportant step in building the universities’confidence was developing common cut scores.As Paul Susen notes:We need to be assuring the university that ourbasic gatekeeper courses are consistent in theircontent and rigor. We couldn’t do that with gatekeepercourses across the system having differentplacement criteria to enter, particularly if your cutscores are 40 percent different entering into thatcourse.The third and arguably most compelling reasonthat cut scores became a top priority was alegislative mandate. In 2007, Special Act 07-7required “a report on the establishment ofplacement scores for the Community-TechnicalCollege System and the Connecticut StateUniversity System that establish specific proficiencylevels for all matriculated studentsentering college level courses.” The GeneralAssembly required that cut scores be establishedwithin six months, a startlingly short timelinegiven the faculty’s inability to agree on cutscores after more than twenty years of discussions.Setting the ScoreDespite what initially seemed an unreachabledeadline, the faculty agreed on statewide placementcut scores. The fact that Connecticut had astrong faculty leadership structure in placemade all the difference. The process to set thecut scores was led by the mathematics andEnglish committees, comprised of facultymembers and led by respected academic deans.Susen described some of the reasons for theirsuccess:The committee members have worked together;they know each other, and most importantly, theyrespect each other and are highly regarded bytheir peers. They’ve collaborated professionallyand they’ve gotten together socially. But it still tooka lot of compromise. No one got exactly what theywanted. But I think we were fairly successful, eventhough no one was entirely comfortable with theoutcome. Everybody had to move off the dime.As the committees began to review placementpolicies and procedures, it became clear that thereal issue was broader than cut scores.We concluded that we were looking at cut scoresbecause we thought it was important to look at cutscores, but we were missing the forest for thetrees—we didn’t have good alignment [between,for example, developmental education and gatekeepercourses]. We really needed to go backwardand look at the gatekeeper courses, then go backwardto developmental education.In addition to alignment, other complex policydecisions had to be made in light of the new cutscore policy. These included: which studentscolleges would have to assess; protocols forreturning students designated as college readyunder the previous policy; protocols for the useof calculators; and protocols for retesting.The ImpactIn addition to these challenges, the setting ofcommon cut scores had major fiscal implications.The colleges’ budgets are automaticallyadjusted for their enrollment mix. AsConnecticut funds developmental education at ahigher rate than some college-level courses, adramatic increase in the number of studentsentering developmental education could significantlyincrease the costs to the state:We ran the projections using the newly proposedcut score and found that some colleges wouldneed to add up to 10 additional sections of developmentaleducation.Changes in the numbers of students needingdevelopmental education would also havestaffing implications. In some cases, faculty whowere teaching college-level courses would needto teach developmental education. For somefaculty, this change would represent, Susennotes, “a pedagogical, personal, and culturalmismatch.” While professional developmentcould smooth the transition in some instances,difficult staffing decisions would be inevitable.IT’S NOT ABOUT THE CUT SCORE 9

The system also projected major implicationsfor professional support staff. To help studentsreach the higher standards, the CCCS wouldneed to dramatically decrease the ratio ofstudents to professional counselors and increasethe number of tutors and other support servicesfor struggling students.When you are confronted with huge numbers ofstudents, and you don’t have enough staff to helpthem, it’s hard to help those students besuccessful.The Work Ahead: Setting the Score, ThenSettling the BillThere are still kinks to work out and decisionsto make with regard to placement cut scoreprotocols, but for many the looming question issimple: who pays for all of this? Setting theinitial cut score is only the first step in a longtermprocess of calibrating placement cut scoresat a point that maximizes student success. TheCCCS will be monitoring student success at thecurrent cut scores to inform future adjustments.Additionally, the CCCS does not expect itscolleges to simply squeeze more from theirbudgets to serve more developmental educationstudents; the system will make the case to theGeneral Assembly for additional resources toaddress the fiscal impact of common cut scores.This is a tricky proposition: it will likely drawattention to the number of students in need ofdevelopmental education. While this may not bean easy conversation to have, Susen says thatCCCS officials believe that it is the rightconversation:In the end cut scores will have risen, communitycollege standards will have risen, and it will beeasier for students to transfer because the fouryearswill have more confidence in the content andrigor of gatekeeper courses.North Carolina: A Process ofContinuous CalibrationIn fall 2007, North Carolina community collegesbegan assessing college-readiness proficiencyunder the system’s uniform placement policy,seemingly concluding a process that began in1993 with a mandate from the General Assembly.The process does not end with the establishmentof statewide placement cut scores, however.Instead, it is iterative as North Carolina seeks toset a placement cut score that maximizes studentsuccess through continuous adaptation and refinement.The following description of North Carolina’sexperience setting common placement standardsis based on an interview with Marc Williams,Coordinator of Counseling Services at GuilfordTechnical Community College and former chair ofthe statewide placement committee.BackgroundBefore the mid-1980s, the placement polices forNorth Carolina community colleges were, in thewords of Marc Williams, “hit or miss.” Collegesused departmental exams and other homegrowntests to place students, none of which werenationally normed. Complicating mattersfurther, the colleges—in some cases within closeproximity of one another—offered differentlevels of developmental education. Williamsgives a good sense of the complexity:There may have been 50 different English coursesbeing taught, and 50 different numbers and evenhere at Guilford Tech, we were operating on threelevels, offering an English for vocational students,an English for technical students, and an Englishfor college transfer students. And there’s a lot ofmoving from college to college. Right here inJamestown, there are five other colleges within 30to 40 minutes of each other.By the 1990s, the colleges’ relationships withone another were changing from an associationof relatively autonomous institutions into acomprehensive system offering transfer. Whenthis combined with a move to the semestersystem and standardized curricula across thestate, pressure increased to standardize placementpolicies. In 1993 the North CarolinaGeneral Assembly mandated that the StateBoard for Community Colleges study which10 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

placement tests and cut scores should be usedas standards to determine college readiness andto report findings to the General Assembly inMay 1994.As a first step, the board appointed a statewideplacement committee to guide the state’s efforts.One of its first actions was to survey colleges tofind out their placement assessment practicesand policies. In 1997, the survey was followedby a cut score validation study conducted bynational testing companies, including theEducational Testing Service, ACT, and theCollege Board. This effort culminated inOctober 1999 with a statewide policy requiringcolleges to use cut scores at about the 50thpercentile on ACCUPLACER.Symptomatic of the entire process, that policywas continuously revisited and tweaked. Forexample, the colleges were required to submitcut score-related outcome data for fall 2001-03to ACT, which would validate the cut scores.However, as Williams describes, and againindicative of the challenges inherent in settingplacement policies, particularly cut score policies,there were hurdles in collecting the data.We didn’t really have the data needed to makethe first recommendation in 1999. We finally gotaround to doing the studies using actual gradesof students. We tried to collect as much data aspossible, including providing course grades forstudents for a four-semester period to get a largegroup. Since colleges were doing things differentlyas far as cut scores, there was a little variation inthe data. We had a bit of trouble getting the gradedata to the testing organizations. As a matter offact, we had to do it three times before we got itright.Luckily, the third time’s the charm and ACTcompleted the validation study in February2005. The placement committee met in April toevaluate the validation study outcomes andmake recommendations using the cut scoresestablished in 1999 as a springboard. The validationstudy data were critical, but the placementcommittee also concluded that, inWilliams’s words, “setting cut scores is notpurely scientific—political, social, philosophicaland fiscal issues” are at play. For the committee,the core of the cut score issue was embodied inthe following questions:At what point along the continuum of cut scores isthe greatest number of students who are preparedfor college-level work placed in college-levelcourses? And, at what point along the continuumare students who are not prepared to besuccessful in college-level work without remediationplaced in developmental education?The placement committee took the position thatit was the best use of student and faculty timeand state resources to move the maximumnumber of students capable of college-levelwork into college-level courses. The committee’spractical approach to erring on the side of inclusionwas met with resistance from some facultygroups and from some colleges that argued thatthey were focused on excellence even if itresulted in more students needing remediation.Some were particularly alienated by the placementcommittee’s rationale for where to set thecut score:Our guiding principle was that we should identifythe point where students can reasonably besuccessful and not incur the cost of remediation ifit’s not needed. . . . [L]looking at what the data said. . . if any of the scores could be lower and wecould keep the same . . . probability of successwith students, then we saw no reason not to lowerthe score because . . . [getting as many studentswho are capable of college-level work into collegelevelcourses is] the best use of our taxpayers’money, and the best use of our students’ and ourfaculty’s time.In the end, the placement committee maintainedthat its goal was placing as many students aspossible in college-level courses, while simultaneouslyensuring that students who could notsucceed without developmental educationreceived the support they needed. Somewhatironically, they ultimately recommended cutscores that were similar to those set in 1999.To vet the committee’s recommendations, theyIT’S NOT ABOUT THE CUT SCORE 11

were posted to the Web site of the NorthCarolina Community College System andreviewed by instructional administrators,student services administrators, the Presidents’Association, and others. Finally, the Presidents’Association voted, and the result was that mostcolleges would use the COMPASS and ASSETplacement tests, as well as the existing cut scoresestablished in 1999, while colleges using ACCU-PLACER would adjust their cut scores for themath section.Next Steps: Continuing to Refine and CalibrateIn August 2006, the State Board of CommunityColleges adopted the new, standardized placementpolicy. Although the policy made fewchanges in terms of cut scores, it established anew committee charged with monitoring theimpact of the scores. The committee will workwith the testing companies to conduct validationstudies every three years, and it will advisethe state board on revisions as needed.POLICY RECOMMENDATIONSAdditional research is needed to understand theimpact of common placement standards onsuccess outcomes, but as this brief hasdiscussed, there are many good reasons forembarking on this process. For example, clearand coherent placement policies can help ensurethat colleges assess and place all students asaccurately as possible, thereby decreasing theincidence of students failing to receive the developmentaleducation intervention they need.Clear standards that are common acrosscolleges also make it easier for states to comparethe rates at which students progress throughdevelopmental education. This analysisimproves the ability of states to benchmarkperformance and identify best practices to replicateand scale statewide. Clear placement policiescan translate into clear expectations aboutthe skills that high school graduates will need tosucceed in college, and common practices willreduce confusion among entering students whowant to understand their options. Commonplacement assessment policies will also signal tofour-year institutions that there is consistencyamong community colleges, thereby facilitatingtransfer.The following recommendations—informed bylessons learned by, and best practices in,Achieving the Dream states and other states—describe what states can do to develop anoptimal placement policy environment from theperspective of improving student outcomes incommunity colleges.Rigorously examine the impact of existingplacement policies on student success.The first step in developing common placementpolicies in Virginia, Connecticut, and NorthCarolina was to understand current placementpractices throughout the system. The systemsprofiled here learned of campus practicesthrough internal surveys and by commissioningnational testing companies to perform cut scorevalidation studies. After reviewing the extent ofvariation in practices and policies, states movedforward, in some cases in collaboration with thetesting organization and a system-level placementcommittee.Establish common standards for success incollege-level work.States should convene math and English facultygroups to develop a consensus on the levels ofacademic preparation students need to succeedin entry-level college courses. Optimally, thesestandards would be calibrated with high schoolgraduation requirements and signal readinessfor college and work. However, many stateshave yet to fully align their education systems toa college- and work-ready standard. In these,specifying the assessment instruments that canbe used to place students and establishing aplacement cut score standard for entry-levelcollege work may be an interim step toward fullalignment of state-level high school graduation,college entrance, and college success standards.12 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

Vigorously communicate college entry standardsto high school students and related audiences.States should develop robust communicationsstrategies to inform entering students of thestandards necessary for admittance into collegelevelcourses. Optimally, states would provideearly assessment options, such as placementtests administered to high school students, toindicate if they are on track to enter collegelevelcourses. States should consider options likeCalifornia State University’s Early AssessmentProgram, geared to high school juniors. If theyare not prepared, California students haveadequate time to take courses in the senior year,and the program provides on-line tutorials forstudents who need to raise their skills beforeenrolling in college.Enhance the capacity of data and performancemeasurement systems to track and analyzedevelopmental education outcomes.State performance measurement and accountabilitysystems should include indicators thatcapture placement cut score data, developmentaleducation starting points, and progressrates. Performance measurement systems shoulddisaggregate developmental education outcomesby student subgroups, including students ofcolor, low-income students, English languagelearners, and non-traditional-age collegestudents. This would provide the state with amore fine-grained picture of how studentsprogress through developmental education. Italso would allow states to differentiate policyinterventions to meet the needs of diversegroups of learners. Finally, states should trackoutcomes over the long term and mine longitudinaldata for successful strategies that could beimplemented and scaled across all colleges inthe state.Build consensus in the“quality versus access” debate.opposites certainly oversimplifies a complexdebate. Nevertheless, the hyperbole captures theintensity of the divide between staunch advocatesfor open access and equally staunch advocatesfor high academic standards. Each of thestates profiled here has found a balance toensure that students with a reasonable chance ofsuccess in college-level courses are not placed indevelopmental education and that studentsunable to succeed without developmentaleducation are appropriately placed. Better dataon student outcomes can help as states considerthe spectrum of arguments in the quality-versusaccessdebate. While both positions have merit,a cut score policy that maximizes studentsuccess is likely to fall in the middle.This process is not about the cut score. It is reallyabout developing a common understanding ofwhat college readiness means.Each state profiled here entered into placementpolicy deliberations focused on identifying aspecific cut score or cut score range. In eachcase, the cut scores themselves—while important—werein the end peripheral to the processof defining standards that students must meet toenter college-level courses. A coherent placementassessment policy requires specifyingassessment instruments, cut score standards,and protocols and procedures to be useduniformly.Ultimately, cut scores are the by-product of astatewide process to determine the academicskills students need to begin entry-level collegecourses. While the discussions in each of thefeatured states began with cut scores, theiroutcomes suggest broader systemic issues.Indeed, one of the most important outcomes isthe process of bringing stakeholders together toagree on academic proficiency standards forsuccess in the first year of college.Each of the states featured in this brief ran upagainst two schools of thought during theirdeliberations to set common cut scores: theacademic excellence camp and the access camp.Describing these two perspectives as polarIT’S NOT ABOUT THE CUT SCORE 13

APPENDIX ARESOURCES:Placement Assessment Policies in Achieving the Dream StatesTable 2: Achieving the Dream State Placement Assessment PoliciesPlacement Policy Options Yes NoAre students required totake placement tests?Does the state requirespecific tests?Does the state specify whichstudents are exempt?Is there a commonstatewide placement cutscore?Are students required toenroll in or completedevelopmental educationwithin a specified timeperiod?AR, CT, FL, HI, MA,NC, OH, OK, SC,TX, VAAR, CT, FL, HI, MA,NC,OH, OK,TX, VAAR, CT, FL, HI, MA,NC, OH, OK, TX, VAAR, CT, FL, HI, MA,NC, OH, OK, TX, VAAR, FL, OKMI, NM, PA, WAMI, NM, PA, SC, WAMI, NM, PA, SC,WAMI, NM, PA, SC, WACT, HI, MA, MI, NC,NM, OH, PA, SC, TX,VA, WATable 3: Achieving the Dream States that Require Placement TestsStateArkansasConnecticutFloridaHawaiiMassachusettsNorth CarolinaOhioOklahomaTexasVirginiaState-Required Placement TestsCOMPASS and ASSETACCUPLACERFlorida Entry-Level Placement Test (commonly referred to asCollege Placement Test or CPT)COMPASSACCUPLACERACCUPLACER, ASSET, COMPASS, CPTCOMPASSACTACCUPLACER, ASSET, COMPASS, THEACOMPASSThe challenges of serving large numbers ofstudents who need developmental educationbefore attempting college-level courses are strikinglysimilar across state lines, but states haveimplemented a wide array of policy options forassessing and placing students in developmentaleducation. Table 2 gives an overview of placementassessment policies in the Achieving theDream states, showing a clear delineationbetween those states with more comprehensiveplacement assessment policies and those stateswith few policies in place.These data are self-reported by the states andwere collected from Achieving the Dream statesby JFF via interviews and a review of statutes,administrative rules, and policy manuals in late2007 and early 2008. See Appendix B:Methodology for further information andAppendix C: State Cut Score Charts for detaileddata on cut scores by state and institution.Does the state require students to takeplacement tests? Does the state requirespecific tests?Of the 15 Achieving the Dream states, 11 havestate-level policies in place requiring communitycollege students to take placement tests beforeenrolling in college-level courses. All of thosestates except South Carolina require theircolleges to use specific placement tests. ACCU-PLACER and COMPASS are the primary placementassessment tools, but some states also useASSET.Florida and Texas have their own state-specificplacement tests. Florida uses the CollegePlacement Test (CPT), a computerized andcustomized version of ACCUPLACER. Texasallows students to be placed by the TexasHigher Education Assessment (THEA) in additionto ACCUPLACER and COMPASS.Oklahoma is unique in using the ACT as a “firstcut” assessment to determine if a student is14 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

eady to enroll in college-level classes.Institutions may choose a secondary assessmentfor students who do not pass the “first cut.”ACCUPLACER and COMPASS are among thetests that institutions choose as a “second cut,”but institutions may use any test so long as it isa valid assessment.Does the state specify which studentsare exempt?Ten states specify exemptions or methods ofbypassing their placement assessment policy(Arkansas, Connecticut, Florida, Hawaii,Massachusetts, North Carolina, Ohio,Oklahoma, Texas, Virginia). The most commonexemptions are for students who have earned acollege-ready score on the SAT or ACT. Thestandard scores for SAT and ACT exemptionsvary from state to state depending on collegeentrance standards negotiated within the state.In some cases, students must demonstrate stillhigher levels of proficiency to enter higher-levelcollege courses. In Florida, for example, thecolleges are not required to honor the stateexemption and can require students to demonstrateadditional proficiency before admittingstudents into college-level courses. Texasappears to have the greatest number of exemptionmethods, including earning a college-readyscore on the Texas high school exit exam,enrolling in programs that last less than oneyear, demonstrating previously earned collegereadystatus, or being on active military duty.Does the state require students to enroll indevelopmental education within a specifictime period?While the majority of Achieving the Dreamstates require students to be assessed beforeenrolling in college-level courses, only Arkansas,Florida, and Oklahoma require students tocomplete developmental education courseswithin a specified period. Arkansas requiresreading deficiencies to be remediated in the firstor second semester and in each subsequentsemester until met. Florida requires full-timestudents to begin developmental education intheir first term. Part-time students mustcomplete developmental education within 12semester credit hours, and students who fail todo so must remain continuously enrolled indevelopmental education until their remedialrequirements are met. In both states students areallowed to complete their remediation whileconcurrently enrolled in college-level courses. InOklahoma, students must remove deficiencies incertain subjects before taking college-creditcourses in those subjects.Is there a statewide common cut score?Ten Achieving the Dream states—Arkansas,Connecticut, Florida, Hawaii, Massachusetts,North Carolina, Ohio, Oklahoma, Texas, andVirginia—have set statewide common cutscores. COMPASS and ACCUPLACER are themost commonly used placement assessmentinstruments in these states. Each of these assessmentshas multiple sections, and states mustchoose which sections will be required for placementpurposes.COMPASSCOMPASS offers only one test for readingcomprehension, but states must make choicesfor the writing and mathematics sections. Forwriting, states have the option of assessingsentence skills and/or essay writing. Themajority of states require only the sentenceskills assessment, but a number of statesaugment this with the essay-writing section.For mathematics, states have the option ofspecifying placement standards on pre-algebra,algebra, college algebra, geometry, and/ortrigonometry. Most Achieving the Dream statesonly require students be assessed on the algebraportion of the mathematics assessment. The sixAchieving the Dream states using COMPASSreported the following cut scores, which are thebottom of the range for those states reporting arange:IT’S NOT ABOUT THE CUT SCORE 15

Table 4: Common Cut Scores for COMPASS 4COMPASS AR HI NC OH TX VAReading 82 79 81 69 81 81Writing 75 74 70 — 59 76Math-Algebra 41 50 * 65 39 44*47 on Pre-Algebra sectionTable 5: Common Cut Scores for ACCUPLACER 5ACCUPLACER CT MA NC TXReading 83 68 80 78Writing 88 * 86 80Math-Elementary Algebra 54 82 55 63*Institutions encouraged to assess writing samples on site• Reading: Five states maintain cut scoresbetween 79 and 82; Ohio is the only statewith a lower cut score, at 69.• Writing: Most states require a placement cutscore of at least a 70 to bypass developmentaleducation, though in Texas the threshold islower than the other states, at 59.• Math-Algebra: Cut scores vary widely. Ohio’scut score for math-algebra is the highest, at65; Texas has the lowest, at 39. NorthCarolina’s policy differs from the other statesin that it requires the pre-algebra section ofthe assessment rather than algebra.ACCUPLACERLike COMPASS, ACCUPLACER offers a singlereading assessment, but states have choices onwhich writing and mathematics tests they willuse. Similarly, ACCUPLACER also has twowriting tests: sentence skills and written essay.Most of the states require the sentence skillstest, but, as with COMPASS, a number of statesaugment this assessment with the written essay.In Texas, students who do not meet the cutscore requirement on the sentence skills portionof the assessment may use the essay score todemonstrate readiness for college-level courses.For the mathematics assessment, states have theoption of specifying placement standards onarithmetic, elementary algebra, or college-levelmath. The Achieving the Dream states thatrequire ACCUPLACER reported the followingcut scores (these scores are the bottom of therange for those states reporting a range):• Reading: Cut scores fall within a 15-pointrange, with a high of 83 in Connecticut and alow of 68 in Massachusetts.• Writing: Cut scores fall within a range of 8points, from a high of 88 in Connecticut to alow of 80 in Texas.• Math-Elementary Algebra: Cut scores demonstratethe greatest variation by far, with a 28-point range, from a high of 82 inMassachusetts to a low of 54 in Connecticut.Cut Scores for Other Statewide PlacementAssessmentsFlorida, Oklahoma, and Texas use assessmentsother than COMPASS or ACCUPLACER.• Florida uses a computerized placement test,the College Placement Test (CPT), which is aversion of ACCUPLACER customized forFlorida. Florida sets a single cut score of 83for the reading, math-elementary algebra, andwriting sentence skills sections.• Texas allows institutions to use a state-specificassessment instrument, the Texas HigherEducation Assessment (THEA). Studentsscoring a 230 on reading and math and a 220on writing can avoid further assessment fordevelopmental education.• Oklahoma’s cut score policy is unique amongAchieving the Dream states, using the ACT asa “first cut” placement score. Students whofail to score at least 19 on the math andEnglish sections of the ACT must undergoadditional testing to determine whether developmentaleducation is necessary. Colleges arefree to determine which secondary assessmentsto use to place students who do notpass the state’s first cut.16 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

Does it all matter?The underlying questions in all this research are:What effect do cut score policies have on institutionalbehavior, and what is their impact onstudent success? Is the variation between thelowest and highest cut score smaller in states thatset standard cut scores or ranges? Do cut scorepolicies lead to more coherence and consistency?To answer these questions, we asked communitycolleges in Achieving the Dream states to reporttheir cut scores on the reading, writing, andmath sections of the COMPASS and ACCU-PLACER exams. We identified the median cutscore reported for each test section in each state,and averaged these medians across states withcommon cut scores and states with flexible policies.We used this average median cut score tocompare differences in cut scores across thesegroups. Similarly, we identified the rangebetween the lowest and highest cut scores foreach test section reported by the communitycolleges in each state, and averaged these rangesacross states, comparing the average ranges forstates with common cut scores against theaverage ranges for states with flexible policies.Additional research is needed to draw definitiveconclusions about the impact of common placementassessment policies on the variation in cutscores from college to college within states.With this caveat and the limitations of the datain mind, the preponderance of the data suggeststhat—as one might expect—there is more variationin cut scores in states with flexible cut scorepolicies. This outcome suggests that, absent acommon cut score standard, the expectationsindividual colleges have about the academicskills necessary for success in entry-level coursesoften vary broadly. It is also worth noting that,for both the COMPASS and ACCUPLACERexams, states’ cut scores were most similar onthe reading section and least similar on the mathsection, regardless of which policies were inplace. Several Achieving the Dream states haveattempted to narrow the variation in expectationsand standards of the colleges by systematizingtheir cut score policies; others aremoving in the same direction.COMPASSThe COMPASS cut scores reported byAchieving the Dream states suggest that policiesestablishing common cut scores or rangesnarrow variation in scores across institutions.However, all institutions—whether in stateswith common cut score policies or not—hoveraround a similar median score for both readingand writing.COMPASS Cut Score Range:States with Cut Score Policy vs.States with No Policy4035302520151050Reading Writing Math-AlgebraStates with Cut Score PolicyStates with No PolicyThe existence of common cut score policies doesappear to reduce the variability of cut scores onthe COMPASS test within a state. States withcommon cut score policies reported a narrowerrange of cut scores than states with flexible,institution-set cut scores. This pattern held forthe reading, writing, and math-algebra sections.States with common cut score policies exhibitedaverage ranges of 7, 17, and 20 points on thereading, writing, and math-algebra sections,respectively. In states with flexible policies, theseaverage ranges were considerably larger and inthe case of reading and writing more thandoubled: 20 points for the reading section, 35points for the writing section, and 34 points forthe math-algebra section.However, there were no major differences in themedian cut scores used in states with commonIT’S NOT ABOUT THE CUT SCORE 17

cut scores in comparison to states with flexiblepolicies, except for those in math. For thereading section, both states with common cutscore policies and states with flexible policieshad a median cut score of 80. For the writingsection, states with common cut score policieshad an average median cut score of 72, ascompared with 73 in states with flexible policies.For the math-algebra section, the averagemedian cut score reported by states with flexiblepolicies was 57, somewhat higher than theaverage median cut score of 46 in states withcommon cut score policies (see Table 6).ACCUPLACERVery few colleges in Achieving the Dream stateswith flexible cut score policies use the ACCU-PLACER exam. Among the six states in thisgroup from which we received data, only sixcolleges reported cut scores for any givensection of ACCUPLACER, and only threereported data on all three identified sections.Consequently, it is difficult to make comparisonsbetween states in this group and thoseusing common placement policies. The data areprovided here for reference, but we urge cautionin interpreting the results.Table 6: Median Cut Scores for COMPASS, States with Cut Score Policy vs. States with No PolicyCOMPASS—Cut Score States Count Range MedianArkansasReading 22 2 82.0Writing 22 6 75.0Math - Algebra 22 30 51.5HawaiiReading 7 0 79.0Writing 7 0 74.0Math - Algebra 7 0 50.0North CarolinaReading 58 8 81.0Writing 58 10 70.0Math - Algebra 4 9 41.0OhioReading 12 18 77.5Writing 12 27 75.5Math - Algebra 10 39 51.0TexasReading 35 3 81.0Writing - Sentence Skills 36 46 59.0Math - Algebra 36 32 39.0VirginiaReading 20 12 78.0Writing 19 12 76.0Math - Algebra 18 11 44.0CUT SCORE STATES AVERAGEReading 7.2 79.8Writing 16.8 71.6Math - Algebra 20.2 46.1COMPASS—States with No Policy Count Range MedianMichiganReading 18 38 77.0Writing 17 69 70.0Math - Algebra 17 40 45.0New MexicoReadingWriting 11 8 71.0Math - Algebra 10 15 63.0OklahomaReading 11 12 80.0Writing 11 58 74.0Math - Algebra 10 36 53.0PennsylvaniaReading 4 16 77.5Writing 3 39 70.0Math - Algebra 3 48 44.0South CarolinaReading 9 16 83.0Writing 12 15 75.0Math - Algebra 12 34 66.0WashingtonReading 16 20 81.0Writing 33 21 78.0Math - Algebra 11 31 71.0FLEXIBLE STATES AVERAGEReading 20.4 79.7Writing 35.0 73.0Math - Algebra 34.0 57.0Key:Count=Number of community colleges reportingRange=Range between high and low cut scores used by institutions in the stateMedian=Median cut score for each test section in each state18 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

For the ACCUPLACER exam, there was noevidence that common cut score policies had aneffect on the variability of scores within a state,likely ue in part due to the small sample size.States with policies in place exhibited ranges of16, 9, and 34 points on the reading, writing,and math (algebra) sections, respectively; stateswith flexible policies had ranges of 10, 7, and30 points on these sections, respectively (seeTable 7).There were minimal differences in averagemedian cut scores between these groups. For thereading section, states with common cut scorepolicies had an average median cut score of 78,as compared with 77 in states with flexible policies.For the writing and math sections, stateswith flexible policies had slightly higher mediancut scores: 87 on the writing section and 72 onthe math-algebra section, as opposed to 84 and65, respectively, in states with common cutscores.Tables 7: Median Cut Scores for ACCUPLACER, States with Cut Score Policy vs. States with No PolicyACCUPLACER—Cut Score States Count Range MedianConnecticutReading Comprehension 11 31 81.0Writing - Sentence Skills 10 28 87.0Math - Elementary Algebra 11 36 58.0MassachusettsReading Comprehension 10 28 68.0Writing - Sentence Skills 4 19 74.0Math - Elementary Algebra 5 20 82.0North CarolinaReading Comprehension 55 0 80.0Writing - Sentence Skills 55 0 86.0Math - Elementary Algebra 1 0 57.0OhioReading Comprehension 2 2 82.0Writing - Sentence Skills 2 0 92.0Math - Elementary Algebra 2 69 62.5TexasReading Comprehension 44 18 78.0Writing - Sentence Skills 41 0 80.0Math - Elementary Algebra 43 43 63.0CUT SCORE STATES AVERAGEReading Comprehension 15.8 77.8Writing - Sentence Skills 9.4 83.8Math - Elementary Algebra 33.6 64.5ACCUPLACER—States with No Policy Count Range MedianMichiganReading Comprehension 4 8 75.5Writing - Sentence Skills 1 0 86.0Math - Elementary Algebra 4 43 50.5New MexicoReading Comprehension 5 13 80.0Writing - Sentence Skills 6 6 85.0Math - Elementary Algebra 4 37 88.0OklahomaReading Comprehension 3 2 80.0Writing - Sentence Skills 3 2 80.0Math - Elementary Algebra 3 32 73.0PennsylvaniaReading Comprehension 4 22 64.5Writing - Sentence Skills 2 21 90.5Math - Elementary Algebra 4 27 53.0WashingtonReading Comprehension 2 3 85.5Writing - Sentence Skills 6 8 94.5Math - Elementary Algebra 4 11 93.5FLEXIBLE STATES AVERAGEReading Comprehension 9.6 77.1Writing - Sentence Skills 7.4 87.2Math - Elementary Algebra 30.0 71.6Key:Count=Number of community colleges reportingRange=Range between high and low cut scores used by institutions in the stateMedian=Median cut score for each test section in each stateIT’S NOT ABOUT THE CUT SCORE 19

APPENDIX BMETHODOLOGYJobs for the Future asked Achieving the Dreamstate lead organizations if they kept data on theplacement cut scores for each community college.Connecticut, Florida, New Mexico, Arkansas,Hawaii, and Oklahoma provided matrices of theplacement cut scores used by their communitycolleges. 6 JFF then contacted each communitycollege to verify the data in the matrices.For states without central repositories of placementcut scores, JFF collected data by calling communitycollege assessment directors. In interviews, weasked them for information on the placementassessments used and the minimum cut score thatstudents must earn to be placed into college-levelcourses. Our research questions focused on theACCUPLACER and COMPASS placement assessments,used by the vast majority of communitycolleges. 7 Specifically, we requested the placementcut scores for the following test sections:• ACCUPLACER: Provided by the College Board,ACCUPLACER is a computer-adaptive placementtesting program that uses a student’sperformance on one question to determine thedifficulty of the next question. ACCUPLACER’sgeneral assessments cover reading comprehension,sentence skills, arithmetic, elementaryalgebra, and college-level mathematics. Each testis scored on a 120-point scale. ACCUPLACERalso offers WritePlacer, which electronicallyscores students’ writing samples. Scores on theWritePlacer Plus range from 2 to 12, while scoreson the Texas WritePlacer range from 2 to 8.• COMPASS: Provided by ACT, COMPASS is acomputer-adaptive placement testing programthat uses a student’s performance on one questionto determine the difficulty of the next question.Its placement measures assess mathematics (prealgebra/numericalskills, algebra, college algebra,geometry, and trigonometry), reading, andwriting skills. Each of these tests is scored on a100-point scale. COMPASS also offers e-Write,which electronically scores students’ writingsamples. COMPASS e-Write exams can be scoredon a 2 to 8 scale or 2 to 12 scale.In addition to collecting placement cut score data,JFF collected information on state-level placementcut score policies through interviews with statepolicy officials and a review of statutes, administrativerules, and policy manuals. We conducted interviewsto identify the actions states and/or institutionsrequire when students fall short of minimumplacement cut scores. Specifically, we sought toanswer the following questions:• Are new students required to take assessmenttests for placement into developmental education?• Does the state require specific tests?• Does the state specify which students are exempt(using SAT/ACT) and, if so, who is exempt?• Is there a statewide placement cut score?• Are students required to enroll in or completedevelopmental coursework within a certain timeperiod?JFF conducted follow-up interviews to ensure theaccuracy of the campus-level placement cut scoredata. However, because these data are selfreported,there is room for error; they should beinterpreted with caution. We often found discrepanciesin reported cut scores within institutions andoffices, in large part because many institutions usemultiple assessments for a single subject area.While we conducted multiple rounds of calls tocollect the most accurate data possible, we cannotguarantee that these data are completely accurate.As a supplement to these data, JFF conducted abrief national survey of developmental educationpolicies in spring 2008. This survey consisted ofreviews of published statutes, administrative rules,and policy manuals, as well as interviews withstate-level education administrators. Data weregathered for 31 of the 35 states that do not participatein Achieving the Dream. These data, in combinationwith data gathered from Achieving theDream states, provided information on the policiesand practices in 46 states—a thorough picture ofnationwide trends in developmental educationplacement and assessment policies.20 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

APPENDIX CSTATE POLICIES ON DEVELOPMENTAL EDUCATION IN ACHIEVING THE DREAM STATESSources Does the state require newstudents to be assessed forplacement into developmentaleducation?Does the state requirespecific tests?Does the state specify whichstudents are exempt (usingACT/SAT)? If so, who isexempt?ARKANSAS Arkansas HigherEducation CoordinatingBoard PoliciesYes.All first-time enteringfreshmen at statesupportedcolleges anduniversities must betested in Englishcomposition, reading, andmathematics.Yes.Schools administer theCOMPASS and ASSETexams.Yes.ACT: students with a 19on math may enroll incollege-level math, 19 inEnglish in college-levelEnglish, and 19 inreading are exempt fromdevelopmental reading.SAT: 460 on quantitativefor math, 470 on verbalfor composition andreadingCONNECTICUT Connecticut CommunityColleges Board ofTrustees Policy ManualYes.At minimum, institutionsmust assess incomingstudents in readingcomprehension,sentence skills, andmathematics.Yes.Institutions are instructedto use the ACCUPLACERfor readingcomprehension,sentence skills, andmathematics.Yes.Students scoring 550 orhigher on the mathsection of the SAT areeligible for college-levelmath. Students with aSAT verbal or SAT essayscore of 500 or higherwould be eligible to enrollin ENG 101.FLORIDA Rule 6A-10.0315 ofFlorida AdministrativeCode2007 Florida Statute: TitleXLVIII, Chapter 1008Yes.First-time-in-collegedegree applicants aretested in reading, writing,and mathematics prior tocompleting registration.Yes.Schools administer theFlorida College Entry-Level Placement Test(the CPT). They also mayidentify optionalsupplemental tests forplacement.Yes.The state determineswhich ACT/SAT scoresqualify for exemption, butthe college president maychoose whether or not toaccept the ACT/SAT.HAWAII University of HawaiiCommunity CollegeSystem - COMPASS TestPlacement ScoresMemo: System-wideCOMPASS scoresYes.The University of HawaiiSystem is the publicsystem of highereducation in Hawaii andadministers placementtests in all of itscommunity colleges.Yes.All University of Hawaiicommunity collegesadminister theCOMPASS.Yes.Beginning in fall 2007,SAT/ACT exemptions willbe standardized acrossthe system. Prior to fall'07, decisions were madeat the individual schoollevel.Is there a statewide placementtest cut score?Yes.COMPASS: 41 onalgebra, 75 on writing, 82on readingASSET: 39 onintermediate algebra, 45on writing, 43 on readingYes.ACCUPLACER: 54 onintermediate algebra or40 on college-level math;88 on sentence skills; 83on EnglishcomprehensionYes (but colleges can sethigher scores).CPT: 72 on elementaryalgebra, 83 on readingcomprehension, 83 onsentence skillsYes (though there areslight differences ofinterpretation across theschools).COMPASS: 50 onalgebra, 79 on reading,74 on writingDoes the state requirestudents to enroll in orcomplete developmentalcoursework within a certaintime period?Yes.Reading: 1st or 2ndsemester and eachsubsequent semester untilcompleteMath: before college-levelmath takenEnglish comp: beforebeing awarded credit forfreshman English, butfreshman English can betaken concurrentlyNo.However, a majorconference is planned forNovember 16, after whichthere might be greaterstandardization.Yes.Full-time students (12+credits) must begindevelopmental courses in1st term, part-timestudents beforecompleting 12 credits.Students must completedevelopmental courses bythe time they have 12credits or must maintaincontinuous enrollment indevelopmental courses.Students are permitted totake courses in otherareas concurrently.No.No state policy explicitlystipulates when studentsmust enroll in or completedevelopmental courses.However, system-wide,COMPASS scores have atwo-year shelf-life beforethey become invalid.IT’S NOT ABOUT THE CUT SCORE 21

Sources Does the state require newstudents to be assessed forplacement into developmentaleducation?MASSACHUSETTS Massachusetts Board ofHigher Education:Assessment PoliciesCommon AssessmentreportYes.Institutions are requiredto assess the basicacademic skills ofentering students (MGLChapter 15A Section 32provides authority for thisaction).MICHIGAN None No.Individual institutionsmake these decisions.NEW MEXICO New Mexico Departmentof Higher Education cutscore matrixNo.Individual institutionsmake these decisions.NORTH CAROLINA Validation of PlacementTests ScoresRecommendation -NCCCS (authority fromSL 1993-321, Section108)Yes.Placement testing inreading, writing, andmath is mandatory for allstudents takingcurriculum-level courseswith a developmentalprerequisite.Does the state requirespecific tests?Yes.Institutions areexpected to use theACCUPLACER.No.Individual institutionsmake these decisions.No.Individual institutionsmake these decisions.Yes.Colleges must use one ormore of the followingtests: ASSET,COMPASS, CPT, orACCUPLACER. ThePlacement TestingCommittee must approveother tests.Does the state specify whichstudents are exempt (usingACT/SAT)? If so, who isexempt?Yes.SAT: 500 on the verbalsection exempts studentfrom readingassessment, 600 onverbal exempts studentfrom both reading andwriting; all students areassessed in mathNo.Individual institutionsmake these decisions.No.Individual institutionsmake these decisions.Yes.Colleges may grantexemptions to studentsdemonstrating proficiencythrough ACT or SATscores, but individualcolleges can determinewhich scores to accept.Is there a statewide placementtest cut score?Yes.Cut scores exist for bothmath and reading, butinstitutions areencouraged to assesswriting samples on site(with scoring guidanceprovided by the state).ACCUPLACER: 82 onelementary algebra or 40on college-level math, 68on readingNo.Individual institutionsmake these decisions.No.However, a task force isworking on alignment andcut score issues, sogreater standardization ispossible in the future.Yes (with a few schoolsexempted).COMPASS: 47 on prealgebrafor math, 70 onwriting, and 81 onreading for EnglishcomprehensionASSET: 41 on numericalskills for math, 41 onwriting, and 41 onreading for EnglishcomprehensionACCUPLACER: 55 onarithmetic for math, 86 onsentence skills, and 80on readingcomprehension forEnglish comprehensionDoes the state requirestudents to enroll in orcomplete developmentalcoursework within a certaintime period?No.Board policy states thatinstitutions areencouraged to follow 'bestpractice' in assessmentand placement, including .. . completion ofdevelopmental courseworkduring the spring orsummer prior toregistration, but no statelevelpolicy requiresdevelopmental courses tobe completed in a certaintime period.No.Individual institutions makethese decisions.No.Individual institutions makethese decisions.No.Individual institutions makethese decisions.22 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

Sources Does the state require newstudents to be assessed forplacement into developmentaleducation?OHIO University System ofOhio Strategic Plan forHigher Education, 2008-2017Yes.Institutions use the ACTor SAT to determinereadiness. Students whodo not have a qualifyingscore in Math or Englishon one of these examsmust meet the state cutscore on the COMPASS.OKLAHOMA Oklahoma State Regentsfor Higher EducationPolicy ManualYes.Institutions must use ACTscores as a first cut, andstudents scoring belowminimum ACT scoresmust undergo furthertesting or take developmentalcourses.PENNSYLVANIA None No.Individual institutionsmake these decisions.SOUTH CAROLINA South Carolina StateBoard for Technical andComprehensiveEducation Procedure No.3-2-101.1Yes.Institutions must establishminimum placementcriteria for admission intodevelopmental studieswhich reflect an analysisof students' entry levelskills for each curriculum.Does the state requirespecific tests?Yes.The ACT, SAT, andCOMPASSYes.Oklahoma requires thatall schools accept theACT. In addition to this,institutions are free tochoose their ownsecondary exams forstudents failing to getACT exemptions.No.Individual institutionsmake these decisions.No.Individual institutionsmake these decisions.Does the state specify whichstudents are exempt (usingACT/SAT)? If so, who isexempt?Yes.Students scoring 18 orhigher on the Englishsection of the ACT or 22or higher on the Algebrasection of the ACT areexempt. Equivalent SATscores may also be used.Yes.Students scoring below19 on the math, English,reading, or sciencereasoning sections of theACT must undergoadditional testing or enrollin the correspondingdevelopmental course(s).Institutions may establishhigher standards byrequiring additionaltesting of thosemeeting/exceeding thesescores.No.Individual institutionsmake these decisions.No.Individual institutionsmake these decisions.Is there a statewide placementtest cut score?Yes.Students must score 69or higher on the Englishor 65 or higher on thealgebra section ofCOMPASS to be placedin non-remedial courses.Yes.Institutions must followthe state's ACT scoreminimum for placementtest exemptions.Individual schools canselect their own cutscores for secondaryassessments.No.Individual institutionsmake these decisions.College presidents arediscussing a voluntarymove to a placementtest/cut score policy, but itwill take time toaccomplish if they decideto go in this direction.No.Individual institutionsmake these decisions.Does the state requirestudents to enroll in orcomplete developmentalcoursework within a certaintime period?No.Individual institutions makethese decisions.Yes.Developmental coursesshould be completed asearly as possible andwithin the first 24 collegelevelhours attempted.Students continuouslyenrolled in developmentalcourses may be allowed tocontinue enrollmentbeyond 24-hour limit.Students must removedeficiencies in a subjectbefore taking college-levelwork in that subject.No.Individual institutions makethese decisions.No.Individual institutions makethese decisions.IT’S NOT ABOUT THE CUT SCORE 23

Sources Does the state require newstudents to be assessed forplacement into developmentaleducation?Does the state requirespecific tests?TEXAS Texas Success Initiative Yes.Students must beassessed prior toenrollment or, inexceptionalcircumstances, duringtheir first semester.Schools have discretionin the creation ofacademic plans.Yes.The Texas HigherEducation CoordinatingBoard approves the useof ASSET, COMPASS,ACCUPLACER, and theTexas Higher EducationAssessment.VIRGINIA None Yes. Yes.VA community collegeshave agreed to use theCOMPASS.WASHINGTON State Board forCommunity and TechnicalColleges Policy ManualNo.Individual institutionsmake these decisions.No.Individual institutionsmake these decisions.Does the state specify whichstudents are exempt (usingACT/SAT)? If so, who isexempt?Is there a statewide placementtest cut score?Yes.ACT: composite 23 with aminimum 19 on Englishand/or math exempts onefrom the correspondingsection(s)SAT: combined 1070 witha minimum 500 verbaland/or math exempts onefor correspondingsection(s)Yes (but colleges can sethigher scores).ASSET: 41 reading, 38elementary algebra, 40writing skills, 6 essayCOMPASS: 81 reading,39 algebra, 59 writingskills, 6 essayACCUPLACER: 78reading, 63 elementaryalgebra, 80 sentenceskills, 6 essayTHEA: 230 reading, 230mathematics, 220 writingYes. Yes.Institutions use theseCOMPASS cut scoreranges (some keep theranges, others usespecific scores withinthem):Reading: 81 (76-80decision zone)Writing: 76 (69-75decision zone)Algebra: 44 (39-43decision zone)College Algebra: 41 (36-40 decision zone)No.Individual institutionsmake these decisions.No.Individual institutionsmake these decisions.Does the state requirestudents to enroll in orcomplete developmentalcoursework within a certaintime period?No.Individual institutions makethese decisions.No.Individual institutions makethese decisions.No.Individual institutions makethese decisions.24 ACHIEVING THE DREAM | JOBS FOR THE FUTURE

ENDNOTES1 These data are self-reported and were collected byJobs for the Future via interviews and a review ofstatutes, administrative rules, and policy manuals inlate 2007 and early 2008. See Appendix B:Methodology for further information.2 Learn more about Achieve, Inc.’s AmericanDiploma Project Network at www.achieve.org.3 The “quality vs. access” debate is an ongoingconversation about how to continually widenaccess to postsecondary education while maintaininghigh academic standards.4 These scores are the bottom of the range for thosestates reporting a range.5 These scores are the bottom of the range for thosestates reporting a range.6 Oklahoma uses a placement cut score of 19 on theACT, which is referred to as a “first cut.” Studentswho don’t meet the first cut are subject to furtherplacement testing at the institution level.7 Florida uses a customized version of ACCU-PLACER called the CPT, and Texas institutions canuse the Texas Higher Education Assessment in lieuof another test.REFERENCESAdelman, Cliff. 2006. The Toolbox Revisited:Paths to Degree Completion from High SchoolThrough College. Washington, DC: U.S.Department of Education, Office of Vocationaland Adult Education.Bashford, Joanne & Doug Slater. 2008.Assessing and Improving Student Outcomes:What We Are Learning at Miami Dade College.New York: Community College ResearchCenter, Teachers College Columbia University,Report No. 2 in the Culture of Evidence Series.Parsad, Basmat & Laurie Lewis. 2003.Remedial Education at Degree-GrantingPostsecondary Institutions in Fall 2000. NCES2004-010. Washington, DC: U.S. Department ofEducation, National Center for EducationStatistics.Prince, Heath. 2005. Standardization vs.Flexibility: State Policy Options on PlacementTesting for Developmental Education inCommunity Colleges. Boston, MA: Jobs for theFuture.Shulock, Nancy & Colleen Moore. 2007. Rulesof the Game: How State Policy Creates Barriersto Degree Completion and Impedes StudentSuccess in the California Community Colleges.Sacramento, CA: California State University.IT’S NOT ABOUT THE CUT SCORE 25