Setting a Passing Standard for the Maryland Functional Tests.

More documents

Recommendations

Info

The second step in t.he test development process took place in Mar~h of 1981 when a group of Maryland math~uatics teachers and curriculum specialists used the 3~ obj ectives in ~.he Declared Competemcies Index ~ develop test specifications for an assessment system. Seven different assessmen~ domains were identified and t.he rules for generating test items tm measure each domain carefully delineated. The resulting doculuent, "Test Specifications for t.he Maryland Functional Mathematics Test," Maryland State Depar~ent of Education, Baltimore, NovemDer 1981, was used to guide the production of 546 mathematics test items, 78 items per assessment domain. Cne-half of these items were developed for use in the upper level version of the M/T~T. This upper level form of the measure is given for the first ~/me in t~he ninth grade and is used for graduation certification. The other half of the test items were deve!oped, for use in the lower level version of the MFMT, a seventh-grade examination that serves a diagnostic and instructional function. In t-he Fall of 1981, all of the mathematics test its~s were field tested by representative samples of Maryland s1~x~en~s. A ~al of 24,000 students took par~ in t.his field test, including 9,000 ninth-graders and 3,Z0@ eleventh-graders who also participated in ~.he t_~lout of the upper level MFMT i~ems. Data analyses from t-his field test provided various indices of item quality which were subsequently used to guide the construction of the first operational forms of the MFMT. The field test results also yielded s~udent performance da~a. With an operational test form in place, MSDE now must present information t_hat will help guide the State Superintendent of Schools as he makes this important decision regarding an appropriate passing standard. CRITERIA FOR S T ~ S ~ Establishing t.he passing standard for a certification examined/on can be the most vulnerable operation in the entire development of ~.hat instrument. Since the introduction and widespread use of minimu~ competency tests, the issue of cat-score determination has received a great deal of attention. Two key issues s=~m to emerge in all such considerations. First, st~andard-setting involves political as well as psychometric c~ncerns. To be credible, a standard-setting operation should involve as many different concerned constituencies as is practically possible. The more individuals who participate in the process of recommending a cut-score, the more credibility that process will possess. Second, standard-setting is at heart a judgmental operation. Although there are many different procedures for standard-setting currently in use, all of them rely au some point on human judgment. Judgmentally-based decisions, if not made carefully, will always be subject ~ the criticism t_hat they are arbitrary and inconsistent. Cne strategy for avoiding such charges is to provide t.he individual or group who must make the final decision regarding a cat-score with as much information as possible. Multiple sources of information will, hopefully, allow for the most informed judgment possible.
The standard-setting procedures used for the Maryland Functional Mathematics Tests were designed ~ meet the ~o key criteria of maxim~ participation and mul~/ple ~fpes of information. The question "How well must a student do in order to pass the Mmryland Functional Mathemat/cs Tests?" will ultimately be answered by ~_he State Superintendent of Schools. The decision-maker will be provided wi~h information of three different types; namely, test i ~an-based judgments, test specifications-based judgments, and student performance dana. The following pages elabora=e on the types of informer.ion collected and the results. ~-T~M-KASED ~ u-JEC, MEVI~ On Saturday, June 12, 1982, at three separate sites in Maryland, 61 individuals reviewed the Maryland Functional Mmt~hematics Test (MFMT] , and then offered recommendations regarding an appropriate ~uassing star~ard for ~.hat test. Three categories of standard advisors were represented at t~he workshops, r~mely, ci tizens, mathematics teachers, and school administrators (including coumselors). Ln all, ~here were 28 citizens, "19 mat.hema~/cs teachers, and !Z administrators who served as standard advisors. The three sites were Annapolis (19 participants), Easrmn (28 par~:icipants), and Frederick (14 participants). A substantially larger number of individuals ~d been invited to participate in the workshops via invitations directed toMaryland's 24 dis~ric'~. Reccn'~nendation Cne ~he first recon~nemdations secured at each site were based on individuals' taking an actual form of the test, then answering Yes or No, on an item-by-i~.am basis, the following question: "Should every high school graduate in Maryland be able to answer this item correctly?" Of the 78 items rated, each advisor's total of Yes ratings (that is, the items which the advisor t~hought should be answered correctly by every high school graduate) constituted a reccn~nended standard based on a review of the tes~ items. Resul~s of these initial r--~endatio~-~r~ presen~e~[ ~-n "TaD--~ ~. TABLE I Initial Reco~endations: of the Tesu Items. Suggested Passin~ Standards Based on a Review Citizens (n=35) Teachers (n=14) Aduinistrato rs (n=12) Items 69.9 Percent 89% items Percent 68.9 88% Itzms Percent 69.7 89%
Page 1 and 2: . . . . . . . . ~,~ ~vl~ " ~ ~M ...
Page 3: S~-~ A STANDARD FOR STAND~/%D SETTL
Page 7 and 8: TABLE 3 Third Reco~endations: Suage
Page 9 and 10: Table 2. Means and Standard Deviat/
Page 11 and 12: 3) "Should Persons Who Rave Passed
Page 13 and 14: A ~ F~EC~MMLNDED P.%SSD~ STA~F4~S F
Page 15 and 16: =-~. r.r. 1982 -- MARYZAND FUNCTZ~N

Setting a Passing Standard for the Maryland Functional Tests.

Create successful ePaper yourself

Delete template?

Save as template?