Technical b r Report - International Military Testing Association

More documents

Recommendations

Info

BRIEF CF FCL’R RESEARCH STUD?ES USItX THE SFLF-FVALUATICN TfCHNIQUE (SET STUDIFS) US Army Crdnancc Center and School Doctrine and Training Development Givision hJrFOSc of t.he SET Studleg The purpose of the four SET stxeies was to determine if student stlfevaluations of their performance tests improved student ptrformar.ces on required gerformance tests. Procedure of the SET Studies This section descrihrs the subiects, course requirements, testing it,;trumrnts used. hou the tests were administered, and how the scores vtre obtained. The subjects used in these SFT studies numbered three hundred and fifty-four Ftudencs vho performed tutnty-seven hundred and four performance tests. Tt-ese studtn:s vert enrolled in the L&C20 Vtldtr Course, 61ClO/20 Fire Control instrument Repairman Course, 6LE20 Hachfnist Course. and 63C70 FIJQ~ and Eler’tric Systems Repairman Course. These courses were conducted at the US Army Crc’nancr Centrr and School. Course Recuiremcnt s Gnc of the prereouiFites for each of thQ courses IS as fsllovs: Weltier. General “aintenance (G?!) score of 90, Fire Control Instrument Repairman. General ?!aintenance (G’I) score of 100, Machinist. Central Maintenance (C!f) score of 100. an8 FUPI an+ Electric Systems Repairman, Yotor !reak t?ous several of the performance tests so that some of the require? tasks coulci he tested at a prescribe+ ccsttnp station. A COPY of thic instrument is attache8 as Appendix E. 257
For the evaluation of tht Fire Control Instrument Kepairaan and the Apprtnt ice Hachinlst a separate performance evaluation Instrument was designed to cover each of the performance tests. Copies of there instruments art attathtd as Appendix C and D. <strong>Testing</strong> Procedure Several classes were used to validate. all self-evaluation instrumtnt 5. After the instruments wcrt validated, students who did not make self-evaluations were dtsignattd as the control group and students making self-evaluations were considered to be the experimental group. The SET test project dirtctcr matched each student in the control group vith a comparable student in the txptrimtntal group using the GM or .% scores as a basis for equating these matched pairs. In some instances it was impossible to make comparable matches for each of the matched pairs vher. matching the students by classes. The overall results on the total of matches for each course showed a very close mean for the matched pairs. The experimental groups made a self-evaluation for each of Cht ptrformanct tests they performed. k technical ly qua1 if fed grader evaluated each performance test performed by both the experimental md control groups. ‘rhtst evaluations wert forvardtd to tht SET study project director who scored the self-evaluator’s score sheets. the grader’s evaluation of the self-evaluators’ score sheets and the pradtrs’ evaluations of the non-self-evaluatiqns. The student’s self-evaluation and the grader’s evaluation sheets were a’ttachcd toEtther and returned to the students 4th tither critical or complimentary remarks which were influenced by the scores they made on that performance test. X:0 score was rtcotded on the returned student or prac’er’s evaluation sheets acd the comments reflzc:cd only his weak or strcnp tasks on rht Ftrfornance test. The Fral’ers’ scores for the non-self-evaiuators wtrt recorded by the project 2irectar opposite his match from the self-evaluators. The performance test scores for each of the matched pairs were used to test for a sign! ficant difference between the means of the control and experimental groups. For the purpose of evaluation the SET studies were designed to test for significant tiifierenct bervetn tSe grac’crs’ scores for students uho made se1 f-rvaluat ions and the praders’ scores for students who did not make se1 f-evaluat ions. 258 .
Page 1 and 2:
UNCLASSIFIED,‘UNLIMITEDw “‘$1
Page 3 and 4:
-* . ‘: .;‘*nn. :: ;.... 1. . .
Page 5 and 6:
:: FOREWORD The 15th Annual Confere
Page 7 and 8:
.- . . . _ -._* _.-.,. ..-_. __...,
Page 9 and 10:
JJ Hemisfair Room Ground Floor 0845
Page 11 and 12:
1525 - 1515 1545 - 1600 1600 - 1630
Page 13 and 14:
1330 - 1350 1350 - 1420 1420 - 1435
Page 15 and 16:
1045 - 1100 1100 - 1130 1130 - 1145
Page 17 and 18:
Il 0845 - 0900 0900 - 0920 0920 - 0
Page 19 and 20:
1. II. III. IV. V. VI * VII. VIII:
Page 21 and 22:
XXII. &tics 06 &?&I .:CAdcfl’dl A
Page 23 and 24:
11 XLV. XLVI. XLVI I. XLVIII. XIX.
Page 25 and 26:
!’ For these reasons, then, I wel
Page 27 and 28:
effort, or some other factor, the f
Page 29 and 30:
This means we have to CJoid arbitra
Page 31 and 32:
44 Which means what, specifically?
Page 33 and 34:
-\ * . . Ladies and gentlemen, if w
Page 35 and 36:
. Participants: Synopsis MTA Sympos
Page 37 and 38:
Both recruiters aid counselors lost
Page 39 and 40:
.- -- . rests are timed. 1) e 8. Te
Page 41 and 42:
10. Research needs identified by hi
Page 43 and 44:
DEXEtOPHEX~ OF THE ARNED SERVICES V
Page 45 and 46:
I. ’ 2. 3. 4. 5. 6. 7. 8. 9 .I EX
Page 47 and 48:
Characteristics of the Battery The
Page 49 and 50:
magniture, with the perceptualmotor
Page 51 and 52:
the time, the development of these
Page 53 and 54:
ut rcoding comprehension item leads
Page 55 and 56:
DLL p,m PH~OKITY 1 (DLISDA, p - 0 0
Page 57 and 58:
Introduction * Enlisted Selection a
Page 59 and 60:
u tive to engage in relevant activi
Page 61 and 62:
operational use of the new ACB The
Page 63 and 64:
The quality of new accessions in th
Page 65 and 66:
Ii same job training, and therefore
Page 67 and 68:
I:* . Test.' .'. Ce~wxal 'Ability T
Page 69 and 70:
This is done by raring the speech o
Page 71 and 72:
the areas of electronics, mintenanc
Page 73 and 74:
A great amount of effort is being e
Page 75 and 76:
;Lv EXPERIMEXTAL, iWLTIMEDIr\ CAREE
Page 77 and 78:
(3) \~olumc introductions, chapter
Page 79 and 80:
*ro establish that the three groups
Page 81 and 82:
Discussion In this study, 20th expe
Page 83 and 84:
Optimal Utilization of On-the-Job-T
Page 85 and 86:
in the instrument, but most of the
Page 87 and 88:
Table 2 (Cont’d) ;I Week Producti
Page 89 and 90:
Tables 1 through 6 summarize the mo
Page 91 and 92:
Table 5 - - -Alternative Survey App
Page 93 and 94:
. Phase II of the OJT costing contr
Page 95 and 96:
still primitive, so measurements us
Page 97 and 98:
Air Force Specialty CodelJ 551X6 57
Page 99 and 100:
Table 19 F Ratios for Test Specialt
Page 101 and 102:
_-. . . ._A .-. Table 9 summarizes
Page 103 and 104:
Figure 3 shows how the other test s
Page 105 and 106:
_ I I / REFERENCES f i i Dunham, Al
Page 107 and 108:
FOR OFFICIAL USE ONLY OJT COST SURV
Page 109 and 110:
. . . . . . *. ’ . -. . . *’ .
Page 111 and 112:
Ji PART A 1. How many trainees do y
Page 113 and 114:
Part A (cont’d) i / Q Number \Jl-
Page 115 and 116:
SYMPOSILW TRi.“cLATION OF TRAININ
Page 117 and 118:
This lra~s ;IS to our third set 01
Page 119 and 120:
SYMPOSIUM I! "TRANSIXCION OF TRAINI
Page 121 and 122:
When the report is subnitted, it is
Page 123 and 124:
. additional Army effort to adapt i
Page 125 and 126:
Perh3ps those of vou i:> trnining m
Page 127 and 128:
- ..\ . . i I Co ny dismay I found
Page 129 and 130:
You can see, the block to the right
Page 131 and 132:
108
Page 133 and 134:
. i I / Without cross-validation or
Page 135 and 136:
, , . ., \: ; -* All TRAB projects
Page 137 and 138:
SYMPOSIUM $RANSLATIOII OF TRAINIXG
Page 139 and 140:
Dr. Kerr's major poi::::: are that
Page 141 and 142:
TEST FEEDBACK AND TRdlNINC 0EiJECT:
Page 143 and 144:
areas in which changes can be made-
Page 145 and 146:
All quizzes were given in well ligh
Page 147 and 148:
ESULTS A Kruskal - Wallis one way a
Page 149 and 150:
we know that it does enhance perfor
Page 151 and 152:
Ii 1. 2. 3. 4. 5. 6. BIBLIOGRAPHY C
Page 153 and 154:
-L - - t. -. .‘.. .) -- .’ -a-
Page 155 and 156:
I Knew I edce of -31 Product i s i
Page 157 and 158:
Clerical-technical Al I narsmeters
Page 159 and 160:
I . 2. 3. 0. 1335 1943 1948 1963 9.
Page 161 and 162:
ASSESSMENT SYSTEMS, I CCORPCSATE2
Page 163 and 164:
ASI -1?34567E(3 ,’ . I 140 .- .:
Page 165 and 166:
Ii interested in building personal
Page 167 and 168:
II Enclosure (3.) to CO TRACRN Ltr
Page 169 and 170:
Friday a. Educational opportunities
Page 171 and 172:
partlcularl:I eco-centered ~Jals. U
Page 173 and 174:
- not suggest t3 you ieasibic stud:
Page 175 and 176:
. agree with his own brand of cUn.l
Page 177 and 178:
'. "IMPLICATIONS OF CARREL INSTRUCT
Page 179 and 180:
,.‘j :,;. ‘.: :. .:‘. ,_:1...
Page 181 and 182:
1.’ .; :,: ‘. . :, .,.. >.:..;;
Page 183 and 184:
(S) The carrying hooks... 25 (cj By
Page 185 and 186:
:: . . i: ,:>::. . . / : ,. :... .
Page 187 and 188:
Ji &“l.:~ : i ; r t .,... : : ..,
Page 189 and 190:
: ._ EVALUATE Instructor evaluating
Page 191 and 192:
_ 7.. ;,;,,-;;,T-. :,;,,-;;,T-. ..
Page 193 and 194:
PROPICIE::CY KEXS:J.Scr*.-. """UT I
Page 195 and 196:
generally t!lcse In which the teste
Page 197 and 198:
REFERENCES 1. McClelland, D. C. "Te
Page 199 and 200:
i ILS LANDING WC1 RiL”l 3 6 INITI
Page 201 and 202:
TlIE PERSOXU'NEL ASD TRAA..T"IX:G E
Page 203 and 204:
-_... ..-..-. “._..--_-- --_ . ..
Page 205 and 206:
with a Course Achievement Test. Few
Page 207 and 208:
Y PTEP ACCEPTAXE Uhan first introdu
Page 209 and 210:
. i 2 " : these results could give
Page 211 and 212:
PT&P has made a constructive contri
Page 213 and 214:
0. i ., -, L- .- . * m i! . .‘.c
Page 215 and 216:
and skill requirements for each tec
Page 217 and 218:
numbers of sub-items under each ite
Page 219 and 220:
t - . . *. . .: _. . 196
Page 221 and 222:
. The System Achievement Test, beca
Page 223 and 224:
c . Ccn1raI rcj: me (crs) LA- . Exm
Page 225 and 226:
. . i f : . - ! .-----L - _ _ _ .-
Page 227 and 228:
5. Query subsystem. The query subsy
Page 229 and 230: Looking to the future, we are worki
Page 231 and 232: I .._ Lilt group oi hip:i achlcvc;S
Page 233 and 234: variety of areas related directly 0
Page 235 and 236: tr**is avail,;h:tb for measuring rc
Page 237 and 238: WUork-up” (scoring, interpretat i
Page 239 and 240: , I I . . . -‘- .- -. 216 \
Page 241 and 242: %tLitlS:iCdll)’ significant Gilfc
Page 243 and 244: SUBORDINATE MTINGS: Iv’liP NOT? W
Page 245 and 246: Note: Lt is interesting to note tha
Page 247 and 248: TABLE III Representative Comment:;
Page 249 and 250: LCDR “I rton’t: t~~:l a .PO is
Page 251 and 252: 9 5 18 Effective Naval Officer 106
Page 253 and 254: to you or other parsons in the bran
Page 255 and 256: Subordinate rating% of a sort, have
Page 257 and 258: Thc~ Kecslcr Study - Electronic Tec
Page 259 and 260: II. After approximately thr\bC _vca
Page 261 and 262: . ! . a Appendix A Sample 0: After
Page 263 and 264: Appendix C SXlp IC of Delayed Fit?
Page 265 and 266: 1. GRADUATE’S JOI PROFICIEMCI: bd
Page 267 and 268: 8. In your judgmentwMch of the foUm
Page 269 and 270: -- ----- - _--_- ..__ _____ Append
Page 271 and 272: I Observation: I c Appendix E ; 1 S
Page 273 and 274: performance testing dram in these t
Page 275 and 276: 1: order drill is a good military e
Page 277 and 278: e found in the area of in:&personal
Page 279: in a similar vein, the Amy is now e
Page 283 and 284: adopted for the evaluation. illis s
Page 285 and 286: _ AI’PFKDIX A ! EVALUATION CF STU
Page 287 and 288: Student’s same Class 41 7. 3 . 1.
Page 289 and 290: APPENDIX C SIXTH PERFOR~NCE EXAMINA
Page 291 and 292: 1 LLI:70 ?lar!Iinic: curse I- . - -
Page 293 and 294: 44 INTRODUCTION A new, more specifi
Page 295 and 296: The Enlisted Evaluation Center is t
Page 297 and 298: The mcijor iIrc3J arc wcightcd ncco
Page 299 and 300: - step 1 PERFORM JOB ANALYSIS Ident
Page 301 and 302: . . . ‘* , . . . . . .- . . *. .
Page 303 and 304: -. ..- - .~... _ . _ __, , . . ..-
Page 305 and 306: The Comprchcnsivc Occupational Data
Page 307 and 308: ~hc evaluation plans only cover tho
Page 309 and 310: To appreciate the possible savings
Page 311 and 312: m. study of output measurement cond
Page 313 and 314: jobs. Table 8 defines officer duty
Page 315 and 316: . . It Kas not possible to dcscriI)
Page 317 and 318: RECOMMENDED APPROACHES AND Pl.ANS !
Page 319 and 320: ‘. Nevertheless, there arc severa
Page 321 and 322: JJ intentions, a situation ripe for
Page 323 and 324: ,, - Systems engineering documentat
Page 325 and 326: \ _ . ‘\ . , The test psychologis
Page 327 and 328: and his Harvard colleagues (4) in t
Page 329 and 330: i I / A comparison of experiences a
Page 331 and 332:
.f . 8 .,d *. I .*. \ . ‘,’ *.,
Page 333 and 334:
(SLIDE OFF) ‘f . .‘.J. Gomparis
Page 335 and 336:
! _..“. , (Sl,IDE 11 ON) ’ _’
Page 337 and 338:
means other than the objective type
Page 339 and 340:
Relatively few, including those of
Page 341 and 342:
psychologists who were unfam.iliar
Page 343 and 344:
. Popham’s reason 8 (6e) is shown
Page 345 and 346:
: ) ‘1‘. ‘. ,---. .’ \ : i
Page 347 and 348:
In our experience. some instructors
Page 349 and 350:
t . ’ -.* ., .! as punishment and
Page 351 and 352:
- -.. . bEcause they attempted more
Page 353 and 354:
3b. 3c. 1. 5. 6. Ga. 6b. (iC. 6d. G
Page 355 and 356:
“V’ SYSTEAfS ENGINEERING PROCES
Page 357 and 358:
. . . - ‘-..’ ‘. . . - - A ,
Page 359 and 360:
, 336 , L 0 A u a > - a . . u a* =
Page 361 and 362:
. S:,IDE 7
Page 363 and 364:
-- ‘ . -. ..’ ._:._. . . _ . .
Page 365 and 366:
342 . ,I i / .-. - . _ . 3LIUC 11
Page 367 and 368:
344 U SL:IDE 13
Page 369 and 370:
3 _ . _ , : . _‘, . . 346 SLIDE 1
Page 371 and 372:
. . 348 / I 1
Page 373 and 374:
REA SQW 80 -0 ” WtlILE LOOSE GENE
Page 375 and 376:
.: _._’ _ . . . .:.-*..‘. . - -
Page 377 and 378:
: ’ . . 1 t I ,! . f / \ . SLIDE
Page 379 and 380:
. ‘. . . . I... _ . _ - 1 1 . -0
Page 381 and 382:
, . : CRITERIOX REFERESCED PERFOftW
Page 383 and 384:
fie first is that we start by defin
Page 385 and 386:
In the design of our performance te
Page 387 and 388:
now provide realistic cues which il
Page 389 and 390:
test situation one time will not ne
Page 391 and 392:
The experimental design plan called
Page 393 and 394:
. The Combat Performance sub-portio
Page 395 and 396:
that these are problems of techniqu
Page 397 and 398:
A major corporation gives gold char
Page 399 and 400:
exyar.8 t5is to excmi::e I-.cu her
Page 401 and 402:
Ii A listing of al.1 hoxw addresses
Page 403 and 404:
The dynamic at work in Factor III c
Page 405 and 406:
detailed ar.d comprehensive analysi
Page 407 and 408:
Although numerous compiaints were v
Page 409 and 410:
The various group comparisons made
Page 411 and 412:
REFERENCES Belt, John A., and Parro
Page 413 and 414:
Factor I: Passive alienation/integr
Page 415 and 416:
-. Factor V: Within subculture iden
Page 417:
-I - -. . . . . . ‘. Variable Cum
show all

Technical b r Report - International Military Testing Association

Create successful ePaper yourself

Delete template?

Save as template?