Abstracts - Earli

Peer reviewers’ perspectives of their contribution to quality research in education and physicalsciencesYanping Lu, The University of Newcastle, AustraliaEditorial peer review has existed for more than 200 years and achieved universal application.However, although much is determined empirically about editorial process, little is known aboutthe way in which reviewers approach review for journals. There is also concern about theeffectiveness of peer review in improving the quality of manuscripts. Similarly, while there aremany assumptions about differences in various aspects of peer review between disciplines, little isbased on empirical evidence. This paper addresses both issues, with particular attention to themuch discussed ‘differences’ between peer review practice in Education and that in PhysicalSciences. A mail questionnaire was conducted of senior academics (professors, associateprofessors, and some senior lecturers) in Education, Physics and Chemistry in the 37 universitiesin Australia who had served as journal reviewers. A total of 232 academics in roughly equalnumbers by discipline were approached and 54 agreed to be involved. The survey gatheredreviewers’ demographic information, their impression of the effectiveness of peer review, theirperceptions of their contribution as reviewers, etc. The data were analyzed qualitatively with somelevel of quantification to produce a profile of reviewers’ perspectives. Selected findings include:editors and long-serving reviewers thought it was not up to peer review but the author to improvemanuscript quality; reviewers’ expression of the effectiveness of peer review was significantlycorrelated with their own experience with it; the most frequently cited reasons for declining toreview were lack of expertise or time and conflict of interest; reviewers regarded reviewing moreas part of professional obligation than a means to improve quality. There is a lack of compellingevidence to show that there was a difference in reviewers’ perspectives of the role of peer reviewby discipline.Unused potential – On some misconceptions about the value of Dynamic AssessmentJens F. Beckmann, University of Sydney, School of Psychology, AustraliaDynamic Assessment, as an alternative approach to assessing intellectual capacities, focuses onexaminees’ ability to benefit from learning opportunities provided within the assessment process.The level of appreciation of the potential advantages of this assessment concept is not mirrored bythe extent of its utilisation in practice. One reason for this constraint might be that, allegedly, theproponents of this approach have not yet succeeded in showing sufficient evidence for the validityof their assessment tools. The diversity within the field of dynamic assessment, characterised by avariety in the goals pursued, the methods employed, and philosophical perspectives onmeasurement makes it difficult to pass sentence regarding its validity. In the study presented, a setof dynamic tests has been used to exemplify a suggested strategy to evaluate dynamic tests interms of validity. This strategy emphasised: (1) the explication of the construct dynamic tests areaiming at, and its relationship to other constructs; (2) the definition of construct representativeexternal criteria; and (3) the demonstration of both predictive and incremental validity.Furthermore, it was conceptually argued and empirically demonstrated that (4) a variation ofcorrelations between tests and external criteria across different (sub-)samples or studies can beexplained as differential validity of dynamic tests, which is compatible with the conceptualunderstanding of the construct addressed in such learning tests and would be misinterpreted as anindicator of inconsistencies in validity-related findings in dynamic tests.– 729 –