Reviewing Conference Papers - Mark Bernstein

More documents

Recommendations

Info

Conflicts of InterestBecause acceptance or rejection of paperssometimes has significant personal or professionalconsequences, it is important that reviewers avoidconflict of interest.WHAT IS A CONFLICT?You have a conflict of interest if an author of a paperassigned to you is:• an employee of your institution or company(including subsidiaries and other faculties), or abusiness partner• a collaborator or co-author of a book, chapter, orarticle within the past five years• a former doctoral student, supervisor, or directreport• engaged in a professional rivalry or feud with you• has close personal or family ties with youShould you find yourself having a conflict of intereston a paper to which you have been assigned, simplylet me know. We have a large and capablecommittee, and I should have no trouble locating analternate reviewer.Some contributions will be submitted by members ofthe program committee. The committee should takecare to ensure that these papers are neither favorednor disadvantaged by this circumstance.RETURN PAPERS YOU FEEL UNABLE TO READ WITHINTELLIGENCE AND SYMPATHYIn assigning reviews, I’ll do my best to take intoaccount what I know of your interests and yourbackground. If I have erred, and you find yourselfassigned to review a paper that you really cannotread, please let me know and I’ll arrange for adifferent reader.Occasionally, though, I may be seeking your opinionspecifically because a paper is not precisely up youralley.Obligations of aReviewerSYMPATHYDo your best to read papers with care and sympathy.This is easy to do when the paper is good, but is stilldesirable even when the paper is not.Many hours of work — in some cases, years of work —have gone into research and writing this paper. If thepaper is very bad indeed, this committee may be itsonly audience. Do your best to approach it with abroad interest and generous spirit.This does not argue for lenience or laxity, andcertainly we cannot accept bad papers or encouragedefective research.Authors are most apt to listen to your reviews whenthey're thoughtful and constructive rather thancaustic and dismissive.WHEN AN UNSYMPATHETIC COMMENT ISAPPROPRIATEWhile crushing wit and spectacular denunciation area staple of the press critic, they are usually unhelpfulin conference reviewing. An exception might bemade, though, when the reviewer is certain that theauthor is consistently engaged in an erroneouscourse, of which the paper under review is merely thelatest example. Where a mild correction might be tooeasily dismissed, emphasis may attract attention orprovoke second thoughts.It is usually better to attempt this correction throughprivate correspondence or, occasionally, throughpublic discussion — such as a conference panel or aQ&A session. But the anonymity of conferencereviewing is, at times, an important safeguard of theliterature.ANONYMITYReferee reports are anonymous, and in writing themit is best to be courteous but frank, candid anddirect. Reviews are sent to the primary author ofeach paper, but the identity of the reviewer is notdisclosed to authors.A few academic reviewers prefer to sign theirreviews; if you wish to do so, please include asignature in the body of the review.© Copyright 2008 by Mark Bernstein. All Rights Reserved.
In practice, the anonymity of reviews should not betoo heavily relied upon. Although we take care toavoid accidental disclosure, mistakes do happen.More tellingly, authors may often be able to discernthe identity of their reviewers through their interests,concerns, and styles.COMMENTS TO THE COMMITTEEOccasionally, you may have pertinent comments tomake to the committee which you do not wish to bebrought to the attention of the authors. A separatesection of the review form is provided for thispurpose.In general, it is better to confine your comments tothose that can assist the authors in improving theirwork. Some useful comments to the committeemight be:I assume that equation 7, as derived in section 3.6, is correct.To be honest, I don’t understand it, and I’d be morecomfortable, before the paper is accepted, if someone whoremembers tensor analysis could check it.I found the paper unobjectionable but extremely dull; acceptingit will cause no harm, but rejecting it would not cause muchharm, either.It might not be clear from the paper, which seems to have beenadapted from some sort of proposal, that the system has actuallybeen implemented. I have seen it, and it is actually quitepromising; if the paper cannot be salvaged, a demonstrationwould be very rewarding.CONFIDENTIALITYPapers submitted to the conference are confidentialuntil they are published.Do not use results reported in these papers in yourwork until the paper is published, or comes to yourattention in some other way.Rating PapersFor each paper, please provide a a rating and areasonably complete comment. The comment is farmore important than the rating, but choosing theappropriate rating will significantly assist thecommittee.SCORES: THEIR LIMITATIONSSome conferences place considerable weight onnumerical scores and rating. We won't do this; I find itleads to excessive emphasis on small differences.But some ratings have a specific meaning.The top rating is intended for very fine papers --papers that are among the best papers published inthe field, or that contain results that demandimmediate publication.The lowest rating is meant for papers that areactively pernicious. This would include those that aredeceptive, duplicitous, plagiarized, or fraudulent. Itwould also include papers that are simply wrong.In former times (and still in a few conferences today),acceptance was contingent upon public endorsementof the work by a member of the committee. Imagine,if you would that you would be expected to stand atthe side of the stage while the paper was presented.You would rate a paper a "A" if you would beextremely proud to be associated, even in this smallway, with the first presentation of this work; a ”D"would mean that you'd blush to have your friends acolleagues blame you for having to sit through this.Two additional marks are available to you to qualifyyour scores. An “X” represents exceptionalconfidence in your judgment, because the paperdirectly addresses topics of which you are particularlyknowledgeable. A “Z” indicates that you mistrustyour judgment in some degree, because aspects ofthe work lie outside the realms in which you feelconfident.AX: A remarkable result! One of my top students tried andfailed to achieve this synthesis, and the route adopted here isremarkably clever.AZ: The pattern language proposed in the conclusion will beinfluential and widely adopted; I have already begun toreconsider our firm’s management practice. I admit, though,that I cannot follow the mathematics of Section 3.5 and hopethat another reader will check this section with care.DX: Prior to [Clump 96], it was widely thought that wodgetswould deteriorate in the field. That there is no evidence forwodget deterioration is now clear [Dogget 99] [ Egret 01], butwe continue to see it used as a motivation (as here) and in grantapplications.© Copyright 2008 by Mark Bernstein. All Rights Reserved.
Page 1: Reviewing Conference PapersMark Ber
Page 5 and 6: ERRORS OF INTERPRETATIONOne of the
Page 7 and 8: physical concepts. Remember, extern
Page 9: Thanks!Thank you once more for unde

Reviewing Conference Papers - Mark Bernstein

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?