Clinical Trials

❘❙❚■ Chapter 37 | Critical Appraisal of a ReportTable 1. The quality of evidence as defined by the US Public Health Task Force Guide to clinical preventiveservices [15].Quality of evidenceIII.1II.2II.3IIIType of studyProperly designed randomized controlled trialEvidence obtained from well-designed controlled trials without randomizationEvidence obtained from well-designed cohort or case-control studiesEvidence obtained from multiple time series/observational studies, with or withoutintervention. Dramatic results in uncontrolled experiments (such as the results of theintroduction of penicillin treatments in the 1940s) could also be regarded as this typeof evidenceOpinions of respected authorities, based on clinical experience, descriptive studies,case reports, or reports of expert committees1. Where was the report published?Studies published in high-profile journals are likely to be widely read, evaluated,and debated. One formal measure of the influence of a journal is its impact factor.The impact factor is based on how often articles from that journal are referencedby subsequent publications in the 2 years following publication, and the impactfactor of the journals in which the paper is referenced. The theory is that work ofextraordinary merit will be referenced often. Thus, there is considerablecompetition to publish work in high-profile journals. It is generally believed thatwork published in such journals will have been carefully vetted for bias and majorerrors in methodology and design by both the editors and the reviewers invited toevaluate the paper by the journal. The reviewers – who are invariably experts inthe subject being considered – provide excellent input and feedback to authorsto improve their work where needed, or reject work of low quality, and thuspromote a self-perpetuating mechanism for raising the standard of the articlespublished in these journals. The journal Circulation (impact factor 15) receivesabout 600 articles a month, but publishes less than 60 each month.While clinical trials are the most scrutinized and valued reports in termsof clinical evidence, most journals will also publish reviews, hypotheses, andstudies with small numbers of subjects if the work is novel or controversial;therefore, not all space is devoted to randomized trials. Table 1 demonstrates theimportance placed on randomized trials as the best source of evidence for theevaluation of the efficacy of therapies. A letter in the BMJ noted that the impactfactor of top journals dropped as they began to publish more articles such asresearch letters, since the calculation process does not fully account for thesetypes of article [5]. Therefore, the impact factor alone should not be used to assessthe usefulness of an article published in a particular journal.428