Guide for Developing a Cochrane Protocol - Cochrane Public Health ...

The Cochrane 

Public Health Group 

Guide for developing a Cochrane protocol 

This Guide has been produced by the Cochrane Public Health Group (CPHG) to 

make the process of preparing a protocol for a public health review in Cochrane 

format as clear as possible. The editorial base has developed guidelines that detail 

each section of the protocol and what should be included. These guidelines are a 

distillation of the Cochrane Handbook for Systematic Reviews of Interventions, 

review group policies, recommendations from the CPHG Editorial Team and also 

incorporate the Methodological Standards for the Conduct of Cochrane 

Intervention Reviews, released in November 2011. 

Please refer to The Cochrane Collaboration Training website at 

http://training.cochrane.org/ to access online training modules for Cochrane 

review authors.

Table of contents 

TABLE OF CONTENTS ............................................................................................................................................. - 2 - 

GETTING STARTED................................................................................................................................................. - 3 - 

FILLING OUT THE PROTOCOL TEMPLATE ............................................................................................................... - 4 - 

BACKGROUND ....................................................................................................................................................... - 4 - 

Description of the condition .......................................................................................................................... - 4 - 

Description of the intervention ...................................................................................................................... - 4 - 

How the intervention might work .................................................................................................................. - 4 - 

Why it is important to do this review ............................................................................................................. - 5 - 

OBJECTIVES .......................................................................................................................................................... - 6 - 

Resources ..................................................................................................................................................... - 6 - 

METHODS ............................................................................................................................................................ - 6 - 

Criteria for considering studies for this review ............................................................................................... - 6 - 

Types of studies ............................................................................................................................................ - 6 - 

Types of participants..................................................................................................................................... - 7 - 

Types of interventions ................................................................................................................................... - 7 - 

Types of outcome measures .......................................................................................................................... - 7 - 

SEARCH METHODS ................................................................................................................................................. - 9 - 

DATA COLLECTION AND ANALYSIS ............................................................................................................................. - 10 - 

Selection of studies ..................................................................................................................................... - 10 - 

Data extraction and management .............................................................................................................. - 10 - 

Assessment of risk of bias in included studies .............................................................................................. - 12 - 

Measures of treatment effect ..................................................................................................................... - 13 - 

Unit of analysis issues ................................................................................................................................. - 14 - 

Dealing with missing data ........................................................................................................................... - 14 - 

Assessment of heterogeneity ...................................................................................................................... - 15 - 

Assessment of reporting biases ................................................................................................................... - 15 - 

Data synthesis ............................................................................................................................................ - 16 - 

Subgroup analysis and investigation of heterogeneity ................................................................................. - 17 - 

Sensitivity analysis ...................................................................................................................................... - 17 - 

ACKNOWLEDGEMENTS .......................................................................................................................................... - 17 - 

CONTRIBUTIONS OF AUTHORS ................................................................................................................................. - 18 - 

POTENTIAL CONFLICT OF INTEREST ............................................................................................................................ - 18 - 

REFERENCES - OTHER REFERENCES ........................................................................................................................... - 18 - 

Additional references .................................................................................................................................. - 18 - 

Other published versions of the review ........................................................................................................ - 19 - 

REFERENCES ....................................................................................................................................................... - 20 - 

APPENDIX 1 – GUIDANCE FOR ESTABLISHING A REVIEW ADVISORY GROUP ....................................................... - 21 - 

APPENDIX 2 - PUBLIC HEALTH GROUP EDITORIAL PROCESS ................................................................................ - 24 - 

APPENDIX 3 – RISK OF BIAS ASSESSMENT ........................................................................................................... - 26 - 

APPENDIX 4 – DATA EXTRACTION AND ASSESSMENT FORM ............................................................................... - 29 - 

APPENDIX 5 – CHECKLIST FOR SUBMISSION ........................................................................................................ - 48 - 

Last updated: 24 November 2011 - 2 -

Getting started 

Now that your title has been registered there are several resources you will need to help you to 

complete your protocol. These are listed below and we have provided relevant links throughout 

these documents. All are available at www.cochrane.org and www.ims.cochrane.org. If you have 

problems downloading these files please contact the Review Group Coordinator. You should also 

have received an email notifying you that a user account has now been set up for you in Archie. 

Once you have activated this account you will be able to check out your review. 

1) Cochrane Handbook for Systematic Reviews of Interventions 

The Cochrane Handbook explains the Review process (parts have been summarised in this 

document but you will need to refer to The Cochrane Handbook for more complete guidance) 

(Higgins and Green 2008). The handbook is incorporated in the RevMan 5.1 software and also 

available at http://www.cochrane-handbook.org. This document includes a chapter specific to 

public health reviews (Chapter 21). 

2) RevMan 5.1 

This is the program which you must use to write your protocol and later your review. It is free 

software that you can download easily to your computer. A tutorial is accessible from the help 

menu in RevMan. 

ALL AUTHORS ARE ASKED TO WORK THROUGH THE REVMAN TUTORIAL BEFORE WORKING ON 

YOUR PROTOCOL. 

3) RevMan user guide 

This guide can be accessed as a PDF from the help menu in RevMan and provides a detailed 

description about how to use RevMan. You should continue to refer to this user guide throughout 

the review production process. 

4) Cochrane style guide 

This guide explains the style conventions for a Cochrane review. 

www.cochrane.org/training/authors-mes/cochrane-style-guide 

5) Other resources 

There are a number of other resources available to support you. Many are listed on the Cochrane 

Public Health Group website (http://ph.cochrane.org/resources-and-guidance) 

6) Advisory group 

The CPHG recommends that all review teams establish a Review Advisory Group (RAG) to advise 

them on the scope of their review and its relevance to end-users. One author, usually the lead 

author, should take responsibility for establishing the RAG prior to commencing work on the 

protocol. The CPHG can provide you with assistance in identifying relevant RAG members but 

ultimately it is the author team’s responsibility. Appendix 1 outlines guidance for establishing and 

working with a RAG 


Filling out the protocol template 

This document provides advice on what to include in each of the sections presented to review 

authors in a RevMan protocol template. The headings described below will align with the set 

headings of the protocol template. The advice reflects CPHG editorial policy and following these 

guiding principles should ensure fewer modifications are requested at editorial revision stage. 

See Appendix 2 for an overview of the stages each review goes through prior to publication. You 

should also refer to Chapter 4 of the Cochrane Handbook as you complete the protocol. This 

document does not repeat information in the Cochrane Handbook but rather provides 

supplementary material specific to CPHG requirements. 

Background 

Description of the condition 

This section should describe the condition or issue that the intervention/s under review are 

aiming to address, including information on the historical, (and perhaps political), social, 

economic, geographical and biological perspectives of the problem or issue so as to set a context 

for the review. This will help to establish the rationale for the review and explain the importance 

of the questions being asked. It may be appropriate to change this heading to ‘Description of the 

Issue’. 

Description of the intervention 

Define all terms and interventions clearly and try to set a tone that does not pre-judge the value 

of the intervention (i.e. the likely effectiveness of the intervention/s). Provision of examples of 

interventions and their components here will help the reader gain a better understanding of the 

interventions the authors refer to thereafter in the “Types of Interventions” of the Methods 

section. The complexity of the intervention in seeking to address the problem or issue (especially 

if it is delivered in many different contexts, using different methods and tools or includes many 

interventions to meet the desired outcome/s) needs to be acknowledged, even if the review is 

only looking at a component of the problem or issue. Areas of uncertainty about the intervention 

and issues that may be controversial or the subject of public concern should be highlighted. While 

this section may require you to cover technicalities of the intervention, it is important to write 

this clearly and in plain language to aid reader comprehension. We recommend including 

definitions of key concepts in this section. 

How the intervention might work 

In this section identify the theoretical underpinnings and refer to literature that identifies a 

potential pathway of effect between intervention and outcomes. You might like to include a logic 

model here which shows the connections between determinants of health, interventions and 

outcomes. Logic models can also help to identify the interventions of interest and important 

outcomes, or provide a logical rationale for why only a component of an intervention is being 

reviewed (and point to where other reviews may need to be carried out to complete the 

evidence picture). The following diagram from the published protocol for Community-based 


interventions for enhancing access to and/or consumption of fruits and vegetable access among 

5-18 year olds (Ganann et al 2010) is an example of how a logic model might be constructed. 

Why it is important to do this review 

Clearly describe the justification for doing this review. Cochrane reviews should include a focus 

on exploring uncertainty about effects and/or uncertainty about effects within particular 

subgroups. You should include supporting references to existing and ongoing primary research 

and reviews (including non-Cochrane reviews) of the research topic, to highlight what has been 

learned from past efforts as well as to point out any inconsistencies, methodological strengths 

and weaknesses, and evidence ‘gaps’ that still remain. The contribution of your planned review 

should be emphasised by clearly stating the unresolved questions and controversies that will be 

addressed. To instruct the end-user on the potential application of review findings, include a brief 

statement on how this could inform practice or policy decisions. 

This section should be comprehensive but concise (1-2 paragraphs per subsection). 

Resources 

Cochrane Handbook - Chapters 4 and 5 

The Cochrane Public Health Group Training Handbook (http://ph.cochrane.org/resourcesand-guidance) 

– see How to Ask an Answerable Question 


Objectives 

The objective/s of the review should be clear and specific with a precise statement that covers 

both intervention/s or exposure/s to be reviewed, the population of interest (e.g. whole 

population or particular sub-group/s) and the types of outcome measures of interest. One 

sentence may be enough; however, for some review questions it may be more appropriate to 

break it down into primary and secondary objectives. The intention to identify and report on 

adverse consequences or effects of the intervention/s in the review also should be included in 

this section. 

The objectives should be consistent with the review title. 

Resources 

Methods 

Cochrane Handbook – Chapter 4 

The purpose of the methods sections of a protocol is to describe how the review will be 

conducted and to theoretically allow for the review to be replicated by others. 

Please refer to the Cochrane-Campbell Methods Group Equity Checklist 

(http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist2011.pdf) 

throughout the development of your protocol. This highlights methodological aspects to consider 

at both the protocol and the review stage to help you identify the impact of interventions on 

equity in your review. 

Criteria for considering studies for this review 

This section should make it clear how studies will be selected for inclusion in the analysis. All 

inclusion/exclusion criteria must be explicitly stated within this section, even if mentioned again 

elsewhere throughout your protocol. The protocol needs to reflect your intention to include 

information that will allow the review author, and hence the reader, to judge what factors may 

increase or decrease effectiveness, and under what circumstances/contexts. You do not need to 

provide any text under this heading. Rather complete each of the sections below. Review authors 

should note that if non-randomised studies are to be included detailed descriptions of your 

approach will be needed throughout your protocol. Suggestions are provided throughout this 

guidance document. Note that any changes to eligibility criteria or outcomes studies will need to 

be justified in the review. 

Types of studies 

In this section you need to list the study designs you will include in your review. Whilst you do not 

need to document your study selection in detail, study design/s chosen should be determined by, 

and appropriate for, the intervention and the research question. You should consider the risk of 

bias of each study design and value added to the review. Clear justification for choice of eligible 

study designs must be included. Randomised controlled trials should be included if they are 

feasible for the interventions and outcomes of interest. If an RCT is not able to be used as a study 


design due to ethical or other reasons the review needs to be explicit about which study designs 

are appropriate to include and why. It may be appropriate to include more than one study design, 

even if it is likely that RCTs are available, to increase external validity of the review findings. 

Studies should be included irrespective of their publication status, unless explicitly justified in 

your protocol. 

The CPHG accepts the following study designs as appropriate to the review question: RCTs, 

cluster RCTs, non-randomised controlled studies (including controlled before and after studies 

and interrupted time series studies (with three time points before and after the intervention). 

Whilst there are challenges associated with using study labels, authors are encouraged to refer to 

Box 13.1.a of the Cochrane Handbook to ensure they are clear about the types of study design 

descriptors they might find in the literature. You could consider including a list of definitions of 

your chosen study designs. The focus for eligibility must be on the features of a study’s design, 

rather than the design labels used. You should state in this section if you will be collecting and 

extracting information from qualitative studies emanating from included intervention studies. 

Qualitative research studies can be used in the review to help contextualize the major findings, 

rather than provide causative understandings. The extent to which you search for and include 

qualitative studies will depend on the questions you wish to answer in your review. 

Types of participants 

Clearly describe the population for which the intervention under review is targeted at or likely to 

have an effect on, and clearly specify inclusion and exclusion criteria for your review. These may 

be based on individuals or communities. Examples of defining characteristics may be geographic 

(e.g. where they live or work), demographic (e.g. age, sex) and/or social factors (e.g. education 

level, ‘at-risk’ groups (as specified by the authors). Any restrictions with respect to specific 

population characteristics, settings or factors needs to be justified and reflect information 

presented in the Background section. Many reviews published within the remit of the CPHG will 

have community as their participants. Reviews should provide a definition for community that is 

appropriate for the intervention. It is important to distinguish between ‘community-wide’ and 

‘community-based’ interventions, if these terms are used in your review. 

Types of interventions 

List the intervention/s that will be included in the review, and specify clear inclusion and 

exclusion criteria (e.g. any essential components, minimum duration of the intervention, etc.). 

This section should not describe or define interventions - this detail should be included in the 

Background section (Description of the intervention). This section should therefore be brief and 

outline the types or groups of included interventions, as referred to in the Background section. 

Authors should also include a list of the interventions that will be excluded from the review. 

Include specific examples of relevant interventions/programs of both included and excluded 

studies when possible. If relevant to the inclusion criteria, identify any specific intervention/s 

your intervention/s of interest should be compared against (i.e. what the control group will be). 

Types of outcome measures 

The outcomes you plan to report for your review should be pre-specified in your review to avoid 

bias in reporting your findings. If possible, you should also define in advance the details of what 

will be considered acceptable outcome measures for your review and also how outcome 


measures will be selected when there are several possible measures (e.g. multiple definitions, 

assessors or scales). 

You should include all important outcomes in the protocol, even if you believe there will be 

insufficient data present in the existing research to include in the synthesis. As well as potential 

benefits, you must consider any potential adverse consequences of the intervention and ensure 

that these are addressed in your review. Public health questions often have a large number of 

outcomes of interest – we encourage authors to think carefully about what the main outcomes of 

interest are likely to be for end-users. Once you have a complete list of the outcomes of interest, 

you should identify a short list of main outcomes. The Cochrane Handbook suggests no more 

than seven main outcomes should be chosen, however this should be examined on an individual 

review basis. These outcomes will be used to summarise the key findings of your completed 

review, for example in the abstract and Summary of Findings tables (if you include this in your 

review). For some review questions, it may be appropriate to summarise findings for groups of 

similar outcomes. 

From among the main outcomes, select a small number of primary outcomes (usually 3 or less). 

A primary outcome is of core interest to your review – i.e. the outcome that best answers your 

effectiveness question. You should include at least one undesirable outcome (adverse effect), 

known or hypothesised, with a rationale for why you think these may arise and why they are 

important to include in the review. 

All other main outcomes become secondary outcomes, along with any other additional outcomes 

the review intends to address. These may include unintended outcomes that occur as a result of 

the intervention. These are considered helpful in explaining intervention effects or the integrity 

of the intervention. If you have a large number of secondary outcomes you should consider the 

most appropriate ways of categorising these. If you are planning to include a Summary of 

Findings table in your review (not compulsory but strongly recommended in The Cochrane 

Handbook), you should state the outcomes you will be prioritising for this here. 

For public health reviews, it may often be appropriate to include measures of changes in social 

and environmental determinants of population health, as opposed to or in addition to direct 

health outcomes. For example, changes in children’s purchasing habits (as a result of new 

canteen policies in a school), and reduction in violent behaviour (as a result of adoption of 

responsible drinking policies at a sports club). 

You must clarify in advance whether any outcomes will be used as criteria for including studies 

(rather than as a list of the outcomes of interest within included studies). Outcome measure 

often do not necessarily form part of the criteria for including studies in a review, however some 

reviews do legitimately restrict eligibility to specific outcomes. Furthermore, if relevant to your 

topic, you may wish to specify requirements for the timing of outcome measurement, such as a 

minimum period after the intervention before which an effect would not be detectable (e.g. the 

effect of an intervention in pregnant women on the rate of babies born with low birth weight 

could not be observable for several months after the intervention). 

Resources 

Cochrane Handbook – Chapters 4, 5, 13 and 21 


Chapter 21 in Cochrane Handbook on Reviews in Health Promotion and Public Health 

(www.cochrane-handbook.org) 

The Cochrane Non-Randomised Studies Methods Group has provided information on how 

non-randomised studies should be dealt with in Cochrane reviews (see Chapter 13 in 

Cochrane Handbook (www.cochrane-handbook.org). 

Cochrane Consumers and Communication Group have useful information on study 

designs at : 

www.latrobe.edu.au/cochrane/assets/downloads/StudyQualityGuide050307.pdf ) 

Refer to Cochrane Health Equity website for information on assessing issues relevant to 

equity, along with a checklist for authors that can be used and submitted along with the 

draft protocol (http://equity.cochrane.org) 

Search methods 

Include a description of the search strategy to be used to retrieve studies. You should list the 

sources to be searched (e.g. reference databases, personal contacts, hand searches of journals). 

At a minimum you should search Medline, Embase and CENTRAL. Include a rationale for the 

choice of literature sources (e.g. ERIC for reviews of educational interventions) if possible and the 

years to be covered. It may be useful to refer to your logic model to justify your approach. 

Consider including databases relevant for health equity. Relevant trial/study registers and 

repositories of results must also be included in your search to reduce the risk of publication bias 

and identify ongoing studies. Reference lists of included studies and any relevant systematic 

reviews identified must also be searched and this should be noted in your protocol. 

You should also describe the mechanisms you intend to use or resources that you have available 

to enable you to retrieve potentially relevant documents, especially ones that are unpublished 

and/or written in languages other than English. For example, you may plan to contact relevant 

individuals and organisations for information about unpublished or ongoing studies. You should 

not apply inclusion restrictions based on publication status or language. The CPHG can assist 

authors in identifying potential options for translations if they are required. You should consider 

mechanisms for searching grey literature, for example using databases such as OpenSIGLE. If 

searching restrictions are necessary, a rationale must be provided. 

You should at least make a first attempt at a search strategy (ideally utilising an information 

specialist available to the review author team). The CPHG Trials Search Coordinator, Ruth Turley 

(TurleyRL@cardiff.ac.uk) will provide feedback once you have submitted your protocol for 

editorial review. Your protocol should include (as an appendix) a complete electronic search 

strategy for one database (usually Medline). Consider including terms or concepts in your search 

strategy that are relevant for health equity. Note that generally CPHG review authors will need to 

conduct their own searches. We therefore recommend that an information specialist be recruited 

to the author team. You should outline in your protocol who will conduct the searches. 

Resources 

Cochrane Handbook – Chapters 6 and 21 




Data collection and analysis 

Selection of studies 

This section should outline how results of the various searches will be assessed for inclusion, by 

whom and what you will do if more information is required to determine eligibility (e.g. 

contacting the authors for additional information or finding a process paper associated with the 

study which outlines further information). Specify how many review authors will independently 

review all abstracts and titles and how disputes regarding inclusion will be resolved (there must 

be at least two, and we recommend a third being available to resolve disputes). Specify that all 

irrelevant titles should be excluded and that full-text papers will be obtained where titles are 

deemed to be relevant or where eligibility is unclear. Again, state how many review authors will 

assess these full text papers and how disagreement will be resolved. You should note that you 

will keep a record of reasons for excluding studies. Documentation regarding inclusion decisions 

must be sufficient to complete a PRISMA flow chart and a table of ‘Characteristics of excluded 

studies’. If any statistics will be used to determine inter-rater agreement (e.g. using Cohen's 

kappa) include these details. Please indicate if articles in a language other than English will be 

translated. The reference management software you will be using to manage the records 

retrieved from searches of electronic databases should be named. The most commonly used 

software programs are Endnote and Reference Manager. 

It is common for included studies to refer to a process evaluation or other methodological detail 

that is published elsewhere in a separate paper. These additional papers must be obtained (when 

referred to in the primary paper) and considered as part of the included study. They are likely to 

contain important information needed to understand the implementation of the intervention and 

adequately assess the risk of bias of the study. Multiple reports of the same study must be 

collated, so that each study (rather than each paper) is the unit of interest in the review. 

Data extraction and management 

This section should provide an adequate description of methods used to collect data from 

included studies. You should mention how many people will conduct data extraction and how any 

disagreements will be resolved. It is highly desirable that that study characteristics are extracted 

in duplicate (i.e two review author’s extract data and a third review author be consulted where 

there is disagreement), but not essential. It is essential that outcome data is extracted in 

duplicate. 

You also need to identify the data extraction form/methods you will be using. Appendix 4 

provides a sample data extraction form that you can modify to meet the needs of your review. It 

is recommended that you pilot your data extraction form, to ensure that all participating authors 

are retrieving comparable results, and this should be noted in the protocol. Inclusion of the data 

extraction form that your author team has developed would be useful to include as additional 

information in a figure or additional table. 

In the text of the protocol you should include a brief description of the categories of data you 

intend to collect. In order to provide essential detail for decision makers, information about 

context, implementation factors, equity, cost and sustainability must be collected from included 

studies where available (in addition to data on effectiveness). As mentioned in the above section, 

included studies may refer to a process evaluation or other methodological detail that is 


published elsewhere. These additional papers must be obtained and relevant information 

extracted, as a minimum requirement. In this section of the protocol you should state that you 

will collect this information (i.e. context, implementation, cost, sustainability etc) and what you 

will do with it thereafter. i.e. at a minimum, this information should be reported in the 

Characteristics of included studies table. If you plan to use this information to help synthesise 

your findings (for example, grouping studies by measure of intensity or level of implementation), 

you should state this in your protocol (with the caveat that this would only be done if adequate 

data was available). You may also choose to conduct additional searches for contextual 

information, such as implementation factors, cost and sustainability, beyond that linked with 

included studies. Any plans to do this should be noted in your protocol. 

Within your data extraction form, you should identify studies that report on socio-demographic 

characteristics known to be important from an equity perspective. For this process, use the 

PROGRESS (Place, Race, Occupation, Gender, Religion, Education, Socioeconomic status, Social 

status) framework and, at a minimum, report for each included study which of the eight 

PROGRESS factors were reported for participants at baseline and which were reported at 

endpoint. In addition to the PROGRESS framework, also collect whether or not interventions 

included particular strategies to address diversity or disadvantage. 

We strongly recommend that you incorporate the Cochrane-Campbell Methods Group Equity 

Checklist in your data extraction form and you should note this in your protocol 

(http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist2011.pdf). 

This may help you to identify the impact of interventions on equity later on in your analysis, if this 

is an important consideration for your review. 

In this section you should state that all potential moderators/confounders of study outcomes will 

be included in the extraction form (even if some of these characteristics are not expected to be 

formally tested or discussed in the final review). Consider potential confounders and adjustment 

processes. When extracting data, include this information. 

It can be helpful to refer to The Quality Assessment Tool for Quantitative Studies 

(www.myhamilton.ca/nr/rdonlyres/6b3670ac-8134-4f76-a64c-9c39dbc0f768/0/qatool.pdf), 

developed by The Effective Public Health Practice Project (EPHPP), to be used in conjunction with 

their Quality Assessment Tool for Quantitative Studies Dictionary 

(www.myhamilton.ca/nr/rdonlyres/f5944f3b-15a9-46e7-8afd-1cd67628e33d/0/qadictionary.pdf) 

to check for items that you may wish to consider including as part of your data extraction form. If 

you use items from this tool, you need to state this in your protocol. 

Document how you intend to handle instances in which a single study of effectiveness provides 

data on multiple measures of the same or similar outcomes and you may need to choose what to 

report (e.g. when several variations on an outcome are measured (e.g. weight and BMI) or when 

the same outcome is measured at multiple points in time). An explanation of the criteria used to 

determine whether multiple outcomes from the same or related studies are independent data 

points should be included. 

Authors of primary studies should be contacted where information is missing or clarification is 

needed. You should note this in the later section on Dealing with missing data. You should outline 


how you will attempt to avoid duplicate publication bias and state how multiple reports and if 

publications of the same study will be assembled and compared for duplication, completeness 

and possible contradictions. 

Please note in this section if you will use RevMan to manage data storage and analysis. If you 

intend to use alternative software you must discuss this with the CPHG, and note your intention 

in the protocol. 

Resources 




Refer to Cochrane Health Equity website for information on assessing issues relevant to 

equity, along with a checklist for authors that can be used and submitted along with the 

draft protocol (http://equity.cochrane.org) 

Assessment of risk of bias in included studies 

In this section, you should provide an adequate description of the tool(s) you will use to assess 

the risk of bias of included studies. You should also describe how the tool(s) will be implemented 

and the criteria used to assign studies, for example, to judgements of low risk, high risk and 

unclear risk of bias. The risk of bias assessment must be conducted in duplicate with a clear 

process for resolving disagreements. Supporting information to justify all risk of bias judgements 

must be included in the risk of bias tables. You can consider including the source of the 

information, for example, direct quotes from the study paper. If you are only including 

randomised controlled trials, we recommend that you use the Cochrane Collaboration’s Risk of 

Bias (RoB) tool. This includes selection bias, performance bias, attrition bias, detection bias and 

reporting bias. With regard to the assessment of blinding, we recommend that you consider 

separately the risk of bias due to lack of blinding for (i) participants and study personnel 

(performance bias) and (ii) outcome assessment (detection bias). It is also often appropriate to 

consider the risk of bias due to lack of blinding separately for different types of outcomes. When 

assessing attrition bias, it is recommended to consider the impact of missing data separately for 

different outcomes. 

If you are including non-randomised studies we recommend you use the Effective Practice and 

Organisation of Care (EPOC) RoB Tool for studies with a separate control group instead. This can 

be used for randomised controlled trials as well as controlled before and after studies and other 

nonrandomised designs that include a control group (with the exception of interrupted time 

series studies). This tool includes the standard Cochrane RoB tool items as well as an additional 

item to consider the likelihood of contamination. Importantly for nonrandomised studies, it also 

includes additional items to assess the risk of selection bias and subsequent confounding (“were 

baseline outcome measurements similar?” and “were baseline characteristics similar?”). We 

recommend supplementing this with another additional item, ”did the study authors 

appropriately adjust for important confounders in their analysis?”. 


If you are including ITS studies we recommend that you assess the risk of bias of these studies 

using the EPOC RoB tool for ITS study designs which includes four items from the Cochrane 'Risk 

of bias' tool to assess performance, attrition, detection and reporting bias as well as the following 

additional items relevant for ITS studies: “was the intervention independent of other changes?”, 

“was the shape of the intervention effect pre-specified?” and “was the intervention unlikely to 

affect data collection?”. 

Both of the EPOC RoB tools mentioned above can be found on the EPOC website: 

http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/Suggested%20risk%20o 

f%20bias%20criteria%20for%20EPOC%20reviews.doc as well as guidance for how to prepare a 

RoB 

table: 

http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/How%20to%20prepare 

%20a%20risk%20of%20bias%20table%20for%20reviews%20that%20include%20more%20than%2 

0one%20study%20design.doc. Note that for some items in the RoB tool, for example blinding, it 

may make sense to provide a different assessment for different outcomes (for example outcome 

assessors may be blinded for some outcome measures and not others, or some measures may be 

more objective/subjective than others). Where appropriate, add additional items into the RoB 

table to allow for this. 

Alternative approaches should be discussed and agreed with your contact editor. See Appendix 3 

for general information on Risk of bias tables (referenced from the Cochrane Handbook). This is 

an area of methodological development and any new tools will be available on the CPHG website. 

In this section you also need to describe the method by which you will summarise the risk of bias 

assessments. Rather than at the study level, it is recommended that you do this at the outcome 

level. This is due to the fact that the risk of bias may be different for different outcomes within 

the same study. To do this you can provide an overall risk of bias assessment for the main 

outcomes within each study, then provide an overall risk of bias assessment for relevant 

outcomes across studies, so that outcomes will be judged overall as ‘Low’, ‘Medium’ or ‘High’ risk 

of bias given overall considerations of the study designs, and the potential impact of the 

identified risks noted in the table for each study that contributed results for that outcome. Also, 

consider how you will address risk of bias in the synthesis of your results, considering how 

potential study biases might affect your conclusions. 

Measures of treatment effect 

In this section you should state how outcomes will be reported (e.g. dichotomous data) and how 

you will analyse and compare them (e.g. using Risk Ratios). It is likely that a number of 

quantitative outcome measures may be identified. You need to firstly identity the types of data 

you will obtain from your included study designs. For controlled studies, with continuous data, 

we recommend reporting means or changes in mean scores. Weighted mean difference can also 

be reported for continuous outcomes. Standardised mean differences should be reported when 

different studies use different scales to report the same outcome (e.g. quality of life scales). 

Where a number of outcome measures are identified, authors should use the ratio of means 

method (Friedrich 2008). Dichotomous (or binary) outcomes can expressed as relative risks (RR), 

odds ratio (OR) or risk difference (RD), however the CPHG recommends using RR (Deeks 2002). 

We also recommend using RR for categorical data (e.g. outcomes reported on a short Likert 


scale). 

Unit of analysis issues 

It can be difficult to identify the unit of analysis issues that may emerge from your included 

studies. However, it is important to consider and document in this section what these might be, 

based on your included study designs. 

Special issues in the analysis of studies with non-standard designs, such as cross-over trials and 

cluster-randomised trials, should be described here. Where these studies are not analysed 

appropriately this may result in unit of analysis error, for example where cluster-randomised 

trials are analysed at the individual participant level without taking into account the effect of 

clustering. Effects can be recalculated using the approximately correct analysis or by inflating 

standard errors which is equivalent to calculating new sample sizes. If you plan to include these 

studies you should consult a statistician to identify how unit of analysis errors will be treated, and 

note this in the protocol. 

If you find studies with multiple intervention groups, you will need to consider and document 

how you will deal with multiple groups in your analysis. Report outcomes for all groups of interest 

in your review, however only include groups that meet the eligibility criteria. If you identify more 

than one intervention group of interest, you may need to divide your analysis into pair-wise 

comparisons (e.g. Group A vs control, Group B vs control, Group A vs Group B) and conduct a 

meta-analysis for each comparison, if appropriate to do so. Be careful not to include a group of 

participants twice in the same meta-analysis. One way to include two pair-wise comparisons 

against the same control group in one meta-analysis is to simply halve the number of participants 

in the control group. Document your intentions regarding these issues in this section. 

Resources 

Cochrane Handbook - Chapter 7 and 9 (data extraction and methods of analysis by type 

of outcome) 



Non-standard designs are discussed in detail in Chapters 9 and 16 of the Cochrane 

Handbook for Systematic Reviews of Interventions, including cluster-randomised trials 

(Section 16.3), cross-over trials (Section 16.4), and studies with multiple intervention 

groups (Section 16.5) 

Dealing with missing data 

Strategies for dealing with missing data should be described in this section. This will principally 

include missing information on the methods used in included studies, missing participants due to 

drop-out (and whether an intention-to-treat analysis will be conducted), and missing statistics 

(such as standard deviations or correlation coefficients or where data is reported at baseline but 

is not reported at outcome). It may be necessary for you to contact authors where data are 

missing. Your strategy for contacting authors (e.g. using email addresses on the study’s 

publication or accessing phone directories from author’s documented affiliated organisation) 

should be outlined in the protocol. Section 16.1.2 of the Cochrane Handbook outlines the options 

available to review authors if missing data is not found. Refer to Section 16.1.2 of The Cochrane 


Handbook which outlines the options available to review authors if missing data is not found and 

state in your protocol which option you will use. The option chosen will depend on what is 

practical and whether or not it can be assumed that the data is missing at random. 

Resources 

Cochrane Handbook - Chapter 16 (Sections 16.1 and 16.2) 

Assessment of heterogeneity 

This section should outline how you plan to assess the heterogeneity (or differences) between 

your included studies. It is helpful to consider and refer in this section to methodological 

heterogeneity (e.g. how similar or different your included studies are in terms of study design, 

types of participants, interventions and outcomes), as well as statistical heterogeneity (i.e 

variability in the intervention effects being evaluated in the different studies) which is a 

consequence of methodological heterogeneity. 

If you plan to conduct a meta-analysis you should consider and document how you will assess 

statistical heterogeneity. You should state that this analysis will be used to determine whether it 

is suitable to conduct a meta-analysis or to analyse your studies qualitatively. The CPHG 

recommend that you should use the I 2 statistic to quantify the level of heterogeneity present. 

Review authors may also consider using the Chi square test for heterogeneity (p


Data synthesis 

The protocol should outline the procedures you intend to use to analyse and summarise the 

study results, including whether or not you intend to carry out meta-analyses. This section should 

contain a clear rationale for any choices, considering the potential impact of each choice on the 

outcomes of the review. Meta-analyses should only be undertaken if participants, interventions 

and comparisons are judged to be sufficiently similar to ensure an answer that is meaningful. It 

may be useful to organise this section by study type and/or by type of outcome data (e.g. 

continuous or binary). 

More specifically, if the intention is to carry out a quantitative analysis of results, you should 

outline in the protocol: 

• the software package that will be used to conduct the analyses (this should 

be RevMan and alternatives should be discussed with the CPHG); 

• how statistics describing the overall literature will be presented; 

• why a particular effect size metric is to be used; 

• if adjustments to effect sizes will be made for any reason; 

• techniques to be used to combine results of separate tests; 

• techniques to be used to assess and then analyse variability in findings 

across tests. 

As a default option, the CPHG recommend using the random-effects model to incorporate 

heterogeneity into your meta-analyses (as opposed to the fixed-effect model). The randomeffects 

model allows for a greater level of natural heterogeneity between studies, assuming that 

each study is estimating a unique ‘true’ effect applicable to the time, population and context in 

which the study was conducted. Fixed effects meta-analysis may be included in subsequent 

sensitivity analysis or when significant justification is provided in the protocol. 

For qualitative syntheses (with or without a meta analysis), you need to provide a rationale for 

how you intend to organise studies/findings to arrive at reasonable conclusions and present 

information in a useful way for end-users of the review. For example, you may like to synthesise 

studies grouped by the type of intervention, the length of the intervention, the type of outcome, 

or by study design. It may be difficult to identify what is most appropriate at protocol stage but 

we encourage you to report your anticipated approach or to identify the options for analysis and 

how the decision on synthesis structure will be reached. 

You must include information about how you will summarise the findings of the review. Use the 

five GRADE considerations (study limitations, consistency of effect, imprecision, indirectness and 

publication bias) to assess the quality of the body of evidence for each outcome, and to draw 

conclusions about the quality of the body of evidence within the text of the review. Note that this 

does not mean you have to provide a GRADE rating for the quality of the body of evidence (high, 

moderate, low, very low), however you must consider the five factors mentioned above in your 

review. Be sure to justify and document all assessments so that they can be included in your 

review. These factors must be addressed irrespective of whether your review will include a 

‘Summary of Findings Table’ (SoFT). If you plan to include a formal SoFT in your review, you 

should include this information in this section of your protocol. It is good practice to list the 


outcomes (up to seven) that you plan to include within the SoFT, as well as which comparisons 

and subgroups will be covered, if appropriate. A SoFT should include the number of participants 

and studies for each outcome, summarise the intervention effect and include a measure of the 

quality of the body of evidence addressing the considerations listed above. 

Subgroup analysis and investigation of heterogeneity 

In this section you should list your proposed sub-group analyses, selecting key factors that may 

differ between or within your included studies, and that you anticipate will be associated with 

differences in the effect observed, such as differences in the population, intervention or context 

of the studies. These should be restricted in number with a rationale provided for each. Issues of 

equity may be a key consideration. Include information about any formal statistical tests that will 

be used to compare subgroups, in the event that there are sufficient studies for this to be 

meaningful. It may be possible to undertake a meta-regression. Please note if this is an option for 

your review. 

Resources 


Campbell and Cochrane Equity Methods Group Equity Checklist: 

http://equity.cochrane.org/sites/equity.cochrane.org/files/uploads/equitychecklist.pdf 

Sensitivity analysis 

You should document in this section under what circumstances a sensitivity analysis will be 

undertaken. Sensitivity analysis is often confused with subgroup analysis – although they are 

different. Sensitivity analysis is undertaken to identify whether review findings are dependent on 

the decisions made during the review process, such as about study inclusion/exclusion, the 

selection of data to analyse, the analysis methods used, imputed data, and the impact of studies 

at high risk of bias. Some examples of decisions that may indicate a need for sensitivity analysis 

are listed in The Cochrane Handbook (Chapter 9, section 9.7). Where it is known in advance that 

decisions have been made that may affect the results of the review, sensitivity analysis should be 

planned and noted in the protocol. During the course of the review, other study characteristics 

may also be identified for inclusion in the sensitivity analysis. This is acceptable and should be 

stated as a possibility in the protocol. 

Resources 


Acknowledgements 

Acknowledge any individuals or organisations who may have made a sufficient contribution to 

developing the protocol, (e.g. secretarial support, protocol referees, review advisory group 

members) but are not included in the Contributions of Authors section. Please ensure you have 

obtained permission to name these people in the protocol, otherwise a general acknowledgment 

based on roles (‘our advisory group members’, ‘external peer referees of the protocol’). It is not 

necessary to acknowledge the support of specific people within the CPHG team who have helped 


you in developing the protocol. A broad acknowledgement of the CPHG team is quite adequate if 

you wish to acknowledge the group. 

Contributions of authors 

Identify who has or will perform each of the following tasks (authors’ initials only): 

Draft the protocol 

Study selection 

Extract data from studies 

Enter data into RevMan 

Carry out the analysis 

Interpret the analysis 

Draft the final review 

Disagreement resolution 

Update the review 

Potential conflict of interest 

You should report any conflict of interest of any of the authors, which may influence their 

judgments in conducting the review, at protocol stage (refer to the summary of the 

Collaboration’s policy on conflicts of interest in Chapter 2 (Section 2.6) of the Cochrane Handbook 

for Systematic Reviews of Interventions for more information). This will include any present or 

past affiliations or other involvement in any organisation or entity with an interest in the review 

that might lead to a real or perceived conflict of interest. Areas of uncertainty should always be 

discussed with the CPHG. Situations that might be perceived by others as being capable of 

influencing a review author’s judgements include personal, political, academic and other possible 

conflicts, as well as financial conflicts. Authors must state if they have been involved in a study 

that may be included in the review. 

This section should reflect the information contained in authors’ ‘Declarations of interest 

statement’ within the Disclosure of Potential Conflicts of Interest Forms. You can access these 

forms from within your Revman file by clicking File -> Reports -> Declaration of Interest (you will 

receive a reminder from the editorial office to fill out electronically during the course of 

completing your protocol if you have not done so already). 

If there are no known conflicts of interest, this should be stated explicitly, for example, by writing 

‘None known’. 

References - Other references 

Additional references 

References cited in the text should be listed here. Other reference categories (such as Included 

Studies) are generally not used in a protocol, but will be used in the completed review. 

References are referred to throughout the review using a Reference ID, usually comprising the 

first author’s surname and year of publication (e.g. Smith 2001), and all references must be linked 


from the text of the protocol using this ID. You can enter the references individually, or import 

them from reference management software. 

Instructions on entering, importing and linked references are available from the Help menu in 

RevMan. 

Please run the Citation wizard in Revman, to help identify other Cochrane reviews of potential 

relevance to your review, to cite where appropriate in the protocol (and review thereafter). 

Other published versions of the review 

If the review has previously been published (e.g. in a journal or textbook) it should be listed here. 

Completing our Checklist 

Appendix 5 is a checklist we would like the author team to complete and send back when 

submitting their protocol for editorial review. This will help ensure you have followed our 

guidance and ensure speedier progress through this editorial stage. 


References 

Deeks JJ. Issues in the selection of a summary statistic for meta-analysis of clinical trials with 

binary outcomes. Stat.Med. 21 (11):1575-1600, 2002. 

Evans T, Brown H. Road traffic crashes: operationalizing equity in the context of health sector 

reform. Inj Control Saf Promot. 2003 Mar-Jun;10(1-2):11-2. 

Friedrich JO, Adhikari NKJ, Beyene J. The ratio of means method as an alternative to mean 

differences for analyzing continuous outcome variables in meta-analysis: A simulation study. BMC 

Medical Research Methodology 2008;8(32). 

Ganann R, Fitzpatrick-Lewis D, Ciliska D, Dobbins M, Krishnaratne S, Beyers J, Fieldhouse P, 

Delgado Noguera MF, Gauvin FP, Tort S, Hams SP, Martinez-Zapata MJ, Wolfenden L, Bonfill Cosp 

X, Clay F. Community-based interventions for enhancing access to or consumption of fruit and 

vegetables (or both) among five to 18-year olds (Protocol). Cochrane Database of Systematic 

Reviews 2010, Issue 8. Art. No.: CD008644. DOI: 10.1002/14651858.CD008644. 

Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 

2003; 327(7414):557-60. 

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions 

Version 5.0.0 [updated February 2008]. The Cochrane Collaboration. 2008. Available from 

www.cochrane-handbook.org. 

Kavanagh J, Oliver S, Caird J, Tucker H, Greaves A, Harden A, et al. Inequalities and the mental 

health of young people: a systematic review of secondary school-based cognitive behavioural 

interventions. London: EPPI-Centre, Social Science Research Unit, Institute of Education, 

University of London.; 2009. 

National Health Service: Centre for Reviews and Dissemination. Undertaking systematic reviews 

of research on effectiveness. CRD's guidance for those carrying out or commissioning reviews. 

CRD Report Number 4 (2nd Edition). 2001 [cited 2007 19 September]; Available from: 

http://www.york.ac.uk/inst/crd/report4.htm. 

Rees R, Oliver S, Harden A, Shepherd J, Kavanagh J, Burchett H, et al. Use of an advisory group to 

ensure relevance: reflections on participation of stakeholders in a review of sexual health 

promotion for men who have sex with men (MSM), in XII Cochrane Colloquium. 2004: Ottawa, 

Canada. 

Thomas BH, Ciliska D, Dobbins M, Micucci S. A process for systematically reviewing the literature: 

providing the research evidence for public health nursing interventions. Worldviews Evid Based 

Nurs, 2004. 1(3): p. 176-84. 

Thompson SG and Pocock SJ. Can meta-analysis be trusted. Lancet 1991 Nov 2;338(8775):1127- 

30. 


Appendix 1 – Guidance for establishing a Review Advisory 

Group 

The Cochrane Handbook suggests the establishment of Review Advisory Groups to help reviewers 

outline the parameters of their review. The establishment and management of an advisory group 

can be difficult and potentially time-consuming but it is an important component in the initial 

stages of review development and it can help establish appropriate review parameters so that 

the end product reflects the needs of end users. 

Systematic reviews are likely to be more relevant to the end user and of higher quality if they are 

informed by advice from people with a range of experiences, in terms of both the topic and the 

methodology (Rees 2004, Thomas 2004, NHS CRD 2001). 

This guidance has been developed by the Cochrane Health Promotion and Public Health Field. 

What is the role of a review advisory group? 

Review Advisory Groups are established to help reviewers outline the parameters of their 

proposed review to ensure that the end product reflects the needs of its potential readers and 

users. Systematic reviews are likely to be more relevant to the end user and of higher quality if 

they are informed by advice from people with a range of experiences. 

Examples of opinions sought from the review advisory group: 

Does the review question seem to capture the essence of the topic under review (will it 

sound interesting and useful to its target audience)? 

What interventions should be included in the review? 

Which populations should be included in the review and which should be excluded? 

What types of outcomes should the review include? 

How should equity issues be highlighted in the review? 

Are the needs of developing countries considered in the review? 

How do I establish an advisory group? 

The Cochrane entities with whom you are involved, especially Fields 

(www.cochrane.org/contact/entities.htm#FIELDLIST), may be a useful source of advisory group 

members. Many entities keep a database of members tagged with skills, special interests and 

expertise. You should email relevant entity contacts and request for them to contact relevant 

members. 

Alternately, it may be appropriate for you to make contact directly with potential members with 

known interest or expertise in the topic under review. It is important that you have set up tasks 

or terms of reference for your advisory group prior to making contact with potential members. 

This will ensure that roles and responsibilities are known from the outset. 

Who should the members of the advisory group be? 

Advisory groups are not intended to be another layer of peer-review. Nor are they intended to 

assist with technical review support (e.g. statistical expertise). If you require this level of 

assistance you should contact your review group or consider recruiting an additional reviewer 


with these skills to assist you. Advisory Group members should only be used to provide contentrelated 

support, highlighting what end users of the review will want to have included in the 

review. They may be able to direct you to additional studies and/or to provide background 

information on the topic, particularly within the context of their local situation. This latter point 

is a good reason why membership of the advisory group should be inclusive of people from 

different parts of the world, to ensure the end review has relevance globally. . Experience by 

Effective Public Health Practice Project in Canada suggests 6 members is an appropriate size for 

the advisory group. 

The members of your advisory group will vary depending on your review question. However, it 

may be useful to consider members in the following categories: 

Consumers (those with whom the intervention/s under review are targeted) 

Content experts 

Policy-makers 

Practitioners (those implementing the intervention/s under review) 

Who is responsible for coordinating the advisory group? 

The lead author should take primary responsibility for coordinating the advisory group and 

establishing a communication strategy that is acceptable to all (and reflective of the resources 

available to the review team) . Contact should be made via the lead author in the first instance. 

Lead authors should cc all authors into correspondence with advisory group members. It may also 

be appropriate to cc others in (e.g. any Cochrane entities with whom you are involved). 

What information does the lead author need to provide prospective advisory group members 

with? 

Potential members should be provided with adequate details about the review (title, authors 

etc), preferably before the title is registered with a CRG. 

They also may need information about the Cochrane Collaboration as all members may not be 

familiar with Cochrane and the review process. Call for advisory group members should include a 

clearly defined role, remits and boundaries (potentially a terms of reference document) and 

timeline of tasks. 

What processes need to be established for the advisory group to work effectively? 

To ensure that your advisory group works effectively it is important that you establish roles and 

responsibilities (You may want to formalise this in a terms of reference document). This will 

ensure that authors and advisory group members are clear about the role of the advisory group. 

Again, processes may differ although you should consider the following: 

What is the role of each advisory group member (for example, will each answer concerns 

about their area of expertise or experience only or the whole review)? 

What tasks do you want them to complete? 

What method of communication will be used and how frequently will the advisory group 

members be consulted? 

What workload is involved? 

Are there timelines that need to be considered? 

When does the work of the advisory group end (once the parameters for the protocol 

have been accepted by the registering CRG?) 


How will advice be managed and what will happen if conflicting advice (or that quite 

contrary to the reviewer’s beliefs) is offered? 


Appendix 2 - Public Health Group editorial process 

Registration of title 

3-6 

months 

Integration into 

Public Health Group module 

Submission of 

Protocol 

(editorial 

base) 

6-12 

months 

Submission of 

completed 

review 

(editorial base) 

4 weeks 

Note: 

Protocol/review 

may undergo 2-3 

drafts/reviews 

before 

acceptance for 

publication 

6-8 weeks 

Review andcomments 

Review and comments 

Publication in 

Cochrane Library 

The overall process of review development can be seen below: 

On completion of the draft protocol it should be submitted to the Managing Editor who then 

organises for an internal review of the protocol by your assigned Contact editor, methods 

advisor, statistical editor, the trial search coordinator and, if appropriate, the CPHG’s developing 

countries editorial consultant. All the comments are compiled and sent to the author. Prompt 

reply and amendment will ensure timely progression to the next stage – external review. 

Generally 3-4 external referees are recruited (composing of content experts,potential end users 

and if appropriate and possible, a consumer). Once these referees have agreed to referee the 

protocol the relevant documents are sent and there is an understanding that the comments 

should be returned to the CPHG within two weeks. 

All the comments are compiled by the Contact editor (with guiding comments added) and sent to 

the author. Again, prompt reply and amendment will ensure quicker inclusion into The Cochrane 

Library. On resubmission, the Contact editor and the Coordinating Editor will then review the 

changes. Once the changes have been approved, the protocol will be sent to Wiley Publishes for 

copy editing and thereafter the protocol marked by the Managing Editor for publication. 

Thereafter all authors will be sent a Permission to Publish Form, along with the final version of 

the protocol for approval, to sign. The protocol can NOT be published until this form is signed 

(electronically) by ALL authors (therefore all authors need to be available to do so before 

publication deadline). 


The same editorial processes apply for review development and publication. 


Appendix 3 – Risk of bias assessment 

See Table 8.5.c (Criteria for judging risk of bias in the ‘Risk of bias’ assessment tool) in Cochrane Handbook 

for Systematic Reviews of Interventions. 

Extract from Cochrane EPOC Data Collection Checklist for assessing risk of bias in controlled studies and 

ITS 

Source: 

http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/Suggested%20risk%20o 

f%20bias%20criteria%20for%20EPOC%20reviews.doc 

6.4.1 Risk of bias for studies with a separate control group (RCTs, CCTs, CBAs) 

Nine standard criteria are used for all RCTs, CCTs and CBAs. Further information can be obtained from the Cochrane 

handbook section on Risk of Bias and from the draft methods paper on risk of bias under the EPOC specific resources 

section of the EPOC website. 

Was the allocation sequence adequately generated? 

Score “Yes” if a random component in the sequence generation process is described (eg Referring to a random number table). 

Score ”No” when a nonrandom method is used (eg performed by date of admission). CCTs and CBAs should be scored “No”. Score 

“unclear” if not specified in the paper. 

Was the allocation adequately concealed? 

Score “Yes” if the unit of allocation was by institution, team or professional and allocation was performed on all units at the start 

of the study; or if the unit of allocation was by patient or episode of care and there was some form of centralised randomisation 

scheme, an on-site computer system or sealed opaque envelopes were used. CBAs should be scored “No”. Score “unclear” if not 

specified in the paper. 

Were baseline outcome measurements similar?* 

Score “Yes” if performance or patient outcomes were measured prior to the intervention, and no important differences were 

present across study groups. In RCTs, score “Yes” if imbalanced but appropriate adjusted analysis was performed (e.g. Analysis of 

covariance). Score “No” if important differences were present and not adjusted for in analysis.** If RCTs have no baseline 

measure of outcome, score “Unclear”.** 

Were baseline characteristics similar? 

Score “Yes” if baseline characteristics of the study and control providers are reported and similar. Score “Unclear” if it is not 

clear in the paper (e.g. characteristics are mentioned in text but no data were presented). Score “No” if there is no report of 

characteristics in text or tables or if there are differences between control and intervention providers. Note that in some cases 

imbalance in patient characteristics may be due to recruitment bias whereby the provider was responsible for recruiting patients 

into the trial. 

Were incomplete outcome data adequately addressed?* 

Score “Yes” if missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the 

intervention and control groups or the proportion of missing data was less than the effect size i.e. unlikely to overturn the study 

result). Score “No” if missing outcome data was likely to bias the results. Score “Unclear” if not specified in the paper (Do not 

assume 100% follow up unless stated explicitly). 

Was knowledge of the allocated interventions adequately prevented during the study? * 

Score “Yes” if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcomes are 

objective, e.g. length of hospital stay. Primary outcomes are those variables that correspond to the primary hypothesis or 

question as defined by the authors. Score “No” if the outcomes were not assessed blindly. Score “unclear” if not specified in the 

paper. 


Was the study adequately protected against contamination? 

Score “Yes” if allocation was by community, institution or practice and it is unlikely that the control group received the 

intervention. Score “No” if it is likely that the control group received the intervention (e.g. if patients rather than professionals 

were randomised). Score “unclear” if professionals were allocated within a clinic or practice and it is possible that communication 

between intervention and control professionals could have occurred (e.g. physicians within practices were allocated to 

intervention or control) 

Was the study free from selective outcome reporting? 

Score “Yes” if there is no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are 

reported in the results section). Score “No” if some important outcomes are subsequently omitted from the results. Score 


Was the study free from other risks of bias? 

Score “Yes” if there is no evidence of other risk of biases 

* If some primary outcomes were imbalanced at baseline, assessed blindly or affected by missing data and others were not, each 

primary outcome can be scored separately. 

**If “UNCLEAR” or “No”, but there is sufficient data in the paper to do an adjusted analysis (e.g. Baseline adjustment analysis or 

Intention to treat analysis) the criteria should be re scored to “Yes”. 

6.4.2 Risk of bias for interrupted time series studies 

Seven standard criteria are used for all ITS studies. Further information can be obtained from the Cochrane 

handbook section on Risk of Bias and from the draft methods paper on risk of bias under the EPOC specific resources 

section of the EPOC website. 

Note: If the ITS study has ignored secular (trend) changes and performed a simple t-test of the pre versus post 

intervention periods without further justification, the study should not be included in the review unless reanalysis is 

possible. 

Was the intervention independent of other changes? 

Score “Yes” if there are compelling arguments that the intervention occurred independently of other changes over time and the 

outcome was not influenced by other confounding variables/historic events during study period. If Events/variables identified, 

note what they are. Score “NO” if reported that intervention was not independent of other changes in time. 

Was the shape of the intervention effect pre-specified? 

Score ”Yes” if point of analysis is the point of intervention OR a rational explanation for the shape of intervention effect was given 

by the author(s). Where appropriate, this should include an explanation if the point of analysis is NOT the point of 

intervention;Score “No” if it is clear that the condition above is not met 

Was the intervention unlikely to affect data collection? 

Score “Yes” if reported that intervention itself was unlikely to affect data collection (for example, sources and methods of data 

collection were the same before and after the intervention); Score “No” if the intervention itself was likely to affect data 

collection (for example, any change in source or method of data collection reported). 

Was knowledge of the allocated interventions adequately prevented during the study?*** 

Score “Yes” if the authors state explicitly that the primary outcome variables were assessed blindly, or the outcomes are 

objective, e.g. length of hospital stay. Primary outcomes are those variables that correspond to the primary hypothesis or 

question as defined by the authors. Score “No” if the outcomes were not assessed blindly. Score “unclear” if not specified in the 

paper. 

Were incomplete outcome data adequately addressed?*** 


Score “Yes” if missing outcome measures were unlikely to bias the results (e.g. the proportion of missing data was similar in the 

pre- and post-intervention periods or the proportion of missing data was less than the effect size i.e. unlikely to overturn the 

study result). Score “No” if missing outcome data was likely to bias the results. Score “Unclear” if not specified in the paper (Do 

not assume 100% follow up unless stated explicitly). 

Was the study free from selective outcome reporting? 

Score “Yes” if there is no evidence that outcomes were selectively reported (e.g. all relevant outcomes in the methods section are 

reported in the results section). Score “No” if some important outcomes are subsequently omitted from the results. Score 


Was the study free from other risks of bias? 

Score “Yes” if there is no evidence of other risk of biases. 

e.g. should consider if seasonality is an issue (i.e. if January to June comprises the pre-intervention period and july to December 

the post, could the “seasons’ have caused a spurious effect). 

*** If some primary outcomes were assessed blindly or affected by missing data and others were not, each primary outcome can 

be scored separately. 


Appendix 4 – Data Extraction and Assessment Form 

This form is to be modified in keeping with the following instructions. Some questions may be 

changed from open-ended questions to specific data items where appropriate. Refer to the 

Cochrane Handbook when undertaking modifications to this form. 

Sections can be expanded and irrelevant sections can be removed. It is difficult to design a single 

form that meets the needs of all reviews. It is therefore important that you consider your needs 

carefully prior to data extraction and pilot your process. Elements within the template are not 

intended for use as a scoring system, but rather suggests elements which should be addressed in 

the Table of Included Studies and Comparisons and Data in RevMan. The components of the Risk 

of Bias Table have been incorporated into this form. If you are using an additional quality 

assessment tool you will need to add appropriate questions to reflect the additional components. 

Notes on using a data extraction form: 

Pilot the Data Extraction Form you plan on using (and note in your protocol that it will, or has, 

been piloted) 

Be consistent in the order and style you use to describe the information. This will make it 

easier to complete the Table of Included Studies, prevent you from overlooking information 

and make reading of the review easier. 

Highlight any missing information as unclear or not described, to make it clear to the reader 

of your review that the information was not included in the description of the study, not that 

you forgot to extract it. 

You should include instructions and decision rules on the data collection form. It is crucial that 

you practice using the form and receive, or give, training if the form was designed by 

someone other than the person using it. 

Assessment of Risk of Bias has been adapted from Cochrane Handbook Table 8.5.a: The 

Cochrane Collaboration’s tool for assessing risk of bias. 

http://epocoslo.cochrane.org/sites/epocoslo.cochrane.org/files/uploads/How%20to%20prepare 

%20a%20risk%20of%20bias%20table%20for%20reviews%20that%20include%20more%20than%2 

0one%20study%20design.doc. Criteria for judging risk of bias as well as examples of appropriate 

methods of addressing each form of bias are provided in Chapter 8 of the Cochrane Handbook, 

particularly Table 8.5.c. For tips on how to enter data into RevMan 5, see “Risk of Bias” tables in 

the RevMan User Guide. 


Data Extraction and Assessment Form – template (cut and paste and modify to suit your 

review) 

Study ID: Report ID : Date form completed: 

First author: Year of study: Data extractor: 

Citation: 

1. General Information 

Publication type 

Journal Article Abstract Other (specify e.g. book chapter)___________________ 

Country of study: 

Funding source of study: 

Potential conflict of interest from funding? Y / N / unclear 

2. Study Eligibility 

Study Characteristics 

Page/ 

Para/ 

Figure # 

Type of study 

(Review authors 

to add/remove 

designs based on 

criteria specified 

in protocol) 

Randomised Controlled Trial (RCT) 

Cluster Randomised Controlled Trial 

(cluster RCT) 

Interrupted Time Series (ITS) 

At least 1 time point before 

and 1 after the intervention 

Clearly defined intervention 

point 

A process evaluation of an included 

study design 

Description in text: 

Controlled Before and After (CBA) study 

Contemporaneous data collection 

Comparable control site 

At least 2 x intervention and 2 x 

control clusters 

Other design (specify): 

Does the study design meet the criteria for 

inclusion? 

Yes 

No Exclude Unclear 


Participants 


insert inclusion 

criteria as 

defined in 

Protocol) 

Describe the participants included: 

Are participants defined as a group 

having specific social or cultural 

characteristics? 

How is the geographic boundary 

defined? 

Do the participants meet the criteria 

for inclusion? 

Yes No Unclear 

Details: 

Details: 

Specific location (e.g. state / country): 

Yes No Exclude Unclear 

Types of 

intervention 



criteria as 

defined in 


Duration of 

intervention 

Types of 

outcome 

measures 



criteria as 

defined in 


Strategies included in the 

intervention 

Focus of the intervention 

Does the intervention meet the 

criteria for inclusion? 


Start date: Stop date: Intervention duration: 

Is the duration of intervention 

adequate for inclusion? 

List outcomes: 

Outcome measured at a population 

level or individual level? 

Do the outcome measures meet the 

criteria for inclusion? 


Details: 



Summary of Assessment for Inclusion 

Include in review 

Exclude from review 

Independently assessed, and then compared? 

No 

Yes 

Differences resolved 

Yes No 

Request further details? Yes No Contact details of authors: 

Notes: 

DO NOT PROCEED IF PAPER EXCLUDED FROM REVIEW 

3. Study details 

Study intention Descriptions as stated in the report/paper Page/ 

Para/ 

Figure # 

Aim of intervention What was the problem that this intervention was designed to address? 

Aim of study 

What was the study designed to assess? Are these clearly stated? 

Equity pointer: 

Social context of 

the study 

Start and end date 

of the study 

Total study 

duration 

e.g. was study conducted in a particular setting that might target/exclude specific 

population s? See also Inclusion/exclusion criteria under Methods, below. 

Identify which elements of planning of the intervention should be included 

Methods Descriptions as stated in the report/paper Page/ 

Para/ 

Figure # 

Method/s of recruitment of participants 

(How were potential participants approached and 

invited to participate? Where were participants 

recruited from? Does this differ from the intervention 

setting?) 

Inclusion/exclusion criteria for participation in study 

Representativeness of sample: Are participants in the 

study likely to be representative of the target 

population? 

Total number of intervention groups 

Assumed risk estimate 

(e. .baseline or population risk noted in Background) 

Sample size calculation: 

What assumptions were made? 

Were these assumptions appropriate? 

References: 

(Yes/No/Unclear) 


What was the unit of randomisation? 

Allocation by individuals or cluster/groups 

What was the unit of analysis? 

Is this the same as the unit of randomisation? 

Statistical methods used and appropriateness of 

these methods 

(Yes/No/Unclear) 

(Check with your statistician if unsure about 

appropriateness) 

Results 

Participants 

Include if relevant 

Include information for each group (i.e. intervention and controls) 

under study 

Page/ 

Para/ 

Figure # 

What percentage of selected 

individuals agreed to 

participate? 

Total number randomised (or 

total pop. at start of study for 

NRCTs) 

Number allocated to each 

intervention group (no. of 

individuals) 

For cluster trials, number of 

clusters, number of people per 

cluster 

Where there any significant 

baseline imbalances? 

Number and reason for (and 

sociodemographic differences 

of) withdrawals and exclusions 

for each intervention group 

Were patients who entered the 

study adequately accounted 

for? 

What percentage of patients 

completed the study? 

What percentage of participants 

received the allocated 

intervention or exposure of 

interest? 

Is the analysis performed by 

intervention allocation status 

(intention to treat) rather than 

the actual intervention 

received? Have any attempts 

been made to impute missing 

data? 

Age (median, mean and range if 

possible) 

Sex 

Yes 

Details: 

No Unclear 


Race/Ethnicity 

Principal health problem (incl. 

stage of illness) 

Diagnostic criteria 

Co-morbidity 

Other sociodemographics (eg. 

Educational level, literacy level, 

soci-economic status, first 

language. Also consider possible 

proxies for these e.g. low 

baseline nutritional status ) 

PROGRESS categories reported 

at baseline (indicate letters of 

those reported: Place of 

residence, race, occupation, 

gender, religion, education, SES, 

social capital) 

Subgroups 

Enter a description of any participant subgroups from this paper to 

be analysed in the review. 

Intervention Group 1 

(copy and paste table for each Intervention group) 

Group name: (State brief name for this intervention group.) Page/ 

Para/ 

Figure # 

Details of intervention or control condition (Include if relevant in sufficient detail for replication) 

Setting eg multicentre, university 

teaching hospitals, rural, 

metropolitan, school, workplace, 

community, GP clinic, etc. 

Theoretical basis (include key 

references) 

Content (list the strategies 

intended and delivered) 

Did the intervention include 

strategies to address 

diversity/disadvantage? 

Delivery (eg. Stages (sequential 

or simultaneous), timing, 

frequency, duration, intensity, 

Enter a description of any relevant strategies 


fidelity – process indicators) 

Providers (who, number, 

education/training in 

intervention delivery, ethnicity 

etc. if potentially relevant to 

acceptance and uptake by 

participants 

Co-interventions 

Duration of intervention 

Duration of follow-up 

Was sustainability discussed by the 

authors? Was is a consideration in 

study development? 

Economic variables 

ie costs of the intervention, and 

changes in other (eg health care) 

costs as result of intervention 

Other economic information (from a 

societal, non-healthcare view – e.g. 

lost wages, time) 

Yes List in Outcome section if appropriate 

No Unclear 

Details: 

Yes 

No 

Details: 

Resource requirements to replicate 

intervention (e.g. staff numbers, 

hours of implementation, 

equipment?) 

Subgroups 

What are the moderators/mediators 

of changes stated in the study? 

Do the authors describe any political 

or organisational context? 

Were any partnerships referred to? 

Was a process evaluation 

conducted? 

Control/comparison (what 

information is provided about what 

the control or comparison group 

received?) 

Enter a description of any intervention subgroups from this report to 

be analysed in the review. 

List relevant dot points 

List these as dot points 

What components were included in the process evaluation? (eg. 

dose, frequency, consistency, implemented as intended etc) 

Enter a description of what was provided for the control group, if 

applicable 

Outcomes 

(This table is set up for 2 outcome measure to save spaces, copy and paste table as often as required) 

Costs associated with the intervention can be linked with provider or participant outcomes in an 

economic evaluation (depends on the type of economic evaluation) 


Question Outcome 1 Page/ 

Para/ 

Figure # 

Is there an analytic 

framework applied (e.g. 

logic model, conceptual 

framework)? 

Outcome definition 

(with diagnostic criteria 

if relevant) 

Type of outcome: Is this 

a modifiable variable 

(Community level, 

neighbourhood level, 

individual level) or 

desired health outcome 

Time points measured 

Time points reported 

Is there adequate 

latency for the outcome 

to be observed? 

Is the measure repeated 

on the same individuals 

or redrawn from the 

population / community 

for each time point? 

Unit of measurement (if 

relevant) 

For scales – upper and 

lower limits and indicate 

whether high or low 

score is good 

How is the measure 

applied? Telephone 

survey, mail survey, in 

person by trained 

assessor, routinely 

collected data, other 

How is the outcome 

reported? Self or study 

assessor 

Is this outcome/tool 

validated? 

…And has it been used 

as validated? 

Is it a reliable outcome 

measure? 

Is there adequate power 

for this outcome? 

Outcome 2 

Page/ 

Para/ 

Figure # 


Were PROGRESS 

categories analysed by 

outcome? Indicate the 

letters of those that 

outcomes were 

analysed by (place of 

residence, race, 

occupation, gender, 

religion, education, SES, 

social capital) 


Results 

Copy and paste the appropriate table for each outcome and subgroup at each timepoint, 

including baseline 

For RCT/CCT 

Dichotomous outcome 

Comparison 

Outcome 

Subgroup 

Timepoint 

Results Intervention Comparison 

Events No. participants Events No. participants 

page/para/fig 

No. of missing 

participants 

and reasons 

Any other 

results 

reported 

Reanalysis 

required? 

(specify - 

(e.g. correlation 

adjustment) 

Reanalysis 

possible? 

Reanalysed 

results 

yes/no/unclear 

For RCT/CCT 

Continuous outcome 

Comparison 

Outcome 

Subgroup 

Timepoint 

Postintervention 

or change 

from 

baseline? 

Results Intervention Comparison 

Mean 

SD (or 

other 

variance) 

No. 

participants 

Mean 

SD (or 

other 

variance) 

No. participants 

page/para/fig 

No. missing 

participants 

and reasons 

Any other 

results 


eported 

Reanalysis 

required? 

(specify) 

Reanalysis 

possible? 

Reanalysed 

results 


For RCT/CCT 

Generic inverse variance method 

Comparison 

Outcome 

Subgroup 

Timepoint 

Results Effect estimate SE (or other variance) Intervention no. Control no. 

Page/para/figure 

No. missing 

participants 

and reasons 

Any other 

results 

reported 

Reanalysis 

required? 

(specify) 

Reanalysis 

possible? 

Reanalysed 

results 

For CBA 

Comparison 

Assignment 


How were control and treatment groups selected?? Is there likely to be an 

effect if these were the opposite way? 

Page/para/fig 

Outcome 

Subgroup 

Timepoint 


or 

change from 

baseline? 

No. 

participants 

measured 

Contemporaneous data collection? 

Intervention 

Comparison 


No. missing 

participants 

and reasons 

Baseline result 

(with variance 

measure) 


results (with 

variance 

measure) 

Change (Post – 

baseline) (with 

variance 

measure) 

Difference in 

change 

(intervention – 

control) (with 

variance 

measure) 

Any other 

results 

reported 

Reanalysis 

required? 

(specify) 

Reanalysis 

possible? 

Reanalysed 

results 


For ITS 

Generic inverse variance method 

Page/para/fig 

Comparison 

Outcome 

Subgroup 

Length of 

timepoints 

measured 

Snapshot or 

interval 

measured 

No. 

participants 

measured 

No. missing 

participants 

and reasons 

No. of 

timepoints 

measured 

Pre-intervention 

Post-intervention 


Mean value 

(with variance 

measure) 

Difference in 

means (post – 

pre) 

Percent 

relative 

change 

Result 

reported by 

authors (with 

variance 

measure) 

Reanalysis 

required? 

(specify) 

Reanalysis 

possible? 

Individual time 

point results 

Read from 

figure? 

Reanalysed 

results 


yes/no 

Change in level SE Change in slope SE 


Other relevant information 

Were outcomes relating to harms/unintended 

effects of the intervention described? Include any 

data for these in the outcomes tables above 

Potential for author conflict ie. evidence that 

author or data collectors would benefit if results 

favoured the intervention under study or the 

control 

Key conclusions of the study authors 

Could the inclusion of this study potentially bias 

the generalisability of the review? Equity pointer: 

Remember to consider whether disadvantaged 

populations may have been excluded from the 

study. 

Is there potential for differences in relative effects 

between advantaged and disadvantaged 

populations? (e.g. are children from lower income 

families less likely to wear bicycle helmets) 

Are interventions likely to be aimed at the 

disadvantaged? (e.g. school meals aimed at poor 

children). 

Issues affecting directness 

(Note any aspects of population, intervention, etc. 

that affect this study’s direct applicability to the 

review question) 

References to other relevant studies 

Additional notes by review authors 

Correspondence required for further study 

information (from whom, what and when) 


Risk of bias assessment 

Please refer to Chapter 8 - Table 8.5.c: Criteria for judging risk of bias in the ‘Risk of bias’ 

assessment tool and to the Cochrane EPOC Group’s guidance for assessing Risk of bias for studies 

with a separate control group (RCTs, CCTs, CBAs) and Risk of bias for interrupted time series 

studies (Appendix 3) for additional guidance for scoring Yes/No/Unclear. Note that the table 

below includes items from both EPOC tools. The ITS tool has been incorporated into the bottom 

of the table and all items for ITS studies are denoted by ITS preceding the risk of bias question. 

Domain 

Was the allocation 

sequence 

adequately 

generated? 

Review 

authors’ 

judgement* 

Yes / No / 

Unclear 

Description 

Describe the method used to generate the allocation sequence in 

sufficient detail to allow an assessment of whether it should 

produce comparable groups. 

Page/ 

Para/ 

Figure # 

Was allocation 

adequately 

concealed? 

Yes / No / 

Unclear 

Describe the method used to conceal the allocation sequence in 

sufficient detail to determine whether intervention allocations 

could have been foreseen in advance of, or during, enrolment. 

Were baseline 

outcome 

measurements 

similar? 

Yes/No/Uncl 

ear 

Note whether baseline outcome measurements were reported and 

whether there were any important differences between groups. If 

there were important differences between groups, note whether 

appropriate adjusted analysis was performed to account for this. 

Were baseline 

characteristics 

similar? 

Yes/No/Uncl 

ear 

Note whether baseline characteristics were reported and whether 

there were any important differences between groups. 

Were incomplete 

outcome data 

adequately 

addressed? 

Assessments should 

be made for each 

main outcome (or 

class of outcomes). 

Yes / No / 

Unclear 

Describe the completeness of outcome data for each main 

outcome, including attrition and exclusions from the analysis. State 

whether attrition and exclusions were reported, the numbers in 

each intervention group (compared with total randomized 

participants), reasons for attrition/exclusions where reported, and 

any re-inclusions in analyses performed by the review authors. 

Was knowledge of 

the allocated 

intervention 

adequately 

prevented during 

the study? 

Yes / No / 

Unclear 

Describe all measures used, if any, to blind study participants and 

personnel from knowledge of which intervention a participant 

received. Provide any information relating to whether the intended 

blinding was effective, or whether blinding was appropriate. 

Participants – yes, no, unclear [record supporting statement 

from study]. 

Separate 

assessments should 

be made for 

relevant groups of 

Investigators – yes, no, unclear [record supporting statement 

from study]. 

Outcomes assessors – yes, no, unclear [record supporting 


people involved in 

the study i.e 

participants, 

outcome assessors, 

investigators, data 

assessors etc 

statement from study]. 

Data assessors – yes, no, unclear [record supporting statement from 

study]. 

Was the study 

adequately 

protected against 

contamination? 

Yes/No/Uncl 

ear 

State whether and how the possibility of contamination was 

minimised by the study design/implementation. 

Are reports of the 

study free of 

suggestion of 

selective outcome 

reporting? 





Yes / No / 

Unclear 

Yes / No / 

Unclear 

State how the possibility of selective outcome reporting was 

examined by the review authors, and what was found. 

State any important concerns about bias not addressed in the 

other domains in the tool. 

Other sources of 

bias 

ITS: Was the 

intervention 

independent of 

other changes? 

Yes/No/Uncl 

ear 

Describe whether or not the intervention occurred independently 

of other changes over time and whether or not the outcomes may 

have been influenced by other confounding variables/historic 

events during the study period. 


ITS: Was the shape 

of the intervention 

effect pre-specified? 

Yes/No/Uncl 

ear 

State whether or not the point of analysis was the point of 

intervention. If not, describe whether a rationale for the shape of 

the intervention effect was given by the study authors. 

ITS: Was the 

intervention 

unlikely to affect 

data collection? 

Yes/No/Uncl 

ear 

Describe whether or not the intervention was likely to affect data 

collection and what the potential impact might have been. 

ITS: Was knowledge 

of the allocated 

interventions 

adequately 

prevented during 

the study? 

Separate 

assessments should 

be made for 

relevant groups of 

people involved in 

the study i.e 

participants, 

outcome assessors, 

investigators, data 

assessors etc 

ITS: Was incomplete 

outcome data 

adequately 

addressed? 





ITS: Was the study 

free from selective 

reporting? 

Yes/No/Uncl 

ear 

Yes/No/Uncl 

ear 

Yes/No/Uncl 

ear 

Describe all measures used, if any, to blind study participants and 

personnel from knowledge of which intervention a participant 

received. Provide any information relating to whether the intended 

blinding was effective, or whether blinding was appropriate. 

Participants – yes, no, unclear [record supporting statement 

from study]. 

Investigators – yes, no, unclear [record supporting statement 

from study]. 

Outcomes assessors – yes, no, unclear [record supporting 

statement from study]. 

Data assessors – yes, no, unclear [record supporting statement from 

study]. 

Describe the completeness of outcome data for each main 

outcome, including attrition and exclusions from the analysis. State 

whether attrition and exclusions were reported, the numbers in 

each intervention group (compared with total randomized 

participants), reasons for attrition/exclusions where reported, and 

any re-inclusions in analyses performed by the review authors. 

State how the possibility of selective outcome reporting was 

examined by the review authors, and what was found. 


ITS: Was the study 

free from other 

risks of bias? 

Yes/No/Uncl 

ear 

State any important concerns about bias not addressed in the 

other domains in the tool. 

* Note: For each section above ‘Yes’ indicates a ‘low risk of bias’; ‘No’ indicates a ‘high risk of bias’; ‘Unclear’ 

indicates an ‘uncertain risk of bias’. When entering the data into RevMan, the options to choose from will be ‘Low’, 

‘High’ and ‘Unclear’ 


Results 

Comparison: 

Outcome: 

Subcategory: 

Treatment group: 

Control group: 

Observed (n) total (N) observed (n) total (N) 

Total randomised 

excluded* 

Observed 

lost to follow up* 



*Reasons for loss/exclusion: 

Subcategory: 



Observed (n) total (N) observed (n) total (N) 

Total randomised 

excluded* 

Observed 

lost to follow up* 



*Reasons for loss/exclusion 


Appendix 5 – Checklist for submission 

This checklist should be cut and pasted, completed and sent with your protocol to the referees. Please ensure all 

sections have been addressed by your protocol. This will speed up the process and ensure your protocol is quickly 

included on the Cochrane Library. Please remove this section and use to check your protocol before submitting to 

the Coordinator. 

Title 

Is the title consistent with the one originally approved by the CPHG? 

Background 

Does the background support the need for a systematic review by providing sufficient 

information on the frequency and severity of the problem/public health issue and the 

uncertainties in its management? 

Objective/s 

Is the main objective of the review specified in terms of intervention(s), issue being addressed, 

population and outcomes (both beneficial and harmful)? 

To evaluate the benefits and harms of various initiatives that aim to ensure adequate access for all to food in 

communities within developed countries. 

Selection criteria 

Types of studies 

Have you adequately identified which study designs will be included? 

Is there a rationale provided for why the study designs have been included/excluded? 

Types of participants 

Are the characteristics of the issue being addressed and the population with whom the 

intervention is targeted described adequately? 

Have the population groups to be excluded been specified? 

Have the appropriate population groups been excluded? 

Types of interventions 

Have the study interventions been described? 

Have the control interventions been described? 

Have all relevant interventions for the issue being addressed and question asked been 

identified? 

Have the interventions to be excluded been described? 


Are the interventions to be excluded appropriate? 

Types of outcome measures 

Are the outcome measures for benefits and harms of the intervention(s) clearly defined in 

nature and in timing? 

Are the outcome measures used important to the population with whom the intervention is 

targeted? 

Have all relevant outcomes (both beneficial and harmful) been included? 

If specific outcomes have not been included, does this conform to the question asked? 

Search methods for identification of studies 

Has the search strategy been included and are the search terms appropriate? 

Are the dates that each source will be searched indicated? 

Does the search strategy include contacting experts in the field? 

Have the appropriate subject headings, key words and text words for the clinical problem and 

population been used? 

Has the Cochrane Collaboration search strategy to identify RCTs been used? 

Has the Trials Search Coordinator been contacted? 

Are studies in languages other than English to be included? 

Has the team considered how duplicate publications of the same study will be identified and 

dealt with? 

Will the following data sources be searched? 

The Cochrane Public Health Group specialised register 

Cochrane Central Register of Controlled Trials (CENTRAL) (most recent) 

MEDLINE (from 1950 - ) 

EMBASE (from 1980 - ) 

Other databases relevant to the review topic 

Reference lists of textbooks, reviews (including previous systematic reviews), and previous 

trials 

Conference proceedings 

Assessment of risk of bias 

Have the criteria to be used to assess the individual studies risk of bias been reported? 

Does the criteria to be used to assess study bias include:- 

Sequence generation 


Allocation concealment 

Blinding 

o Participants 

o Investigators 

o Outcome assessment 

o Data assessors 

Incomplete outcome data 

Selective outcome reporting 

Other potential biases 

Methods of the Review 

Will at least two authors of the review:- 

Perform the literature search? 

Determine study eligibility? 

Assess study quality? 

Extract data? 

Enter data in RevMan? 

Will authors work independently? 

Will consensus and/or liaison with a third author be used to resolve disagreement between 

the primary authors? 

Will authors of primary studies be contacted for clarification of unclear data or to obtain 

missing information? 

Will you attempt to analyse for possible publication bias using funnel plots or other methods? 

Will plausible explanations for variations in treatment effect be explored using subgroup 

analysis based on study quality, population and interventions? 

Statistical analysis 

Will the results of primary studies be reported with 95% confidence intervals using relative risk 

(RR) for dichotomous outcomes and mean difference (MD) for continuous outcomes? 

Have the methods used to pool the results of the primary studies been reported? 

Are these methods relevant? 

Will RR and MD summary statistics be calculated using a random effects model? 

Example of text: Statistical analysis will be performed using RevMan. For dichotomous outcomes (relapse or no 

relapse) results will be expressed as relative risks with 95% confidence intervals. Data will be pooled using the 

random effects model. Where continuous scales of measurement are used to assess the effects of treatment (e.g. 

time to relapse), the weighted mean difference will be used, or the standardised mean difference if different scales 

have been used 

Have you stated how you will test for heterogeneity? 

Example of text: Heterogeneity will be analysed using the Cochran Q test on N-1 degrees of freedom, with an α of 

0.1 used for statistical significance. 


Have you specified how you will determine the applicability of the results to individuals? 

Example of text: Calculation of absolute risk reductions with intervention in relation to different baseline risk of 

the event with no intervention or a different intervention. 

Acknowledgements 

Have you acknowledged all the relevant people and/or organizations? 

Declarations of interest 

Have you and your coauthors declared any potential conflicts of interest? 

References 

Have you checked your references? 

Sources of support 

Have you listed your internal sources of support (e.g. hospital, university)? 

Have you listed your external sources of support (e.g. scholarship, bursaries)? 

OTHER 

Have you completed and are including with your protocol submission, the Health Equity 

Checklist?

Guide for Developing a Cochrane Protocol - Cochrane Public Health ...

Create successful ePaper yourself

Delete template?

Save as template?