Methodology reference guide

A Cochrane tool for assessing risk of bias in randomised controlled trials (RCTs). It replaced the original RoB tool (RoB 1).

Risk of bias for the specific outcome being assessed, rather than for the study as a whole, through answers to signalling questions.

• Domain 1 — the randomisation process.
• Domain 2 — deviations from intended interventions.
• Domain 3 — missing outcome data.
• Domain 4 — measurement of the outcome.
• Domain 5 — selection of reported results (selective reporting).

Systematic reviews and meta-analyses of RCTs, primarily Cochrane reviews; clinical guidelines.

A Cochrane tool for assessing risk of bias in non-randomised studies of the effects of interventions. It is built around the concept of a target trial: a hypothetical RCT against which the observational study is compared.

Risk of bias across 7 domains, including biases arising before, during, and after the intervention.

• Confounding factors (confounding).
• Selection of participants into the study.
• Classification of interventions.
• Deviations from intended interventions.
• Missing data.
• Measurement of outcomes.
• Selection of reported results.

Cohort, case–control, and quasi-experimental studies of intervention effects; reviews that include non-randomised data.

A counterpart to ROBINS-I adapted for studies of exposures rather than interventions. It is used when exposure cannot be assigned, such as harmful factors, lifestyle, or the environment.

Risk of bias in observational studies of exposure–outcome associations.

• Confounding factors.
• Measurement of exposure.
• Selection of participants.
• Post-exposure interventions.
• Missing data.
• Measurement of the outcome.
• Selection of reported results.

Environmental epidemiology, toxicology, hygiene, studies of occupational and environmental risks; systematic reviews of exposures.

A simple star-based scale for assessing the quality of observational studies. It is one of the most popular tools because it is quick to apply.

Methodological quality of cohort and case–control studies.

• Selection — sample representativeness and definition of groups.
• Comparability — control of confounding factors.
• Outcome / Exposure — quality of outcome or exposure assessment.

Systematic reviews and meta-analyses of observational studies; often used as an alternative to the more complex ROBINS-I.

An index of methodological quality for non-randomised studies, originally developed for surgery.

Methodological quality of non-comparative and comparative non-randomised studies.

• 8 items for non-comparative studies (aim, patient inclusion, data collection, endpoints, objectivity of assessment, follow-up duration, loss to follow-up, and sample size calculation).
• +4 additional items for comparative studies (adequacy of the control group, concurrent groups, baseline comparability, and statistical analysis).

Surgery and other fields where RCTs are difficult to conduct; systematic reviews of non-randomised interventions.

An updated tool for critically appraising systematic reviews themselves. Unlike the first version, it assesses reviews of both RCTs and non-randomised studies.

Methodological quality and reliability of a systematic review.

• 16 items, of which 7 are considered critical (for example, protocol registration, search completeness, justification of exclusions, risk of bias in included studies, consideration of risk of bias when interpreting results, and publication bias).

Quality assessment of published systematic reviews; overviews of reviews; clinical guidelines and HTA.

A short scale for assessing the quality of narrative reviews, which do not follow a strict systematic methodology.

Quality of the presentation and substantiation of a narrative review.

• 6 items (justification of the topic's importance, statement of aims/questions, description of the literature search, referencing, scientific reasoning, and presentation of data), each scored on a 0–2 scale.

Journal editors and peer reviewers; authors of narrative reviews; educational purposes.

A tool for assessing risk of bias and applicability in diagnostic accuracy studies (comparison of an index test with a reference standard).

Risk of bias and applicability across four domains.

• Patient selection.
• Index test.
• Reference standard.
• Patient flow and timing (flow and timing).

Systematic reviews and meta-analyses of diagnostic accuracy; assessment of diagnostic technologies (HTA).

A system for assessing the certainty of the body of evidence for each outcome and determining the strength of recommendations, rather than assessing an individual study. It is the de facto standard in clinical guidelines.

Certainty of evidence (high / moderate / low / very low) and strength of recommendations (strong / conditional).

• Factors that lower certainty: risk of bias, inconsistency, indirectness, imprecision, and publication bias.
• Factors that may increase certainty for observational data include a large effect, a dose–response gradient, and all plausible residual confounding that would attenuate the observed association or produce an association where none was observed. See GRADE handbook, section 5.3.3.
• Presentation in Summary of Findings and Evidence Profile tables.

Development of clinical guidelines, systematic reviews (including Cochrane reviews), health technology assessment, and policy development.

The leading reporting standard for systematic reviews and meta-analyses. It is a guide to what a publication should describe, rather than a quality assessment tool.

Completeness and transparency of a review report (not risk of bias).

• A 27-item checklist (with an expanded version containing sub-items) covering the title, abstract, introduction, methods, results, discussion, and other information.
• Study selection flow diagram: identification, screening, and inclusion.

Authors, peer reviewers, and editors of systematic reviews and meta-analyses across all fields of medicine and beyond.

A PRISMA extension for scoping reviews, which aim to map the extent and nature of the literature rather than answer a narrow clinical question.

Completeness of scoping review reporting.

• 20 required and 2 optional items adapted to literature mapping (without mandatory risk of bias assessment or synthesis of effects).

Authors of scoping reviews; fields with extensive or heterogeneous literature where an overview of the field is needed.

Reporting standard for observational epidemiological studies.

Completeness and transparency of an observational study report.

• A 22-item checklist covering the title, introduction, methods, results, and discussion.
• Separate adaptations are available for cohort, case–control, and cross-sectional designs, along with extensions (STROBE-MR for Mendelian randomisation and others).

Authors and peer reviewers of cohort, case–control, and cross-sectional studies.

A reporting standard for randomised controlled trials, specifying which details about an RCT must be included in a publication.

Completeness and transparency of an RCT report.

• A 25-item checklist (in the 2010 version) covering the title/abstract, introduction, methods, results, discussion, and other information.
• Participant flow diagram (enrolment, allocation, follow-up, and analysis).
• Numerous extensions (for cluster trials, pilot trials, non-inferiority, harms, and others).

Authors, peer reviewers, and journals publishing RCTs.

Guidance on the content of clinical trial protocols. It complements CONSORT: SPIRIT addresses the protocol before the study, and CONSORT addresses the report afterwards.

Completeness of an interventional trial protocol.

• A 33-item checklist: administrative information, introduction, methods (participants, interventions, outcomes, and analysis plan), ethics, and dissemination.

Development and review of clinical trial protocols; ethics committees; sponsors and regulators.

A reporting standard for case reports: detailed descriptions of individual patients.

Completeness and quality of a case report.

• A 13-item checklist: title, keywords, abstract, introduction, patient information, clinical findings, timeline, diagnosis, intervention, outcomes, discussion, patient perspective, and informed consent.

Authors and journals publishing case reports and case series.

A reporting standard for studies that develop or validate multivariable prognostic (prediction) and diagnostic models.

Completeness of a report on the development/validation of a prediction model.

• A checklist (22 items in the original) covering the title/abstract, introduction, methods (data sources, predictors, sample size, and statistics), results, and discussion.
• TRIPOD+AI (2024) extends the requirements to machine learning and AI models.

Prediction modelling, clinical risk calculators, and AI/machine learning models.

A set of simple checklists for critically appraising studies with different designs. It is intended for education and the practice of evidence-based medicine.

Study validity, results, and applicability, assessed through a series of questions answered yes / no / cannot tell.

• Separate checklists for systematic reviews, RCTs, cohort, case–control, qualitative, diagnostic, and economic studies, and clinical prediction rules.

Critical appraisal training, journal clubs, and rapid appraisal in systematic reviews.

A family of critical appraisal checklists from the Joanna Briggs Institute covering a wide range of designs, including qualitative and prevalence studies.

Methodological quality and risk of bias of the studies being included.

• Separate tools for RCTs, quasi-experimental, cohort, case–control, analytical and descriptive cross-sectional studies, case series, case reports, qualitative studies, prevalence studies, diagnostic accuracy studies, economic evaluations, and text/opinion.

Systematic reviews using JBI methodology (including qualitative and mixed-methods reviews); widely used in nursing and public health.

A single tool for critically appraising different study types within one mixed review: qualitative, quantitative, and mixed-methods studies.

Methodological quality of heterogeneous studies within a common framework.

• 2 screening questions + 5 criteria for each of five categories: qualitative, randomised, non-randomised quantitative, quantitative descriptive, and mixed-methods studies.

Mixed methods systematic reviews, where studies with different designs need to be assessed consistently.

A tool for planning and characterising trial design along a pragmatic ↔ explanatory continuum, rather than assessing risk of bias. It helps assess how applicable the results will be in real-world practice.

The degree of pragmatism of a trial design across 9 domains, each rated 1–5 and plotted on the PRECIS-2 wheel.

• Eligibility, recruitment, setting, organisation, flexibility of intervention delivery, flexibility of adherence, follow-up, relevance of the primary outcome, and primary analysis.

Clinical trial design; choosing between explanatory and pragmatic approaches; assessment of external validity.