Introduction to Systematic Reviews

Data Extraction and Screening Tools

Covidence
Online software to improve the efficiency and experience of creating and maintaining evidence synthesis projects including systematic reviews and scoping reviews. Register with stanford.edu, stanfordhealthcare.org, or stanfordchildrens.org email.

DistillerSR
DistillerSR is a systematic review software that manages, tracks, and streamlines the screening, data extraction, and reporting processes of reviews, including automatic generation of the PRISMA flowchart. This software requires a fee.

JBI SUMARI
Set of tools for creating, conducting, and disseminating research reviews. Includes tools to import and manage references, perform appraisals, extract data, and perform data analysis.
Click JBI SUMARI; Users must create Ovid login and password while on Stanford network; login with Ovid personal login

The Systematic Review Toolbox
The Systematic Review Toolbox is a community-driven, searchable, web-based catalogue of tools that support the systematic review process across multiple domains. Use the search option to limit to specific tools.
Microsoft Excel
Excel is the most basic tool for the management of the screening and data extraction stages of the systematic review process. However, it may be difficult to use for large reviews.

Critical Appraisal and Risk of Bias Tools

As you gather a body of literature, it is important to learn how to critically appraise a clinical research paper. This helps

Does this study address a clearly focused question?
Did the study use valid methods to address this question?
Are the valid results of this study important?
Are these valid, important results applicable to my patient or population?

From the Centre for Evidence Based Medicine

As part of critically appraising an article, you also need to assess the risk of bias. Risk of bias is defined as the risk of systematic error in results or inferences. This is important as the development of sophisticated quasi-experimental study designs, and systematic reviews are increasingly incorporating non-randomized studies into evidence syntheses.

Latitudes Network
Search a library of validity assessment tools for use in evidence syntheses. You can browse recommended tools by study type. The website also includes training resources.

Below are some methodological tools you can use for critically appraising an article and assessing the risk of bias, based on study design.

A MeaSurement Tool to Assess systematic Reviews (AMSTAR) Checklist
Risk of Bias in Non-randomized Studies – of Interventions (ROBINS-I)
Developed by members of the Cochrane Bias Methods Group and the Cochrane Non-Randomised Studies Methods Group.
Critical Appraisal Tools by Joanna Briggs Institute (JBI)
Includes checklists for cross-sectional studies, case-control studies, case reports, case series, cohort studies, diagnostic test accuracy studies, economic evaluations, prevalence studies, qualitative research, experimental studies, randomized controlled trials, systematic reviews, and text and opinion.
Critical Appraisal Tools from CEBM, University of Oxford
Includes appraisal sheets for systematic reviews, diagnostics, prognosis, randomized controlled trials, qualitative studies, and IPD.
CATMaker from CEBM, University of Oxford
A computer-assisted critical appraisal tool for key articles regarding therapy, diagnosis, prognosis, etiology/harm, and systematic reviews of therapy.
Critical Appraisal Skills Programme (CASP) Tools and Checklists
Includes critical appraisal checklists for systematic reviews, randomized controlled trials, cohort studies, case-control studies, economic evaluations, diagnostic studies, qualitative studies, and clinical prediction rule.
Critical Appraisal Checklists by BMJ Best Practice
Includes appraisal checklists for randomized controlled trials, diagnostic test studies, and systematic reviews.
Grading of Recommendations Assessment, Development and Evaluation (GRADE)