Session 4: Intervention Effectiveness and the Role of Systematic Reviews
- Rationale for Systematic Reviews: The Cochrane Collaboration Logo
- Defining Systematic Reviews
- Meta-Analysis
- Standards for Systematic Reviews
- Haphazard Reviews
- Systematic Review Consortia
- The Cochrane Collaboration
- The Campbell Collaboration
- Other Review Groups
§ SCIE: The Social Care Institute for Excellence
§ 3ie: The International Initiative for Impact Evaluation
§ The EPPI-Centre: The Evidence for
Policy and Practice Information and Co-ordinating
Centre
§ Centre
for Reviews and Dissemination
·
Finding Systematic Reviews
o The Cochrane Library
o
The
Campbell Library
·
Reading Systematic Reviews
o
Anatomy
o
Standards: PRISMA and AMSTAR
Objective: to give you a
better understanding today of systematic reviews, meta-analysis, systematic
review collaborations, and how to approach trying to read them without being
overwhelmed by the volumes of information they contain.
As you probably have
realized by now, I am passionate about systematic reviews. With the glut
of information on the Internet and in journals, the amount of material
available to practitioners can be quite overwhelming. While I have long
preached about being a wise consumer of what’s out on the net because there isn’t
a censoring process for low quality research, I have come to realize that even
what is available in peer-reviewed journals is not necessarily high quality.
While every primary research study is flawed in one way or another, systematic
reviews, especially as conducted by the Cochrane and Campbell Collaborations,
search for ALL research on a given PICO question, appraises the quality of
includable studies.
Systematic reviews then
synthesize the information from included studies, taking into account study
quality, sample size, participant characteristics and other study
characteristics, so that we can begin to answer questions of what treatments
work for what people with what problems under what circumstances. Systematic
reviews conducted by most or all of these consortia are updated with new
searches and appraisals of any additional studies as often as every two years.
Still, remember that this is only one or possibly two legs of a three-legged stool called evidence-based practice. Interventions found to work within the majority of studies may still not be applicable to your client, their wishes, their values, and your expertise.
Systematic reviews may or may not be synthesized using a statistical process called meta-analysis. Meta-analysis may seem very intimidating to you at this point, and teaching these statistical processes goes beyond the realm of this class; I still don’t teach it in the PhD program, although I know much more about it than I used to. Like Rubin, even as recently as 5 years ago, I equated systematic reviews to meta-analysis. Meta-analyses may or may not be based on a thorough cataloging of both confirming and disconfirming research studies, and therefore, may be biased in some subtle ways.
Standards for systematic reviews are beginning to be established through cooperative ventures of people involved in both the Cochrane and Campbell Collaborations as well as others. Extensions of these standards are currently being developed for study designs which lend themselves more to complex psychosocial interventions, to examine adverse effects of interventions, to make use of qualitative information, and to account for contextual and econometric factors. These extensions, as they become established, are being integrated into the conduct of systematic reviews, and I remain optimistic that the conclusions reached by systematic reviews which adhere to them will become increasingly more relevant to social work practice.