Does a rapid review version of a large epidemiological systematic review fail to identify many eligible studies, and what implications does this have for the results of the review? Barnish M. Oral Presentation, Society for Social Medicine and Population Health and International Epidemiology Association European Congress Annual Scientific Meeting 2019
Really interesting oral presentation (abstract below). Lots of really interesting data:
“114 studies were eligible for inclusion, of which SR identified 101 (89%, due to absence of forward citation chasing) and RR 64 (56%, McNemar test p<0.001). SR reviewed 35,262 records (0.3% were included) and RR reviewed 92 records (70% were included).”
- For the SR 35,262 records were screened and 101 included. For the RR, 92 records were screened and 64 records included.
- 0.26% of the articles were screened in the RR compared with the SR (which I guess means 0.26% of the screening workload).
- The RR found 63.4% of the articles included in the SR.
- Effect on the results “…there were no significant differences in the review results“
A fraction of the work and the same result!
Time and again we get situations where you use significantly less workload and still get the same result. If a SR is undertaken, when a RR would suffice, is a waste of resource and arguably unethical. So, we urgently need to understand when a RR will suffice and when you need fuller methods – simple really!
Background Systematic reviews (SR) are the gold standard evidence synthesis method. Rapid reviews (RR) have been proposed as an alternative method that may provide evidence in a more timely fashion to inform clinical decision making and policy making. However, RR may fail to identify all relevant evidence, which may bias the review conclusions. An analysis was conducted to compare SR and RR versions of a large epidemiological review in terms of completeness and efficiency of evidence retrieval and any differences in overall review findings.
Methods A SR on the political determinants of health was conducted with searches in November 2017 on 10 scholarly bibliographic databases using a combination of MeSH terms and key words, accompanied by a search on Google Scholar (GS) and backward citation chasing. Internationally comparative studies assessing the relationship between any of four political themes (democracy, globalisation, political tradition, and welfare state) and any population health outcome, excluding healthcare expenditure, were eligible for inclusion. A RR version of this review was conducted with the same search dates. The RR comprised a GS search for health plus each of ‘politics’, ‘political’ and the four political themes plus backward and forward citation chasing. The SR and RR were compared on completeness (% of total included studies identified), efficiency (% of reviewed records that were included) and results profile (% of included studies with positive vs non-positive results). Analysis was descriptive in terms of n(%) and used chi-square and McNemar test as appropriate in SPSS v.25.
Results 114 studies were eligible for inclusion, of which SR identified 101 (89%, due to absence of forward citation chasing) and RR 64 (56%, McNemar test p<0.001). SR reviewed 35,262 records (0.3% were included) and RR reviewed 92 records (70% were included). For the welfare state theme, 54(77%) studies had positive results in SR vs 31(78%) in RR (chi-square=0.002, p=0.966), for political tradition theme 3(60%)vs 2(50%, chi-square=0.090, p=0.764), for democracy theme 14(78%) vs 14(82%, chi-square=0.114, p=0.735), and for globalisation theme 3(17%) vs 5(38%, chi-square=1.873, p=0.171).
Conclusion RR identified significantly fewer included studies than SR, but there were no significant differences in the review results. RR offered greater efficiency with far greater% of reviewed records being included. This analysis benefited from using data from a large scale epidemiological review. However, it only assessed one broad topic area. Further research and evidence synthesis is needed to assess the value of RR in an epidemiological setting.