This is the sixth article in the series exploring the reasons for undertaking a systematic review [1-5].

With 28.5% of the votes, this was the most popular reason for undertaking a systematic review: ‘To see what has been done before, to see if new research is needed‘. In hindsight the ‘see if new research is needed‘ is too narrow, it should also be to learn from previous research, but you get the idea!

I think this is the most rationale reason for undertaking a comprehensive systematic review.  However, this has lots of caveats.  I actually think all the other reasons given in this series have merit, but the question always ‘boils down’ to what method of evidence synthesis is appropriate?

Iain Chalmers has written extensively on the need for undertaking a systematic review of previous research to inform future research.  His 2014 Lancet article ‘How to increase value and reduce waste when research priorities are set‘ [6] is as good a place as any to start!  Three snippets to whet your appetite:

• …worrying evidence that previous research is being ignored
• Cumulative meta-analyses of clinical trials clearly show why systematic reviews need to be done—eg, trials of whether a short course of corticosteroids in pregnant women expected to give birth prematurely improved neonatal mortality were repeatedly undertaken even after a clear reduction in risk of death had been shown.
• Research funders and regulators should demand that proposals for additional primary research are justified by systematic reviews showing what is already known, and increase funding for the required syntheses of

So, without doubt, when considering undertaking research (primary research or indeed secondary evidence synthesis) we need to understand and appreciate what has gone on before.  Without doing so can lead to significant waste.  In the recommendations section, Iain reports:

“Increased recognition of the need to assess systematically what is already known will mean that researchers need to be trained in research synthesis methods. The high expected standards for systematic reviews are undoubtedly challenging. If systematic reviews of relevant existing evidence are to inform proposals for additional primary research, work is needed to identify how trustworthy results can be generated with methods that are less resource intensive than are those expected now, and how computers can be used to increase the efficiency of the preparation of systematic reviews“.

You will hopefully have spotted my less than subtle highlighting in the above passage.  As alluded to higher in the article and is at the heart of this whole website – what method is appropriate?  I can see three main types of reviews:

1. A systematic review of all trials/data – including unpublished trials, CSRs etc.
2. Systematic review of all published trials, as exemplified by Cochrane.
3. Systematic reviews of all published trials undertaken using a rapid methodology.

The top method is hugely time consuming.  Is this appropriate for planning new research?  I don’t think Iain is suggesting this method.  That leaves option 2 or 3, which creates the interesting possibility of the arbitrary difference.  I contend that there is a continuum of methods with time/resource on one axis and accuracy/confidence on the other, as stylised in the image below:

At what stage would Iain (if he were to arbitrate) say ‘enough is enough’?  How does one make that decision?  If you’re relying on published journal articles you’re already compromising quite heavily (by missing lots of unpublished data), so the decision becomes pragmatic and to an extent arbitrary.  Who defines what is reasonable?  The difference between a so-called systematic review and a rapid review is arbitrary (that word ‘arbitrary’ again).  So yes, undertake a systematic review of the literature but let’s better define what that systematic review might look like.  What benefit is there in spending 1000 hours over 25 hour?  Is that an ethical use of valuable resource?  Without doubt we need to look at less resource intensive methods!

References

1. Why do we do systematic reviews?
2. Why do we do systematic reviews? Part 2
3. Why do we do systematic reviews? Part 3
4. Why do we do systematic reviews? Part 4
5. Why do we do systematic reviews? Part 5
6. How to increase value and reduce waste when research priorities are set. Chalmers I et al. Lancet. 2014 Jan 11;383(9912):156-65
7. Standing on the shoulders of giants. Wikipedia
8. George Santayana. Wikipedia