Single-reviewer abstract screening missed 13 percent of relevant studies: a crowdbased, randomized controlled trial. Gartlehner G et al. J Clin Epidemiol 2020 Jan 20

Conclusions: Single-reviewer abstract screening does not appear to fulfill the high methodological standards that decision makers expect from systematic reviews. It may be a viable option for rapid reviews, which deliberately lower methodological standards to provide decision makers with accelerated evidence synthesis products.

Interesting paper that frustrated me due to the use of outcomes. Over the years I have bemoaned the use of process measures (number of missed studies) as opposed to outcome measures such as, did the effect of this rapid intervention alter the results of the systematic review?  I particularly dislike the missing trials figure as we already know the effect of reporting bias that sees around 50% of trials unreported – so most SRs using dual screeners are likely to miss a large number of trials. To summarise, missing 50% of trials is unimportant but the 13% here is some methodological big deal, a non sequitur if ever there was one.  To illustrate, there may have been 20 trials for a given intervention:

• A full SR misses 50% due to reporting bias (this itself is based on some assumptions, but it’s only an illustration) e.g finds 10 trials
• A RR finds 13% less e.g 9.
• The full SR gets 10/20 of the trials = 50%
• The RR gets 9/20 trials = 45%

A difference of 5%.

This is one reason I dislike the process measure and prefer the end results.

Two other things to throw in to the mix:

• I’d be interested to see an analysis of the missed studies. For instance, I’ve seen studies suggesting poorer studies tend to be more hidden, so perhaps the missed trial were more likely to have high risk of bias. Was this the case here? Or, were they smaller, non-English?
• There was a difference in missed studies between the public health topic (16.8%) and pharmacological topic (10.5%) – this concurs with my own findings due to things like the lack of standardised language, methods etc in public health. So, for pharmacological reviews the figure is less (10.5%) than the headline of 13.4%.

In summary, dual screening invariably finds more articles than single screening and this is an issue if you’re interested in process and less interested in the accuracy of any estimates of effect size (which this paper does not answer).