Screening studies for reviews: two papers

Single screening versus conventional double screening for study selection in systematic reviews: a methodological systematic review. Waffenschmidt S BMC Medical Research Methodology 2019;19:132

This concludes:

Single screening of the titles and abstracts of studies retrieved in bibliographic searches is not equivalent to double screening, as substantially more studies are missed. However, in our opinion such an approach could still represent an appropriate methodological shortcut in rapid reviews, as long as it is conducted by an experienced reviewer. Further research on single screening is required, for instance, regarding factors influencing the number of studies missed.

This paper was reviewed via the BMJ EBM VERDICT series.

Assessing the accuracy of machine-assisted abstract screening with DistillerAI: a user study. Gerald Gartlehner et al. Systematic Reviews December 2019

Concludes:

The accuracy of DistillerAI is not yet adequate to replace a human screener temporarily during abstract screening for systematic reviews. Rapid reviews, which do not require detecting the totality of the relevant evidence, may find semi-automation tools to have greater utility than traditional systematic reviews.

 

Comment: What these papers indicate is that double screening is superior to single screening but the importance of any missed studies, especially when single screening is carried out by an experienced reviewer, might not be that impactful. At the other end of the spectrum – using AI doesn’t seem quite up to the job. This certainly reflects my own experience of using automated screening tools!

Invariably the ‘quality’ gap will close between and AI approach and human and when it does it’ll be single (human) screening all round.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s