Systematic reviews in five minutes?  That’s the suggestion from Trip with their Trip Rapid Review System (TRRS)

Conflict of Interest: I run Trip and designed the TRRS and wrote the posts I’ll link to below!

The system is designed to replicate the results of a systematic review but using a number of shortcuts.  It doesn’t rely on a full search, it doesn’t critically appraise the articles.  But, that aside, if you’re interested in the bottom line result of whether an intervention has some ‘worth’ it gives surprisingly good results.

The principle stems from sentiment analysis, a technique that “…aims to determine the attitude of a speaker or a writer with respect to some topic or the overall contextual polarity of a document.”  In short, it says whether a section of text is saying positive or negative things.  So, from an abstract we can say if the article is saying positive or negative things about the intervention (currently, the system only works on placebo-controlled trials).  So, these are the steps:

• A user goes to a special TRRS interface and is presented with two search boxes (1) ‘P’ for the population (ie the disease) and (2) ‘I’ for intervention (eg the drug) and press search.
• The user gets presented with a list of potential controlled trials taken from Trip’s controlled trials collection.  The user selects those that are pertinent to their question.  They then press ‘analyse’ and the system does it’s ‘magic’.
• In the background the system ‘reads’ the abstract and decides if it’s positive or negative about the intervention.  If it’s positive the articles score +1 and if it’s negative it scores -1. The system also tries to ‘understand’ the size of the trial and uses this to adjust the score, if the trial is small or medium in size (as these are more unreliable).  So, a small trial score +0.25 or -0.25 and a medium trial scores +0.5 or -0.5.
• A score is presented which represents the average scores of all the trials.  At the top of the page was a sample image from a review of oral tapentadol for cancer pain.  But I’ll reproduce it below:

So, how does it compare?  There are three write-ups to see:

1. Ultra-rapid reviews, first test results
2. Trip Rapid Review worked example – SSRIs and the management of hot flashes
3. Oral tapentadol for cancer pain

In short, I would say Trip is approaching around 85% agreement with Cochrane Systematic Reviews.  Considering how little resource was available, this is remarkable.

The next steps are as follows:

• Improve the sentiment analysis.
• Improve the trial size detection.
• Develop heuristics to better aid effect estimate.
• Work on head-to-head trials (not just placebo).
• Start scanning for trials, as they are published, which should be part of pre-existing reviews and semi-automatically update the review.

All the above is exciting and the technological side seems relatively straightforward!  But acceptance is the bigger issue and that’s the real challenge.  But, if you need a quick overview of trials and a pretty decent estimate of effect, then the TRRS (possibly version 2) would surely be suitable!