Autosynthesis

Can you do evidence synthesis automatically?

The following was posted on the Trip Database blog….:

As part of the KConnect work (EU funded Horizon 2020 project) we have been doing a fair bit of work exploring the automatic extraction of various elements from RCTs and systematic reviews.  If we can automatically understand what a paper is about it can open up all sorts of avenues with regard search and evidence synthesis.

The KConnect output is virtually ready for Trip to use and it will allow us (with decent, but not perfect accuracy) the following elements from a RCT or systematic review:

  • P – population/disease
  • I – intervention
  • C – comparison (if there is one)
  • Sentiment – does the trial favour the intervention or not
  • Sample size – is this a large or small trial
  • Risk of Bias – via RobotReviewer, which is already on the site (see this post)

So, what can we do with this?  A few examples:

  • For a given condition we can identify all the trials in this area and what the interventions are.
  • We can rank the interventions on likely effectiveness
  • For a given intervention we can look at what conditions it’s been used it.
  • We could present graphic like Information is Beautiful’s Snake Oil for a given condition and/or intervention.
  • We can massively increase the coverage of our Answer Engine.

Also, all this will be fully automatic, as new trials are added to Trip they will get processed and added to the system.

We’ve got a few technical issues to go (integrating the various systems) but we are so close. You will have no idea how long I’ve fantasised about the system.  And, even though it won’t be perfect, it should stand as a very good proof of concept.

3 thoughts on “Can you do evidence synthesis automatically?”

  1. Pingback: Can you do evidence synthesis automatically? – auxifp's Blog

  2. This looks great. But what about outcomes? To me, this is the most important aspect of a trial. Surely a rating of effectiveness can only be given on one outcome (or one outcome type) at a time, so this needs to be specified in an automatic system like this too. If you are a patient using this, you will be able to choose which outcome is important to you. Sadly, the likelihood is that patient-important outcomes are less well researched. But a system like this would be a great way of bringing attention to the problem.

    Like

    Reply

  3. Hi Caroline,

    All great points. The reasons for not including outcomes are numerous, for instance:

    1) We will typically use the abstracts – which skips over outcomes even more than full-text.
    2) Even if the outcomes are mentioned it’s another level of complexity to find them – each variable adds additional demands on resources.

    However, I think the over-arching reason is that this is highly experimental and I wanted to make it as manageable (and therefore deliverable) as possible. The release will be stage one and if it gains enough interest there are lots of ways of improving it (outcomes being one).

    Like

    Reply

Leave a comment