Commentary

Where’s the evidence?

In my quest for understanding I keep coming back to some fundamental questions relating to evidence synthesis and I’m often left wondering about the evidence underpinning what appears to be assumptions. So, the first question I’m requesting evidence for is:

 

What is the evidence that systematic reviews give an accurate assessment of the effectiveness of an intervention?

 

There, simple.  We’re always extolling people to ‘ask for evidence’ so the above is me ‘asking for evidence’.

3 thoughts on “Where’s the evidence?”

  1. Seems SR as “best evidence” is axiomatic, not evidence based – but in order to challenge that axiom, one would need some underlying definition of “accurate assessment of effectiveness”…Hmm…

    Like

    Reply

  2. To me there is a cost-effectiveness issue here. With limited resource available for evidence synthesis we should surely be trying to ensure the most bang for the buck.

    We can look at the cost of various methods fairly easily but what about the benefits – that’s really problematic. Which is better one SR (taking 100 days), 5 rapid reviews (each taking 20 days) or 25 RRs (each taking 4 days).

    We seem incapable of answering this question and very few seem to be interested.

    Like

    Reply

  3. A few thoughts… 1) Axiomatic – if we want to provide the best and most comprehensive “Review” of the evidence, we need to be systematic – methods continue to evolve, but a ‘comprehensive’ review of the evidence for a question would (axiomatically) SEEM the best approach 2) Historical – 20 years ago SRs could be done, but RR’s could not (or not very easily) due in large part to the absence of tech (eg, TRIP!) that now enable RRs, 3) There is also the question of PRECISION – a “SR” may “get it wrong” if the wrong approach is used to answer a question, or a SR answering one question may be (inappropriately) used to answer another, 4) Fundamentally, has to do with one’s assessment of what evidence is needed to answer a question, and I can’t find any way around the answer that short of a belief-based or axiomatically grounded answer. (Answering on Twitter too, to [hopefully] generate discussion)

    Like

    Reply

Leave a comment