Coming soon….Cochrane Overviews of Reviews

In Evidence-Based Medicine on February 5, 2009 by Sarah Tagged: , ,

In Issue 1 2009 of the Cochrane Library, the release notes point to a new type of review, an “Overviews of Reviews”. The aim of such a review is to “primarily to overview multiple Cochrane intervention reviews addressing the effects of two or more potential interventions for a single condition or health problem. Cochrane Overviews highlight the Cochrane reviews that address these potential interventions and summarize their results for important outcomes.” So rather than assessing a single intervention, these “umbrella” reviews will instead look at a condition and include more interventions. At the moment there’s one protocol included for this type of review and I’m a little confused about the value of them, given the criteria for inclusion and the search methods.

Looking at the methods section for Interventions for neovascular age-related macular degeneration, it states that relevant Cochrane reviews will be included as well asNon-Cochrane reviews of randomised controlled trials (RCTs) of interventions not currently the subject of a Cochrane“. Although Cochrane Reviews are seen as a gold standard, what if there’s a non-Cochrane systematic review out there on the same subject which eclipses the Cochrane review in terms of search date, potentially including additional studies not included in older Cochrane Reviews?

Additionally, the same issue arises from the following criteria “Where there is overlap between Cochrane and non-Cochrane reviews, we will compare and comment on the findings, however, the Cochrane review only will be included in the overview.” , again, potentially excluding newer studies only found in non-Cochrane reviews in a summary of the systematic review evidence on a particular condition.

The search sources for non-Cochrane systematic reviews are pretty straightforward and systematic, but there is a date limit. “We will restrict our searches to January 2006 onwards. This is because we feel that reviews published before that time are likely to be out of date.”

This seems rather contradictory as ALL Cochrane reviews will be included regardless of age. Of course, we know that Cochrane reviews are more likely to be updated compared with those published in non-Cochrane journals but a quick scan down the list of Cochrane reviews on asthma reveals one “Last assessed as up-to-date: 2 March 1999“. To get into even deeper nitpicking, there’s currently a Cochrane review on Radiotherapy for neovascular age-related macular degeneration which falls into the scope of this review of reviews protocol. This review on radiotherapy was published in 2004 with search dates around May 2004. With this criteria, this review would be included but as I understand it, no other non-Cochrane reviews would be included even if they were published in the four and a half years subsequently and possibly include studies not in the Cochrane review.

I’ll reserve my judgement until the review is published. The description in the release notes does state that Overviews of Reviews have also been developed outside the Library and can be viewed in the Evidence-Based Child Health Journal: A Cochrane Review Journal at”.

I should add at this stage that I probably have a conflict of interest as my “day job” is to find and assess systematic reviews and RCTs on a variety of topics for a regularly updated, evidence-based resource. If Cochrane were to start publishing overviews of reviews it would possibly make my life a little easier, though judging by these methods, it might be easier to ignore them at the moment as we would include ANY relevant, robust systematic review regardless of source. Although the aim in the release notes clearly states that highlighting Cochrane reviews is the main objective, if so, why bother including non-Cochrane reviews at all?


2 Responses to “Coming soon….Cochrane Overviews of Reviews”

  1. All sounds rather ill thought-through to me. And if you ask anyone who works in Cochrane whether or not all Cochrane reviews are quite as gold standard as they like to think, well…!

  2. If anyone should know about publication bias, it should be Cochrane! And how it isn’t great to put your own publications before other more up to date ones, etc. Why bother using Lilacs (database largely in Spanish) and translating papers, then?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: