Climategate and the peer-review process

What the science says...

Exhibit No. 1 of the climate conspiracy theory is a collection of emails stolen from the Climatic Research Unit (CRU) of the University of East Anglia (UEA), which appeared on the internet in November 2009. Though some of these "Climategate" emails can sound damning when quoted out of context, several inquiries have cleared the scientists. The most comprehensive inquiry, the Independent Climate Change Email Review, did something the media completely failed to do: it put the emails into context by investigating the main allegations. Its general findings (summarised here) were that the scientists' rigour and honesty are not in doubt, and their behaviour did not prejudice the advice given to policymakers, though they did fail to display the proper degree of openness.

One set of allegations examined by the Review is the potential corruption of the peer review process. Contrarians claim that a small group of scientists, including those at CRU, attempted to hijack the peer review process, pressuring journals to reject papers whose conclusions contradicted their own. There are three main instances in which this is alleged to have happened.

The first involved a paper by Soon and Baliunas published in Climate Research in 2003, reviewing the literature on temperature change during the recent millennium. It concluded that late 20th century Northern Hemisphere temperatures were not unprecedented, contradicting the majority of the other analyses which came before and after it. The paper was approved by four reviewers and one of the journal’s ten review editors, Chris de Freitas, but received a hostile reception from the climate science community, as is reflected in the CRU emails. For example, Jones wrote in an email dated 11/3/2003:

I think the skeptics will use this paper to their own ends and it will set paleo back a number of years if it goes unchallenged. I will be emailing the journal to tell them I’m having nothing more to do with it until they rid themselves of this troublesome editor, a well-known skeptic in NZ. A CRU person is on the board but papers get dealt with by the editor assigned by Hans von Storch.

Michael Mann replied:

This was the danger of always criticising the skeptics for not publishing in the “peer-reviewed literature”. Obviously, they found a solution to that — take over a journal! So what do we do about this? I think we have to stop considering “Climate Research” as a legitimate peer-reviewed journal. Perhaps we should encourage our colleagues in the climate research community to no longer submit to, or cite papers in, this journal. We would also need to consider what we tell or request of our more reasonable colleagues who currently sit on the editorial board…

Contrarians have used these email quotes to argue that a group of scientists including Jones and Mann deliberately hijacked the peer review process to promote a favoured conclusion.

The second incident involved the editor of Energy and Environment, Dr Boehmer-Christiansen, who claims “[t]he hacked emails revealed attempts to manipulate peer review to E&E’s disadvantage, and showed that libel threats were considered against its editorial team. Dr Jones even tried to put pressure on my university department.”

The third involved Briffa’s actions as the editor of Holocene. In an email dated 4/6/2003, Briffa wrote:

I am really sorry but I have to nag about that review — Confidentially I now need a hard and if required extensive case for rejecting — to support Dave Stahle’s and really as soon as you can. Please

Based on this email, contrarians accuse Briffa of colluding with the reviewer to reject a contradictory paper.

As well as investigating these individual cases, the Review also commissioned Dr Richard Horton, editor of distinguished medical journal The Lancet, to write an essay about the context of peer review, published as an appendix to the inquiry report.

Dr Horton told the Review that some of the questions raised by the CRU emails “may be based on a misinformed view of the peer review process”. Peer review is quality control, not censorship. Although it is obviously impossible for reviewers to be purely objective, the decision to accept or reject is the editor’s responsibility alone; what an editor seeks from a reviewer is “a powerful critique of the manuscript”. Peer review has an important role to play: it prevents over-interpretation and ensures discussion of uncertainty and context — things which contrarians claim to be in favour of. However peer review is not infallible: “Many well-founded concepts are rejected and many erroneous ideas accepted.”

Horton wrote:

Authors and reviewers are frequently passionate in their intellectual combat over a piece of research. The tone of their exchanges and communications with editors can be attacking, accusatory, aggressive, and even personal. If a research paper is especially controversial and word of it is circulating in a particular scientific community, third-party scientists or critics with an interest in the work may get to hear of it and decide to contact the journal. They might wish to warn or encourage editors. This kind of intervention is entirely normal. It is the task of editors to weigh up the passionate opinions of authors and reviewers, and to reflect on the comments (and motivations) of third parties. To an onlooker, these debates may appear as if improper pressure is being exerted on an editor. In fact, this is the ordinary to and fro of scientific debate going on behind the public screen of science. Occasionally, a line might be crossed. [Appendix 5]

So the question becomes: did the CRU scientists cross that line? It turns out the answer is probably not. Let’s look at the three individual cases named above.

In the case of Soon & Baliunas 2003, it was not only CRU which reacted strongly to the paper. The Review recounts:

A number of review editors resigned as a reaction against the publication of what they regarded as a seriously flawed paper. The journal’s publisher admitted that the journal should have requested appropriate revisions of the manuscript prior to publication. The Editor in Chief resigned on being refused permission by the publisher to write an editorial about what he regarded as a failure of the peer review system. [8.3]

Although de Freitas described this reaction as “a mix of a witch-hunt and the Spanish Inquisition”, the Review pointed out that there were scientific grounds given (namely, the paper “conflated qualitative data on temperature and precipitation from many sources that could not be combined into a consistent proxy record”). These counter-arguments “are strongly put, and suggest that the reaction was based on a belief, for which evidence was adduced, that the science was poor. In light of the reaction of the Journal’s publisher, we do not believe that any criticism of Jones can be justified in this regard.” [8.3]

Considering this in the context provided by Richard Horton’s paper, the Review concluded that “this scale of reaction is not unusual in contested areas […] The Review makes no judgement or otherwise about the correctness or otherwise of the Soon and Baliunas paper, but we conclude that the strong reaction to it was understandable, and did not amount to undue pressure on Climate Research.” [8.3]

In the case of Energy and Environment, the Review Team “see nothing [in] Boehmer-Christiansen’s evidence that supports any allegation that CRU has directly and improperly attempted to influence the journal that she edits.” Furthermore, the emails actually show that Boehmer-Christiansen had been accusing CRU of scientific fraud, and “Jones’ response to her accusation of scientific fraud was appropriate, measured and restrained.” [8.4]

In the case of Briffa’s actions, when the Review examined the full email exchange they found nothing to support the interpretation of collusion in rejecting contradictory ideas:

It appears to reflect an Editor with a strongly negative review in hand, and who presumably had read the paper, asking for confirmation that the paper should be rejected, possibly to reduce one of the many complications that assail an editor; and in view of the delay in communicating to authors, hoping for  a strong decision from the referee. On receiving a second, more equivocal review, he offers the authors the opportunity to re-submit. [8.5]

The Review’s conclusion on the peer review allegations was as follows (its emphasis):

On the allegations that there was subversion of the peer review or editorial process we find no evidence to substantiate this in the three instances examined in detail. On the basis of the independent work we commissioned (see Appendix 5) on the nature of peer review, we conclude that it is not uncommon for strongly opposed and robustly expressed positions to be taken up in heavily contested areas of science. We take the view that such behaviour does not in general threaten the integrity of peer review or publication. [1.3.3]

Despite being heralded as “the final nail in the coffin of anthropogenic global warming”, Climategate did not even demonstrate small-scale corruption of the peer review process, let alone on the scale of the climate science community. In any case, the CRU scientists reviewed only a small part of the large body of evidence for anthropogenic global warming. That mountain of evidence cannot be explained away by the behaviour of a few individuals.

Last updated on 24 December 2010 by James Wight.