Who's your expert? The difference between peer review and rhetoric
Posted on 17 June 2011 by John Cook
CLEARING UP THE CLIMATE DEBATE: Director of the Global Change Institute, Ove Hoegh-Guldberg submits some climate “sceptics” to peer-review and finds them wanting.
Peer review is the basis of modern scientific endeavour. It underpins research and validates findings, theories and data.
Submitting scientists' claims to peer review is a straightforward way to assess their credibility.
The Climate Commission was established by the Australian government to help build consensus around climate change.
Chief Commissioner Professor Tim Flannery handed the first major report, The Critical Decade to Julia Gillard on May 23.
Peer-reviewed by internationally respected scientists, the report summarises key evidence and conclusions regarding climate change for Australia and the world.
Rising temperatures, changing rainfall, threats to human health and agriculture, and deteriorating ecosystems are carefully documented from the scientific literature. The report makes compelling reading and a solid case for rapid action on greenhouse gases such as CO2.
But are all experts really in agreement with the Climate Commission’s report?
Enter an alternative group of experts.
Writing in Quadrant Online Bob Carter, David Evans, Stewart Franks and Bill Kininmonth stated, “The scientific advice contained within The Critical Decade is an inadequate, flawed and misleading basis on which to set national policy.”
Carter and his colleagues dispute the major findings and assert that “independent scientists are confident overall that there is no evidence of global warming” or unusual “sea-level rise”.
According to them “there is nothing unusual about the behaviour of mountain glaciers, Arctic sea ice or the Greenland or West Antarctic ice sheets.”
You would be forgiven for concluding that firm action on carbon dioxide might not be warranted if the experts can’t agree.
But is there really so much scientific dispute over the facts of climate change?
One way to resolve this is to ask a simple question. If Carter and company hold different views to those expressed in the majority of the peer-reviewed, scientific literature, then have they submitted their ideas to independent and objective peer-review?
This is a critical process that sorts opinion and rhetoric from scientific knowledge and consensus.
If the answer is “yes”, there are legitimate grounds for concern over the report’s conclusion.
If the answer is “no”, the arguments against the Climate Commission’s report fall away as unsubstantiated opinion.
The Web of Science is maintained by Thomson Reuters and covers 10,000 journals across the sciences, social sciences, arts and humanities.
You can search this database for papers by different authors within reputable, peer-reviewed journals.
I used the Web of Science to see if Carter, Evans, Franks and Kininmonth were legitimate experts in the areas in which they claim superior knowledge.
Given such strong opinions, you would expect that the four individuals would have published extensively in the peer-reviewed, scientific literature on subjects like climate change, oceanography, and atmospheric physics.
After all, if they have such strong opinions, then surely these ideas have been treated like all other valid scientific ideas?
The Climate Commission and its scientific advisory panel survive this type of scrutiny extremely well. For example, Climate Commissioner Professor Lesley Hughes has at least 39 peer-reviewed publications since 2000.
Many of these articles focus on the terrestrial ecosystems on climate change, an area for which Professor Hughes is internationally recognised.
Similar conclusions can be made for Professors Will Steffen, Matt England, David Karoly, Andrew Pitman and the others associated with the Climate Commission.
Searching for peer-reviewed articles by “R. M. Carter”, however, revealed plenty of peer-reviewed articles on unrelated topics within geology.
Only one paper turns up that could be remotely related to climate change.
This brings us back to zero for the number of credible papers published by Carter on climate change in the Web of Science.
Searching for articles by David Evans and William Kininmonth revealed no peer-reviewed scientific literature that tests their claim that climate change is not happening.
Lastly, searching for peer-reviewed papers from Stewart Franks yielded a number of articles (>50) on hydrology and climate variability since 2000.
None of these peer-reviewed articles presented data or tested the idea that climate change is or is not happening, or any of the other “errors” that Carter and his co-authors claim are associated with the conclusions of the Climate Commission.
The number of articles by Franks since 2000 that involve peer review of his claims that climate change is not happening is also zero.
So the number of peer-reviewed papers that adequately expose the ideas of Carter and co-authors to the scientific peer-review system on the climate change issue is 0, 0, 0 and 0.
We are left, then, with the observation that the Climate Commission’s report, peer-reviewed and assessed by scientists with appropriate expertise, is being challenged by four individuals who refer to websites and blogs and who have not had their core claims about climate change tested in the peer-reviewed scientific literature.
Don’t get me wrong, discussion is important, but on serious matter such as climate change, let us hope we listen carefully to the experts and not the unsubstantiated opinions of those that are not.