Climate Science Glossary

Term Lookup

Enter a term in the search box to find its definition.


Use the controls in the far right panel to increase or decrease the number of terms automatically displayed (or to completely turn that feature off).

Term Lookup


All IPCC definitions taken from Climate Change 2007: The Physical Science Basis. Working Group I Contribution to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change, Annex I, Glossary, pp. 941-954. Cambridge University Press.

Home Arguments Software Resources Comments The Consensus Project Translations About Support

Bluesky Facebook LinkedIn Mastodon MeWe

Twitter YouTube RSS Posts RSS Comments Email Subscribe

Climate's changed before
It's the sun
It's not bad
There is no consensus
It's cooling
Models are unreliable
Temp record is unreliable
Animals and plants can adapt
It hasn't warmed since 1998
Antarctica is gaining ice
View All Arguments...

New? Register here
Forgot your password?

Latest Posts


Scientists Respond To Tol’s Misrepresentation Of Their Consensus Research

Posted on 22 September 2015 by CollinMaessen

To quote John Reisman, “Science is not a democracy. It is a dictatorship. It is evidence that does the dictating.” It’s this evidence based ‘dictatorship’ that is the basis for a scientific consensus. Based on this ‘dictatorship’ of evidence we know that global warming is real, we’re causing it, and that it’s a problem if we don’t act. This presents a real problem for those denying that there is a problem or want to minimize the consequences.

In science careers are made by overturning existing ideas and findings. This is why a consensus can only arise in science when other scientists cannot find flaws in earlier findings. As Richard Alley has said about showing that global warming isn’t a problem: “Is there any possibility that [among] tens of thousands of scientists there isn’t one of them that got the ego to do that? It’s absurd!

Falsifying a well established scientific theory or concept advances the career and the reputation of a scientist far more than confirming it does.

Attacking the consensus

This is why the paper published in 2013 by Cook et al. that analysed the scientific consensus on human caused global warming in the scientific literature is attacked so much. Finding a consensus of 97% in the scientific literature that we’re causing most of the rise in temperature is inconvenient for them. If they manage to discredit these studies, or sow doubt about them, you’ll prevent the public from acting:

Like Oreskes said, spreading doubt is the most effective strategy a science denier has. This type of attack is crucial to maintain a gap between what scientists agree on and what the public thinks scientists agree on so you can delay action. This tactic is how the tobacco industry successfully delayed action against the harmful effects of smoking for decades.

The consensus gap

A consensus is dangerous for those who deny human caused global warming, or minimize its consequences, as it shows, that they are in the minority. They are a very small percentage compared to the scientists who say, based on the evidence, that human caused global warming is a reality.

That’s why one of the most often used tactics is trying to make the consensus seem tiny. During an interview with me Dana Nuccitelli, one of the authors of Cook 2013, explained how this tactic is used on Cook 2013:

Tol’s nonsensus

Which brings me to the latest attack by one of the most persistent attackers of Cook 2013: Richard Tol.

He has a history of attacking Cook 2013 with strange claims about flawed methodologies and saying that the data doesn’t show a 97% consensus. Which is rather odd as the 97% was also found when Cook asked authors what the position of their paper was.

What makes Tol’s persistence in attacking Cook 2013 still stranger is that Tol has said that “There is no doubt in my mind that the literature on climate change overwhelmingly supports the hypothesis that climate change is caused by humans. I have very little reason to doubt that the consensus is indeed correct” and that “The consensus is of course in the high nineties.”

But then he publishes this chart:

Richard Tol's nonsensus

He published this graph in his blog post More nonsenus [sic] (archived here) in which he announced a response to Cook 2013 that he has written (a comment to Cook 2013 that’s currently being reviewed by the journal Environmental Research Letters). This graph strongly implies that Cook 2013 is the outlier with the consensus it found, when it isn’t.

I’ve already mentioned to Tol that context is everything when talking about consensus percentages. This context is what makes it clear that what Tol has produced is a perfect example of nonsensus (not the other way around). He uses the tactics I mentioned earlier, and a lot more that I haven’t mentioned yet, to make it look like the consensus is lower than it is.

So how did Tol manage to make it appear like Cook 2013 is an outlier? When I asked Nuccitelli this question he gave me the following answer:

What Tol has essentially done is to define “consensus” in a number of different ways, most of them making no sense whatsoever, and put the incomparable results together on a single chart in a grossly misleading manner.  For the literature surveys, in some cases he’s included papers that don’t take a position on what’s causing global warming (i.e. in Oreskes’ 75%), and in others he’s omitted them (i.e. in Cook’s 97%).  For the scientist surveys, in some cases he’s included both expert and non-expert opinion, sometimes just experts, and in some cases he’s even included numbers that only represent the ‘consensus’ among those who reject human-caused global warming, most of whom have no expertise in climate science.  These are not comparable numbers, and putting them together on one chart makes no sense.

Yes, it’s indeed this bad. It’s something I immediately noticed when I looked at the graph and the studies he cited. But you don’t have to take Nuccitelli’s or my word on this. I contacted the authors of the cited consensus studies and I got back some scathing responses.

Misrepresenting research

When I asked Naomi Oreskes if Tol had used her data and findings correctly she wasn’t happy with Tol changing a consensus of 100% into one of 75%:

No it is not accurate. As usual, he is misrepresenting scientific work, in this case mine.

Obviously he is taking the 75% number below and misusing it. The point, which the original article made clear, is that we found no scientific dissent in the published literature. This demonstrates that the “dissent” that was being reported in the media was politically-driven, not scientifically driven, which was, of course, the point of the paper, and led to our book, Merchants of Doubt, which explains where the political dissent comes from.

Oreskes is referring to a passage in her article where she said that of the papers she investigated 75% either explicitly or implicitly accept the consensus view (which is in her paper defined as most of the observed warming over the last 50 years is likely to have been due to the increase in greenhouse gasses). However, 25% of the papers she investigated didn’t say anything on human caused global warming. But none disagreed with the consensus position.

This is how Tol changed a consensus of 100% into 75%, by counting papers that did not say anything about the question Oreskes was trying to answer. Bart Verheggen, another cited author, publicly stated to Tol about this that “You can’t just divide the number of affirmative statements by all papers in the sample, if many papers didn’t actually stake out any position on the question at hand. The latter should logically be excluded, unless you want to argue that of all biology papers, only 0.5% take an affirmative position on evolution, hence there is low consensus on evolution.”

The effect of an evidence based consensus is that what already is established will seldomly be mentioned in a paper, an effect Cook 2013 very clearly found with their large sample. Verheggen gave more detail when I corresponded with him about Tol’s usage of his research:

Tol selectively quotes results from our survey. We provided results for different subsamples, based on different questions, and based on different types of calculating the level of agreement, in the Supporting Information with our article in ES&T. Because we cast a very wide net with our survey, we argued in our paper that subgroups based on a proxy for expertise (the number of climate related peer reviewed publications) provide the best estimate of the level of scientific consensus. Tol on the other hand presents all subsamples as representative of the scientific consensus, including those respondents who were tagged as “unconvinced”. This group consists to a large extent of signatories of public statements disapproving of mainstream climate science, many of whom are not publishing scientists. For example, some Heartland Institute staffers were also included. It is actually surprising that the level of consensus in this group is larger than 0%. To claim, as Richard Tol does, that the outcome for this subsample is somehow representative of the scientific consensus is entirely nonsensical.

Another issue is that Richard Tol bases the numbers he uses on just one of the two survey questions about the causes of recent climate change, i.e. a form of cherry picking. Moreover, we quantified the consensus as a fraction of those who actually answered the question by providing an estimate of the human greenhouse gas contribution. Tol on the other hand quantifies the consensus as a fraction of all those who were asked the question, including those who didn’t provide such an estimate. We provided a detailed argument for our interpretation in both the ES&T paper and in a recent blogpost.

Cherry picking is the tactic of focussing on specific pieces of data, often out of context, while excluding any data that conflicts with the desired conclusion. Verheggen’s research was in a similar waymisrepresented by Rick Santorum during an interview.

Verheggen also raised very clearly what I also already mentioned to Tol: expertise matters. It’s these detailed questions that made it possible for Verheggen to find the interesting result that attribution experts, scientists who investigate what is changing temperatures and by how much, say that humanity has caused more than 100% of the warming (natural trends and factors would make temperatures drop slightly if we weren’t increasing greenhouse gasses).

The problem with Tol ignoring expertise for his consensus percentages was spelled out by William Anderegg when I asked him if Tol had used his results correctly:

This is by no means a correct or valid interpretation of our results. For our sampling strategy, we bent over backwards to include as many doubters as possible within our sample, so analyzing the whole sample is completely misleading and misrepresenting our study. We showed that 50% of the doubter group had *zero* publications in the peer-reviewed climate literature whatsoever, and 80% had fewer than 20 publications, which was our cut-off to be included as an expert. The basic premise of analyzing expert consensus is that you should only count the views of true experts in the subject. You wouldn’t count the opinions of astronomers on the best heart surgery technique. Thus it makes no sense at all to count the vast numbers of non-experts doubters included in our sample. We showed in a follow up example that a large fraction didn’t have a Ph.D. at all and those that did were primarily in fields almost entirely unrelated to climate science.

Neil Stenhouse, another cited author, had the same issue with how Tol calculates his consensus percentages and highlights a point why not all studies can be directly compared:

Tol’s description omits information in a way that seems designed to suggest – inaccurately – that the consensus among relevant experts is low. This is contrary to the conclusion we took from our data, which is that there is a high level of consensus among actively publishing climate experts. Tol’s reference to “subgroups” generally, as if the nature of the subgroups were irrelevant, omits the fact that the subgroup with the highest level of expertise is also the subgroup with the highest level of agreement that global warming is human-caused.

Tol also omits something else we mentioned – that our estimates of consensus may be conservative, given that (due to an oversight) we asked about the past 150 years of global warming, rather than the past 50 or so years (the period commonly studied for signs of human causation). Several respondents emailed to suggest their answer would have been different if we had asked about the last 50 years.

Because he omits this kind of information that is centrally relevant to interpreting the numbers correctly, despite clear discussion of it in the article text, I have to wonder about his commitment to clarifying these matters for readers – his claimed motive for writing the comment.

Peter Doran really hammered the point about how Tol is incorrectly comparing datasets and results. He also very clearly spells out why expertise matters:

Well, I would never express it that way. I’ve attached the EOS paper which is very short and readable.

We sent a survey to 10,257 Earth Scientists listed in the AGI directory. [Of these scientists] 3146 people responded, 90% of this group answered that they agreed temperatures were increasing, 82% expressed the view that humans have played a significant role (exact questions in the attachment). […]

Our results showed, that when you focus on the most knowledgeable group with regards to climate – those who self-identify as climate scientists and are active in climate research and publication […] – this subset has the strongest response to Q2 about the human influence. They are >97% in support.

But all 97% are not equal. The 97% in the Anderegg study and the 97% in the Cook study all address slightly different things and groups. To properly state our 97% it would be “>97% climate scientists, who are actively publishing in the field, think human activity is a significant contributing factor in changing mean global temperatures since the industrial revolution.”

All the other numbers Tol throws out are subsets from less qualified people in our survey. It’s also not all of them. The EOS paper was based on my student’s thesis (which is referenced in the EOS paper) where the full survey and details are kept. It was much too big for an EOS article. So Tol lists the opinion of our lowest groups only with equal weight to our highest group and overall result.In fact, in the survey there were 25 categories of expertise. So if he wants to take the approach he’s taken, he should be pulling 27 numbers from our survey (these 25 categories, plus the overall number, plus the 97% for publishing climatologists). But wait, why stop there, to be complete you would have to include another 25 numbers for all of these groups for people who are not active publishers, and another 25 for people who are active publishers, and another 25 for those who have PhD’s, another 25 for those who have MS degrees, etc. That’s not how you do statistics. Our study focused on expertise. The conclusion was that the more expertise you have in climate science and/or being an active researcher, the stronger your support for humans playing a significant role. To pull out a few of the less expert groups and give them the same weight as our most expert group is a completely irresponsible use of our data. It would be like me having a medical team tell me I need surgery to remove a life-threatening malignant growth, but going to my local Starbucks to get the opinion of the team of baristas and giving both recommendations the same weight.

The two citations of the work of Hans von Storch and Dennis Bray are the only ones that are the least problematic citations (one of the percentages is correct). Though here Tol again didn’t account for no responses or correct the data in any way so the different consensus percentages can be compared.

There are more issues in the comment Tol has written. The cited authors responded to one paragraph in Tol’s comment, there are four more paragraphs.

Click here to read the rest

0 0

Printable Version  |  Link to this page


Comments 1 to 8:

  1. Is Toll being deliberately dishonest because he can be (or is paid to be perhaps), or does he really believe what he writes?  

    0 0
    Moderator Response:

    [PS] No accusations of fraud etc. Please review the Comments Policy

  2. What is intellectual fraud if not what I've just read described and documented above?

    0 0
    Moderator Response:

    [PS] Dealing with fact is better than dealing with intent/motive or speculation on what someone may or not know/believe. An accusation of fraud implies a knowing attempt to deceive. Let's stick to fact.

  3. I apologise.  Let me re-state my question.  Does Toll really believe what he says or is he just saying because he can?

    0 0
  4. uncletimrob @3,

    The alternatives you present in you question aren't a dichotomy - or .

    The classic lie, presenting something known to be wrong to knowingly deceive others becomes less stark as either the error becomes less obviously wrong or as it becomes seen as less important to the message (and note Tol considers his overaching message very important). Add to that the ambiguity of who the message is intended for, and the dichotomy is replaced by a whole set of possible situations. Note that the "because he can" may not entirely be about getting the peer review to accept this nonsense. Tol appears mindful that his work's rejection provides an opportunity (for him or others) to bash on about a scientific conspiracy. What fun he has!

    Myself, I'm of the view that if somebody inappropriately presents egregious nonsense (like Tol is doing here) he is plainly a liar but whether he is lying to us or to himself (your question) or (in my view) to what degree the nonsense is a product of incompetence, who can say. You'd need to examine his comments in detail to see how stupid/deceitful such a man is.

    In Tol's case, note the comment on his blog "That was a bit of an eye-opener." It seems he hadn't thought to carry out this analysis before which demonstrates s pretty rubbish grasp of the subject he purports to analyse.

    It is also evident from Tol's blog that he holds in very low regard both the journal in which he is seeking publication and the referees he is seeking to persuade with his work.

    Interesting stuff, but none of that helps in answering your/my question.

    0 0
    Moderator Response:

    [PS] Enough please. This kind of speculation serves little purpose and is borderline for the comments policy. Anyone who wants to pursue this should take it up with Toll directly.

  5. Is this a Tol-free website?

    Sorry, I couldn't resist!

    0 0
  6. uncletimrob asked: Does Toll really believe what he says or is he just saying because he can?

    I was wondering the same, whether he was perhaps pulling a practical joke. I asked him via twitter. His response was rather evasive, on the one hand claiming that these studies don't measure anything, but on the other hand seemingly happy to draw conclusions based on (his misrepresentations of) them. See e.g. towards the end of storify.

    0 0
  7. Tol said at one point:

    a. shut up
    b. destructive comment
    c. constructive comment

    a. is wrong
    c. is not an option. I don’t have the resources to redo what they did, and I think it is silly to search a large number of papers that are off-topic; there are a number of excellent surveys of the relevant literature already, so there is no point in me replicating that.

    that leaves b

    But even here he's not being honest since he's spent vastly more time on this issue than is required to replicate the entire Cook study. His rationale for his choices should be: a) I don't want to, b) I want to, and c) I don't want to.

    0 0
  8. Even if 55 percent of climate scientists accept the global warming theory, that is still high enough for us to take it seriously.  However, I find it funny that science deniers can´t make a good list of climate researchers that disagree with the basics of the global warming theory. 

    0 0

You need to be logged in to post a comment. Login via the left margin or if you're new, register here.

The Consensus Project Website


(free to republish)

© Copyright 2024 John Cook
Home | Translations | About Us | Privacy | Contact Us