Arguments
Skeptical Science Study Finds 97% Consensus on Human-Caused Global Warming in the Peer-Reviewed Literature
Posted on 16 May 2013 by dana1981, John Cook
A new survey of over 12,000 peer-reviewed climate science papers by our citizen science team at Skeptical Science has found a 97% consensus among papers taking a position on the cause of global warming in the peer-reviewed literature that humans are responsible.
Lead author John Cook created a short video abstract summarizing the study:
The Abstracts Survey
The first step of our approach involved expanding the original survey of the peer-reviewed scientific literature in Oreskes (2004). We performed a keyword search of peer-reviewed scientific journal publications (in the ISI Web of Science) for the terms 'global warming' and 'global climate change' between the years 1991 and 2011, which returned over 12,000 papers. John Cook created a web-based system that would randomly display a paper's abstract (summary). We agreed upon definitions of possible categories: explicit or implicit endorsement of human-caused global warming, no position, and implicit or explicit rejection (or minimization of the human influence).
Our approach was also similar to that taken by James Powell, as illustrated in the popular graphic below. Powell examined nearly 14,000 abstracts, searching for explicit rejections of human-caused global warming, finding only 24. We took this approach further, also looking at implicit rejections, no opinions, and implicit/explicit endorsements.
We took a conservative approach in our ratings. For example, a study which takes it for granted that global warming will continue for the foreseeable future could easily be put into the implicit endorsement category; there is no reason to expect global warming to continue indefinitely unless humans are causing it. However, unless an abstract included (either implicit or explicit) language about the cause of the warming, we categorized it as 'no position'.
Note that John Cook also initiated a spinoff from the project with a survey of climate blog participants re-rating a subset of these same abstracts. However, this spinoff is not a part of our research or conclusions.
The Team
A team of Skeptical Science volunteers proceeded to categorize the 12,000 abstracts – the most comprehensive survey of its kind to date. Each paper was rated independently at least twice, with the identity of the other co-rater not known. A dozen team members completed most of the 24,000+ ratings. There was no funding provided for this project; all the work was performed on a purely voluntary basis.
Once we finished the 24,000+ ratings, we went back and checked the abstracts where there were disagreements. If the disagreement about a given paper couldn't be settled by the two initial raters, a third person acted as the tie-breaker.
The volunteers were an internationally diverse group. Team members' home countries included Australia, USA, Canada, UK, New Zealand, Germany, Finland, and Italy.
The Self-Ratings
As an independent test of the measured consensus, we also emailed over 8,500 authors and asked them to rate their own papers using our same categories. The most appropriate expert to rate the level of endorsement of a published paper is the author of the paper, after all. We received responses from 1,200 scientists who rated a total of over 2,100 papers. Unlike our team's ratings that only considered the summary of each paper presented in the abstract, the scientists considered the entire paper in the self-ratings.
The 97% Consensus Results
Based on our abstract ratings, we found that just over 4,000 papers expressed a position on the cause of global warming, 97.1% of which endorsed human-caused global warming. In the self-ratings, nearly 1,400 papers were rated as taking a position, 97.2% of which endorsed human-caused global warming.
We found that about two-thirds of papers didn't express a position on the subject in the abstract, which confirms that we were conservative in our initial abstract ratings. This result isn't surprising for two reasons: 1) most journals have strict word limits for their abstracts, and 2) frankly, every scientist doing climate research knows humans are causing global warming. There's no longer a need to state something so obvious. For example, would you expect every geological paper to note in its abstract that the Earth is a spherical body that orbits the sun?
This result was also predicted by Oreskes (2007), which noted that scientists
"...generally focus their discussions on questions that are still disputed or unanswered rather than on matters about which everyone agrees"
However, according to the author self-ratings, nearly two-thirds of the papers in our survey do express a position on the subject somewhere in the paper.
We also found that the consensus has strengthened gradually over time. The slow rate reflects that there has been little room to grow, because the consensus on human-caused global warming has generally always been over 90% since 1991. Nevertheless, in both the abstract ratings and self-ratings, we found that the consensus has grown to about 98% as of 2011.
Percentage of papers endorsing the consensus among only papers that express a position endorsing or rejecting the consensus. From Cook et al. (2013).
Our results are also consistent with previous research finding a 97% consensus amongst climate experts on the human cause of global warming. Doran and Zimmerman (2009) surveyed Earth scientists, and found that of the 77 scientists responding to their survey who are actively publishing climate science research, 75 (97.4%) agreed that "human activity is a significant contributing factor in changing mean global temperatures." Anderegg et al. (2010) compiled a list of 908 researchers with at least 20 peer-reviewed climate publications. They found that:
"≈97% of self-identified actively publishing climate scientists agree with the tenets of ACC [anthropogenic climate change]"
In our survey, among scientists who expressed a position on AGW in their abstract, 98.4% endorsed the consensus. This is greater than 97% consensus of peer-reviewed papers because endorsement papers had more authors than rejection papers, on average. Thus there is a 97.1% consensus in the peer-reviewed literature, and a 98.4% consensus amongst scientists researching climate change.
Why is this Important?
Several studies have shown that people who correctly perceive the scientific consensus on human-caused global warming are more likely to support government action to curb greenhouse gas emissions. This was most recently shown in McCright et al. (2013), recently published in the journal Climatic Change. People will defer to the judgment of experts, and they trust climate scientists on the subject of global warming.
However, research has also shown that the public is misinformed on the climate consensus. For example, a 2012 poll from US Pew Research Center found less than half of Americans thought that scientists agreed that humans were causing global warming. One contributor to this misperception is false balance in the media, particularly in the US, where most climate stories are "balanced" with a "skeptic" perspective. However, this results in making the 3% seem much larger, like 50%. In trying to achieve "balance", the media has actually created a very unbalanced perception of reality. As a result, people believe scientists are still split about what's causing global warming, and therefore there is not nearly enough public support or motivation to solve the problem.
Such false balance has long been the goal of a dedicated misinformation campaign waged by the fossil fuel industry. Just as one example, in 1991 Western Fuels Association conducted a $510,000 campaign whose primary goal was to "reposition global warming as theory (not fact)." These vested interests have exploited the media desire to appear "balanced."
Open Access for Maximum Transparency
We chose to submit our paper to Environmental Research Letters because it is a well-respected, high-impact journal, but also because it offers the option of making a paper available by open access, meaning that for an up-front fee, the paper can be made free for anybody to download. This was important to us, because we want our results to be as accessible and transparent as possible.
To pay the open access fee, in keeping with the citizen science approach, we asked for donations from Skeptical Science readers. We received over 50 donations in less than 10 hours to fully crowd-fund the $1,600 open access cost.
Human-Caused Global Warming
We fully anticipate that some climate contrarians will respond by saying "we don't dispute that humans cause some global warming." First of all, there are a lot of people who do dispute that there is a consensus that humans cause any global warming. Our paper shows that their position is not supported in the scientific literature.
Second, we did look for papers that quantify the human contribution to global warming, and most are not that specific. However, as noted above, if a paper minimized the human contribution, we classified that as a rejection. For example, if a paper were to say "the sun caused most of the global warming over the past century," that would be included in the less than 3% of papers in the rejection categories.
Many studies simply defer to the expert summary of climate science research put together by the Intergovernmental Panel on Climate Change (IPCC), which states that most of the global warming since the mid-20th century has been caused by humans. According to recent research, that statement is actually too conservative.
Of the papers that specifically examine the human and natural causes of global warming, virtually all conclude that humans are the dominant cause over the past 50 to 100 years.
Net human and natural percent contributions to the observed global surface warming over the past 50-65 years according to Tett et al. 2000 (T00, dark blue), Meehl et al. 2004 (M04, red), Stone et al. 2007 (S07, light green), Lean and Rind 2008 (LR08, purple), Huber and Knutti 2011 (HK11, light blue), Gillett et al. 2012 (G12, orange), Wigley and Santer 2012 (WS12, dark green), and Jones et al. 2013 (J13, pink).
Most studies simply accept this fact and go on to examine the consequences of this human-caused global warming and associated climate change.
Another important point is that once you accept that humans are causing global warming, you must also accept that global warming is still happening; humans cause global warming by increasing the greenhouse effect, and our greenhouse gas emissions just keep accelerating. This ties in to our previous posts noting that global warming is accelerating; but that over the past decade, most of that warming has gone into the oceans (including the oft-neglected deep oceans). If you accept that humans are causing global warming, as over 97% of peer-reviewed scientific papers do, then this conclusion should not be at all controversial. With all this evidence for human-caused global warming, it couldn't simply have just stopped, so the heat must be going somewhere. Scientists have found it in the oceans.
Spread the Word
Awareness of the scientific consensus on human-caused global warming is a key factor in peoples' decisions whether or not to support action to reduce greenhouse gas emissions. However, there is a gap here due to the public's lack of awareness of the consensus. Thus it's critical that we make people aware of these results. To that end, design and advertising firm SJI Associates generously created a website pro-bono, centered around the results of our survey. The website can be viewed at TheConsensusProject.com, and it includes a page where relevant and useful graphics like the one at the top of this post can be shared. You can also follow The Consensus Project on Twitter @ConsensusProj, and on Facebook.
Quite possibly the most important thing to communicate about climate change is that there is a 97% consensus amongst the scientific experts and scientific research that humans are causing global warming. Let's spread the word and close the consensus gap.
Coming tomorrow, details about a feature that will let you test our results by rating the papers directly yourself. The Consensus Project results have also been incorporated into the rebuttals to the myths There is no consensus and IPCC is alarmist.
s_gordon_b:
You wrote:
"Categories 2 and 3 are too ambiguous - as defined in table 2 - to equate to the consensus that humans are not only a cause of industrial era global warming but the dominant cause since the middle of the last century (at least). Am I not reading that right? And if so, shouldn't only those abstracts that take an unambiguous stand on the consensus of dominant human causation be counted and the others excluded in the same way that abstracts that take no stand at all were excluded?"
From looking at the paper, what I find as the first statement of the consensus (in the abstract) is: “Among abstracts expressing a position on AGW, 97.1% endorsed the consensus position that humans are causing global warming.” So the definition of the consensus position is that “humans are causing global warming”. But how is this to be understood?
In my view, it has to be understood as "common speech": When the proverbial “man in the street” asks the question, “Are we (humans) causing global warming?”, he is NOT necessarily asking, “Is the degree of responsibility for global warming that is due to human activity greater than 50%?” or "Is human activity the dominant cause of global warming?" He is asking something much closer to, “Is human activity contributing to global warming big enough that we ought to be doing something different?” And I believe most scientists would agree with this understanding of the AGW issue: Not that the human contribution is necessarily greater than 50%, but rather that it is important enough that we should be seriously thinking hard about whether we want to change what we are doing, on a global scale.
My understanding of the Levels of Endorsement (LOEs) 2 & 3 is that they indicate agreement, explicit or implicit, with the spirit of the underlined statement above, which I regard as a filled-out "common speech" version of the statement in the abstract.
Therefore, I have no doubts about including LOEs 1 - 3 together as indicating support for the AGW consensus.
@Barry 201:
Speaking for myself, I do not find it terribly useful for you to define one gneral term, i.e., academic merit, with another general term, i.e., quality of scholarship. What is generally accepted set of standards which are associtaed with these terms? (Source please.)
In your opinion, which of the standards were violated by Cook et al 2013?
Do you believe that the editors of Environmental Research Letters deliberately ignored certain standards when they published Cook et al 2013?
nealjking,
In my view, it has to be understood as "common speech"
Precisely. There is a reason why the email asked the original authors whether their paper endorsed such a simple, "common speech" statement without definitions of each word — it's so that their answer can be reported to the general public in the same terms.
Consider the fact that laws are full of such undefined terms, like "reasonable". Probably the most famous of these is the phrase "guilty beyond reasonable doubt". The reason that they don't tell you, as a jury member, how you are "supposed" to interpret that is precisely because they want you to bring your own definition to the table. Get a large enough group of people together and the opinion of the majority will be an effective "poll" on what the word is supposed to mean.
Likewise, ask a few thousand scientists whether or not their papers endorse the proposition that human activity is causing global warming and you can be sure that the majority are going to interpret that in the same way as the majority of the populace. It doesn't matter if a few scientists here and there have odd interpretations, or a few members of the public have odd interpretations, overall they will agree.
The only exception to this rule is when the scientists have a reason for systematically interpreting it differently to the general population, and in this case it could well be true, because these scientists are climate scientists, and they can hardly be unaware of the IPCC's statement of consensus, which is that "human activity is very likely causing most of the current GW". Given that, it is entirely reasonable to assume that those scientists answered an even stronger interpretation of the statement than what the general population would assume it to mean, but this possibility does no favours to those who wish to nitpick.
The bottom line is that several thousand scientists, when asked to asses their own papers — so there was no need for them to try to interpret what they wrote, they knew exactly what they were trying to say — responded by claiming that 97.2% of those that addressed the question at hand endorsed the statement that human activity is causing global warming.
This is profound for two reasons: 1) It shows that a large number of scientists, independently interpreting both the meaning of that phrase and their own papers, arrived at the same level of endorsement as Cook et al did looking only at their abstracts, and 2) it did so while at the same time enormously growing the percentage of papers that actually addressed the question, that Cook et al were forced to categorise as "neutral". Incredible.
Arguments over whether levels 2 or 3 endorse the proposition that humans are the "dominant" cause, or a "signficant" cause, or "> 50%", are seriously missing the point, just like those insisting that "reasonble" be precisely defined. If the scientists wanted to convey lesser degrees of human involvement, they could have chosen level 5, or level 6, or level 7, depending on how it was conveyed in their paper. The fact that the overwhelming majority of them didn't speaks volumes. Levels 2 and 3 cannot be read in isolation, and the fact that both have the worse "endorse" in the title cannot be ignored.
There is a reason why the email asked the original authors whether their paper endorsed such a simple, "common speech" statement without definitions of each word — it's so that their answer can be reported to the general public in the same terms.
Actually, there was one definition: "human activity (i.e. anthropogenic greenhouse gases)". This narrowing of "human activity" to just anthropogenic GHGs allows them to state that the consensus relates to GHGs without anyone being able to muddy the waters by claiming that some scientists might have been talking about other activities like land clearing.
I keep submitting this, but it keeps not showing up. I'm sorry if it ends up appearing in triplicate:
Rob, again, based on my reading of the paper (please correct me if I'm wrong), your statement that the paper indicates that "97% of the research supports AGW" is the right conclusion. But that's not how the study is being (forgive me) spun. Evidence of 97% support/endorsement of a mostly* unquaintified degree of human causation (not the consensus) is being conflated with evidence of a 97% agreement with the consensus of predominant human causation. We criticize deniers and contrarians - indeed, SkS excels at this - for such sleights of hand. We should apply the same standard to "our" research. It's a good study, but unless someone can explain how and why I (and others who I now see have raised the same question) have got it wrong, the presentation of the study is not good.
The study also attributes agreement with the consensus to >97% of the authors, based on their ratings (from the anstract: "Among self-rated papers expressing a position on AGW, 97.2% endorsed the consensus"). Reading the paper and the supplementary data, it again appears that the authors' 1-7 ratings were then aggregated in the same way as the reviewers' ratings, potentially conflating the popular (non-consensus) view that "people are at least responsible for some of the warming" with the consensus of the IPCC and, I think it's safe to guess (but not from this paper), most climate scientists.**
*I've looked everywhere, but I can't find where the numbers of abstracts assigned to each of the original Table 2 categories is or the category assignments by the study authors. Could you point me to that data?
**It's too bad the authors weren't asked for their views on the consensus, apart from what their papers had or had not said. The last survey had answers from less than 100 climate scientists. This study's sample size of ~1200 would probably have included hundreds more or even all of them, inasmuch as they were publishing papers on climate science regardless of their discipline.
[JH] The two prior duplicate posts have been deleted.]
s_gordon_b:
As stated above, my interpretation of the meaning of the consensus is:
"Not that the human contribution is necessarily greater than 50%, but rather that it is important enough that we should be seriously thinking hard about whether we want to change what we are doing, on a global scale."
s_gordon_b,
Rob, again, based on my reading of the paper (please correct me if I'm wrong), your statement that the paper indicates that "97% of the research supports AGW" is the right conclusion. But that's not how the study is being (forgive me) spun.
Perhaps it would be easier to understand your point if you could point us to some examples of where the study is being spun?
Regarding the authors, again, they were asked to state whether each specific paper endorsed the proposition that anthropogenic GHGs are causing global warming, rejected the proposition that anthropogenic GHGs are causing global warming, or was neutral. If the author of the paper felt that their paper implied humans were having a minimal impact on global warming (e.g. by proposing an alternative as the main cause of global warming), or stated that human impact was minimal or non-existent, or stated that humans were causing less then half of global warming, then they would have categorised their own papers as rejecting the proposition.
I'm fairly confident that anyone who rejected the consensus view would have made damn sure their paper was counted as a rejection if it was at all possible to do so! And let's not forget that the authors of any papers who feel their paper should have been counted as a rejection are free to search for their paper in the results and alert us to the miscategorisation.
Anyway, the bottom line is that the authors of 97.2% of the papers that took a position stated that their papers endorsed the proposition that anthropogenic greenhouse gasses are causing global warming. That's it.
*I've looked everywhere, but I can't find where the numbers of abstracts assigned to each of the original Table 2 categories is or the category assignments by the study authors. Could you point me to that data?
http://www.skepticalscience.com/tcp.php?t=search
Perhaps you should spend some time reading the earlier comments to avoid rehashing the same points over and over again.
For a summary of the category assignments extracted from the database, see my earlier comment. The bottom line is that, ignoring neutral papers, the percentage of papers that endorse the consensus in each category are:
Papers that quantify the human contribution to global warming (i.e. level 1 vs level 7): 88%
Papers that make explicit statements about causation without quantification (2 vs 6): 98.4%
Papers that imply the impact that humans are having (3 vs 5): 98.2%
If we ignore all papers with implicit statements about the impact humans are having and only include those that make explicit statements about causation with and without quantification then we get 97.6%.
No matter which way you cut it, the results keep coming up the same.
John Hartz,
I have posted my thoughts on the paper upthread.
I think that the survey definitons are ambiguous, such that the criterion for rating the papers may have been different between the authors of the study (Cook et al), and the original Authors (the ~1200 that responded to the email and self-rated their papers).
The close match in ratings between the authors and the original Authors is a centrepiece of the corroboration of the results. If enough respondants had a different ratings criteria to others, then the apparent corroboration between the 2 groups is undermined, weakening the results.
barry & John Hartz,
Another factor that can weaken the results is the fact that the authors were rating their papers, while the raters were rating the abstracts. Unless someone were to go through and re-evaluate some papers and check against the abstract rating, this level of weakenning will remain unknown.
Barry @208...
You are presenting a subjective analysis of what you believe about the survey which is going to reflect your personal biases. This is why science doesn't rely on any one piece of research to be the definitive answer on any question. We are all subject to our personal biases.
What makes Cook13 important is the fact that the conclusions independently confirm the previous research of Doran, Oreskes, Anderegg and others. Cook13 was also sufficiently blinded so that it would have been impossible for the raters to inadvertantly bias the ratings to come up with the same figures of previous research. And the data set is sufficiently large enough to also prevent biasing.
Then, on top of that, Cook13 took the added step of being self-skeptical by asking scientists to self-rate their own research based on the definitions used in the study. And the results there came out nearly identical with a 97.1% vs 97.2% consensus.
Could all these pieces of the puzzle falling exactly into place be completely happenstance? Highly unlikely!
Beyond that, it would be up to you, Barry, to come up with what you determined to be less ambiguous phrasing and then test your results. Based on how consistent these results have been I would guess any data you could compile would not be sufficiently different that those of any of these studies.
HJones @209... But I believe with such a large data set, if there were a weakness it would have presented itself as pretty obvious. The big difference you see between the SkS raters and the self-ratings is that there are far fewer neutral papers. And that makes sense because the SkS raters were not reading the full papers.
The fact that the consensus figures for the SkS raters and the self-ratings are nearly identical suggests a high level of robustness in the results.
This debate is getting irritating beyond belief, so I'll take advantage of a moments insomnia to stick my oar in.
Barry, you have shown that it is possible that authors used a more relaxed rating system than did the Consensus Project raters. You have not shown that using the more relaxed rating system significantly increases the number of papers rated as affirming the concensus, nor even that a significant number of authors used the more relaxed rating system. Even if half of papers self rated as affirming the consensus are given a neutral rating, the support for the consensus from self rated papers remains at 94.5% of all relevant papers. That shows how robust the results of the paper are; and that you must not just show the mere possibility of but the actuality of a difference in rating >>50% of affirmations if you wish to call the results into question. To my knowledge, no critic of the paper has even attempted to do so. At most they have argued for the mere possibility a less robust definition of "affirms agw" and then assumed that therefore >>50% of rated affirmations should have been rated neutrals.
Rob (and others), the near match in percentage of abstract rated and self rated papers rated as affirming the consensus (from relevant papers) is a product of a 1.9 fold increase by percentage of papers rated as affirming the consensus, a 2.6 fold increase by percentage of papers rejecting the consensus, the lack of a category for papers uncertain of the consensus, and an increased rate of return on recent papers (more likely to affirm the consensus) than older papers (which are less likely to affirm the consensus, particularly prior to 1995). Given that, it should largely be considered coincidental. What is robust about this result is not the 97%, but the 95%+ of papers which is difficult to eliminate by varying the data to account for potential biases.
I look forward with interest to further developments in this debate, and to when I have sufficient time to actively participate again.
Very briefly, all participants in this debate would do well to read Bray and Von Storch 2010 (particularly question 21, which shows that in 2008, 83.51% of climate scientists where significantly convinced that "most of recent or near future climate change is, or will be, a result of anthropogenic causes"). Note that this is not inconsistent with Cook et al 2013 because (a), climate scientists "convinced by the evidence" publish approximately twice as many papers on climate science on average relative to those unconvinced by the evidence (Anderegg et al, 2010), and (b), those not significantly convinced may well not be convinced for reasons other than the weight of evidence, and hence lag in proportion the balance of evidence as seen in the literature. It should place a constraint on the over interpretations and the absurd underinterpretations of Cook et al 2013.
Thanks Tom. I'm actually less stuck on the specific 97.1% figure that it may seem. The fact that all this research, taken collectively, and taking the fact that Cook13 is such a large sampling of research, should tell people how robust the conclusions actually are.
Again, I would challege any other group to perform a similar research project, using their own wording, and see if they can come up with appreciably different results. The point being, Barry, Lucia, Schollenberg and others can quibble until the cows come home, but until they do their own research (and get it published) they have little ground to stand on.
The case for AGW must be really shaky if a "scientific consensus" is required to substantiate it. "Scientific consensus" is really an oxymoron because true science does not require a consensus. The idea of voting on scientific truths seems totally alien to me. It is just like the political entity that voted to change the value of pi. In mathematics it takes only one counterexample to invalidate a theorem.
William Haas, you should read the post "Is There a Scientific Consensus on Global Warming?" to understand what is meant by "consensus." To make your example from mathematics applicable, you must address mathematicians' consensus on whether that single counterexample really does invalidate a theorem. If you believe that all mathematical counterexamples are correct, you are living in a fantasy world.
William Haas
All science requires consensus. The oft used claim is that it is about evidence not consensus, which is actually a very confused statement.
Of course it is about the evidence for something. Does the evidence support the hypothesis or not? To determine this one has to evaluate the evidence and form a conclusion, an opinion, about whether the evidence supports the hypothesis or not. And each individual forms their own opinion on this. You and I might look at the same evidence and form different opinions about whether that evidence supports the hypothesis. So whose opinion should we accept, yours or mine?
All knowledge is a consensus of the opinions of many people about the significance of evidence. If a large number of people share the same opinion that the evidence supports the hypothesis, we tend to accept that it does.
Was the proof of Fermat's Last Theorem a few years back a valid proof? Yes, because that was the consensus of the many mathematicians who evaluated the proof.
William Haas, scientific consensus of the sort that Cook has done is not evidence for the theory of anthropogenic global warming. Rather it is an index of the scientific support that the theory has. That index is used by people who don't have the time, energy, training, means, and/or motivation to engage the published science effectively. That would be most people. Like it or not, most people rely on opinion-makers for their opinions. They're forced to.
Question for you, William: do you understand my point?
The point of the paper is that the state of the current scientific literature reflects an overwhelming working assumption by relevant scientists that AGW is real and happening; and that, unfortunately, the impression of most Americans (for example) is that the scientific community is in a state of conflict about the matter.
Hi JasonB,
You wrote in reply to my post:
Ironically, in light of the methodology of the paper, the spin (more neutrally, I should call it confusion or conflation) starts with my "rating" of where Cook et al.'s abstract stands on AGW (emphasis below is mine):
"We analyze the evolution of the scientific consensus on anthropogenic global warming (AGW) in the peer-reviewed scientific literature, examining 11 944 climate abstracts from 1991–2011 matching the topics 'global climate change' or 'global warming'. We find that 66.4% of abstracts expressed no position on AGW, 32.6% endorsed AGW..."
AGW is immediately defined as "the scientific consensus on anthropogenic global warming," not in the broadbrushed, imprecise, popular way that ranges from "we're a significant contributor, along with natural causes" to "we're the cause." In the next breath, we're told that "32.6% endorsed" that consensus. Ask anyone who reads climate science papers what the term "scientific consensus on anthropogenic global warming" means in the context of such a paper, and how many will say "simple: humans are a significant cause of global warming" vs "it refers to the consensus of the IPCC and pretty much all of the other major scientific review bodies that humans are the dominant (if not the only) cause of global warming, at least since the 1950s"? In climate science circles "consensus" has a precise meaning. And this is a climate science paper. This is why when I initially read about the paper - and based on my very high level of trust in SkS - I took the claim about support for "the consensus" literally, in the IPCC sense, and represented the paper that way on social media. It's also why I was stunned when I actually read the paper and got to the part where categories 1, 2 and 3 were rolled up together and rated as "the consensus." Not so. Two and 3, by definition, lacked the scope to indicate one way or another whether they support the consensus. Basically, just as most of all of the abstracts that were rated lacked the scope to comment at all on causation; 2 and 3 lacked the scope to comment on the consensus quantification of causation.
You continued:
You're misunderstanding my argument. I'm not in any way suggesting that minimizing or denying papers were lumped into the "endorsing"/consensus-supporting abstract count. For example, if I wrote a paper that (to quote from the choices the authors were given) "... explicitly states humans are causing global warming or refers to anthropogenic global warming/climate change as a given fact," I would concur if it was rated as category 2. Likewise, I would concur with a category 3 rating if my "paper implies humans are causing global warming. E.g., research assumes greenhouse gases cause warming without explicitly stating humans are the cause." But I would concur in both these cases even if my paper had zero to say for or against the IPCC consensus on degree of causation. Indeed, if my paper had supported the consensus, I would have rated it 1.
Exactly. They "endorsed the proposition that anthropogenic greenhouse gasses are causing global warming." "Are causing" can be read the same way as "obesity is causing Type 2 diabetes": it's a contribuitng cause. But, again, only a fraction of the papers were designed to endorse or reject the dominant cause IPCC consensus implied in the abstract and in all manner of coverage that has followed. E.g.:
From the lead in Suzanne Goldberg's story in The Guardian (http://www.guardian.co.uk/environment/2013/may/16/climate-research-nearly-unanimous-humans-causes):
"A survey of thousands of peer-reviewed papers in scientific journals has found 97.1% agreed that climate change is caused by human activity."
When I write (as a journalist) or read "is caused by" in the context of the climate change social debate (it is a debate, in civil society; otherwise there would be no need for this paper and the Consensus Project), my meaning/understanding is primary causation, if not "the cause."
There's no shortage of argument on this board about the semantics used in the study and surrounding it. Cook et al. could simply address this by releasing a clarification, unless they really believe it's scentifically sound to infer that category 2 and 3 papers do endorse the consensus. Alternately, they could go through all their past writings and redefine their use of the word consensus to mean that human activities are at least a significant contributor to global warming.
Quoting and answering me:
I had already gone there. I still don't see how I can use that page to get those figures. Why isn't the data simply published somewhere? It's such basic information for people who want to understand the study's findings.
I read plenty and found little beyond arguments about semantics. But now you've provided me numbers for category 1 in your earlier post, where you write:
I think you've made a fundamental error here. For a paper to unambiguously support the IPCC et al. consensus, it does indeed have to fall into category 1. But for a paper to unambiguously reject or deny it, it only has to fall into category 5, 6 or 7. That's 0.7 of 11,944 papers = 84 (maybe a few more, judging by the responses to some to the queries Popular Technology sent to known anti-AGW scientists whose papers had been rated). So the proper comparison, it seems to me, is 64 vs 84 or 64 out of 148 explicitly, unambiguously endorsing vs rejecting the IPCC consensus.
For me, the lesson of this study is that it's very hard to find robust support for the IPCC consensus just by doing a head count of papers on climate change, because very few papers, so far at least, have explicitly (or implictly, I suppose) sought to test that quantified consensus. Maybe this is analogous to the relationship between any large body of literature and the relatively uncommon major reviews and meta-analyses that attempt to put it all together and draw those larger conclusions of which consensuses are made.
JasonB,
I wrote: "I still don't see how I can use that page to get those figures."
D'oh. I mistook the search page for the "rate abstracts" page. It does insist you enter a search term, though, which limits the results.
s_gordon_b @221, try searching for the terms " " (ie, a single space) or "i" for a generalized search.
s_gordon_b @220, you claim that categories 2 and 3 do not support the consensus because:
First, whether or not the abstracts lack sufficient information to categorize them is irrelevant to the meaning of "endorse AGW". Your argument, if valid, at most shows that categories (2) and (3) should be empty sets.
Seting that argument aside, several cues require us to give "endorse AGW" and "reject AGW" the same meanings where ever they occur in the list of categories. Of these the most important is that if you do not, if you allow "endorse AGW" to mean "endorse the claim that humans have caused most of recent warming" in (1), but "humans have caused at least some part of recent warming" in (2) and (3), then abstracts can logically belong to both categories (2) {or (3)} and category (7) at the same time. Given that the categories are clearly intended to be exclusive, it follows that if "endorse AGW" and "reject AGW" can be interpreted in a way that makes them exclusive it they should be; and that any interpretation that makes them non-exclusive is a misinterpretation. If follows that "endorse AGW" and "reject AGW" must be given the same meaning whereever they occur in the rating system, and that the difference in ratings for endorsements and rejections is a difference not in the level of endorsement (or rejection) but in the clarity of the endorsement or rejection in the abstract.
Several critics of the consensus have bizarrely criticized the paper both on the grounds that both the ratings at different level vary the meaning of "endorse AGW" and that the ratings are inconsistent due to overlap. They do not appear to recognize that by doing so they make their criticism inconsistent. Specifically, they make it clear that they show their criticism to be based on a hostile, out of context interpretation of the ratings and therefore irrelevant.
Leaving that aside, consider how you would rate the following title and abstract:
How would you rate it?
It certainly does not ascribe a specific portion of recent warming to anthropogenic factors, so according to your argument it should be rated as neutral (4) at best. It was actually rated as implicitly endorsing AGW (3)*, a rating I agree with because:
1) It explicitly indicates, "Global climate change will increase outdoor and indoor heat loads" (my emphasis), something we have no reason to believe if anthropogenic factors are not the main driver of recent and near future temperature changes.
2) It implicitly endorses the IPCC A2 scenario as a plausible scenario of future temperature evolution; thereby implicitly endorsing the causal connection between greenhouse gases and temperature rise shown in that scenario including the forcing history and relationship to temperature in recent times. That forcing history, of course, shows anthropogenic factors as the cause of greater than 50% of recent warming.
I think the suposition that categories (2) and (3) cannot endorse anthropogenic factors as causing >50% of recent warming is simply wrongheaded, as shown by the example above. Of course any proposition that can be stated explicitly with quantification can also be stated explicitly without quantification by the use of such terms as "most of" (as in "most of recent warming is due to anthropogenic factors") or "dominant" (as in "the dominant cause of recent warming has been anthropegenic factors"). Further, anything that can be stated explicitly can be stated implicitly by leaving part of the affirmation to background information.
In the end, your objection comes down to the claim that it is easier to make mistakes about categories (2) and (3) than category (1). As the endorsement becomes less explicit and precise, it becomes easier to mistake endorsement for a neutral paper, and vise versa. That, at least, is true. Given the comparison between abstract rated and self rated papers, however, the mistakes have overwelmingly been conservative so the papers conclusions stand.
*One of the author's of the paper has rated it as neutral (4), but in ongoing comments he has shown that he does not understand the rating system by indicating that he thinks there is a "luke warm" category (which is clearly a mistake), and implicitly endorsed a claim by a well known "skeptic" that the consensus position is that "... almost 90-100% of the observed global warming was induced by human emission", which is absurd given the actual statements from the IPCC. It follows that the author (Richard Tol) is so confused about what is being endorsed and the rating system that his self ratings are irrelevant. I discuss his claims further on my blog.
s_gordon_b:
You wrote:
"Two and 3, by definition, lacked the scope to indicate one way or another whether they support the consensus. Basically, just as most of all of the abstracts that were rated lacked the scope to comment at all on causation; 2 and 3 lacked the scope to comment on the consensus quantification of causation."
That is an interpretation you can assert, but from my involvement and discussion with the study evaluators, I know that they would not have checked off on levels of support 2 or 3 without having come to the conclusion that the abstract was assenting to a support for a significant (not minimal) degree of global warming, due to human influence. They bent over backwards to avoid reading too much support into a statement: Why do you think there are so many neutrals? If the evaluators had been willing to interpret perceived lukewarm support or a tiny perceived impact as 2 or 3, there wouldn't be many neutrals at all.
What you're proposing is a conceivable interpretation of the words in the paper, but bears no relationship to how the evaluators actually worked.
Tom,
Are you saying that because 1 and 7 are quantified, that it follows that 2, 3 5 and 6 must reflect that quantification (>/<50% human influence)?
If ratings 2, 3, 5 and 6 are perceived as purely qualitative, then they can be read as exclusive. But, as a supposed reflection of the following paper, they might not be so. 2 and 3 get a tick if there is any suggestion that the globe has warmed/will warm due to human activity. 5 and 6 get a tick even if warming has been acknowledged, but the abstract 'minimises' (qualitatively) the human contribution.
2 and 3 might get a tick if the abstracts contain a statement like "it is expected that increases in atmospheric concentration of CO2 from industrial emissions will cause the lower atmosphere to warm," but the paper's conclusion may be that human activity has contributed 30% to the total warming. This is why I want to know if Cook et al rated 2 and 3 with the >50% AGW in mind (with the original Author in this example self-rating by the lower standard of AGW to any degree). Same rating but different criterion. That's an issue, how much of an issue is difficult to say.
An example of the same potential disjoint for ratings 5 and 6; an abstract may say, "our results indicate a smaller contribution to global warming from human activity than our previous study," but only in the body of the paper do you discover that this contributon amounts to, say, 53% instead of 59%. Qualitatively, in the abstract, AGW has been 'minimised'.
Imagining that there is no more information in the abstract than what I have invented in bold, how would you rate the last one, Tom?
(When I have time again I'll check out relevant category results, and maybe compare to full version papers if possible)
I am not 'hostile,' for the record.
s_gordon_b,
Your entire argument that the abstract of the paper is an example of the spin you're referring to is based on the claim that "the consensus" referred to in the abstract is not the IPCC statement (stated explicity in the introduction) but rather a vague claim that humans are merely a signficant cause of global warming, and your entire evidence for that is the simple fact that categories 2 and 3 are rated as supporting the consensus.
In other words, you believe that categories 2 and 3 cannot be interpreted as endorsing the IPCC's statement of consensus and the only way to claim that they do is by watering down "the consensus" that is being referred to.
This belief of yours, however, has not been demonstrated at all. Where are the examples of papers that cannot be claimed to endorse the IPCC's consensus that were nevertheless rated by Cook et al as category 2 or 3 papers? The descriptions of each category cannot be read in isolation, independently of the other categories that were available, and they cannot be read without examining how they were actually applied.
You seem to believe that categories 2 and 3 were "catch-all" default categories that anything that remotely appeared to endorse the consensus was lumped in to. This is not the case. As a concrete example, here is one of the papers that I rated as a 3 (implicit endorsement) during the first exercise, well before any tiresome semantic argumentation here about how papers should have been categorised, or whether "humans are causing global warming" means something different to "global warming is caused by humans" (!):
Methodology for adapting metal cutting to a green economy
The advent of global warming, as attested by the Intergovernmental Panel on Climate Change (IPCC), has serious ramifications for various facets of the human endeavour. Manufacturing is one such activity that is poised to undergo significant changes for curbing green-house gas emissions. This paper presents schemes for reducing the energy consumed in machining and grinding operations — the workhorses of any typical manufacturing set-up. These schemes are based on the mechanics of the metal cutting process which have been well studied over the years. Besides lowering the energy drawn from national grids, such schemes will pave the way for improved energy-efficient manufacturing processes. In the foreseeable future, alternative fuel systems to power manufacturing processes together with both newer operations and schemes, such as those outlined in this effort, would have a positive impact on the energy consumption of the manufacturing sector.
Searching for the paper in the database showed that Cook et al also categorised this paper as a level 3. The paper is like many others in this category — it is not about the causes of global warming, it is one of many about the impacts or (in this case) steps we can take to mitigate the impacts. Yet it quite clearly accepts the IPCC's findings. Indeed, it's the motivation for the paper. This is the kind of paper that you would not want counted as endorsing the consensus because it doesn't say "humans are responsible for >= 50% of global warming" while at the same time you do want counted papers (such as this) which don't say "humans are responsible for < 50% of global warming" and yet get categorised as level 5 by Cook et al:
I think you've made a fundamental error here. For a paper to unambiguously support the IPCC et al. consensus, it does indeed have to fall into category 1. But for a paper to unambiguously reject or deny it, it only has to fall into category 5, 6 or 7. That's 0.7 of 11,944 papers = 84 (maybe a few more, judging by the responses to some to the queries Popular Technology sent to known anti-AGW scientists whose papers had been rated). So the proper comparison, it seems to me, is 64 vs 84 or 64 out of 148 explicitly, unambiguously endorsing vs rejecting the IPCC consensus.
No. The categories are symmetric. If a paper numerically quantifies human responsibility, it's either a level 1 or a level 7. If a paper makes an explicit statement about human responsibility without numbers, it's either a level 2 or a level 6. If a paper implies or assumes causation, it's level 3 or level 5. You can't count only level 1 and compare it with level 5 + 6 + 7. If you don't trust the implicit categories, fine — ignore them. Measure 1 vs 7, or 1 + 2 vs 6 + 7. And then wonder why the figures come out so similar anyway.
Your mistake is to assume that papers in categories 2 and 3 must be ambiguous, and that unambiguous papers must be category 1. That is not the case. To exclude papers like the one I showed above would not result in an accurate reflection on the level of support for the IPCC consensus within the scientific literature.
To elaborate on the argument of 'overlap' in the ratings - there may be no overlap in the mind of each rater, as each interprets the set differently, but the total results may include plenty of overlap because of each rater interpeting differently. I note that there was a considrable amount of disagreement on rated abstracts for Cook et al, who then got to discuss and amend their ratings according to clarified criteria.
Isn't this strongly suggestive of problems with definitions? One third of endorsement ratings disagreed, and Cook et al had been discussing the criteria during the first phase of rating. Even after the second phase there was still 16% disagreement. There was no such process of clarification for the self-rating Authors. Are these not fair grounds to wonder if the comparative results are not as robust as indicated?
barry,
Out of interest I compared the ratings I gave to the ten papers in the original exercise with the ratings that Cook et al gave.
The paper that I rated as a 1, Cook et al also rated as a 1.
The five papers that I rated as a 4, Cook et al also rated as a 4.
That leaves four papers, all four of which I rated as 3's. Of those four, Cook et al rated one a 2, one a 3, and two as 4. So of the five that I rated as endorsing the consensus, Cook et al only counted three as endorsing the consensus, applying a much stricter test than I did for the most part.
The original authors, on the other hand, had a higher endorsement rate than I did — my average was 3.3, the original authors' average was 2.8. However, knowing that the original authors were assessing the entire paper explains why so many of the 4's were recategorised, such as this one that I (and Cook et al) rated as a 4 based on the abstract, but who's very first sentence in the introduction of the actual paper was:
The anticipated increases in greenhouse gases, especially carbon dioxide, in the atmosphere are predicted to raise temperatures by about 2.1 to 5.0 C globally within the next 100 years (Wigley and Raper, 1992; IPCC, 2001).
Note that the 16% disagreement was over all seven levels of endorsement category. In my case, the paper I rated as 3 that Cook et al rated as 2 would be counted as a "disagreement", even though it would have no impact on the final endorsement vs rejection percentage. I suspect that when lumping the 1+2+3 and 5+6+7 categories together the level of disagreement would have been much lower. And the two that I rated as 3 that Cook et al rated as 4 would actually lower the final endorsement percentage.
I'm not especially concerned that they had to revisit and discuss the interpretations of the criteria as real-world examples were encountered so they could reach a consensus on how the criteria should be applied; I've encountered exactly the same issue when marking exams with written answers. I'm also not concerned that the original authors may have individually been applying a slightly different interpretation, for the same reason that nobody seeks to rigorously define exactly what "beyond reasonable doubt" means to jury members. When you get large numbers of people responding, unusual interpretations tend to cancel out.
All these factors mean is that there is an error margin when it comes to the precise level of endorsement in the literature but nobody should be getting hung up on the precise percentage. The fact is that the results are overwhelming. As Tom mentioned before, in order for the original authors' level of endorsement to drop to 94.5% — a figure I would still consider overwhelming — you would need to believe that half of the original authors mistakenly assessed their papers as endorsing the consensus when they should have been rated as neutral. Now, some may have made a mistake, but to assume that half did beggars belief, and even if they did it still wouldn't change the take-home message.
I have seen statements here and there to the effect that such-and-such is the final nail in the coffin of "CAGW" and that there is a growing opposition within the scientific community to the IPCC's statements or that a "silent majority" of scientists are sick of IPCC alarmism and are gradually coming out of the woodwork to speak out against the "corruption" of science, but there is absolutely no evidence of this, and Cook et al actually shows the endorsement growing with time. If that belief was reflected in reality then we wouldn't need to be arguing over whether "humans are causing global warming" means something different to "global warming is caused by humans", there would be a lot more papers unambiguously rejecting the consensus. No matter which way you cut it, that category is tiny, and that's before we even get into the nitty-gritty detail about the actual quality of those papers.
Jason, can you define, precisely, the consensus position you rated under? If there are caveats regarding the 7 ratings, can you also indicate? I want to understand what you did.
I upthread asked for peopel to precisely define what they think is meant by 'consensus position' in Cook13, relative to rating options 2 and 3. I was interested to see if their was conformity or not.
a) >50% human influence
b) dominant human influence
c) significant human influence
d) some human influence
These are four ways that I can think of to legitimately interpret the consensus position for ratings options 2 and 3. When I did the public survey, I applied criterion d) (I missspoke earlier upthread). I am unsure as to what degree Cook et al would have applied to ratings 2 and 3.
Can you help me understand this? If a paper was rated as neutral, it was not included in the tally. Is that how this supposition is figured? Any case, that is not the argument I'm making. I need to know, definitively, the consensus position Cook et al were rating under. Is it this?
Because if the original Authors rated under the lower bar (any anthro influence) for 2 and 3, then the comparative results are not as robust as indicated.
To put it simply, and assuming;
1) Cook et al rate with the consensus of >50% anthro as the standard for all ratings bar neutral. They get 97% endorsement.
2) Original Authors rate 1) same as Cook et al, but rate 2) and 3) as endorsing any amount of human influence of global warming. 97% endorsement, also.
You don't see a problem with this potentiality?
barry,
Jason, can you define, precisely, the consensus position you rated under? If there are caveats regarding the 7 ratings, can you also indicate? I want to understand what you did.
I'll do you one better, I'll show you the abstracts I rated as 3 in addition to the one shown above.
1. http://www.sciencedirect.com/science/article/pii/S1359431106001980
I rated this as "3. Implicit endorsement" because the first sentence is "Climate change induced by global warming is a result of an excess of energy at the earth’s surface due to the greenhouse effect." My reasoning was that the author was unequivocal about the fact that global warming is due to the greenhouse effect, and since man is responsible for the enhancement of the greenhouse effect, I decided that it implicitly endorses man's responsibility.
Cook et al rated this as "4. Neutral".
2. http://onlinelibrary.wiley.com/doi/10.1002/asmb.530/abstract
I found this one to be tricky but ended up rating it as "3. Implicit endorsement" because the way the paper discusses "‘clean up’ (e.g. reforestation)" and "irreversible emissions" suggested to me that they were taking for granted man's effect with no indication that they thought anything else might be involved.
Cook et al rated this as "2. Explicit endorsement".
3. http://www.inderscience.com/info/inarticle.php?artid=4830
I rated this as "3. Implicit endorsement" because the paper talks about the economics of different greenhouse gas emissions abatement rates in the context of interactions between global climate change and the world economy, implying (to me) that they take for granted the relationship between emissions and climate change (discussing abatement rates would be pointless otherwise) with no indication that they thought anything else might be involved.
Cook et al rated this as "4. Neutral".
In each case, I felt that the authors were making a link between AGHGs and global warming and none of them indicated there was any other factor involved at all, so your four categories of ">50%", "dominant", "significant", or "some" don't apply at all — there is no clue in any of those abstracts that anything other than man is causing global warming.
Can you help me understand this? If a paper was rated as neutral, it was not included in the tally. Is that how this supposition is figured?
1,342 papers were rated by their authors as endorsing the consensus, while 39 were rated by their authors as rejecting the consensus. Hence the rate of endorsement was 1,342/(1,342 + 39) = 97.2%. If half of the "endorsement" papers were mistakenly rated as endorsing the consensus when they should have been neutral, the rate of endorsement would have been 671/(671 + 39) = 94.5%. (We are assuming here that no author is going to mistakenly rate their paper as endorsing the consensus when it actually rejects it, which seems to be reasonable given the motivation to draw attention to a "contrarian" paper that's passed peer review and the availability of levels 5, 6, and 7.)
Any case, that is not the argument I'm making. I need to know, definitively, the consensus position Cook et al were rating under. Is it this?
The consensus position Cook et al were rating under was very clearly spelled out in the paper: "that human activity is very likely causing most of the current GW (anthropogenic global warming, or AGW)"
Given they rated two papers as neutral that I thought were pretty clearly implicitly endorsing that very statement, it seems they used a very strict interpretation of it.
Because if the original Authors rated under the lower bar (any anthro influence) for 2 and 3, then the comparative results are not as robust as indicated.
That's a big "if", and one for which you are yet to provide any evidence whatsoever.
Why would authors rate under a lower bar? Why would they rate their own paper as "endorsing" if, in fact, it assumed or stated a minor human influence, when levels 5 and 6 were available for them to categorise their own paper? After all, the authors were working with their entire paper plus their own knowledge of what they actually meant by what they wrote, so if in their view their paper said nothing to imply something other than humans was the main cause in their entire paper, how can you argue that their paper is "actually" not endorsing the consensus?
Regarding Cook et al's ratings, if you feel that mistakes were made, feel free to check them yourself. They're completely open. There's no point discussing "potentialities" when you can actually check the reality. If you're just going to check a random subset, remember to count how many you check so we can assess the significance of your findings.
What gets me about this whole rigmarole is that in publication, almost no one actually questions the attribution studies that have been done. There aren't actually a large number of people working on this question. As CBD points out on another thread, "Actually, I am not aware of any 'legitimate scientists' who disagree that a doubling of atmospheric CO2 levels would (by itself without feedbacks) cause a little over 1 C of warming." Solar studies are virtually uniform in their pointing out that solar variation is either an insignificant and/or negative forcing for the trend of the last fifty years. Those two pieces of evidence alone should be enough to convince any scientist working in or near climate that anthro is the primary forcing behind the trend of the last fifty years.
The range of responses to the Cook study is telling. The un-engaged mainstream either accept it or don't, without investigation. Few of those aware of the attribution research quibble with the presentation of the study, because they know the above: the dominance of the human factor is pretty obvious. Now, who are these people who quibble? Why do they quibble? For some, it's obviously the same reason they quibble over Mann's "hockey stick" and the Marcott reconstruction. The message is simple and rhetorically powerful in the public domain, and undermining that message is job no. 1 (literally for some--Watts, Singer, Monckton, et al.). For others, I don't know . . . follow the leader, I guess. It's rather obvious, though, that if this level of scrutiny were applied to Soon & Baliunas (2003), Scafetta's work, or Chilingar et al., then there'd be fewer in the ranks of doubters and many more angry at the publication standards of petro journals (e.g. Energy & Environment).
Barry @225:
1) To read categories (2), (3), (5), and (6) as consistent (and hence exclusive) even qualitatively, you must take (2) to include all papers that explicitly state anthropogenic factors as a cause of warming, but do not minimize or otherwise indicate the impact is minimal. If you do not, you can have papers rated as both (2) and (5) or (6). Likewise (3) must also be taken as excluding papers which minimize the anthropogenic contribution. Ergo saying that doubling CO2 causes a 1 C no feedback warming, but a 0.5 C warming after feedbacks would preclude an abstract from being classified as (2) or (3), and would pretty much mandate its classification as (5). However, while you can assess the meaning of ratings (2)-(6) in isolation, doing so excludes relevant information. The description of ratings (1) and (7) makes it clear the benchmark for "minimizing" or "not minimizing" is the 50% mark.
On that basis, a phrase like "... anthropogenic factors are the major cause of recent warming" would get the abstract rated as affirming the consensus. In contrast, a phrase like "... anthropogenic factors are a major cause of recent warming" would downgrade it to the neutral bin in that it is possible with that phrasing that natural factors are equal or larger causes.
2) Because the evaluation of abstracts rated as (2) or (3) are based mostly on qualitative terms, it is likely that they will show more false positives than abstracts rated as (1). That does not justify distorting the meaning of the categories to minimize false positives. It is important to avoid false positives (and negatives); but it is more important that the characterization of the consensus be consistent across categories. Therefore arguing from the fact that the qualititive assessments in categories (2) and (3) will lead to false positives to the conclusion that the definition of the consensus differs for those ratings compared to category (1) puts the cart before the horse. The proper way to proceed to to determine the meaning of "affirms the consensus" from the available data from all rating categories and related comments; and if you do the position I am defending naturally follows. You then go on to assess the likely rate of false positives and/or negatives.
Because it is desirable to be conservative in these assessments, the way to avoid false positives is to be cautios in the ratings. Given the difference between the abstract ratings and the self ratings, I do not think there is any doubt that the Cook et al rating team were conservative. While there are likely to be false positives among the ratings, they are far exceded by false negatives. Arguments that the self rating data do not confirm that false negatives exceeded false positives depend essentially on ignoring what we know about the consensus of scientists and publication rates from other sources.
3) Turning to your first example, given only the data you provide, I would indeed have rated the paper as (3), and it would have been a false positive. It would also be an odd example, with the stronger result hidden in the body of the text. It would be rather like a newspaper article leading with "Dog growls at man" and mentioning in the last paragraph "assassination attempt on president". The reason for classifying the abstract as (3) includes the presumption that stronger, more interesting results will be mentioned in the abstract.
4) Turning to your second example, I would have classified it as neutral (4); and did so before I read the follow on part of the sentence.
5) Instead of checking category results, try doing the self rating excercise for 100 plus papers and check your level of agreement on endorsements with the Cook et al ratings. I am up to 35 so far, with complete agreement using my interpretation of what it means to confirm the consensus. I know from examples presented elsewhere that there are some abstract ratings I disagree with; but given that 12,000 abstracts were rated, it is unsurprising that there are some errors. You cannot check the validity of the rating system by picking out one or two errors; but only by rating a significant sample.
Jason,
my own rating for numbers 1, 2, 3 and the abstract from the previous post (4), were
1) -- 3
2) -- 3
3) -- 4
4) -- 4
That is made difficult by my sudden loss of ESP at a New Year's Eve party. Cook et al are bound to keep ratings of the scientists confidential, unfortunately.
First paper I could access in full under the category paleoclimate is "Pangaean climate during the Early Jurassic; GCM simulations and the sedimentary record of paleoclimate".
For the period modeled they discovered,
[italics are from the paper, page 548 - there is also a chart showing 2 X CO2 leading to a global temp increase of 0.3C; 4 X CO2 = 0.5C, and 6 X CO2 shows 0.7C rise]
The paper could be marked as 4 or possibly 7, under the rubric of >50% human influence, or 5 and 6 supposing 'minimising' the human influence is a purely qualitative rating. A rater may make choices under certain assumed knowledge or not (eg, do you extemporise from what you know of modern CO2 rise, or do you strictly focus on what the abstract/paper actually says, and don't incorporate your own knowledge?).
The abstract was rated 3 by Cook et al. The first sentence reads:
I don't have much free time, unfortunately. I'll try to come back to this later.
barry (234):
Based on what you've shown, I would also rate that as a 3 rather than a 4: although the authors are talking about non-CO2-related mechanisms for temperature change, they're concerned with the past - specifically 180 million years ago.
The meaning of the chart relating CO2 increase to temperature change would have to be further examined in the text to derive a clear implication. You can figure out an implied value for the climate sensitivity, but for what time period is it? What's their point?
barry @234, my own ratings and reasons are, in the order you present them (my rating first, then paper, then Cook et al rating followed by reason for my rating):
Note that the discussion of ocean circulation is not relevant to the rating as the changes are due to very large scale changes in continental arrangement which cannot be a factor for recent warming. I also note, with regard to the paper, that the model used specified SST, thereby eliminating the main feedbacks modellable in 1992 (water vapour concentration, sea ice extent) "... thus limiting the CO2 effect to direct radiative heating" (pp 546-7). Thus the additional factors in the full paper do not change the rating of the paper IMO.
nealjking,
The full paper is linked in my post. Here again below.
http://pubs.giss.nasa.gov/docs/1992/1992_Chandler_etal.pdf
Tom,
the highest rating one could have given that paper is a 4, because, as you point out, human influence doesn't factor. The key point in the abstract is that ocean circulation changes "may have been the primary force generating warmer climates of the past 180 m.y.". Nowhere in the abstract is it suggested that CO2 was a primary driver. In the full paper, CO2 is clearly 'minimised' (much less than 50% influence).
Your call for large sample size is well-taken. I wish I had more time to look at other papers. I do not understand how it could be hoped that I could validate my point, the self-ratings beng confidential. I can only explain how discrepancies in ratings may occur. I'm not sure how to proceed on this, and reiterate that it might be well for Cook et al to contact the Authors and get clarification on how they rated.
It seems you have a higher bar for rating than Cook et al. My concern, untestable, is that (some/many?) self-rating authors had a lower bar than Cook et al, making the close comparison less so.
I have no doubt that there is a consensus among self-rating Authors believe more CO2 in the atmosphere will cause warming (confirmation of AGW to some degree). I question whether the comparative results are as robust as Cook et al maintain, if Cook et al rated more stringently. The one example I have checked demonstrates the opposite, but does not increase my confidence in the results. I'll look further when there is time.
barry,
I would give the paper a 3: It doesn't really say ANYTHING about current climate trends, it seems to be restricted to the era 180 million years ago. That he concludes that CO2 may not have played much of a role in the particular warming then actually says nothing about what he would say about the warming now.
In fact, if I had to guess at the author's opinion, I would point out that he seems to use the global climate models (GCMs) with great confidence. So, I would guess that he would be likely to be confident about applying GCMs to the present-day climate as well: and since today's application of GCMs lends credence to the importance of CO2 (today), I would speculate that the author actually would likely support the consensus opinion, which is strongly supported by the GCMs.
But this is going beyond what is directly available in the paper. I would give it a 3, definitely not a 4 or a 2.
FWIW, the abstract of the Chandler paper clearly implies that CO2 should be expected to cause warming. A corollary of this implication is that anthropogenic CO2 emissions sould be expected to cause warming. Suggesting that ocean heat transport can explain the relatively warm Jurassic without CO2 does not logically contradict that. Rating the paper as a three sounds reasonable to me, although the implication is not as direct as in some abstracts.
barry @237, the issue in rating abstracts or papers are the implications regarding the warming from 1900 to 2012, and in particular the implications for the period from 1950-2012 where that can be distinguished seperately. In this case the implications are that rising CO2 causes warming.
The implication about changed ocean circulation are not relevant because the cause of that change is so distinct from anything hapening in the 20th century that it has no direct implications for the 20th century. This is particularly the case as the changed heat transfer due to ocean circulation is merely assumed in the paper as, "... effective full ocean models are not yet available, and the static mixed layer models used in some GCM experiments generate high SSTs that are inconsistent with temperature estimates made using the the geological record." (p 545)
The potential effects of "ground hydrological schemes" are also irrelevant as they are a feedback, not a forcing.
Consequently, of the trio of potential factors that could improve the modelling (according to the abstract), only one is directly relevant to 20th century temperatures; and hence only one can impact on the rating. That one is the influence of CO2. You are then left with the fact that the abstract and paper both indicate that CO2 influence temperature, but do not provide sufficient information to estimate whether that influence is likely to represent at least 50% of warming over the period 1900-2012 or 1950-2012. Hence a (4) IMO.
The only way you could rate the paper or abstract as rejecting the consensus is if you:
1) Forget that it deals with Jurassic continental distributions and hence has no implications for heat transport in the twentieth century;
2) For the paper, forget that the SST changes are imposed rather than modelled (note that this is not mentioned in the abstract, and nor is the generation of the model which cannot be derived from the year which is strictly not available for abstract ratings); and
3) For the paper, forget that the effects of doubling CO2 shown are for a state in which the majority of the feedbacks of the doublings have already been imposed in the initial state,ie, that the additional warming is for direct radiative heating only where some of the direct radiative heating overlaps with that from the pre-existing increased water vapour content in the atmosphere.
Dikran,
For me, the abstract "minimises" the role of CO2. I do not know how it can be read any other way, unless one were to completely ignore the first sentence. In fact, the first sentence strongly suggests a 7 rating might be appropriate. (Leaving aside, for the moment, that the period of interest falls outside the ratings - but then, why is paleoclimate a category at all, if not because lessons from the geological past have implications for today?)
nealjking,
I think most climate scientists would agree that anthro CO2 has caused a significant amount/most of the global warming for the last 50 - 100 years. But are they rating their papers or giving a vote according to their general opinion? This paper did not assess the validity of GCMs for current conditions.
Any case, Cook et al gave it a 3 based on the abstract, which I don't think is supportable when you consider options 5 and 6. How could the abstract not be seen as minimising the role of CO2, considered in the geological period they investigate? Alternatively, how could it not be marked a 4 if the time period is inapplicable?
Tom,
your points are well-taken. For your point 3), however, I believe they ran simulations to test for CO2-specific warming, and the feedbacks were not pre-imposed. Read the bottom of page 557, 1st column ("Table 3...", where they go on to state that feedbacks to CO2 forcing are the primary cause of warmth.
In the conclusions, they mention that their results have implications for "future" warming.
I wonder, Tom, what you make of Dana Nuticelli's comment;
That looks to me like a qualitative statement rated as a qunatitative one.
This paleo abstract (1992), assigning no value to anthro greenhouse warming, or even mentioning anthropogenic influence, was rated 3.
No mention of anthro influence or value in the body of the paper either, but at one point it is remarked that the effect of enhanced greenhouse gases in the Early Eocene may have been 'minor'. No idea how a self-rating Author would rate this one.
This 1993 abstract assigns no value to CO2 warming, saying only that post-industrial increases in CO2, CH4 and N2O "may well have contributed to the observed global warming." Marked as 3 (no full version).
.....
I've been skimming titles in paleoclimate, ignoring ones that do not immediately present themselves as pertaining to the modern period. But I've noted that Cook et al have rated at level 3 any paper which suggests enhanced CO2 levels should cause global warming. Most times, the abstracts I've looked at make no mention of antrho influence or the modern period, and give no quantification on CO2 contribution to warming.
Contrary to my thesis upthread, it appears that Cook et al may have rated at the lower bar (any amount of warming from enhanced CO2). If so, this would make the comparative results Cook13/self-rating Authors less problematic.
Tom, it appears you may have overestimated the ratings stringency of Cook et al. Scroll through the paleoclimate category and I believe you will concur.
barry,
I believe your interpretation of the first sentence of the abstract is absurd and perverse: It's pretty clear that they're NOT including the last 200 years within the period of study, which comprises 180 million years. For one thing, they contrast the warming of the period under study from that of "the present": You can't do that unless they're mutually exclusive. A minimization of the role of CO2 during the Jurassic has precisely nothing to imply about the role of CO2 today; especially since CO2 did not jump from 280 ppm to 400 ppm within 200 years during the Jurassic, as it has in the present era.
Further,
barry, this is the continental configuration during the Jurassic:
This is the continental configuration now:
The paper and abstract asserts that the change in ocean currents brought about by that difference in continental configuration between the modern and jurassic may be enough to account for a 5-10 degrees C difference in GMST. Do you wish to suggest that that is significant (or even relevant) evidence suggesting that the motion of Africa 2.15 meters to the North East over the last century has caused a significant portion of the 0.7 C warming over the last 100 years? If you do not, then the effect of the very large difference in continental positions between the modern and the Jurassic is irrelevant to warming over the last century, and therefore irrelevant to the classification of the paper (and abstract).
I have chosen the case of Africa as it currently has the largest current impact on GMST by my understanding. You may prefer to use the (at most) 10 meter increase in the width of the Atlantic over the same interval, and argue that that has significantly increased GMST. In fact, pick any continental motion, or sum of them and tell me you think there is evidence that their combined effect has caused 0.2 C warming over the last century (ie, sufficient to be statistically distinguishable) and show me a remotely plausible theory as to how it could do so, and I will accept that the paper should have been rated a six or seven.
Absent that, you are merely focussing on an irrelevancy in assessing the paper, and IMO a transparent irrelevancy.
Barry @242:
How is that not clear enough for you?
I have already summarized this quote, including a quote of the final crucial phrase and directed you to the part of the article which contains it; and yet now you want to put your hazy understanding against the author's direct statement.
barry,
You might be better off using a category other than Paleoclimate for your checking.
Firstly, due to the fact that it deals with the ancient past, it has almost the equal-lowest "relevance" rating of the four categories (only 22% of papers were classified as expressing an opinion on the topic, vs nearly 57% of Mitigation papers and 32% of Methods papers) and is more likely to include factors that are going to make it difficult to categorise without a deep understanding of the topic.
Secondly, it is also the smallest category (6.6% of all papers), and so even if you somehow prove that every one of the papers classified in favour of the consensus should have been neutral instead, it will have almost no impact on the final result. (If you want to show a problem in general, and not just with Paleoclimate papers, then I would argue you should randomly select from all papers and not just a particular category.)
If you are going to pick a particular category, I'd suggest Mitigation: it has the highest Consensus rating (99.84% — 1,912 rated in favour, just 3 rated against; 20 × level 1 vs 0 × level 7, 418 × level 2 vs 2 × level 6, and 1,474 × level 3 vs 1 × level 5) as well as the highest percentage of papers rated as being relevant (nearly 57%).
Regarding the four papers we both checked, Cook et al rated two as endorsing the consensus and two as neutral, and you rated two as endorsing and two as neutral. Even though you agreed with Cook et al on two and disagreed on two, the net effect was the same because the two you disagreed on went opposite directions. Of course, this sample is far too small to draw any conclusions, so you might want to use the tool to try a much larger set of papers and compare your ratings with Cook et al, but the point is this: when the numbers are so highly skewed in one direction, even gross errors have very little impact on the final results, and so far there's no indication of gross errors. As Tom originally pointed out, even if half of the original authors mis-classified their papers as endorsing the consensus instead of being neutral, the results would still be overwhelming.
As for finding full papers that should be classified differently to what you would get based on the abstract alone, in and of itself this isn't a problem — it's only a problem if abstract classification is not an unbiased estimator of paper classification. In other words, if papers endorsing the consensus are more likely to say something to that effect in their abstract than papers rejecting the consensus (or vice-versa). The fact that such a large number of papers that Cook et al classified as neutral were rated by their authors as taking a position on the topic can be a simple reflection of the additional information in the paper (or the fact that they don't have to try to figure out what they mean) rather than any indication that the authors used a lower "bar" than Cook et al.
If you are going to keep checking papers, don't forget to report how many you checked that you didn't have a problem with the rating of. If you pre-filter the information to only report those that you disagree with, then we have no way of knowing what the impact of that is. Disagreeing with 100 out of 100 papers is very different to disagreeing with 100 out of 10,000.
barry Did the simulations performed for the paper include the last 200 years, and did they include the rise in atmospheric CO2 over this period due to anthropogenic emissions?
(Hint: I rather doubt they simulated the whole of the last 180 million years as this would have taken rather a lot of time to run on the computer).
Some observations:
At this point, the circling of the arguments re: consensus, and the wash/rinse/repeat cycle of arguing over individual abstracts absent the context of statistics on how many are inarguable, strikes me as excessive repetition as per the Comments Policy.
Perhaps folks could reserve further comments to aspects of this topic not already reviewed ad nauseum?