## Phil Jones - Warming Since 1995 is now Statistically Significant

#### Posted on 14 June 2011 by dana1981

As all Skeptical Science readers are undoubtedly aware, in February of 2010, Phil Jones was asked some loaded questions in an interview with the BBC. Several of the questions were gathered from "climate sceptics", and Jones' answer to the second one has been widely re-published and distorted:

"Do you agree that from 1995 to the present there has been no statistically-significant global warming?""Yes, but only just. I also calculated the trend for the period 1995 to 2009. This trend (0.12C per decade) is positive, but not significant at the 95% significance level. The positive trend is quite close to the significance level. Achieving statistical significance in scientific terms is much more likely for longer periods, and much less likely for shorter periods."

Why choose 1995 as the starting point in this question? Well, that is the closest year for which the answer to this loaded question is "yes". From 1994 to 2009, the warming trend in the HadCRUT dataset was statistically significant at the 95% confidence level (CL). It's also worth noting that there's nothing magical about the 95% CL - it's simply the most commonly-used interval in scientific research, but it's also true that the HadCRUT 1995-2009 trend was statistically significant at a 93% confidence level.

In other words, using Jones' data, we could say with 93% confidence that the planet had warmed since 1995. Nevertheless, this did not stop numerous mainstream media outlets like Fox News claiming that Phil Jones had said global warming since 1995 was "insignificant" - a grossly incorrect misrepresentation of his actual statements. The Daily Mail warped the truth even further, claiming Jones had said there was no global warming since 1995. **These media outlets turned 93% confidence of warming into "no warming".**

Furthermore, the HadCRUT dataset excludes portions of the Arctic where there are no temperature stations. The Arctic also happens to be the fastest-warming part of the planet. NASA's GISTemp, whose data analysis extrapolates for the Arctic temperatures using the nearest temperature stations, did find a statistically significant warming trend at the 95% CL from 1995 to 2009. So not only are the "skeptics" cherrypicking the start date, they're also cherrypicking a dataset which doesn't cover the whole planet.

Deep Climate has detailed the history of the 1995 cherrypicked starting point. It appears to have originated with an email from Richard Lindzen to Anthony Watts, which was subsequently published in a post on WattsUpWithThat (WUWT):

Look at the attached. There has been no warming since 1997 and no statistically significant warming since 1995. Why bother with the arguments about an El Nino anomaly in 1998? (Incidentally, the red fuzz represents the error ‘bars’.)

Best wishes,

Dick

Luboš Motl made a similar argument in December 2009 using UAH satellite data, which was also published on WUWT. Two months later, the question was posed to Phil Jones in the BBC interview, which suggests strongly that it originated from Motl, Lindzen, and/or Watts. Regardless of the source, what really matters is that the question was based on a cherrypicked starting date, and on a somewhat arbitrary statistical confidence level, and that the media subsequently distorted Jones' response.

In January 2009, Tamino at Open Mind analyzed the data after removing the influence of exogenous factors like El Niño, volcanic eruptions, and solar variation from the temperature data. Tamino concluded that "*until 2001 the warming is statistically significant*" (Figure 1).

*Figure 1: HadCRUT3v estimated warming rates from the plotted date to Present with 2-sigma error bars, using exogenous factor-compensated temperature data (Open Mind)*

Another year has passed since the original BBC interview, and in a new BBC article, Jones notes that the HadCRUT warming trend since 1995 is now statistically significant.

"Basically what's changed is one more year [of data]. That period 1995-2009 was just 15 years - and because of the uncertainty in estimating trends over short periods, an extra year has made that trend significant at the 95% level which is the traditional threshold that statisticians have used for many years.

"It just shows the difficulty of achieving significance with a short time series, and that's why longer series - 20 or 30 years - would be a much better way of estimating trends and getting significance on a consistent basis."

As Jones notes, and as scientists like Lindzen and Motl should very well know, trying to assess trends in the noisy global temperature data over periods as short as 10-15 years is pointless. There's just too much short-term noise, which if you're going to look at such short-term data, you at least need to attempt to filter out first, as Tamino did.

So to sum up, a cherrypicked starting date chosen by a couple of "skeptics" (Lindzen and Motl) and published by a "skeptic" blog (WUWT) was picked up and passed along in the form of a loaded question to Phil Jones in the BBC interview. Phil Jones' answer was subsequently (and predictably) grossly distorted by various media outlets, who turned 93% confidence of global warming into "no global warming".

In reality, the HadCRUT warming trend since 1995 was statistically significant above the 90% CL, the GISTemp warming trend (which does not exclude the Arctic) was significant at the 95% CL, and by removing short-term effects, even HadCRUT has been significant at the 95% CL since 2000. One year later, we can now say that the HadCRUT warming trend since 1995 is statistically significant at the 95% CL, even including the exogenous factors.

Unfortunately, the main consequence of this sequence of events was that much of the public was misinformed by media articles claiming that global warming since 1995 was "insignificant" or non-existent, which are both factually incorrect statements. Misleading the public may well have been the goal of those individuals who originally cherrypicked the 1995 starting date and the HadCRUT dataset, and if so, they succeeded. And not surprisingly, Anthony Watts continues to mislead his readers, claiming Phil Jones' comments are "an about face...From the “make up your mind” department", when in reality Jones' comments have been consistent and accurate throughout.

This reactions to this story have revealed a number of media outlets whose aim is not to accurately inform their readers with regards to the climate, but rather to misinform them.

Dikran Marsupialat 22:44 PM on 15 June, 2011Bernat 22:46 PM on 15 June, 2011Dikran Marsupialat 22:59 PM on 15 June, 2011Eric (skeptic)at 23:08 PM on 15 June, 2011MoreCarbonOKat 23:38 PM on 15 June, 2011Moderator Response:(DB) Please take care to formulate your comments to better comply with this site's comment policy.MoreCarbonOKat 00:01 AM on 16 June, 2011Tom Curtisat 00:15 AM on 16 June, 2011JMurphyat 00:15 AM on 16 June, 2011MoreCarbonOK- you only have to look at his website, to see classics like this :A short summary of the basic results of my study: CO2 is insignificant as a greenhouse gas. CO2 is not a poison or a pollutant. CO2 is one of the two main building blocks of ALL plant life on Earth. CO2 concentration has been up to ten times higher in the past CO2 is good.To start off with, I found Svante Arrhenius' formula completely wrong...It appears that we have not even yet reached the level of warmth compared to previous warm periods in known history, such as for example, the Medeviel Warm Period (MWP) - 1000 years ago, when Greenland was really green. See here:(Link to WUWT removed)Now the opposite, global cooling, that really is something to worry about....There are some people there who make admirable attempts at showing him where he is going wrong, but there are none so deep in denial as those who believe that they have the secret answer and who believe they are more intelligent than the rest of us mere mortals !KRat 00:16 AM on 16 June, 2011Tom Curtisat 00:18 AM on 16 June, 2011Tom Curtisat 00:30 AM on 16 June, 2011Dikran Marsupialat 00:31 AM on 16 June, 2011assuming the null hypotheis is true. That does not mean that there is a 1 in 20 chance that the null hypothesis actually is true. This is mixing frameworks, but the p-value is p(x>X|H0), where x is the trend and X is the observed trend and H0 is the event that the null hypothesis being correct, however what you actually want is p(H0|x>X), which you can get by Bayes formula p(H0|x>X) = p(x>X|H0)p(H0)/(p(x>X|H0)p(H0) + p(x>X|H1)p(H1)) however the hypothesis test does not involve p(H0), p(H1) (the prior probability that the null and alternative hypotheses are correct) and p(x>X|H1), which measures how likely such an extreme statistic is assuming that there is a secular trend. Essentially a frequentist hypothesis test does not allow you to assign a numeric probability to either hypothesis being correct as it can only assign probabilities to things that have long run frequencies and the a particular hypothesis is either true or ot isn't. However, that is exactly the form of answer we actually want, which is why the p-value is so often misinterpreted that way. That makes it very difficult to give an accessible answer to the question that is also correct as statistical significance simply doesn't mean what people think it means. JMurphy Absolutely, I gave him the benefit of the doubt, but he showed he was a denialist by being unwilling to engage with the substantive technical criticism of his methodology. The hubris was priceless, though ;o)Dikran Marsupialat 00:38 AM on 16 June, 2011KRat 00:42 AM on 16 June, 2011don'thave multiple publications in statistics): Would this be more correct?"Statistical significance at the 95% level means that there is a 1 in 20 chance of seeing a trend as large or larger assuming the null hypotheis is true, while statistical significance at the 93% level means that there is a 1 in 14 chance of seeing a trend as large or larger assuming the null hypotheis is true. So again, while 93% significance over that period is suggestive, the short time frame data does not meet the statistical significance threshold, which is why 20-30 years of data is preferred to clearly identify trends."The major point aimed for here is to emphasize that while the data ispointing in a particular direction, it's not doing so strongly enough to meet the standard test - hopefully not leaving room for the interviewer (or onlookers) to say"No warming since 1995"...Albatrossat 00:46 AM on 16 June, 2011John Russellat 00:55 AM on 16 June, 2011"Media training is vital for any scientist that need to have contact with the press, but nothing can provide a 100% bulletproof defense."Well then, I think we agree; because I was never saying there is abullet proofway to respond. Rather I was saying that, in his innocence, Phil Jones made it much too easy and there would have been ways of expressing an honest answer that would have been more difficult to twist -- or even if twisted, easier to demonstrate as obfuscation. I'm sorry I can't come up with one, but that's because I'm not an expert on the science. Cheers!macolesat 00:56 AM on 16 June, 2011Dikran Marsupialat 01:04 AM on 16 June, 2011KRat 01:16 AM on 16 June, 2011"I don't mean to be argumentative, it is just that Jones did a very good job of dealing with this very difficult question. I don't think I could have done any better without the use of diagrams and a page or two of text."I agree. I find itextremelydifficult to talk without a whiteboard sometimes.dana1981at 01:35 AM on 16 June, 2011Rob Honeycuttat 01:44 AM on 16 June, 2011Rob Honeycuttat 01:56 AM on 16 June, 2011dana1981at 02:07 AM on 16 June, 2011lesat 02:11 AM on 16 June, 2011Daniel Baileyat 02:13 AM on 16 June, 2011"That's an interesting question, and let me tell you why it's an interesting question..."First align with the questioner, then differentiate to reveal the fallacy of the question.Philippe Chantreauat 03:27 AM on 16 June, 2011Dikran Marsupialat 03:30 AM on 16 June, 2011Daniel Baileyat 03:50 AM on 16 June, 2011Rob Honeycuttat 04:05 AM on 16 June, 2011JMurphyat 04:26 AM on 16 June, 2011Dikran Marsupialat 05:08 AM on 16 June, 2011Rob Honeycuttat 05:19 AM on 16 June, 2011Daniel Baileyat 06:22 AM on 16 June, 2011Charlie Aat 06:41 AM on 16 June, 2011Charlie Aat 06:50 AM on 16 June, 2011Rob Honeycuttat 07:34 AM on 16 June, 2011Charlie Aat 08:02 AM on 16 June, 2011Moderator Response:(DB) Actually, Tamino's analysis showed that the warming since 2000 and earlier was statistically significant.Tom Curtisat 10:08 AM on 16 June, 2011Albatrossat 14:09 PM on 16 June, 2011Butnow that he has determined that the warming in the HadCRUT3 (variance adjusted data) does meet the 95% criterion, they suddenly do take issue. Note too that they ignore the fact that the warming does meet the 95% confidence level in the GISTEMP data, not to mention all the other signs that the planet is warming: [Source]Charlie Aat 15:41 PM on 16 June, 2011Charlie Aat 15:44 PM on 16 June, 2011Albatrossat 16:06 PM on 16 June, 2011Albatrossat 16:14 PM on 16 June, 2011Also, have you asked yourself what type of statistical analysis and data Lindzen used to arrive at his conclusion that there had been no stat sig warming? Don't see you questioning his stats analysis, never mind the content of his email to Watts. He was showing Watts how to cherry pick and distort. Are you OK with that? I find your insinuations that Jones behave nefariously in this whole fiasco manufactured by Lindzen to be beyond the pale."Why choose 1995 as the starting point in this question? "Dikran Marsupialat 17:17 PM on 16 June, 2011even during long term forced global warmingof a decade or two, for precisely this reason. In other words the effectivepowerof the test is very low. Lastly, a test of statistical significance provides evidence for making some claim based on the value of a single statisticalone. That is,ifthat one statistic were the only evidence you had that the climate was warming, you could only claim the planet was warming if there were a statistically significant trend. But the trend from 1995-2010 is not the only evidence we have, so Jones is perfectly justified in having 100% belief that the world is warming. Basically anyone who is making a big song and dance about the trend not being significant is defficient in their statistics for not understanding the multiple-hypothesis testing issue involved in the cherry picking of the start point to suit one particular argument, and defficient in their understanding of climate physics for not understanding why you wouldn't expect the trend to be statistically signficant with high probabilityeven if there is a secular trend of the expected magnitude.Dikran Marsupialat 17:27 PM on 16 June, 2011Dikran Marsupialat 17:39 PM on 16 June, 2011Bernat 19:44 PM on 16 June, 2011Dikran Marsupialat 20:44 PM on 16 June, 2011John Russellat 20:55 PM on 16 June, 2011Moderator Response:[Dikran Marsupial] Done (I have made a long and detailed post, I'll happily discuss it further if it was over the top ;o)snappleat 21:25 PM on 16 June, 2011