Arguments
How contrarians used pal review to publish contrarian papers
What the science says...
The lone documented case of true 'pal review' was committed by climate contrarians in the journal Climate Research from 1997 to 2003, during which time editor Chris de Freitas accepted 14 papers from a select group of contrarians. The journal had not published any papers from that group of authors previously, and only published 2 more papers from the group of 'pals' after de Freitas left.
Climate Myth...
Climate science peer review is pal review
"Peer review has become ”pal review.” Send a paper to one of the very many journals published by the American Geophysical Union–the world’s largest publisher of academic climate science–and you can suggest five reviewers. The editor doesn’t have to take your advice, but he’s more likely to if you bought him dinner at the last AGU meeting, isn’t he? That is, of course, unless journal editors are somehow different than government officials, congressmen, or you." (Patrick Michaels)
We often hear claims from climate contrarians that climate scientists are guilty of what they describe as "pal review." The conspiracy theory goes something like this - climate scientists conduct biased research with the goal of confirming the human-caused global warming theory. They then submit their biased results to a peer-reviewed journal with friendly editors ("pals") who pass their paper along to friendly reviewers (other "pals") who give their fraudulent work the green light for publication. Thus, the contrarians argue, the preponderance of peer-reviewed literature supporting human-caused global warming is really just a sign of corruption amongst climate scientists.
However, while climate contrarians are never able to produce any evidence to support their conspiracy theory, John Mashey has thoroughly documented a real world example of true pal review. Contrary to the standard conspiracy theory, the pal review did not involve mainstream climate scientists, but instead the climate contrarians themselves.
The True Story of Climate Research Pal Review
Mashey has done an excellent job documenting a real life case of pal review, which happened at the journal Climate Research between 1997 and 2003. That particular journal was once again brought to the forefront in the recent second Climategate stolen email release.
In those emails, various climate scientists had expressed concern that Climate Research was publishing shoddy papers by a small group of climate contrarians, and discussed what they could do about it. The most infamous of these papers was one by Soon and Baliunas (2003) which concluded that current global temperatures are not anomalous compared the past 1,000 years. After publishing this paper, Soon was invited by Senator James Inhofe to testify before US Congress, and the Soon and Baliunas paper was used by Congressional Republicans to justify opposition to climate legislation.
However, the paper contained numerous major fundamental flaws, such as equating dryness with hotness, and was subsequently roundly refuted by an article in the American Geophysical Union journal Eos written by a number of prominent climate scientists. This paper, and Climate Research's refusal to revise or retract it, led to the resignation of five of the journal's editors, including recently-appointed editor-in-chief Hans von Storch, who explained the reason for his resignation:
"..the reason was that I as newly appointed Editor-in-Chief wanted to make public that the publication of the Soon & Baliunas article was an error, and that the review process at Climate Research would be changed in order to avoid similar failures. The review process had utterly failed; important questions have not been asked....It was not the first time that the process had failed, but it was the most severe case....I withdrew also als editor because I learned during the conflict that [Climate Research] editors used different scales for judging the validity of an article. Some editors considered the problem of the Soon & Baliunas paper as merely a problem of "opinion", while it was really a problem of severe methodological flaws. Thus, I decided that I had to disconnect from that journal, which I had served proudly for about 10 years."
In short, the journal's chief editor voiced the exact same concerns as the climate scientists in the Climategate 2 emails - that certain Climate Research editors were systematically publishing methodologically flawed papers in their journal. Soon and Baliunas were far from the only climate contrarians to benefit from the journal's friendly editorial policy. In fact, the biggest pal review beneficiary bears a very familiar name.
Patrick Michaels and Pals
Mashey has examined the publications in Climate Research in great detail, and has produced a spreadsheet of its publications and a report summarizing his findings.
Prior to Hans von Storch's promotion to Climate Research editor-in-chief in 2003, the journal did not have a chief editor, and so authors sent their manuscripts to an Associate Editor of their choice. One particular Associate Editor, Chris de Freitas, published 14 separate papers from a select group of 14 climate contrarians during the 6 year period of 1997 to 2003:
Sallie Baliunas, Robert Balling, John Christy, Robert Davis (both Climate Research author and editor), David Douglass, Vincent Gray, Sherwood Idso, PJ "Chip" Knappenberger, Ross McKitrick, Pat Michaels, Eric Posmentier, Arthur Robinson, Willie Soon, and Gerd-Rainer Weber.
As Mashey shows, from 1990 to 1996, Climate Research published zero papers from this group. From 1997 to 2003, the journal published 17 papers from this group, 14 with de Freitas as the Associate Editor. Serial data deleter Patrick Michaels was an author on 7 of the 14 pal reviewed papers, which also accounted for half of his total peer-reviewed publications during this timeframe. During this period, 14 of the 24 (58%) papers accepted by de Freitas came from this group of contrarians. After von Storch's resignation in 2003, de Freitas published 3 more papers from authors outside this group before leaving the journal in 2006.
Another on the list of 'pals', Robert Davis, was another Associate Editor at Climate Research who accepted 36 papers during his tenure, two of which were co-authored by another pal, Robert Balling. The journal also published 5 other papers from this group by non-pal editors. However, in total, at least 16 of the 21 (76%) of the papers published by Climate Research which were authored by this group of climate contrarians had pal review editors, mostly de Freitas (67%) during this six year window.
After von Storch's resignation, Mashey documents that the pals' Climate Research publications dried up. Davis accepted one of Balling's papers submitted in 2004, and papers co-authored by Balling and by de Freitas were published by the journal in 2008 (Table 1). 18 of the 21 (86%) of the 15 pals' Climate Research publications were submitted in the 1997 to 2003 timeframe.
Table 1: Climate Research publications grouped by Associate Editor. Grey bars show approximate editor tenure as derived from received dates of papers. The "pals" papers are shown in red capitals, 14 accepted by de Freitas (bold), and 7 handled by others (red, underlined italics). De Freitas also accepted 13 seemingly normal papers from other authors (lowercase black).
Mashey also finds that the 15 'pals' were closely connected in climate contrarian activities outside of Climate Research as well, for example working for various anti-climate think tanks, most being connected with either Fred Singer or Patrick Michaels.
"all have shown persistent involvement with organizations that do climate anti-science, most of which also have tobacco connections."
There is also substantial overlap with the pals joining together to author these papers (Figure 1).
Figure 1: Overlap between pal authors of the 14 de Freitas Climate Research pal review publications between 1997 and 2003. The node numbering represents the Climate Research volume and page number of the pal publications, while the node connections represent papers written by the same pal authors (i.e. 9.3p14 and 23.1p15 were both authored by Michaels and Knappenberger). Image by jg and Kevin C.
The Purpose of the Mainstream Pal Review Myth
For those who oppose the prudent path forward with regards to climate change, which involves major global greenhouse gas emissions reductions, the scientific consensus on human-caused global warming is a very inconvenient thing. Despite the public relations damage resulting from Climategate, people still trust climate scientists' opinions about climate science (although political conservatives' trust in scientists in general has declined). However, much of the public (at least the American public) doesn't realize that there is a scientific consensus on human-caused climate change. Polls in October 2010 and September 2011 found that 44% and 37% of the American public believes that scientists are divided regarding the cause of global warming, respectively.
According to the March 2012 George Mason Center for Climate Change Communication (CCCC) national poll, climate scientists are the most trusted source for climate science information, with 74% of public trust (Figure 2). However, a large segment of the population believes there is a major scientific debate on the subject, no doubt thanks to the false media balance which gives the ~3% minority of experts who think humans aren't the dominant cause of the current climate change (and their non-expert surrogates) ~50% of the media attention. Therefore, many people don't believe that humans are the primary cause of global warming (approximately 41% of Americans).
Figure 2: Responses to the George Mason CCCC poll question "How much do you trust or distrust the following as a source of information about global warming?"
The numbers reveal a stark picture: 76% of Americans trust climate scientists, but 41% think scientists are divided on the causes of the warming, and 41% think the observed warming is mostly natural.
Thus as Ding et al. (2011) concluded, if a larger percentage of people realized that there is a scientific consensus on the issue amongst the group they trust most on the subject (and rightly so), more people would believe that humans are causing global warming, and more people would demand that we do something about it. The lack of public awareness of the scientific consensus on human-caused climate change is one of the biggest obstacle to taking climate mitigation action.
For this reason, climate contrarians have attacked the scientific consensus from many different angles. Some have tried to attack the credibility of the many different surveys and studies documenting the consensus. Others simply ignore this documentation and deny the consensus exists at all.
The third group, discussed in this post, attacks the credibility of the consensus itself, claiming it's all part of a massive fraudulent conspiracy of thousands of corrupt climate scientists (note that conspiracy theories are one of the five characteristics of scientific denialism). Ironically, this conspiracy theory has been most recently voiced by pal review beneficiary Patrick Michaels.
"Peer review has become ”pal review.” Send a paper to one of the very many journals published by the American Geophysical Union–the world’s largest publisher of academic climate science–and you can suggest five reviewers. The editor doesn’t have to take your advice, but he’s more likely to if you bought him dinner at the last AGU meeting, isn’t he? That is, of course, unless journal editors are somehow different than government officials, congressmen, or you."
Michaels of course provides no evidence whatsoever to support this conspiracy theory of peer-review corruption. He expects us to swallow his tale of "pal review" - the conspiracy theory that thousands of climate scientists are publishing thousands of biased papers every year in order to keep the human-caused global warming theory propped up - based on nothing more than his say-so.
While Michaels is indeed something of an expert on the subject, his expertise comes from himself being one of the individuals most guilty of engaging in climate research pal review.
Pal Review Summary
While Patrick Michaels has accused mainstream climate scientists of a vast conspiracy involving pal review (and exposed his own characteristic of scientific denialism in the process) without any substantiation or supporting evidence, in reality Patrick Michaels himself was the biggest beneficiary in the one actual demonstrated case of climate science pal review, as documented by Mashey.
A group of 14 climate contrarians found a sympathetic journal editor who proceeded to publish a large number of papers from this group over a very short timeframe, many of which were scientifically flawed, some of which were subsequently used by politicians to oppose climate legislation.
Ironically, the climate scientists who tried to do something about this problem have themselves been accused of trying to "hijack" or "subvert" the peer-review process. And of course the guiltiest party of all, Patrick Michaels has accused thousands of climate scientists of the sort of pal review he himself engaged in.
Our tale is one of irony, hypocrisy, and projection. The next time you see a complaint about the fairy tale of rampant climate science "pal review", direct the accuser to John Mashey's documentation of a pal review true story.
Intermediate rebuttal written by dana1981
Update July 2015:
Here is a related lecture-video from Denial101x - Making Sense of Climate Science Denial
Last updated on 19 July 2015 by MichaelK. View Archives
I think this is somewhat misleading. We dont use Peer Review in science to prove that what we write is correct. No peer reviewer can do that. No - what peer review is for is to ensure that the conclusions given in the paper are properly supported by the evidence presented. A peer reviewer cannot be expected to go behind the scenes to check from the raw data. ( I myself have published papers with errors which a peer review process did not detect). The point of publishing is to air your views and allow others, who may have different conclusions to refute your work. This is what scientific debate is all about. By polarizing peer review using words such as "pal review" we are missing the point. I see no problem in a "pal review" - thats just asking your friends to check your paper and ensure its based on sound logic. Let them publish thier stuff - and then others (opponents) can publish why they are mistaken. Lets keep the debate where it belongs - on the technical issues.
MartinG @1, in the early days of science, anyone recognized as able to contribute could ask to present a paper to a learned society, and would do so. Their paper would then be published in the societies proceedings. This system worked because there were so few scientists that, even with this all comers approach, any interested person could read all the new papers in a given year, at least for a given country. As the number of scientists grew, the system broke down and peer review was introduced. The need that peer review satisfied, and hence its purpose, was to restrict publication so far as was possible to those papers that were worth reading. Above all else, peer review is a mechanism to ensure that scientists are able to devote their attention to papers of merit. It is a mechanism for filtering out scientific spam.
A paper can have merit without being correct, or indeed, without being free from error. But it must be free from obvious error. Further, a paper can be without error and correct, and lack merit; either because the subject has been covered many time before and the paper adds nothing new, or because the question addressed by the paper is simply of no scientific interest.
The problem with pal review is that it subverts the system as a filter of scientific spam. Your pals may do you a favour in getting your publications up even though they think the paper is without merit. More troubling, if they are politically (or has happened, religiously motivated) they may conspire to ensure the publication of your paper because it says the right things, from their perspective, and without regard to the actual scientific merit of the paper. Certainly some creationist papers have been published by this means, and the evidence is fairly clear that papers that would not otherwise have been published, have been published by pal review simply on the basis that they are critical of the concensus.
IMO, that AGW deniers have so conspired is a tacit admission that their work lacks merit. Had they been confident of their works merit, they would have spurned pal review in favour of the genuine article. But they appear to have decided it was more important politically to have the various papers in press than that they should be good enough to go through the normal rigours of peer review. (Please note, there are some skeptical papers that have gone through normal peer review. Those papers deserve the same respect that any other paper that goes through peer review deserves.)
MartinG, you did read the article, yes? Do you accept what de Freitas did? The Wegman affair is another example--perhaps the richest in irony. When we say "pal review," we don't mean getting one's friends to check one's math. We mean getting one's friends to pass one's work through the peer-review process with just a glance.
DSL - My point is with the idea of "Pal review". I actually have no problem with what de Freitas did - he got his papers peer reviewed and published. Its actually no error on his part if the reviewers were all climate sceptics any more that if they had all promoted AGW, though personally if I was editor I would ensure a balance. Its only when we label everybody as either pro- or anti that we get a problem. I cannot judge if all the papers involved were worthy of publication or not since I havnt read them, but I assume they were if they got past the peer review process. If they were bad then there should have been refused by the editor and reviewers, and the result will be that the editor, peer reviewers, and the journal will fall into disrepute. But I refuse to condemn them just because they have a different opinion of whats hapenning in the world. But if somebody shows me that the papers de Freitas got published were poorly written, with unsupported conclusions etc. then I will agree they shouldnt have been published.
In my discipline the literature is unfortunately full of poor quality papers and we have to learn to sift through to find the goodies just as we do with internet searches. Peer review was initially a mechanism to ensure that scientists are able to devote their attention to papers of merit. The days when an article was worth reading just because it had been peer reviewed are long gone I fear - and thats why I believe we should look at content, not on whether something has gone through peer review or not. This article and its comments have not been peer reviewed - and I am reading them!
MartinG - I would have to disagree, as I have serious concerns about de Freitas getting published in the first place (not to mention his editorial practices). "Pal review" emphasizes ideological/personal approval over scientific criteria, meaning that de Freitas's papers did not receive the critical and constructive opinions of disinterested people familiar with the field.
"Pal review" is closely related to "no review" on the science. And the appearance of such papers (which I consider junk) in reviewed publications is thus a false appearance of respectability. Those papers did not pass peer review at all - if they had they might be taken seriously. And later examination by those peers indicates that the papers are nonsense.
MartinG wrote "But if somebody shows me that the papers de Freitas got published were poorly written, with unsupported conclusions etc. then I will agree they shouldnt have been published"
If you actually read the article above, you will find that the paper in question was refuted in a paper appearing in the journal EOS, and that several of the editors of the journal (including the editor in chief) resigned precisely because the paper was faulty and should not have been published. What more evidence do you want?
What more evidence do I want??. Well, for a start I want some evidence. Come on guys – you are not going to convince anyone with a critical mind of anything with phrases like “pal review” is “no review” , and “ I consider junk”. If those papers were not subjected to a proper review then they shouldn’t have been there – no question - and the reviewers, and especially the editor did not do their jobs.
But the papers are not “junk” just because you say they are, or because many people disagree with the conclusions. As far as I am aware de Freitas still considers them legitimate, and we don’t have any evidence from the reviewers that they didn’t do a proper review. That the contents are attacked in subsequent papers by providing new (or old) scientific arguments proving that the original authors were in error – why that’s just great – that’s how the literature is supposed to work – an interchange of ideas until we reach a consensus. As long as there are well founded scientific questions to the subject – or to the established consensus we should welcome them and deal with them based on the science.
That was the original point I made in this thread. A peer review cannot be used as support for the validity of a paper any more than the absence of a peer review proves that an article is bad. But I agree that all published papers should be subject to a peer review for the purposes of weeding out unsupported or unserious papers. Nor is a peer review by people in your own technical field that you know (pal review) necessarily bad. I would get my "pals" to peer review my next article without any bad conscience - just because I know they have my best interests at heart and will help me to improve the quality and validity of the paper. (but I would also invite a couple who had different ideas to pitch in.) It all depends if you are interested in using the literature to convince everybody out there that your ideas and conclusions are correct, or if you use it in a common search for the truth. I feel we should be looking at content and not get hung up with superficial jargon.
For good measure I must state that I haven’t read the papers involved so I can’t comment on their quality.
MartinG if you don't think that a published refutation is evidence that a paper shouldn't have been published, then you have a rather odd idea about what is considered evidence. Note the problem isn't that "because many people disagree with the conclusions" (your downplaying of the criticism did not escape me) but that fundamental methodological flaws were identified with the work. That is clear evidence of a failure of the review process as the reviewers should have picked up on them.
Similarly if you think that the resignation of the EIC and five other editors, explicitly because of a failure in the review process is not evidence of a problem in the review process, then you have rather stringent views on what you consider "evidence"! Editors (plural) do not resign without good cause.
Nobody is claiming that peer reviews supports the validity of a paper, that is a straw man. The claim is that a group of skeptics were exploiting a friendly editor to get work published that was not of a sufficiently high quality to get through the usual peer review for that journal.
"It all depends if you are interested in using the literature to convince everybody out there that your ideas and conclusions are correct,"
You mean like Soon being invited to testify before congress? This is another straw man, and is just evading the issue of whether the paper was sufficiently sound to justify publication.
So tell me, exactly what would you consider to be evidence that the paper should not have made it through peer review?
MartinG - Compare and contrast:
You first stated: "I actually have no problem with what de Freitas did - he got his papers peer reviewed and published."
and then: "If those papers were not subjected to a proper review then they shouldn’t have been there – no question - and the reviewers, and especially the editor did not do their jobs."
You will pardon me if I point out the contradiction displayed in those statements. The papers were not subjected to a proper review, have been found to be junk under examination, and de Freitas did not do his job.
"For good measure I must state that I haven’t read the papers involved so I can’t comment on their quality." - Enough said.
MartinG:
You can go on and on about wanting evidence, and not wanting to accept others opinions on the matter. To me, your statement "For good measure I must state that I haven’t read the papers involved so I can’t comment on their quality." says that the next step you should take is to read the papers. At that point, I can see three possibilities:
1) You have the background and knowledge to understand the papers and their methodologies, and feel that de Freitas' paper should have been published.
2) You have the background and knowledge to understand the papers and their methodologies, and feel that de Freitas' paper should not have been published.
3) You do not have the background and knowledge to understand the papers and their methodologies, and cannot pass judgement on your own. You will either have to accept a judgement from a source that you trust, or you will have to accept that you just don't know.
Until you have read the papers, I will consider the default to be that you fall into class 3. You don't trust others judgement, and you just don't know.
It is expected in a good journal that reviewers will fall into classes 1 or 2. A good reviewer will realize that (s)he falls into class 3, and will tell the editor "I don't know the subject well enough - you'll have to find another reviewer". Good editors like reviewers that tear papers apart - bad stuff will either get rejected or get revised. Mistakes still happen, but there are a lot fewer of them.
Fake skeptics seem to manage to find poorer journals where reviewers end up in class 3. In "Pal Review", the major problem is that the editors and reviewers don't realize that they fall into class 3, and because they only trust their like-minded pals they continue with their confirmation bias and motivated reasoning. Bad editors like reviewers that say "Hey! Great paper from Fred! I only had time to do a quick glance, but the conclusions are just what I like to hear! Publish it ASAP! Are we still on to do lunch next week?"
To Bob and you others. Thanks for the feedback. I dont mind falling into class 3 on this one. I repeat that peer review should be real - to help the authors. If by pal-review you all mean no-review - then we agree. But you can also get a paper reviewed by your "pals" and get some real help in getting it into a fit state to publish, so Pal review is a poor term to use. Since the identity of reviewers is normally unknown we dont know if they were just "pals" who nodded it through - on these or any other papers. So I remain sceptical about the claims that the papers were not subject to a peer review, or that they did not deserve to be published. (Sceptical science is not believing until you are convinced by the evidence).
But I hear you all proclaiming that the de Freitas papers should never have been published because they were bad, and secondly that there was no real peer review, and they have since been refuted. I would remind you that Mann et.al's original "hockey stick" paper was refuted on numerous well documented technical counts (M&M). These were not picked up in the peer review process. Now I would not suggest that that paper should not have been published. I believe that all voices should be heard, and that the suggestion that some papers should never have been published is superfluous. In climate science there are few hard facts, and most conclusions will (or should) have to have a probability attached to them. Often a different set of assumptions may give a different answer. So none of us should be so naive as to think we know the correct answer. At present we are working with hypotheses. When the correct answer has been established then all reputable scientists working in that field will agree because they will not find any real data or logic to disprove the point - and then they will have nothing to publish - even in 3rd rate journals.
My point is with the purpose of peer review and how it is used in modern literature. And for this I dont need to judge if the numerous papers edited by de Freitas were bad or not - others will have done that in the normal course of the scientific debate in the literature I am sure. Others may uncritically assume that they are junk because sombody said so - thats not really my concern.
Charles Jennings wrote in NATURE (a journal which is definately not 3rd rate):
Whether there is any such thing as a paper so bad that it cannot be published in any peer reviewed journal is debatable. Nevertheless, scientists understand that peer review per se provides only a minimal assurance of quality, and that the public conception of peer review as a stamp of authentication is far from the truth.
Given that many papers are never cited (and one suspects seldom read), it probably does not matter much to anyone except the author whether a weak paper is published in an obscure journal.
http://www.nature.com/nature/peerreview/debate/nature05032.html
Martin, in an ideal world where everyone is trying to advance the science, I agree with Jennings on his first and last statements. This is not an ideal world. There are people trying to get published simply so the publications can be used as an opinion-shaping tool. Jennings probably doesn't have a lot of experience with the Tim Curtins of the world. Publications of misleading and badly done studies can have serious consequences, because there is no accountability where the media is concerned. None. The scientific community ignored the allegations of climategate, and rightly so. Those allegations were, however, extraordinarily damaging to the credibility of climate science as far as a significant portion of the public was/is concerned.
Martin G wrote "So I remain sceptical about the claims that the papers were not subject to a peer review, or that they did not deserve to be published."
If you were skeptical, you would be willing to specify what sort of evidence you would require to be able to conclude that the paper did not deserve to be published. Your most recent comments suggests that there is no evidence that would convince you, which is not skepticism.
Bringing up M&M is probably a bad idea since that paper was itself badly flawed and discredited by subsequent studies. Note also that paper was published in 2003/4, which does not neccesarily imply that the reviewers of the paper in 1997 were in a position to know about the flaws at the time. So M&M is not proof that Mann's paper should not have been published.
"Given that many papers are never cited (and one suspects seldom read), it probably does not matter much to anyone except the author whether a weak paper is published in an obscure journal."
Not in the climate debate, where skeptic papers, whether in obscure journals or not, whether cited or not bu the research community, often get significant exposure in the media, blogsphere and politics. For example, Soon and Bauliunas, hence the need for very thorough and effective peer review.
MartinG - "When the correct answer has been established then all reputable scientists working in that field will agree because they will not find any real data or logic to disprove the point - and then they will have nothing to publish - even in 3rd rate journals."
That statement is demonstrably false - see the Tobacco Control Archives regarding the publishing activities of the tobacco industry: including several thousand references to Dr. Fred Singer, linked, as mentioned in the opening post, to many of the authors of these climate papers.
Or can I interpret your statement to mean you feel that the authors of such papers are not reputable scientists? Because I would agree on that. And when such papers are published in what should be a critically peer reviewed venue, because they were not actually given critical review, those papers have a false imprimatur of respectability and minimal quality.
Opinions are one thing, observations and theories (not hypotheses, I'll note) tied to reality are more harshly evaluated. Opinions contradicted by the facts are simply wrong.
---
Regarding your various statements re: Dr. Mann, I would suggest you take those to this thread - which notes the many confirmations of his results. Results that are notably not refuted.
KR - global ice volume is a pretty good proxy for global temperature. Based on the sea level trend over the last 10,000 years, itself determined by the global ice volume, it is very likely that Dr Mann's reconstructions underestimate how much warmer it is now, when compared the the Medieval Period.
Rob Painting - That is a good point; the "MWP" does not appear to have been a global phenomena, but rather a regional one, and more global reconstructions with additional proxies differ in some regards.
The point I was making was with respect to the overall "hockey-stick" of recent temperatures that is seen in so many other reconstructions, using different statistical techniques and proxies. Not to mention Wahl Ammann 2007 which refuted M&M, and demonstrated that the various complaints about Mann's techniques made no significant difference to his primary conclusions.
MartinG:
The phrase "pal review" was an insulting term proffered by the fake skeptics, accusing mainstream scientsists of bad behaviour. The strongest evidence of such bad behaviour is in the fake skeptics' papers and reviews. Forget the label, and focus on the behaviour.
Even the phrase "peer review" doesn't mean exactly what it says. In the case of the fake skeptics, their "peers" are other fake skeptics, and "peer review" doesn't help much - they are more pals than reviewers. "Peer review" is supposed to mean "expert review". In many disciplines with small global communities, it can be hard to find an expert you don't know at least a bit. Review by pals is fine if the pals do a thorough review - it's when reviewers are more interesting in a group hug that it is a problem. And again, when you look for the evidence, bad reviewing habits show up in the fake skeptics' camp far more strongly than in the mainstream.
I have personally reviewed papers of people that I know well - that is, as a journal reviewer, not a pre-submission review. One had worked as an undergrad with the group I was a grad student in. He later did grad work elsewhere, and I reviewed a paper from his M.Sc. work when I was a prof. I signed the review. When we talked a year or so later, he said he was initially surprised at the rough review (I saw major problems) coming from someone he knew, but then when he went over it all in detail, he realized that the review was the most constructive appraisal of his research that he had seen so far.
I put the "pal" aside, and became a reviewer. Sometimes, the best "pal" you can have is the one that points out where you are going wrong. That is how the review process should work.
I also remember that the department where I did my grad work saved their toughest questioning of "Friday afternoon seminar" speakers for their own grad students. These weren't "boost their ego" sessions - these were "slash and burn" sessions. The profs wanted to produce good scientists, and letting a grad student out into the big wide world with bad results was not in the interest of the department's reputation.