The F13 files, part 3 - the reference list analysis and other problems

Elsevier's journal Renewable and Sustainable Energy Reviews published a paper in 2013 (Florides et al. 2013, "F13" from now on) which we found problematic. We analysed the paper and communicated our findings to Elsevier. Our main findings were that much of F13 text was copy/pasted from other sources without proper attribution and that F13 contained many false claims. In this series of posts I'll go through the problems in F13 and in Elsevier actions. There are four posts:

• The F13 files, part 1 - the copy/paste job
• The F13 files, part 2 - the content analysis
• The F13 files, part 3 - the reference list analysis and other problems
• The F13 files, part 4 - dealing with Elsevier

Part 3 - the reference list analysis and other problems

In addition to content analysis, we analysed F13’s reference list, as that can offer strong quality indicators. In this second part of our analysis, we checked each reference and recorded its properties:

• Type of article (journal article / report / website / etc.).
• Status of peer-review. In addition to journal articles, we included IPCC reports as peer-reviewed. In a few cases, F13 cited a website while the document in question had also been published in a scientific journal. These we also included as peer-reviewed.
• Position on anthropogenic global warming or related issues. References were categorized as supporting (a) the position of mainstream climate science, (b) alternative theories, or (c) contrarian position.

Finally, we commented on general problems and some specific problems elsewhere in F13. These we also show in this post.

Results of the reference list analysis

The references can be categorized in three different ways.

First, by source (shown as a pie chart below), about 40% of the F13 references are scientific journal articles, 19% are scientific reports, 25% are websites, 12% are books or book chapters, and 4% are other articles.

Second, percentage of non-peer-reviewed material in the F13 reference list is 40%.

Third (shown as a pie chart in the beginning of this post), 62% of the references follow mainstream climate science, aligned with IPCC, but 38% support alternative or contrarian views. Cook et al (2013) [2] found that only 0.7% of their sample rejected AGW, so F13 references them 50 times more than one would expect of an unbiased sample. This is problematic in a review article, which generally aims to acquaint non-experts with the field's state of the art, but this seriously misrepresents it.

Analysis of the reference list is presented at Appendix B below.

Some other issues with F13

Although a somewhat minor point, F13 is a review on the climatic effects of CO₂ and the Sun, and yet F13 has a chapter on “CO₂ concentration and life” (F13 Chapter 2.5), which seems to be out of place.

F13 discussion on the climatic effects of the Sun is not covered by our detailed paragraph-by-paragraph analysis, but seems to follow similar paths as the CO₂ related discussion. Alternative and contrarian viewpoints are emphasized and handled uncritically in F13 while ignoring all research showing the weaknesses of those viewpoints. F13 also ignores the evidence that points to weak solar influence to climate change of recent decades.

The hypothesized cosmic ray - climate connection is worth closer inspection. F13 discuss that cosmic rays might have a strong effect on Earth's climate. Dozens of papers have been published on the issue and most find that the effect on Earth's climate is negligible [e.g. 4, 5, 6, and 7]. However, F13 discuss this issue as if strong cosmic ray - climate connection might be on the verge of wide acceptance. The F13 discussion is an example of cherry-picking? they only cite sources in favor of the hypothesis, with no mention of papers reporting contrary results.

When F13 discuss cosmic ray effects on climate, F13 cite references using the name of the institution rather than author names, which seem like independent confirmations, but are really just different texts by the same authors.

F13 also mention CERN Cloud project that studies the cosmic ray issue, but F13 fail to mention that the first results had already been published in 2010 [8] and they were not favorable to the cosmic ray hypothesis. In support of CERN results and against the cosmic ray hypothesis, Kulmala et al. [9] have reported results from atmospheric observations of nucleation events.

Conclusions

About 40% the references are non-peer-reviewed, 38% of them take alternative or contrarian positions on AGW, clearly higher than would be expected in an unbiased review.

The sections of F13 we didn't analyze thoroughly seem to contain similar problems to those sections we analyzed.

Below are shown the full reference list analysis, acknowledgements, and references in our (my and John Mashey's) comment paper as sent to Elsevier. Almost everything in this post is also directly from our comment paper.

Appendix B. The reference list analysis

Table 1. The reference list analysis. Column 1 is the reference number as given by F13. Column 2 is the type of reference. Column 3 is the status of peer-review. Column 4 is the position the reference takes to anthropogenic global warming.

Acknowledgements

Authors wish to thank Peter Jacobs, Kevin Cowtan, Daniel Bailey, Gavin Cawley, Dana Nuccitelli, Bärbel Winkler, Paul W., and Deep Climate for the discussions and help relating to this work.

References:

[1] Florides GA, Christodoulides P, Messaritis V, Reviewing the effect of CO 2 and the sun on global climate, Renewable and Sustainable Energy Reviews 2013?26:63951. doi:10.1016/j.rser.2013.05.062.

[2] Cook J, Nuccitelli D, Green SA, Richardson M, Winkler B, Painting R, et al ., Quantifying the consensus on anthropogenic global warming in the scientific literature, Environ. Res. Lett. 2013?8(2). doi:10.1088/17489326/8/2/024024.

[3] IPCC AR4, Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, et al ., editors. Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA? 2007.

[4] Laken BA, Pallé E, Calogovic J, Dunne EM, A cosmic ray-climate link and cloud observations, Journal of Space Weather and Space Climate 2012?2. doi: 10.1051/swsc/2012018.

[5] Sloan T, Wolfendale AW, The contribution of cosmic rays to global warming, Journal of Atmospheric and Solar-Terrestrial Physics 2011?73(16): 23525. doi:10.1016/j.jastp.2011.07.013.

[6] Kristjánsson JE, Stjern CW, Stordal1 F, Fjæraa AM, Myhre G, Jónasson K, Cosmic rays, cloud condensation nuclei and clouds – a reassessment using MODIS data, Atmos. Chem. Phys. 2008?8:737387. doi:10.5194/acp873732008.

[7] Udelhofen PM, Cess RD, Cloud cover variations over the United States: An influence of cosmic rays or solar variability? Geophysical Research Letters 2001?28(13):261720. doi:10.1029/2000GL012659.

[8] Duplissy J, Enghoff MB, Aplin KL, Arnold F, Aufmhoff H, Avngaard M, et al. , Results from the CERN pilot CLOUD experiment, Atmos. Chem. Phys. 2010?10:163547. doi:10.5194/acp1016352010.

[9] Kulmala M, Riipinen I, Nieminen T, Hulkkonen M, Sogacheva L, Manninen HE, et al., Atmospheric data over a solar cycle: no connection between galactic cosmic rays and new particle formation, Atmos. Chem. Phys. 2010?10:188598. doi:10.5194/acp1018852010.
[10] Visser ME, Keeping up with a warming world? assessing the rate of adaptation to climate change, Proceedings B of the Royal Society 2008?275(1635):64959. doi:10.1098/rspb.2007.0997.

[11] Diffenbaugh NS, Field CB, Changes in Ecologically Critical Terrestrial Climate Conditions, Science 2013?341(6145):486492. doi: 10.1126/science.1237123.

[12] McElwain JC, Wagner PJ, Hesselbo SP, Fossil Plant Relative Abundances Indicate Sudden Loss of Late Triassic Biodiversity in East Greenland, Science 2009?324(5934):15546. doi: 10.1126/science.1171706.
[13] Twitchett RJ, The palaeoclimatology, palaeoecology and palaeoenvironmental analysis of mass extinction events, Palaeogeography, Palaeoclimatology, Palaeoecology 2006?232(24):190213. doi:10.1016/j.palaeo.2005.05.019.

[14] Haug GH, Günther D, Peterson LC, Sigman DM, Hughen KA, Aeschlimann B, Climate and the Collapse of Maya Civilization, Science 2003?299(5613):17315. doi: 10.1126/science.1080444.

[15] Yu S, Zhu C, Song J, Qu W, Role of climate in the rise and fall of Neolithic cultures on the Yangtze Delta, Boreas 2000?29(2):15765. doi:10.1111/j.15023885.2000.tb01208.x.

[16] Plass GN, The Carbon Dioxide Theory of Climatic Change. Tellus 1956?8:140–54. doi: 10.1111/j.21533490.1956.tb01206.x.

[17] IPCC (Intergovernmental Panel on Climate Change), Principles Governing IPCC Work 1998 [cited 2015 Oct 22]. Available from http://www.ipcc.ch/pdf/ipcc-principles/ipcc-principles.pdf .

[18] Svensmark H, Evidence of nearby supernovae affecting life on Earth, MNRAS 2012?423(2):123453. doi: 10.1111/j.13652966.2012.20953.x.

[19] Overholt AC, Melott AL, Pohl M, The Astrophysical Journal Letters 2009?705. doi:10.1088/0004637X/705/2/L101.

[20] Ruddiman WF (2006), Orbital changes and climate, Quaternary Science Reviews 2006?25(23–24):3092–112. doi:10.1016/j.quascirev.2006.09.001.

[21] Hartmann DL, Klein Tank AMG, Rusticucci M, Alexander LV, Brönnimann Charabi SY, et al., Observations: Atmosphere and Surface. In: Stocker TF, Qin D., Plattner GK, Tignor M, Allen SK, Boschung J, et al . editors. Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA 2013.

[22] Hegerl GC, Zwiers FW, Braconnot P, Gillett NP, Luo Y, Marengo Orsini JA, et al., Understanding and Attributing Climate Change. In: Solomon S, Qin D, Manning M, Chen Z, Marquis M, Averyt KB, et al ., editors. Climate Change 2007: The Physical Science Basis. Contribution of Working Group I to the Fourth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA? 2007.

[23] Caillon N, Severinghaus JP, Jouzel1 J, Barnola JM, Kang J, Lipenkov VY, Timing of Atmospheric CO2 and Antarctic Temperature Changes Across Termination III, Science 2003?299(5613):172831. doi:10.1126/science.1078758.

[24] Shakun JD, Clark PU, He F, Marcott SA, Mix AC, Liu Z, et al., Global warming preceded by increasing carbon dioxide concentrations during the last deglaciation, Nature 2012?484:4954. doi:10.1038/nature10915.

[25] Kürschner WM, Leaf sensor for CO 2 in deep time, Nature 2001?411:247248. doi:10.1038/35077181.

[26] Retallack GJ, A 300-million-year record of atmospheric carbon dioxide from fossil plant cuticles, Nature 2001?411:287290. doi:10.1038/35077041.

[27] Crowley TJ, Berner RA, CO 2 and Climate Change, Science 2001?292(5518):8702. doi: 10.1126/science.1061664.

[28] Royer DL, Berner RA, Montañez IP, Tabor NJ, Beerling DJ, CO2 as a primary driver of Phanerozoic climate, GSA Today 2004?14(3):410.

[29] Royer DL, CO2-forced climate thresholds during the Phanerozoic, Geochimica et Cosmochimica Acta 2006?70(23):5665–75. doi:10.1016/j.gca.2005.11.031.

[30] Came RE, Eiler JM, Veizer J, Azmy K, Brand U, Weidman CR, Coupling of surface temperatures and atmospheric CO2 concentrations during the Palaeozoic era, Nature 2007?449:198201. doi:10.1038/nature06085.

[31] Trenberth KE, Fasullo JT, O'Dell C, Wong T, Relationships between tropical sea surface temperature and top-of-atmosphere radiation, Geophys. Res. Lett. 2010:37(3). doi:10.1029/2009GL042314.

[32] Idso SB, The CO2 greenhouse effect on Mars, Earth, and Venus, Science of The Total Environment 1988?77(23):291294. doi:10.1016/00489697(88)900654.

Posted by Ari Jokimäki on Tuesday, 24 October, 2017

The Skeptical Science website by Skeptical Science is licensed under a Creative Commons Attribution 3.0 Unported License.