Posted on 31 October 2017 by Ari Jokimäki

Elsevier's journal Renewable and Sustainable Energy Reviews published a paper in 2013 (Florides et al. 2013, "F13" from now on) which we found problematic. We analyzed the paper and communicated our findings to Elsevier. Our main findings were that much of F13 text was copy/pasted from other sources without proper attribution and that F13 contained many false claims. In this series of posts, I'll go through the problems in F13 and in Elsevier actions. There are four posts:

• The F13 files, part 1 - the copy/paste job
• The F13 files, part 2 - the content analysis
• The F13 files, part 3 - the reference list analysis and other problems
• The F13 files, part 4 - dealing with Elsevier

Part 4 - dealing with Elsevier - first contacts

In parts 1-3 we discussed the possible plagiarism and the problems in the content of F13. In this fourth part we will go through what happened when we communicated our findings to Elsevier. Here is a brief chronology of events, which I explain in more detail below:

• March 14, 2014: we become aware of F13 and start our analysis.
• June 27, 2014: our first contact attempt to Elsevier.
• September 10, 2014: first response from Elsevier - basically just receiving the information from us.
• July 5, 2015: After us asking for the status of the issue a couple of times, and after much waiting, the Editor-in-Chief finally contacts us for the first time - the issue has been communicated to interested parties. We are asked to write and submit a comment paper.
• March 31, 2016: after much work, we submit our comment paper.
• March 21, 2017: after several contact attempts trying to ask about the status of our submission, we get a response from Elsevier saying that they had sent us F13's reply already in May, 2016, which we didn't receive.
• May 14, 2017: The F13 reply is re-sent to us. The Editor-in-Chief rejects our comment paper because F13 didn't like it, and asks us to write another paper where the F13 issue would be reduced to just one example issue among other similar issues. We decline the offer.

After our initial analysis of F13 (which started on March 14, 2014) we made our first contact attempt with Elsevier on June 27, 2014. We used the contact form available on the website of the Journal. It is supposed to create an e-mail (or something) to the Editor-in-Chief of the Journal, who is Lawrence Kazmerski. We sent our analysis about possible plagiarism and a brief overview about the problems of the content.

We waited for a while for a reaction from Elsevier, but didn't receive one. Usually one would expect at least an acknowledgement of the receipt of the communication. On August 4, 2014, we asked them if they had received what we submitted. There was no response.

We decided to try another route to contact the Journal. We used Elsevier's general contact form to ask if something had gone wrong in our contact attempts (we did this on September 9, 2014). This time we got a response first from a customer support person and then from Katherine Eve, the Executive Publisher of the Journal. The Executive Publisher asked us to resend our material to her, which we did. She also asked for some more information about our role in the issue.

After initial exchange of messages with Katherine Eve, we were left waiting for further developments. On January 15, 2015, after no developments, we sent a query about the status of the issue. Katherine Eve sent a note back saying that she was unsure about the status but that she would check it.

Once again we waited for some developments and as there were none, we asked them about the situation on April 27, 2015. We also mentioned our willingness to go public with our analyses and we expressed our concern about the situation, as it had been quite a long time since our first contact.

We waited for a response and after two weeks Katherine Eve sent a note to us (on May 12, 2015), where she urged the Editor-in-Chief, Lawrence Kazmerski, to give us an update about the issue.

Next, the issue went into waiting mode again. Then, finally, on July 5, 2015, we received the first e-mail from the Editor-in-Chief. He explained that he had been traveling and that there had been problems in contacting Florides et al. He had also discussed the issue with F13 reviewers (apparently there had been three reviewers for the paper). The reviewers agreed that some action was needed based on our findings. The Editor-in-Chief then invited us to write a more thorough comment paper on F13.

There were a few things to criticize in this first contact from the Editor-in-Chief. Although he did give some information about what had happened, there was no explanation why it took him well over a year to respond to us. I would have expected him to contact us to let us know that he was handling the issue and was discussing it with the authors and the reviewers. Instead, we were just left hanging with no information about the situation. Another thing is that right from the beginning, the Editor-in-Chief focused on our criticisms about the content and he showed very little interest in the issue of possible plagiarism. He did say that the issue was being dealt with, but he also said that he felt the content criticism to be the more important issue. This is strange considering that plagiarism seems to be a more common retraction reason than flawed content.

We responded that we would like to write the comment paper the Editor-in-Chief was asking for.

The comment paper

We then started working on the comment paper. This took us quite a long time (debunking false claims is far more time-consuming than making them), but eventually we got it finished. We sent the comment paper to Elsevier on March 31, 2016. Again, the response took a while, but on April 4, 2016 the Editor-in-Chief responded and explained that he had been traveling and said he would go through our paper.

After this, things went silent again. We weren't that worried, though, because we thought that peer-review and getting a response from Florides et al. would take some time. Finally, on January 17, 2017, we sent out an e-mail asking about the status of our submission. We got no response. On January 26, we sent another e-mail repeating the query. We got no response. On March 7, we sent out yet another query. This time we added a note: "We submitted the paper almost a year ago and we haven't heard anything about it since." We got no response.

On March 21, we approached Katherine Eve describing our situation. We got a response where she told us that she was under the impression that the Editor-in-Chief was waiting for a revised comment from us.

Soon after that we got an e-mail from the Editor-in-Chief saying that he had sent the Florides et al. reply to us, and subsequent discussion revealed that this apparently happened on May 2, 2016. Then he noted that the issue was a complaint about plagiarism and that he should discuss with Elsevier people about the issue.

We responded with a note where we summarized the previous steps in the process noting especially that the issue was both a plagiarism complaint and a complaint about some flaws in the content. He responded that we should get a chance to look at Florides et al reply and then we would go forward with the publication of our comment. He then said that he would send us the Florides et al reply again, once he had discussed this with Elsevier people.

So once again we were left waiting. On April 3, the Editor-in-Chief sent us a note that he had been unsuccessful in contacting Elsevier people. Then we waited some more. On April 24, we sent an e-mail asking about the situation. We got no response.

The response from Florides et al.

Finally, on May 14, 2017, the Editor-in-Chief sent us an e-mail containing the response from Florides et al. In the response, Florides et al. didn't answer any of our points but instead they assumed the role of a victim and concentrated on attacking our academic credentials and suggested we were working on an agenda.

In their first argument, they suggested that our comment paper was unprofessional. They justified this by highlighting our reference analysis, in which they did not seem to like that we had categorized their references into "mainstream", "alternative", and "contrarian". The last category in particular didn't seem to be of their liking. Nor did they seem to like our use of the term "logical fallacies" to describe their logical fallacies. They even complained that we had categorized the severity of the problems in their paper as "major" and "minor" although we did that for their own benefit (we wanted to acknowledge that some of the problems were only minor ones - which we also mentioned in the comment paper).

Their second complaint was that we had only analyzed their chapters 1 and 2. They also claimed that we didn't mention the sun in our comment paper. This is strange because we did briefly address some of the sun related issues in their paper. We limited ourselves to full analysis of chapters 1 and 2 only, because it would have taken far too much time to do the full paper. Even the two chapters took a lot of time. It is far easier and faster to make false claims (especially if you just copy-paste them from somewhere else) than to correct them.

Their last argument was a complaint that it took us so much time to write the comment paper given that there were two authors and 8 people mentioned in the acknowledgement section working for the paper, and they noted that they did not want to write a reply to our paragraph-by-paragraph analysis because it would be too much work. In other words, we had shown plenty of errors in their paper, some of which were quite serious, but they showed no interest in correcting them.

So, in their reply, Florides et al. didn't answer any of our points but instead made some hand waving type arguments. Furthermore, there was no mention of the plagiarism issue in their response.

The resolution

In his e-mail to us, the Editor-in-Chief also told us what they would like to do next. He had told us on two different occasions that regardless of Florides et al. response, we would get published. But now, he told us that because Florides et al. didn't like our comments, we couldn't proceed as planned. Instead, he suggested that we write yet another paper, where we would review some cases where climate issues have been misrepresented like Florides et al. did. In this new paper, Florides et al. would be just one example among others.

We had already written a brief commentary for them in our initial contact, and we had written a more thorough comment paper when they asked for one. We thought that we had given them plenty already, so we decided not to write another paper for them.

There was one thing left, though. There was no mention of the plagiarism issue in the e-mail from the Editor-in-Chief, so we asked him about it. He told us that they had checked the issue with "iThenticate", which didn't show high similarity. We don't know why this is so, but in this case iThenticate doesn't seem to show correct result, because our own analysis clearly shows high copy/paste percentage, as was shown in the post 1 of this series. But it seems that our plagiarism complaint also was a dead end.

Conclusion and discussion

We have shown that F13 was largely copy/pasted from other sources without indication that the texts in question had been copy/pasted. In our analysis of 27 of F13 paragraphs, 21 were found to contain copy/pasted content.

We found that the content of F13 was largely flawed. In our analysis of the first two chapters in F13, we found 43 individual problems. Of 27 paragraphs analyzed, 19 contained problems. There was a large spectrum of problems of which misleading discussion and logical fallacies were the most common ones. We checked other paragraphs of F13 and discussed some of the problems there as well. It seems that problems with the content continue elsewhere in the paper as well.

We also analyzed F13's reference list. We found that 40 % of the references were non-peer-reviewed documents. We also found that 38 % of references were to documents taking a non-mainstream position on anthropogenic global warming or related issues. This is far more than is expected from a review paper like F13.

Communicating these problems to Elsevier turned out to be very difficult and slow. Elsevier refused to publish our findings. As the end-result, F13 is still in the published content of Renewable and Sustainable Energy Reviews, and the Journal and Elsevier continue as if nothing has happened.

Some criticism on the actions of Elsevier and the Editor-in-Chief:

• It took the Editor-in-Chief over a year to send us the first e-mail on the issue.
• Little serious attention was paid to the detailed plagiarism complaint.
• Little serious attention was paid to the technical flaws either, perhaps because the topic was really outside the journal's focus.
• During the last e-mail exchange with the Editor-in-Chief, it became apparent that no peer-review had been performed for our comment paper.

The publication of F13 in Elsevier's Renewable and Sustainable Energy Reviews is an example of a failure of the peer-review and editorial processes of the Journal. It is our opinion, that F13 should never have been published in a scientific journal. We also think that F13 should have been retracted after we pointed out the problems in it.

Note (October 25, 2017): There was an apparently false statement in this post regarding the usage of iThenticate in checking for possible plagiarism. iThenticate doesn't just compare two documents but checks the document against all sources it can access. One paragraph under "The Resolution" section has been corrected due to this. This was the original paragraph:

"There was one thing left, though. There was no mention of the plagiarism issue in the e-mail from the Editor-in-Chief, so we asked him about it. He told us that they had checked the issue with "iThenticate", which didn't show high similarity. This means that they took F13 and their earlier book chapter and compared them to each other. We had informed them that the issue was not just copying from their own book chapter, but that they had copied from other sources as well. Comparing F13 to only one source gives low similarity but comparing it to all the sources gives, as we have shown in our plagiarism analysis, a high similarity. But it seems that our plagiarism complaint also was a dead end."