Arguments
Lomborg: a detailed citation analysis
Posted on 24 April 2015 by Guest Author
This is a re-post from ConservationBytes
There’s been quite a bit of palaver recently about the invasion of Lomborg’s ‘Consensus’ Centre to the University of Western Australia, including inter alia that there was no competitive process for the award of $4 million of taxpayer money from the Commonwealth Government, that Lomborg is a charlatan with a not-terribly-well-hidden anti-climate change agenda, and that he his not an academic and possesses no credibility, so he should have no right to be given an academic appointment at one of Australia’s leading research universities.
On that last point, there’s been much confusion among non-academics about what it means to have no credible academic track record. In my previous post, I reproduced a letter from the Head of UWA’s School of Animal Biology, Professor Sarah Dunlop where she stated that Lomborg had a laughably low h-index of only 3. The Australian, in all their brilliant capacity to report the unvarnished truth, claimed that a certain Professor Ian Hall of Griffith University had instead determined that Lomborg’s h-index was 21 based on Harzing’s Publish or Perish software tool. As I show below, if Professor Hall did indeed conclude this, it shows he knows next to nothing about citation indices.
What is a ‘h-index’ and why does it matter? Below I provide an explainer as well as some rigorous analysis of Lomborg’s track record.
The h-index stands for the Hirsch index, created by Jorge Hirsch of the University of California, San Diego ten years ago. Put simply, it’s the number of academic (peer-reviewed) papers h one has published with citation number ≥ h. As an example, let’s say I have published ten peer-reviewed journal articles (‘papers’) in my life to date. In descending order, they have been cited by different authors (of other peer-reviewed journal articles) 256, 150, 10, 4, 3, 3, 3, 2, 1, 0 times. Even though I have a total of 432 citations from those 10 papers, giving a mean of 43.2 citations per paper, this is driven largely by only 2 papers. As such, my h-index would be 4 (I have 4 papers with at least 4 citations). For me to increase my h-index to 5, I would need at least one more citation for the paper currently sitting at 4 citations, and at least 2 more citations from one of my other papers that currently only have 3 or fewer citations.
You can see the advantages of using such an index – it’s not influenced to the same degree by wild outliers and it resists artificial inflation by auto-citation (citing your previous papers in your latest ones). The disadvantage of the h-index is that even if you die, it can still increase as time progresses, such that the older you get, the higher your h-index. Some have proposed correcting for this by dividing the h-index by the number of years since your first publication (called the ‘m-index’). This essentially indicates your ‘speed’ of accumulating citations. As a general rule, if your m-index is over 1, you’re an active, publishing researcher. If your m-index is greater than 2, you’re doing very well.
But what can you count as a ‘paper’? This is in fact the crux of the bullshit floating around the web on this particular issue with respect to Lomborg. As a general rule, the two main services to calculate the h-index – Scopus (Elsevier) and Web of Science (Thomson Reuters) – are perhaps a little conservative (i.e., they don’t necessarily count all your works or all the citations to them). This is because they have very stringent rules for what counts as a ‘paper’ – they have to be recognised and accredited, peer-reviewed academic journals (i.e., books, magazine and newspaper articles are disallowed), and in the case of Web of Science, have to be ISI-indexed journals.
Lomborg only has a Scopus profile (you need a subscription to see this), which gives an h-index = 3 based on 31 articles and a total of 71 citations (see adjacent screenshot). He hasn’t set up a public ResearcherID, which would give his Web of Science h-index, but I took the liberty of combing through all of his Web of Science-listed articles and came up with the exact same h-index (3), based on 25 articles and 54 total citations.
Google Scholar is the new kid on the block to calculate researcher citation profiles, but to use this, each individual researcher needs to set up a Google Scholar profile (you can see mine here). Every academic should do this because it’s free to use and browse. If one doesn’t do this (and Lomborg hasn’t), then you have to search for individual publications on Google Scholar.
Herein lies the problem for those who submit that Lomborg’s h-index is 21. Software tools like Harzing’s Publish or Perish are merely Google Scholar aggregators; in other words, they merely act as a front end for the Google Scholar search engine. They are not individual profiles. In fact, ever since Google Scholar introduced profiles a few years ago,Publish or Perish has become entirely obsolete. Why? Because it aggregates everything – including all the inappropriate stuff – and vastly overestimates one’s h-index. Further, it doesn’t distinguish duplicates entries, makes no differentiation between peer-reviewed or non-peer-reviewed articles, and it counts any mention of the author’s name, even if isn’t one of their own articles! In other words, it’s utterly flawed.
So, in the absence of a Google Scholar profile for Lomborg, I combed through his Google Scholar entries and dumped all the duplicates, I ignored all the magazine and newspaper articles (e.g., you can’t count opinion editorials in The Wall Street Journal as evidence of an academic track record), I cut out all non-articles (things Lomborg hadn’t actually written), omitted any website diatribes (e.g., blog posts and the like) and calculated his citation profile.
Based on my analysis, Lomborg’s Google Scholar h-index is 4 for his peer-reviewed articles. If I was being particularly generous and included all of Lomborg’s books, which have by far the most citations, then his h-index climbs to 9. However, none of his books is peer-reviewed, and in the case of his most infamous book, The Skeptical Environmentalist, it has been entirely discredited. As such, any reasonable academic selection committee would omit any metrics based on opinion-based books.
So, the best-case scenario is that Lomborg’s h-index is no more than 4. Given his appointment to Level D (Associate Professor) at a world-class university, the suggestion that he earned it on academic merit is not only laughable, it’s completely fraudulent. There is no way that his academic credentials had anything to do with the appointment.
Addendum
I suppose I should have contextualised what an h-index of 3 or 4 means.
Even a fresh-out-of-the-PhD postdoc with an h-index of only 3 or 4 would have trouble finding a job. As a rule of thumb, the h-index of a Level D appointment should be in the 20-30 range (this would vary among disciplines). Despite this variation, Lomborg’s h-index is so far off the mark that even accounting for uncertainty and difference of opinion, it’s nowhere near a senior academic appointment.
Guidelines are guidelines, and no one is going to commit to a particular h-index as a minimum. Instead, an applicant’s h-index is usually benchmarked against others in the field and/or school of appointment. All one has to do is search for academics of the same or similar academic appointment level on Google Scholar as a point of comparison.
SkS Addendum
Lomborg's highest-cited papers deal with game theory and not climate science.
bozzza @48
Trolling is personally attacking someone with abuse and threats on twitter and in emails where the comments made have absolutely no relationship with the debate being discussed. Also, trolling occurs because the people doing it do not like an opposing argument related to the issue being discussed. It is often done by politically motivated types or just for mischievous purposes. It is designed to drive people involved in legitimate discussion making legitimate points from the debate. The simplest way of dealing with a troll is to block them but quite often people have had to close their online accounts and stop making comments. Trolling is just another strategy to confuse and obstruct a legitimate debate.
There is an opinion piece in today's (April 29th) Australian on Lomborg's appointment, that spends some time on h-factors. The writer seems less enamoured of them than some who have so lucidly commented here. Perhaps posters here will write a comment to the Australian on this opinion piece as it will reach a very wide audience. As commenters here have a very different perspective from that of the journalist, this will add significantly to the understanding of the general public about this somewhat arcane issue.
Pluvial says:
You cannot make statements like that, which is completely against any published science, without at least citing a source for the comment. If you found this erroneous statement (and I'm certain it wasn't in any science paper) then at least let us know where you are getting your information from.
ryland @52, out of interest I did a google scholar search for the author of the opinion piece in The Australian, Jennifer Oriel. It turns out that after a publication in 2005 based on her PhD thesis, she has published two further academic journals plus one for the organ of the Australian Union of Students. From those publications she has a google scholar h-index of 3 (strictly 2 counting purely academic journals alone). She has also published two book reviews and an article for the Institute of Public Affairs, the later suggesting (though not proving) a right wing ideological bias. Her academic affiliation was (and may still be) with Deakin University, who list her research output as zero.
I looked this up because her argument is based on the charge that academic pubication is a poor measure of research output because of a supposed left wing institutional bias in academic publication. Given that is the nature of her argument, the question arises what claim does she have to expertise in this area, and it turns out the answer is none. (That it also shows she has a personal interest in rejecting h-indexes, or publication records as a measure of research merit is a bonus.)
As to her argument I will make three points.
First, it is irrelevant. What the h-index measures is the ability and willingness to make your case to those who have the relevant expert knowledge to actually pick up your mistakes in argumentation, mistatements of data, etc. Anybody unwilling or unable to do that has no place heading an academic think tank at government expense. Indeed, the selection of such a person for that role shows the government is using public money to promote its private ideology - something event the IPA should be able to recognize as bad.
Second, her argument is evassive. There is substantial evidence that that Lomborg in addition to evading scrutiny by his peers, has indulged in a host of academic sins including cherry picking, framing of straw man opponents, misrepresenting data and employing different discount rates when making cost comparisons. That is, on the public record is is reasonable to suppose that his "research" is not only not presented to his peers, but that it would not survive such a presentation because it lacks academic merit. The low h-index is merely a symptom of this larger problem.
Third, her own google scholar record proves her wrong. Specifically, it shows that google scholar cheerfully includes not only peer reviewed publications, but publications from ideological think tanks such as the IPA. It includes 18,200 articles from Australia's most influential, but not peer reviewed journal Quadrant. Ergo, even if her self serving claims about academic publication are accepted as true, there are more than sufficient alternative means of publication with a designed right wing bias to establish a significant publication record, and more than sufficient authors of articles in those journal to establish a respectable h-index.
So, all in all, her argument has no merit.
I will not bother, however, submitting an opinion piece (or even a letter) to The Australian because long experience has shown they have an overwhelming bias against policy relevant opinions they do not like. So much so that on climate change their opinion pieces appear to represent an exact reversal of the AGW consensus, with around 3% of opinion pieces from those supporting the consensus, and 97% from those opposing it.
Joe Rhom at Think Progress reports that UWA has turned down the $4 million and declined to accept the Lomburg center. Any updated from Oz here?
michael sweet @55, the story has been widely reported in Australia, including by the ABC, the Sydney Morning Herald, the Australian, and of course The Gaurdian (Australian Edition) to which Romm links. The Sydney Morning Herald story is echoed across the Fairfax media, as the Australian's version is echoed across the Murdoch press. The only story additional to Romm's report is that Christopher Pyne says he is taking legal advise about the cancellation of the contract.
Australia has been made proud again by the actions of UWA staff: this is a globalised election issue... this action by the staff of a proud university is of historical significance in my opinion and the mere politics of it is that of all human relations.
Bring back the clever country I say. What say you?
Huh! The LNP govs did not give up on this pathetic effort and they want to place Lomborg in Flinders Uni now. Hold your breath about the "Australian pride". Will see if FU (or yet another education institution before the project collapses) accepts the lure of $4m govt grant...