How to bury your academic writing

Inappropriate use of journal impact factors has been much in the spotlight. The impact factor is not only a poor indicator of research quality but it is also blamed for delaying publication of good science, and even encouraging dishonesty.  My own experience is in line with this: some of my most highly-cited work has appeared in relatively humble journals. In the age of the internet, there are three things that determine if a paper gets noticed: it needs to be tagged so that it will be found on a computer search, it needs to be accessible and not locked behind a paywall, and it needs to  be well-written and interesting.

While I'm not a slave to metrics, I am, like all academics these days, fascinated by the citation data provided by sources such as Google Scholar, and pleased when I see that something I have written has been cited by others. The other side of the coin is the depression that ensues when  I find that a paper into which I have distilled my deepest wisdom has been ignored by the world. Often, it's hard to say why one article is popular and another is not. The papers I'm proudest of tend to be those that required the greatest intellectual effort, but these are seldom the most cited. Typically, they are the more technical or mathematical articles; others find them as hard to read as I found them to write.  Google Scholar reveals, however, one factor that exerts a massive impact on whether a paper is cited or not: whether it appears in a journal or an edited book.

I've had my suspicions about this for some time, and it has made me very reluctant to write book chapters. This can be difficult. Quite often, a chapter for the proceedings is the price one is expected to pay for an expenses-paid invitation to a conference. And many of my friends and colleagues get overtaken by enthusiasm for editing a book and are keen for me to write something. But statistical analysis of citation data confirms my misgivings.

Google Scholar is surprisingly coy in terms of what it allows you to download. It will show you citations of your papers on the screen, but I have not found a way to download these data.  (I'm a recent convert to data-scraping in R, but you get a firm rap over the knuckles for improper behaviour if you attempt to use this approach to probe Google Scholar too closely). So in what follows I treated rank order of citations, rather than absolute citation level as my dependent variable. I downloaded a listing of my papers, ranked by citations, and coded them according to whether the article appeared in a journal or as a book chapter. Book chapters tend not to be empirical – they are more often review papers, or conceptual pieces - so to control for that I subdivided the journal articles into empirical and theoretical/review pieces. I also excluded papers published after 2007, to allow for the fact that recent papers haven't had a chance to get cited much, as well as any odd items such as book reviews. To make interpretation more intuitive, I inverted the rank order, so that a high score meant lots of citations, and the boxplots showing the results are in the Figure below.

Citation rank by Publication type. High rank indicates more citations. There is no significant difference between journal reviews and empirical papers, both of which have significantly higher citation rank than book chapters (p < .001)

Because I'm nerdy about these things, I did some stats, but you don't really need them. The trend is very clear in the boxplot: book chapters don't get cited. Well, you might say, maybe this is because they aren't so good; after all, book chapters aren't usually peer reviewed. It could be true, but I doubt it. My own appraisal is that these chapters contain some of my best writing, because they allowed me to think about broader theoretical issues and integrate ideas from different perspectives in a way that is not so easy in an empirical article. Perhaps, then, it's because these papers are theoretical  that they aren't cited. But no: look at the non-empirical pieces published in journals. Their citation level is just as high as papers reporting empirical data. Could publication year play a part? As mentioned above, I excluded papers from the past five years;  after doing this, there was no overall correlation between citation level and publication year.

Things may be different for other disciplines, especially in humanities, where publication in books is much more common. But if you publish in a field where most publications are in journals, then I suspect the trend I see in my own work will apply to you too. Quite simply, if you write a chapter for an edited book, you might as well write the paper and then bury it in a hole in the ground.

Accessibility is the problem. However good  your chapter is, if readers don't have access to the book, they won't find it. In the past, there was at least a faint hope that they may happen upon the book in a library, but these days, most of us don't bother with any articles that we can't download from the internet. 

I'm curious as to whether publishers have any plans to tackle this issue. Are they still producing edited collections? I still get asked to contribute to these from time to time, but perhaps not so often as in the past. An obvious solution would be to put edited books online, just like journals, but there would need to be a radical rethink of access costs if so. Nobody is going to want to pay $30 to download a single chapter. Maybe publishers could make book chapters freely available one or two years after publication  - I see no purpose in locking this material away from the public, and it seems unlikely this would damage book sales. If publishers don't want to be responsible for putting material online, they could simply return copyright to authors, who would be free to do so.

My own solution would be for editors of such collections to take matters into their own hands, bypass publishers altogether, and produce freely downloadable, web-based copy. But until that happens, my advice to any academic who is tempted to write a chapter for an edited collection is don't.

Reference
Eve Mardera, Helmut Kettenmann, & Sten Grillner (2010). Impacting our young Proceedings of the National Academy of Sciences, 107 DOI: 10.1073/pnas.1016516107