Fair Game? A Reviewers Tale

 

Emeritus Professor Peter Armstrong (p.armstrong@le.ac.uk) discusses an episode in the journal reviewing process that led him to believe that power and politics play their part too.

 

Around 1990 I still believed that peer review worked as it is supposed to do. I had begun my career as a reviewer at the journal Work Employment and Society under the editorship of Richard Brown and tried to follow his injunction to ‘let the author write their own paper’. That is, I tried to make a judgment of quality, doing my best to set aside my own preferences for method, theoretical approach and research question. As an author, I was as hurt as anyone else by rejection, but never thought of arguing. Naturally I sometimes believed the referees to be mistaken, but I accepted that they were competent people who had made their judgments in good faith. I was totally naive.

 

It was in the foregoing spirit that I approached the review of a paper which claimed that a dispute between a catering manager and company accountant over the amount allowed for catering purposes could only be adequately understood in the light of a particular social theory. As far as I could see from the author’s account, the dispute could perfectly well be understood as a straightforward difference in priorities rather than requiring any complex theory. I also knew enough about this particular theory to see that the author’s understanding of it was imperfect, to put it kindly. Accordingly I asked for revisions to the exposition of the theory and for the inclusion of any additional case material which might justify the author’s claim that it was necessary.

 

In their reply the editors of the journal enclosed a hand-written rejoinder from the author. A journalist would have described it as a green ink letter. The theory, it declared, MUST STAND, bold, capitals, double-underlined. The case material was perfectly adequate as it stood, and so on for two furious pages. Appended there were a number of references just so I would know who I was dealing with, never mind that this was a clear breach of the ideal of blind refereeing and one in which the editor seemed happy to collude.

 

Faced with a flat refusal to revise anything at all, I didn’t have a great deal of choice. I recommended rejection. The next communication from the editors informed me that, since the two initial reviews had disagreed, the paper would go to a third referee for adjudication. In the ordinary way of things, that would have been the last I heard, except that the adjudication had gone the author’s way. Author 1, reviewer 0.

 

At the time, I was working in a university about 180 miles from my home and was desperate for a job within feasible commuting distance. It so happened that the department at which I had applied for a lectureship was also the one which employed the author and a senior co-author who had also featured in the list of references. Anticipating that I would have problems in working with someone capable of writing a letter of the kind I had received, I considered withdrawing the application but went ahead on the basis of greater need. As luck would have it, neither of the two were on my interview panel. Whether consequently or not, I got the job.

 

On their return I found myself on the receiving end of a little chat with the senior co-author. To the best of my recollection, it went thus.  ‘X doesn’t know it was you, and I won’t tell.’ The evident presumption was that the paper was OK and that it was I who had something to apologise for. The speaker seemed to think they were doing me a favour in keeping my misdeeds from an injured party. Apparently it never occurred to them that there might be something improper about discovering the identity of a supposedly anonymous reviewer. The little chat concluded with, ‘We’ll work on it. It’ll be OK, you’ll see.’

 

And so they did and so it was. The paper was not only published – largely unmodified – but was also awarded a prize by the publisher of the journal. About two years later, through a chance turn in a conversation with another academic, I discovered that they had been the second reviewer and that they had made precisely the same objections to the paper as me, and had asked for similar revisions. In other words, there had been no disagreement between the first two reviews, and so no justification for sending the paper to an adjudicating referee, if indeed this was what had actually happened.

 

At the time the senior of the two collaborators was already a distinguished scholar and, as time went on and the publications piled up, that also became true of their protégé. Both of them were honoured guests at the celebration which marked the tenth anniversary of the journal concerned. Though I worked alongside the pair of them for four years, the affair was never mentioned again and I could never work out how the pair of them saw my part in this episode. Was I some ogre of vindictiveness overcome by the temptation to slap down the aspirations of a rising star, or was I simply too prejudiced or stupid to see the virtue in a piece of work whose objective worth was later attested by the publisher’s prize?

 

Unless there are good reasons to the contrary, my working assumption is always that what I have experienced is not an isolated occurrence. On the basis of my chance behind-the-scenes peek at the review process, I wonder if we who submit to it in good faith are the fools who obey the speed limit whilst the more worldly-wise swish past us in the outer lane with their eyes glued to their speed-trap detectors.

Originally published at http://staffblogs.le.ac.uk/management/

Leave a Reply

Your email address will not be published. Required fields are marked *