Archive for ‘Peers and People’

April 1, 2011

Comedy Peer Review

by JasmineMalone

This Science Media Watch journalist has started a thread of ‘Comedy Peer Review’ on various social media networks, and I eagerly await your stories and experiences, from both journal editors and scientists. I’ll start this post off with a few of my own stories, journals/authors/crucial details will, of course, remain anonymous:

  • Having left a manuscript with a (highly prestigious in his/her field) peer reviewer for quite sometime, I chased and chased. I finally thought I would have to assign the paper to someone else, when I got an email saying that the reviewer had been terribly busy and wanted to devote his complete attention to the paper and had been working on it for ages, but still hadn’t finished and that of course his/her secretary would send it within a weeks’ time. Eager to get this world-renowned experts opinion, I waited a week, and was thrilled when an email came through from his secretary with a scanned attachement. I opened the attachment, only to find the printed out manuscript, with no more than three ticks in only one section of the results and the words ‘ok, but not great’ scribbled at the bottom. I assigned the paper to another reviewer.

Please feel free to comment with your stories below and this post will be updated accordingly!!!


April 1, 2011

Peer review from the journal editor’s perspective

by JasmineMalone

So, if you are having your paper peer reviewed or if you have been invited to peer review a paper, you probably have wondered at some point how the peer reviewer has been chosen.

Some basic steps – ascertained from my chat with the 10 journal current and ex-journal editors discussed in a previous post here – are as follows:

  • Number of peer reviewers needed – in some journals this is two, in others it is more, especially if the article covers a very broad range of sub-specialties
  • Determine a shortlist of specialists in overall field that is covered in the article: this is done by publication history (but outweighed by latest publications), current work, direction of current work if the article is in a new field and seniority
  • Quickly eliminate all potential reviewers from your shortlist that are either listed as authors on the paper or directly affiliated with the authors in one of the ways we discussed in our methodology
  • Scrutinize publication history of peer review candidates: how many papers on topic, have they covered the topic of the article before? Do they have a special interest in it or a particular knowledge or opinion about this field?
  • Geographical location – research published in international journals has to be of international relevance. It is important during peer review, particularly in that of clinical papers, not to forget the geographical relevance of findings/methods/treatments discussed.
  • Any other bias? The million dollar question. This is not always financial bias, as is immediately implied by the word. It can often be promoting the careers of ex-colleagues, or in extreme cases, trying to sabotage the career of rivals.
  • Most importantly, history. If the potential peer review candidate has been used as a peer reviewer before, what is their performance as a referee like? It is surprising how many peer reviewers who are brilliant scientists are quite appalling at judging others’ work objectively, or even fairly for that matter. Most journal editors I know keep a log of poor-performing peer reviewers, those that were tardy with their reports, or not thorough enough. Once blacklisted, these referees are never used again.

On the lighter side, every journal editor has a funny or outrageous peer review story. See future post for ‘Comedy Peer Review’, coming soon!



April 1, 2011

Open review and the trial of 2006

by JasmineMalone

We’ve done a lot of research on what we think is important to highlight with regards to bias and peer review in the articles we have researched from Nature, but it is important to note that we aren’t the first to look into this and Nature has done a pretty good job of responding to criticisms.

In fact, the journal went as far as to trial a different method of review, open review in 2006. In this peer review experiement, a 4-month open review system was tried out and the results published in a web focus.

Dr Phillip Campbell, Nature, published an editorial on the merits of peer review and the trial here.

Disappointingly, the overall conclusion was:

Despite enthusiasm for the concept, open peer review was not widely popular, either among authors or by scientists invited to comment. – Phil Campbell, Nature, 2006

So it’s on to the next alternative, whatever that may be.

April 1, 2011

Journal editors discuss possitive side to peer review

by JasmineMalone

Having worked in journals for 8 years, I thought it a good idea to collate peer review opinions and practices from all my current and ex-collagues. Combined, I talked to 10 current and ex-editors, all of whom have worked on peer-reviewed journals both research and clinical, across several major publishers, including Nature Publishing Group, Elsevier, BioMed Central and the BMJ journals.

Peer review

All the editors I spoke to agree that peer review is essential for quality control of new papers. They agree that there are – in some cases – flaws in the system, and that peer review needs to be managed very carefully, but without it the journals would be biased towards editors’ specialties and knowledge limitations. Several who have also authored papers agreed that they would, as an author, insist on peer review and would be outraged if there wasn’t a system in place to assesses their paper’s merit before publication or, especially, rejection.

read more »

March 30, 2011

Conflict of Interest: Do you have anything to declare?

by Louise Ogden

Are scientists rolling in it?

Funding bodies, such as The Wellcome Trust or the Biotechnology and Biological Sciences Research Council (BBSRC), here in the UK have strict guidelines on how and where they give the money, along with a strict policy on scientific misconduct. All of the funding bodies peer review the grant applications they accept before handing over the dosh. And this is for obvious reasons, funding bodies and charities, like The Wellcome Trust, need to make sure they are funding good science, done by good scientists.

The journal, Science, has also published a handy collection of links for laboratories who are applying for funding on the do’s and don’ts of grant applications. But that mostly applies to US institutions.

read more »

March 28, 2011

The peer’s point of view

by Beki Hill

Reviewers scrutinise the papers they receive

All the while we’ve been talking about peer review, whether we’re lamenting its failures or singing its praises, we’ve only heard from publishers and academics about going through the process (or not in some cases). What we haven’t done is look at it from the reviewer’s point of view. Well all that’s about to change.

Andy Hitchcock is a final year PhD student at the University of Sheffield, and has been lucky enough to review papers for three journals already (and no I’m not saying that sarcastically, it’s not everyone who gets that chance and it’s hugely beneficial if you want to continue your career in academia). I went to speak him about his experiences and to ask what he thinks of this controversial method of scientific publishing.

read more »