[TYPES] OT: Peer-review in a world with rational scientists

[Peter Selinger]

What a bunch of hogwash. That paper makes a number of highly flawed
assumptions. The first such assumption is that papers are accepted or
rejected by some kind of vote of anonymous referees. In reality,
papers are accepted or rejected by named editors (or PC chairs). These
editors use the advice of referees that are not anonymous to the
editors, but are responsible for checking the integrity of the reports
(i.e., whether the recommendations are supported by good reasons), as
well as making the final decision. 

Therefore, the strategy "I reject all papers that are better than my
own work" is not at all rational. Bad decisions will negatively affect
the editor's reputation, as well as the referee's reputation among the
circle of editors (and PC members etc).

In addition, some journals actually publish the name of the
responsible editor along with each paper. I think this is very good
practice and I wish more journals did this. It is a great incentive to
discourage false positives, as well as acknowledging the editor's
(sometimes considerable) work.

In my experience, most people have higher standards in refereeing than
in authoring; it only takes an average referee to point out potential
flaws in an above-average paper. I believe this is simply human
nature: it is much easier to criticize than to produce. From the
viewpoint of increasing overall quality, this tendency is positive for
peer review.

-- Peter

Dr. Claus-Peter Wirth wrote:
> 
> [ The Types Forum, http://lists.seas.upenn.edu/mailman/listinfo/types-list ]
> 
> Dear Marco,
> 
> I was not at all happy about the high bandwidth of the discussion
> on the subject in this list back then,
> but this paper is really interesting.
> 
> Taken seriously, we should stop peer review and use random selection:
> Better quality and less work.
> 
> Unless we are so "rational" that we prefer nepotism to science.
> But I am afraid that might be actually the case.
> 
> Thanks for the most interesting posting,
> CP
> 
> On 17.09.2010, at 17:19, M.C.A. (Marco) Devillers wrote:
> 
> > [ The Types Forum, http://lists.seas.upenn.edu/mailman/listinfo/types-list 
> >  ]
> >
> > A couple of months ago there was a discussion on this list regarding
> > (peer-)reviewing processes. Though, with little experience, I had  
> > nothing to
> > contribute to that discussion, I thought the next paper might be well
> > accepted on this list:
> >
> > "Peer-review in a world with rational scientists: Toward selection  
> > of the
> > average," Stefan Thurner, Rudolf Hanel.
> >
> > The arxiv link: http://arxiv.org/abs/1008.4324v1
> > The link straight to the article (pdf): http://arxiv.org/pdf/1008.4324v1
> >
> > Cheers,
> >   Marco
>