[TYPES] Two-tier reviewing process

[soloviev@irit.fr]

Dear members of the list,

I would like to express my agreement with Roberto.

Surely, we have to think what the -real- interests of
the community are, and not merely follow the current.
Let us be frank, the idea to make all evaluation
and even reputation quantitative is mostly imposed
from outside, by various administrations that want
to have an easy way to control scientific activities.

The flaws of these quantitative methods, their basic
imprecision were discussed a lot. But if the evaluation
method is flawed, it is open to manipulation,
to group politics etc.

Should we add one more possibility of abuse?

So...

All the best,

Sergei Soloviev


> [ The Types Forum, http://lists.seas.upenn.edu/mailman/listinfo/types-list
> ]
>
> On Fri, Jan 29, 2010 at 08:24:19PM -0500, Vijay Saraswat wrote:
> <snip>
>>
>> The only currency relevant is reputation.
>
> Dear Vijay,
> 	thanks for attracting attention to this point...
> I agree that in our small world reputation is one of the
> most relevant components of 'currency'.
>
> Let me humbly offer some food for thought, though.
>
> Reputation is a sophisticated object, and IMHO it does not
> or should not come just from the number of papers published:
> in our field, we know well that designing a powerful abstraction
> is an accomplishment worth hundreds of incremental improvements
> of the state of the art, so one breakthrough paper that subsumes
> dozens of previous works should not be counted as 'just one more paper'.
>
> Unfortunately, the current system does not encourage investing time
> in breakthrough papers, and linking reputation to number of papers
> published is part of the broken mechanism that should be fixed.
>
> Your proposal of publishing a track record of rejects seems
> at first to go in the right direction: the very same bad habit that
> links reputation to number of papers published will probably en up
> in linking bad reputation to number of rejects one author gets.
>
> But if one thinks a bit about it, things are more complex than that: if,
> as
> it has been advocated at large previously, there are not enough slots in
> our
> conferences today to accomodate all the good papers, your proposal will
> end up
> with double punishment for those good papers that do not go through...
> not only
> you dont get accepted (bad enough), but you also get a blame mark (exposed
> to
> the public pillory of authors got rejected).
>
> A natural result would be high resentment from those authors that beleive
> their
> work is really good, and yet got dumped because of what they feel is a
> faulty
> selection process...
>
> I am not sure this is what the community wants...
>
> Yours,
>
> --Roberto
>
>>
>> Rajeev Alur wrote:
>> > [ The Types Forum,
>> http://lists.seas.upenn.edu/mailman/listinfo/types-list ]
>> >
>> > One proposal to fix this is to enforce a rule such as:
>> > You can be a co-author on at most 2 submissions to  top-tier SIGPLAN
>> > sponsored conferences in a given calendar year.
>> > Such a rule would put some burden on authors to submit their best work
>> > to conferences,
>> > and reviewing would be less onerous and better.
>> >
>> Hi Rajeev --
>>
>> The problem of too many poor papers circulating is definitely real. But
>> this seems like a particularly bad way to fix this -- people can be
>> involved in many different lines of work that come to fruition in the
>> same year, a professor is guiding many students, a person is
>> contributing code to many systems etc.
>>
>> Here is an alternate suggestion picking up on your observation that
>> currently there is no cost of submission.
>>
>>
>> What if each of the SIGPLAN conferences also published a tally of who
>> submitted how many paper(*). This would make public the knowledge that A
>> submitted X papers and got Y accepted this year. I think just the social
>> stigma associated with submitting 20 papers and getting 2 accepted would
>> keep people from submitting too many papers :-).
>>
>> (*) The conference should not publish any other information about
>> rejected papers, e.g. title of submitted paper, who co-authored with who
>> etc.
>>
>
> --
> --Roberto Di Cosmo
>
> ------------------------------------------------------------------
> Professeur               En delegation a l'INRIA
> PPS                      E-mail: roberto at dicosmo.org
> Universite Paris Diderot WWW  : http://www.dicosmo.org
> Case 7014                Tel  : ++33-(0)1-44 27 86 55
> 5, Rue Thomas Mann       Fax  : ++33-(0)1-44 27 86 54
> F-75205 Paris Cedex 13
> FRANCE.
> ------------------------------------------------------------------
> Attachments:
>    MIME accepted
>    Word deprecated, http://www.rfc1149.net/documents/whynotword
> ------------------------------------------------------------------
> Office location:
>
> Bureau 6C15 (6th floor)
> 175, rue du Chevaleret, XIII
> Metro Chevaleret, ligne 6
> ------------------------------------------------------------------
>