[TYPES/announce] Two phase reviewing for POPL; a response

[Michael Mislove]

Along the lines of Prakash's comments, I'd like to offer a different
proposal. I take as a starting point the "conventional wisdom" that the
papers accepted at the leading conferences are not the 40 best papers, but
perhaps 40 of the best 80 papers. No reviewing process is perfect and most
select some proportion of the best submissions. I don't have confidence that
changing the reviewing process will cure this - it's inherent in any
subjective process that "human factors" will affect the outcome. This can
range from personal preferences of PC members and their referees to biases
reflected in the composition of the PC.

Given this disparity, the number of papers accepted could be increased by
holding poster sessions, as Prakash suggests, but I have concerns about the
effectiveness of such a venue for communicating new results even to those
who may be interested in them. I also believe poster session papers would
not receive the same regard in promotions and tenure considerations those
selected for full presentation at the meeting. I believe a better approach
would be to make better use of the satellite workshops around the main
conference.

I suggest that an effort be made to see what proportion of the papers in
these workshops were originally submitted to POPL itself. If most of the 40
"best 80" papers that were not accepted at POPL find their way into these
workshops, then I would assert the real problem - if there is one - is the
disparity between the quality of POPL versus that of its workshops. I'm not
as familiar with POPL as I am with other series, but I'd point out that the
mega conference ETAPS has a large number of satellite events, some of which
are regarded as being of a quality rivaling that of the main conferences. If
the same is true of POPL and if there are enough satellite workshops to
accommodate the overflow from POPL, then perhaps there is no problem. If
this is not the case, then POPL could found some new satellite workshops,
directly sanctioned and bearing its imprimatur, and work to make sure their
quality rivals that of the main meeting.

It remains to determine whether papers submitted to POPL but not accepted
actually ended up in one of the satellite events. A first estimate could be
obtained by having the PC chair (or an interested PC member) simply look at
the accepted papers for the satellite events and see how many of the top 80
submissions that were not accepted for POPL found their way onto the program
of one of the workshops. This wouldn't tell the whole story (some papers
that were rated highly but not accepted for POPL might have been found
wanting by a satellite event, and others might not have had an appropriate
venue among the satellites), but it could give some information about how
many papers submitted to POPL actually were orphans. In any case, I think
making better use of the satellite events would result in a better outcome
than tweaking the reviewing process or expanding the number of papers
accepted only to relegate the additional papers to posters at the main
meeting.
  Best regards,
  Mike Mislove


On Tue, Jan 12, 2010 at 8:18 AM, Prakash Panangaden <prakash at cs.mcgill.ca>wrote:

> [ The Types Forum (announcements only),
>     http://lists.seas.upenn.edu/mailman/listinfo/types-announce ]
>
> Thanks to Simon Peyton-Jones for sharing his articulate and
> well-reasoned thoughts with us.  I agree that there are too few papers
> accepted at the best conferences and that the problem is not quality of
> reviewing.  I would like to push for the idea that we accept many more
> papers (perhaps double the present number) and present them at poster
> sessions and have them appear in the proceedings as is done at most of
> the big AI conferences.  Then we could have a smaller number presented
> as conference presentations.  The other point I would like to make is
> that we as reviewers are far too obsessed with polished but incremental
> papers and in the theory conferences (STOC/FOCS/LICS/ICALP) with "hard
> but boring" problems.  It is indeed hard to change the culture, but
> conferences are where we should get the chance to throw out ideas rather
> than participate in a lek.
> Cheers
> Prakash
>
>


-- 

===============================================
Professor Michael Mislove        Phone: +1 504 862-3441
Department of Mathematics      FAX:     +1 504 865-5063
Tulane University       URL: http://www.math.tulane.edu/~mwm
New Orleans, LA 70118 USA
===============================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.seas.upenn.edu/pipermail/types-announce/attachments/20100112/39099535/attachment.htm