[TYPES/announce] Two phase reviewing for POPL; a response

[Jonathan Aldrich]

I appreciate the ongoing conversation, and have some input and ideas to add.

1) There is a *ton* of great work in this community, and since it has 
grown over time, the number of papers accepted should grow also. 
Accepting 25-30% would not lower the quality appreciably, but it would 
help ensure a broader program and focus the decision on "is this of the 
right quality" and reduce the impulse to inject reviewer bias (e.g. 
about topic) into reviews.

2) Presentations are very important.  They're a great way to learn about 
other work.  They are *especially* important as a way way for students 
(who may not already know many people in the field) to get exposure. 
This hasn't been emphasized so far in the conversation.  I'm ok with 
shortening presentations a bit, and going to a multi-track event is a 
good idea.  But let's continue to ensure that all published papers can 
be presented in a main conference session, not a poster session.

3) A journal-like process would be *great* to have as an option for 
submitting to POPL--i.e. submit at any time, review can include more 
than one stage with revisions, and accepted papers appear in the next 
POPL whenever it is.  If this option were available I would use it more 
often than the "regular" POPL process.  I personally am happy to 
contribute to making this happen.

4) Recruiting a single group of papers but accepting them in 2 "tiered" 
categories, i.e. short/long or present/not--is a bad idea.  If I 
submitted a paper to POPL and it got accepted but "demoted" to an 
inferior category I would strongly consider withdrawing the paper and 
submitting to another venue--or not submitting in the first place.

5) More than one paper track, where people submit to different tracks, 
may be worth considering.  OOPSLA has added a track (now a co-located 
conference) called "Onward!" which is specifically intended to accept 
exceptionally innovative, but perhaps not as well validated, papers. 
There are tradeoffs in this approach--less validation often means more 
disagreement about quality--but having a second track recognizes that 
there is more than one dimension to "good" papers and helps to ensure 
that more "out of the the box" work can be considered.

6) Random choice is terrible.  Use it for computing algorithms, but not 
for conference paper or presentation selection.  It is especially cruel 
to submit our students to a random process--even if the current process 
is imperfect, explicit randomness could only make it worse.  I would 
probably never submit to POPL if there were a random choice element.

7) The proposed 2-phase POPL process may improve results, but it will 
also increase reviewing costs and likely increase lead time.  I think a 
journal-like option (perhaps in addition to the normal submission) would 
be a better investment of reviewing effort; then authors can choose if 
they want the greater lead time in exchange for a journal-like process.


Thanks to the POPL steering committee for gathering input on this 
important issue!

Jonathan Aldrich