[TYPES] Two-tier reviewing process

[Norman Ramsey]

 > At the POPL discussion, one goal that was raised was to improve the  
 > number of "expert" reviews per paper. People are dissatisfied when  
 > their paper is rejected by self-proclaimed non-experts.  I believe  
 > that Jens pointed out that this year's POPL had 77% papers with one  
 > "X" review.

I went back and got archival data for ICFP 2007.  ICFP is a
significantly smaller conference which that year had only 120
submissions.  110 of 120 submissions (91%) received at least one X
review.  When comparing these data, here are some points to keep in
mind:

  - ICFP reviewing was double-blind that year.
  - Otherwise ICFP used substantially the same review process that
    POPL uses now.
  - POPL is probably a broader conference than ICFP, which may make it
    more difficult to find expert external reviewers.

I remember great difficulty in finding external reviewers for papers
involving functional programming and XML---many were multi-author
papers, and this is a small community with a lot of cross-
fertilization, so there were quite a few papers for which all the
obvious expert reviewers had conflicts.  (One of the problems with
double-blind review is that it makes a prudent program chair more
cautious about conflicts of interest.)



Norman