MetaReviewScores

Dear all ACs;

(SCs... you should watch out for this if you see it in your own review of all scores handled by your committee)

To add to your flood of email during this busy week while you are gathering the final reviews and composing your meta-review, here are some more hints that will help make this a consistent process.

As you are constructing your meta-review and providing it with an AC score, here are a few things to watch out for (based on some things I currently see across all papers).

First, your meta-review score should reflect where the paper fits in the 1 – 5 range. It should NOT be a binary choice, i.e., don’t do:

The reason you should choose an appropriate value in the range is:

Second, your score should, as much as reasonably possible, reflect the comments of referees as well as your own. This doesn’t mean it should be the average, but it does mean that you have to temper your personal score by considering referee scores as well. Here are some samples of where things are likely unbalanced:

The reasons for being careful are similar to those stated in the first point.

Remember that you can always raise a paper for discussion regardless of the score. However, the score as it appears – especially the AC score – will influence how the 2nd AC and the committee view the paper. If you really do want to put your personal score in it, put it in text, and flag it, i.e., “I scored to the papers 3.5 to reflect the diversity of reviews. However, my personal scoring would have been less – perhaps a 2.5 – due to my personal concerns as detailed below”.

Saul Greenberg and Scott Hudson,
Papers co-chairs