Final Marching Orders for ACs and 2nd ACs

This is a long note, but it should be read carefully. It contains your final 'marching orders' that detail your duties up to and including the PC meeting.

What to do now.

All:

ACs and 2nd ACs

For ACs

For 2nd ACs

Balance your efforts.

The Criteria.

These are the various criteria for papers. The most important ones in the list below are the reject and accept outright. Other criteria are there to help us focus discussion in the PC meeting.

  1. Score <= 2.7, little divergence - standard deviation <= 1 OR standard deviation > 1 but no single score is above 3.0 (auto-reject)
    • These papers are rejected outright.
    • No 2nd AC will be assigned, nor will the paper be discussed at the committee meeting.
    • The AC can elevate a paper in this category to the category 2 discussion level for situations when they believe that there is a realistic chance that the reject decision should be reconsidered. However, this should be done well before the PC meeting, as a 2nd AC will have to be assigned.
  2. Score <= 2.7, significant divergence (tentative reject)
    • A 2nd AC will be assigned.
    • If both ACs can agree ahead of time that the paper should be rejected, then the ‘discussion’ will be limited to report that outcome at the meeting, otherwise it will be discussed in greater detail.
  3. 2.7 < Score < 4, (uncertain outcome)
    • A 2nd AC will be assigned.
    • If the ACs can reach consensus, a summary of the issues will be presented at the meeting along with the tentative decision, followed by discussion as warranted.
    • If the ACs cannot reach consensus, or if there is wide divergence (>=1), a quick summary of the issues will be presented with the likelihood that at least one other AC will be solicited to read the paper and inform the decision process.
  4. 4 <= Score < 4.3 (tentative accept)
    • A 2nd AC will be assigned.
    • If the ACs reach consensus to accept, a quick summary of the issues will be presented at the meeting along with the decision.
    • If the ACs cannot reach consensus, or if there is wide divergence (>=1), a quick summary of the issues will be presented with the likelihood that at least one other AC will be solicited to read the paper and inform the decision process.
  5. 4.3 <= Score <= 5 (accept outright)
    • A 2nd AC will be assigned
    • If the two ACs agrees with accept, this paper will be auto-accepted.
    • If they do not agree, this paper will be scrutinized carefully, with other ACs brought into the process. Rejecting a paper with a high score should be reserved for only the most extreme cases.
    • Discussion will usually be limited to a summary, unless issues are flagged.

The meeting process in brief

The PC Meeting process will cycle between the top- and bottom-rated papers. Doing so will help calibrate decisions. That is, we will have a session starting with the 'best' papers. After a break, we will restart at the bottom, and continue this cycle until we meet in the middle. The process each paper will typically follow is:

  • paper number is announced
  • ACs / SCs in conflict will leave the room
  • The decision is reported by the AC with little or no discussion if:
    • the paper has generally low scores (but still above the auto-reject threshold)
    • the AC/2nd AC agree to a decision
    • the variance of opinion between reviewers is modest
  • If the decision is positive, a brief summary of the paper is reported.
  • If there is disagreement between ACs / 2nd ACs, or if there is high variance, or if any AC wants to discuss issues, then a summary of issues will be presented
    • Depending on the situation, another AC will be assigned to look at the paper and join the discussion (off-line).
    • The above continues until a decision can be reached by the ACs involved. Additional ACs may be brought in as needed.
  • ACs should be prepared to discuss who the reviewers are and why they chose them.

Note that we are following a 'conditional acceptance' practice this year. If a paper is generally sound but requires a modest revision (and we do mean modest) that you can request, this will ensure that the authors have to follow your advice. If they don't we can still reject the paper.