AC Action Items Sept 29

Personal.ACActionItemsSept29 History

Show minor edits - Show changes to output

Added lines 1-47:
All: I have just "thrown the switch" in the PCS system to make assigned papers available to ACs.

ACs: You can find your assigned papers by logging onto the PCS system at:
https://precisionconference.com/~sigchi

You should now proceed to:
* Look over your papers to find any hidden conflicts, and
* Recruit three high quality external reviewers for each paper.

I have pasted in the details instructions for the steps in front of you below (and a complete copy of the detailed instructions is here: http://pages.cpsc.ucalgary.ca/~saul/wiki/pmwiki.php/Personal/CHI2009Planning).

Remember that although we are asking you to do quite a lot in a short period of time, finding good reviewers is probably the single most important aspect of the review process.

Please let us or your subcommittee chair know if you have any questions or any issues arise. And thanks again for your efforts on behalf of the community.

Scott Hudson and Saul Greenberg
CHI 2009 papers co-chairs

-------------------------------------------------------------

Mon 9/29: AC assignments released to you

All work with papers for the conference is coordinated through the web-based PCS conference system which you will find at: https://precisionconference.com/~sigchi As soon as paper assignments are released you should go to the site and look over each paper.

Tue 9/30: Look for conflicts and other difficulties and optionally swap problem papers for others

As a part of your initial review of each paper you should check for any hidden conflicts of interest that the subcommittee chair might have missed. If you find a conflict you will need to "swap" the paper for another one. Also, if you feel you really are not the right person to serve as 1AC for this paper and there is probably someone else on your subcommittee who would obviously be better suited, you may also attempt to swap that paper for another more suitable one (but we don't expect this to happen very frequently). Swapping works on a "give one, take one" basis. For non-conflict swaps, particularly a little later in the process, it might be best to identify and take a paper from the pool before you return yours. For conflicted papers you must put your paper in regardless, so it's best to do that as soon as possible. In that case you may need to return later to find the replacement, but please do try to find one (if papers remain in the pool past a certain point, the
SC will have to force an assignment to a non-conflicting AC).

SPECIAL NOTE: the first time you view the "unassigned paper pool" to find papers to swap with, you will be shown a view which contains information about all papers submitted to the whole conference. However, it's very important that you do not take a paper outside your subcommittee. To avoid potential errors, please be certain to select the view of papers corresponding to just your subcommittee and then select papers only from your subcommittee for swaps. You select this view using the drop down menu near the top left, immediately below the CHI 2009 logo. Although this is obviously not the most usable possible setup, this use of views for subcommittees was a compromise to keep the cost and debugging issues associated with modifying the PCS system to a minimum, so please bear with us. This and the committee meeting view should be the only places this comes up, and you should only have to do this once for each of these views.

Tue 9/30: Recruit three high quality external reviewers for each paper

Once you have determined that you do not have conflicts, and are going to handle a given paper, the most critical part of the review process -- recruiting of high quality reviewer -- begins. The most critical component of the reforms instituted this year lies in getting high quality reviews, from true experts, for every paper. Since you are picking those reviewers, your role in this is absolutely essential. The most important part of your job lies in recruiting the right reviewers -- not just acceptable reviewers, but good reviewers, and not just some good reviewers, but as best you can, all good reviewers. To help focus on the importance of this task, we are asking that you be prepared at the PC meeting to give a explanation to the rest of the subcommittee for why you picked the reviewers you picked.

At the same time, experienced ACs also know that it's important to recruit reviewers fairly quickly; otherwise you may find that some of the more in-demand reviewers for any given topic may have already made commitments to others. These two constraints can make this part of the job difficult. However, that's why we have recruited the best people in the field to do it.

There are a number of strategies for finding good reviewers, but what you seek in the end is someone who really knows the subject matter, will make substantive insightful comments, and has the perspective to evaluate how interesting the results are and whether they are sufficiently relevant to (some part of) HCI as a field. Your first line of attack for finding good reviewers may be your own knowledge -- if the paper is "in your area" (or close to it) you may be able to directly think of a good candidate that you already know to be an expert. Another excellent way to isolate people knowledgeable and experienced in the area is to consider authors of previously published results on the topic. You will likely find some of these publications in the references of the paper itself. Searching for related work in the ACM Digital Library (http://portal.acm.org/advsearch.cfm) and other search engines is also typically very helpful. Keep in mind that publishing a single paper on a topic, even at a good venue such as CHI, might not mean a person is an expert, and that different authors may have contributed different things to a particular paper. For potential reviewers you are not familiar with in advance, it can be helpful to try to have a look at their overall research record through their web presence.

We generally discourage the use of PhD student or convenient "friends down the hall", unless that person really is a highly qualified expert in the area (for example a late stage PhD student might well be extremely knowledgeable in the topic of the dissertation they are completing and mature enough to be a good reviewer). This year there will be no explicit limits placed on use of reviewers from these categories. But remember that you will be asked to publicly justify the choices that you have made and choosing a PhD student or more than one of your nearby colleagues may need extra justification.

Thu 10/7: Rough target date for having all external reviewers signed up

Each paper must have at least three external reviewers beyond yourself and the possible 2AC review (and please don't request reviews from other ACs as a part of this three.) A good strategy for ensuring that you can find three who will commit to reviewing, is to be prepared for some of your reviewers to decline. As you search for good reviewers, don't stop at a list of three, but when possible go on to identify one or two more candidates who can be held in reserve. That way when/if a potential reviewer declines you will be prepared to immediately ask one of your backup candidates. You may find that potential reviewers are slow to respond to requests. You might want to set deadlines for a response based on the target date above, and you might consider sending a request to a backup choice in the case of non-responders. (You may occasionally end up with more than three reviewers this way. However, we ask that you do not seek more than three reviews as standard practice be cause good reviewers are a finite resource and we have many papers that need them.) Note that this date is a rough target only. The important thing is to ensure that you have three good quality completed reviews by the review deadline.