FAQ For Reviewers and Action Editors

This FAQ is intended to help reviewers and action editors navigate OpenReview and the review process.

Timeline

What is the timeline?

  • Days 1-7: AEs are assigned end AEs assign reviewers/desk reject, no notifications go out
  • Day 7: Notifications to reviewers go out (automatically)
  • Day 30: Reviews are submitted
  • Day 35: Meta-Review submitted
  • Day 35: Reviews released to authors

Communication

Who contacts reviewers to kick off reviewing?

The editor-in-chief email all reviewers to kick off reviewing.

How can I communicate with reviewers?

You may communicate with reviewers using the discussion forum. In OpenReview, you should see a button labeled “Official Comment” at the top of the paper forum. Click on it to open a message form. You can also select to exclusively message certain sub-groups, e.g. only reviewer 2.

How can I communicate with the action editor?

In OpenReview, you should see a button labeled “Official Comment” at the top of the paper forum. Click on it to open a message form. For messaging exclusively the action editors, please select only “area chairs” in the “Readers” field of the form.

How do I communicate to the editors in chief that I am unavailable in a given month?

We hope that in Q3 2021 OpenReview will roll out their v2 API, and we hope that API will allow us to create a button or other functionality whereby you can move yourself from “reviewers unavailable” to “action editor available” and back. For now, if you are unavailable in a month, email editors@aclrollingreview.org.

Desk Rejects

One of my assigned submissions uses a workshop / conference template rather than the ARR template. What should I do?

Please inform the action editors. After the trial period for ARR is over this will be a desk reject.

One of my assigned submissions is more than four pages but less than eight pages in length. What should I do?

Treat it as a long paper submission.

Can the ethics statement extend into a 5th/9th page?

Yes.

What is the compliance checklist?

  • Is the content of this paper appropriate to a *ACL venue?
  • Is the paper in English?
  • Does this paper use the ARR template?
  • Does this paper adhere to length constraints for the paper type (long: 8, short: 4, in both cases exclusive of references and appendices)?
  • Is the submission (including paper and any supplemental materials) anonymized?
  • Is this paper already published elsewhere?

How do I actually desk reject a submission?

In OpenReview, you should see a button labeled “Desk Reject”.

Reviewer Assignments

In what order are alternative reviewers for a submission displayed (eg alphabetical, by match score, random)?

We are working on this.

Can we see the number of submissions a reviewer is already assigned?

We are working on this.

Can we see author information?

No, we are managing a two-way anonymized process up through action editors. It is not possible for you to assign a reviewer who has a COI that is automatically identifiable.

How do I assign and unassign reviewers?

On the “Assigned papers” tab, click on “Show reviewers” for a particular paper. You can then search for reviewers for assigning that paper to a new reviewer.
Below each reviewer name, there is an “unassign” link that can be used to unassign a reviewer for that particular paper. If you want to add a reviewer who is not in the system, please email the action editors. This doc from UAI may help: https://auai.org/uai2017/openreview/reviewing.pdf

Meta-Reviewing

How do I enter a meta-review?

Click on the button in the system.

Why can I not press the “meta-review” button?

The meta-review button may not be clickable if not all reviews are submitted yet. In this case, please try to encourage the reviewers to submit their reviews, or recruit a new emergency reviewer.

How do I flag a submission for ethics review?

Please email the action editors.