This FAQ is intended to help reviewers and action editors navigate OpenReview and the review process.
Timeline
What is the timeline?
- Days 1-7: AEs are assigned end AEs assign reviewers/desk reject, no notifications go out
- Day 7: Notifications to reviewers go out (automatically)
- Day 30: Reviews are submitted
- Day 35: Meta-Review submitted
- Day 35: Reviews released to authors
Communication
Who contacts reviewers to kick off reviewing?
The editor-in-chief email all reviewers to kick off reviewing.
How can I communicate with reviewers?
You may communicate with reviewers using the discussion forum. In OpenReview, you should see a button labeled “Official Comment” at the top of the paper forum. Click on it to open a message form. You can also select to exclusively message certain sub-groups, e.g. only reviewer 2.
How can I communicate with the action editor?
In OpenReview, you should see a button labeled “Official Comment” at the top of the paper forum. Click on it to open a message form. For messaging exclusively the action editors, please select only “area chairs” in the “Readers” field of the form.
How do I communicate to the editors in chief that I am unavailable in a given month?
We hope that in Q3 2021 OpenReview will roll out their v2 API, and we hope that API will allow us to create a button or other functionality whereby you can move yourself from “reviewers unavailable” to “action editor available” and back. For now, if you are unavailable in a month, email editors@aclrollingreview.org.
Desk Rejects
One of my assigned submissions uses a workshop / conference template rather than the ARR template. What should I do?
Please inform the action editors. After the trial period for ARR is over this will be a desk reject.
One of my assigned submissions is more than four pages but less than eight pages in length. What should I do?
Treat it as a long paper submission.
Can the ethics statement extend into a 5th/9th page?
Yes.
What is the compliance checklist?
- Is the content of this paper appropriate to a *ACL venue?
- Is the paper in English?
- Does this paper use the ARR template?
- Does this paper adhere to length constraints for the paper type (long: 8, short: 4, in both cases exclusive of references and appendices)?
- Is the submission (including paper and any supplemental materials) anonymized?
- Is this paper already published elsewhere?
How do I actually desk reject a submission?
In OpenReview, you should see a button labeled “Desk Reject”.
Reviewer Assignments
In what order are alternative reviewers for a submission displayed (eg alphabetical, by match score, random)?
We are working on this.
Can we see the number of submissions a reviewer is already assigned?
We are working on this.
Can we see author information?
No, we are managing a two-way anonymized process up through action editors. It is not possible for you to assign a reviewer who has a COI that is automatically identifiable.
How do I assign and unassign reviewers?
On the “Assigned papers” tab, click on “Show reviewers” for a particular paper. You can then search for reviewers for assigning that paper to a new reviewer.
Below each reviewer name, there is an “unassign” link that can be used to unassign a reviewer for that particular paper.
If you want to add a reviewer who is not in the system, please email the action editors.
This doc from UAI may help: https://auai.org/uai2017/openreview/reviewing.pdf
Meta-Reviewing
How do I enter a meta-review?
Click on the button in the system.
Why can I not press the “meta-review” button?
The meta-review button may not be clickable if not all reviews are submitted yet. In this case, please try to encourage the reviewers to submit their reviews, or recruit a new emergency reviewer.
How do I flag a submission for ethics review?
Please email the action editors.