TLDR:
- All authors must complete a form to confirm their OpenReview profile is complete and they will serve if asked.
- Any qualified author may be assigned to review. Our analysis suggests that the average review load will become lower as a result.
- ‘Qualified’ means that a person has (a) at least two papers in main ACL events or Findings, plus (b) at least one more paper in the ACL Anthology or a major ML/AI venue. See the end of this post for lists of venues.
- Review duty exemptions are still possible on a a case-by-case basis. Anyone serving in another capacity (e.g., AC) does not need to review.
- The reviewers or chairs deemed ‘highly irresponsible’ by the program chairs will not be able to commit their work to EMNLP, or (re-)submit their work to the subsequent ARR cycle. This includes missing the review submission deadlines without a warning, egregious violations of guidelines on LLM use and professional tone, extremely terse reviews.
- The great reviewers and chairs will receive more recognition during conferences, and may win a free virtual attendee registration for an *ACL event!
Background
Starting in April 2024, ACL Rolling Review required that at least one author per submission volunteer to review for the same cycle. This helped to make the system more sustainable, but there was still a problem with obtaining sufficient reviewing capacity, and a concern with fair distribution of reviewer load. For example, some big labs had many submissions, but only nominated one senior author, the same one for all submissions. This meant those labs received a lot more reviewer effort than they contributed.
In the October 2024, December 2024 and February 2025 cycles we experimented with various forms of requests to the prolific authors to increase their reviewing load proportionally (while also allowing them to mentor subreviwers instead of performing all reviews personally). In February 2025 we added more specific guidelines for calculating the proportional reviewing contributions, and also allowed non-authors to be nominated (so that e.g. a senior postdoc in a big lab could help with reviewing). Still, so far these approaches have not solved the ‘big lab’ problem, and they became very complex for the authors, adding unnecessary stress over possible desk rejections. In such a lab, it is possible that not everybody even knows about all the submissions, and so they cannot coordinate their reviewer effort to correctly satisfy the requirements.
Meanwhile, our colleagues at CVPR 2025 have successfully experimented with an alternative, simpler approach: all qualified authors on any submission are required to review. They also introduced the penalty for ‘highly irresponsible’ reviews, rejecting 19 papers that would otherwise have been accepted to CVPR. Very recently, NeurIPS also announced a similar policy with desk rejection sanctions, as well as delaying the review release to the authors who are late reviewers.
Reviewer incentives
The ACL peer review committee, in coordination with EMNLP 2025, and learning from the experience of CVPR chairs, developed a comprehensive incentives policy proposal for peer review at *ACL venues, which has been adopted by the ACL exec and will apply at ARR starting from May 2025 cycle.
Positive incentives. The great reviewers and chairs will receive increased recognition during the conferences, and may also win a free virtual attendee voucher for an *ACL event that they may not have attended otherwise. See the full policy for more details.
Negative incentives. The reviewers or area chairs who the program chairs deem ‘highly irresponsible’, will not be allowed to (re-)submit or commit their work during the next ARR cycle (which would typically include the commitment date for the closest upcoming conference). Hence, the teams of papers with such reviewers/chairs as authors will have to sit out a cycle.
We hope that this policy will help to engage the authors to make sure that everyone on their teams submits their (meta-)reviews on time and in accordance with guidelines, and decrease the chasing effort for the chairs. The senior authors can still rely on and mentor secondary reviewers.
What qualifies as ‘highly irresponsible’? The EMNLP 2025 PCs will publish their initial set of criteria at https://2025.emnlp.org/reviewer-policies, based on discussion with CVPR chairs and ARR. As a quick heads-up, this policy will cover the non-submission of (meta-)reviews on time without due warning, extremely terse reviews for good-faith submissions, and egregious cases of unprofessional tone and LLM policy violations. These criteria will be iteratively developed in future ARR cycles based on observed cases of misconduct.
Reviewer recruitment
New reviewer recruitment strategy. An ACL exec member asked us to consider the CVPR approach. Based on our communication with the CVPR chairs and our simulation of this approach for the February 2025 cycle, this would allow for a smaller average review load (about 4 per reviewer). It is also much simpler for the authors than calculating how much effort they need to contribute, given all the papers in the lab. In coordination with the ACL exec, the ACL peer review committee, and the EMNLP 2025 program chairs, ARR will implement this approach in the May 2025 review cycle. We will still allow the primary reviewers to mentor subreviewers instead of performing reviews by themselves. Nominating non-author reviewers will no longer be possible.
Note: ARR will continue to allow qualified authors to ask for an exemption, e.g., if they are on parental leave. If an author is serving in another capacity, e.g., as an AC, then that will be indicated in the form that all authors must complete.
Logistics. After a paper submission the openreview console will show a link to author registration (e.g. for May 2025 cycle the console will be at https://openreview.net/group?id=aclweb.org/ACL/ARR/2025/May/Authors). All authors must register before 48 hours after submission deadline (i.e. May 21 EoD AoE for EMNLP 2025). Based on qualifications, not all authors may actually be assigned reviews, but they all need to sign up. Since the reviewer pool will grow, we expect that the processing of exemption requests and compliance checks may take longer, and so it is possible that desk-rejects due to non-compliance for this requirement could happen later in the cycle. The authors selected as reviewers will receive kick-off emails with instructions and links.
The submitting authors should make sure that all authors on the team are aware of this requirement, and sign up to review on time. There will be an extra warning about this in the submission form. Later in the cycle, they should also coordinate within their teams to check that all (meta-)reviews assigned to the team are submitted on time and in accordance with the guidelines (reviewers, ACs). The planned review deadline for May 2025 cycle is June 18, meta-review deadline - July 15.
Reviewer qualitifactions. Among other feedback we received is that the requirement of having 3 main ACL conference or Findings papers is too stringent a requirement for reviewers. We will relax it to the requirements described at the top of this blog post.
The rationale for requiring prior work in ACL anthology at all is that each scientific community has its norms and expectations, and with too diverse a pool of reviewers the submissions may get judged by the standards of a different scientific community, which is frustrating for the authors. Even interdisciplinary submissions typically have at least some connection to the community in which they intend to publish. That said, when there are no qualified reviewers per paper, ARR does consider reviewer candidates on a case-by-case basis. All authors are expected to review if they receive assignments.
AC qualifications. We are also modifying our Area Chair requirements slightly. ACs need to meet three criteria:
- Either a completed PhD, or a main conference *CL publication from more than 5 years ago.
- Four papers in main ACL events or Findings.
- Two additional papers in either main ACL events or major ML/AI venues.
- Extensive reviewing experience.
Venues considered ‘main ACL’ are: ACL, CL, CoLing, CoNLL, EACL, EMNLP, HLT, IJCNLP / AACL, LREC, NAACL, TACL, *SEM. For conferences that have a main track and Findings, papers in both the main track and Findings count for our purposes.
Major ML/AI venues we consider are: AAAI, CVPR, ECCV, FAccT, ICCV, ICLR, ICML, IJCAI, JAIR, JMLR, NIPS, NeurIPS, TMLR, TPAMI.