Future Improvements to ARR

This page describes some ideas that have been proposed by the community, and that we are considering for the future.

Planned Future Work

An author feedback mechanism not only for reviewers, but also ACs (for action by SACs)
Enabling the joint track work by all track SACs
Improving the workflow to pass reviewed papers to other conferences/workshops
Improving the integration with proceedings generation
Reviewer profiles with service history (in development at OpenReview)
Create a public record of who actually did reviewing, etc for each cycle.
Using ACL Pub Check to assist PCs with desk reject identification

The following ideas have been suggested and may be considered in future ARR development:

Adding a category of “training” reviewers. Assign them to papers in addition to the normal reviewers. This will help grow the pool of reviewers over time.
A new type of desk reject, disallowing resubmission (for some rare cases of procedural violations)

Show AEs and SAEs the meta-review scores in the list of papers.
Show SAEs, AEs, and reviewers the planned schedule in the OpenReview interface.
A formal way to indicate secondary reviewer contributions.
Give authors the ability to download all materials for a paper as a PDF, maintaining anonymisation.
Mechanism for SAEs to nominate good/bad AEs.

On the stats page, show resubmission stats as distributions rather than with box plots (at the moment it is hard to see what the median is).

A mentoring system for new reviewers.
An interactive tutorial for reviewers
Create a system that detects low-quality review comments at the sentence level. This could either (a) be shared with reviewers, encouraging them to make changes, or (b) shared with AEs, who can then nudge reviewers.
Encourage authors of resubmissions to mark up the changes. Maybe do it automatically?
Automatic checks for length violations.
Integrating the review system of ARR into OrcID so that there is a method for showing reviewing contributions.