As an author, reviewer or action editor for OpenReview, when and how should you declare a conflict of interest?
The ACL has a conflict of interest policy. It says: “A person has a conflict of interest with a submitted paper if that person:
- is a co-author of the paper, or
- has been a student or supervisor of one of the authors in the previous five years, or
- has co-authored a paper or collaborated with one of the authors in the previous five years, or
- is employed at the same company or institution as an author, or
- has any other circumstances that could cause a bias in evaluating the paper.”
Everyone
We try to automatically identify conflicts of interest using the OpenReview profiles of authors on a submission and of each potential reviewer or action editor. We check:
- submission co-authorship by looking at the submission
- student / supervisor relationships via your declared relationships in your OpenReview profile
- historical co-authorship by using your publications from your DBLP and Semantic Scholar entries in your OpenReview profile
- employment via your domains / employment in your OpenReview profile, backing off to your email addresses if necessary
- other circumstances using your declared conflicts of interest in your OpenReview profile
This means that it is critical to have a complete OpenReview profile. Now, if you don’t regularly publish, or are a first-time author, you may not have Semantic Scholar or DBLP entries. But as soon as you get that first publication in a computer science discipline (congratulations!) please add those profiles.
These helpful guides show you how to set up your OpenReview profile and so much more:
Reviewers and Action Editors
If you take a look at a submission and think, oh, I might have a conflict of interest here, you should report it immediately. You should not review the submission. This is why it’s a good idea to take a look at your assignments as soon as you receive them. Reviewers should report COIs to their action editor; action editors to their senior action editor. If you are not removed from the submission within 24 hours, email editors@aclrollingreview.org.
Some Frequently Asked Questions
-
Isn’t it a lot of work to set up a profile? - It takes about 10 minutes to set up a profile; and you’d have to have a profile whether we used OpenReview or START.
-
Can’t the system be gamed by providing incomplete or incorrect information? - Yes, malicious people could do this. This would be a clear violation of professional behavior, and a matter for the ACL ethics committee.
-
Can’t you just desk reject every submission where the authors don’t have complete profiles? - First-time authors won’t yet have Semantic Scholar or DBLP entries, so we can’t just desk reject because a profile doesn’t contain this information.
-
My co-author has never published before or is not in our field. Do they need an OpenReview profile? It’s a lot of overhead for making a submission. - This is the flip side of the coin from the previous question. If you have a co-author who couldn’t make a profile for some reason, please add an Offical Comment to the editors (program chairs) on your submission in OpenReview.
-
Can’t AEs just check manually? - Our field is really large; an AE is unlikely to know all or even any of the potential conflicts of interest between authors and reviewers, even if they knew the identities of the authors, which they don’t.
-
The ACL policy is overly broad; I work at a big tech company and shouldn’t have conflicts with 500 other people I’ve never met. - That would be a question for the ACL exec.
-
In the presence of “noise” (e.g. incomplete profiles) how can we minimize the assignment of submissions to reviewers with conflicts of interest? - We do not assign reviewers for whom we don’t have basic information (so if you are wondering why you have never received an assignment, check your OpenReview profile). Beyond this, we could minimize potential conflicts of interest by assigning reviewers based on the inverse of their expertise match to a submission - the opposite of what we do now. However, this would lead to bad reviews from people with no expertise in the subject matter. A more nuanced policy (second best match?) is also possible. However, this shows that great conflict of interest handling and good reviewer assignment are highly related, and the best way to get both is to complete your OpenReview profile.