ARR Mentorship Programme

ARR Mentorship Programme

We expect a large fraction of NLP submissions to go through ARR in the very near future. To make sure that we can handle this growth, and to maintain a diverse and balanced reviewer pool, we would like to set up a reviewer mentorship program in the spirit of previous programs run at ACL. The goal of this program is to help researchers with less reviewing experience to quickly get up to speed with the help of more experienced reviewers.

Overview

The program operates as follows: 1) Mentees and Mentors apply to join the program–see the application forms below.
2) We match mentors and mentees based on availability, review experience, topical alignment etc. 3) Mentees work through online material to prepare for their first review assignment.
4) Once ready, in the following ARR cycle we assign a paper to the mentee (provided there is a good match). 5) In the beginning of the cycle, mentor and mentee meet for an introduction and for high level advice from mentor to mentee 1) This meeting can also be a place for the mentor to raise open questions from reading the online material below. 6) The mentee prepares their review in 14 days (so 10 days earlier than normal) 7) Right after, the mentor and mentee meet to discuss the review and what might need to be improved 1) This can involve the action editor too. 8) The mentee improves the review until stakeholders (mentee, mentor, action editor) decide it has sufficient quality. 9) After the cycle, mentor and mentee decide if more mentorship is necessary or if the mentee can continue to review independently.

Mentees

We are looking for researchers in the area of NLP that have experience with NLP paper authoring and/or reviewing (e.g. have an EMNLP or ACL paper, or reviewed for workshops), but don’t yet feel ready to review for *ACL conferences and ARR. If you fall into this category and are interested in becoming a review mentee at ARR, please fill out the form below:

https://forms.office.com/r/5xyiutt16z Note that we cannot guarantee a place for everyone as the number of mentors is limited.

Mentors

We are looking for ARR reviewers and action editors willing to mentor junior reviewers. If you are interested (and already an ARR reviewer/action editor), please fill out the form below:

https://forms.office.com/r/jxn5eYV2ZW

If you are not yet an ARR reviewer/action editor, please indicate your interest in becoming one using the form on the bottom of https://aclrollingreview.org/reviewers. Feel free to fill out the mentor application form afterwards, but note that we can’t accept you as mentor before we accept you as reviewer/action editor.

Material for Mentors and Mentees

We ask mentees to go through the material below (in full). While we expect mentors to have sufficient review experience, it might still be good for them to review the material too in order to help with communicating what good and bad reviews are.

  • EMNLP 2017 review advice: How to write good review: https://2020.emnlp.org/blog/2020-05-17-write-good-reviews
  • ACL 2017 leaders view on reviewing: https://acl2017.wordpress.com/2017/02/23/last-minute-reviewing-advice/
  • Two example good reviews from NAACL 2018 presented in their reviewing form: https://naacl2018.wordpress.com/2018/01/20/a-review-form-faq/
  • A podcast by Noah Smith about peer reviews: https://soundcloud.com/nlp-highlights/77-on-writing-quality-peer-reviews-with-noah-a-smith
  • Reject if not SOTA? https://hackingsemantics.xyz/2020/reviewing-models/

Other References

  • https://www.aclweb.org/adminwiki/index.php?title=2020Q1_Reports:_ACL_2020
  • https://2021.aclweb.org/blog/reviewer-mentoring-program/
  • https://2021.aclweb.org/blog/reviewer-mentoring-survey/