Author Response

The NeurIPS review period has now passed and the author response phase has started!

Authors have from Thursday July 25th to Wednesday July 31st to draft and submit a rebuttal that addresses the concerns of the reviewers and clarifies any potential misunderstanding/factual errors in the reviews. The author response will then inform the discussion between the reviewers and the Area Chair (AC), that will lead to their final recommendation.

Period between reviewing and author response

Some may have noticed a gap between the review submission deadline (July 15th) and the author response phase (July 25th). This period in the schedule is meant for ACs, Senior Area Chairs (SACs) and the Program Chairs (PCs) to chase overdue reviews and obtain emergency expert reviews when needed.

While the majority of reviewers finish all their reviews on time, a significant number of reviewers submit their reviews late or entirely drop out at the last minute (sometimes for legitimate reasons, but often not). The consequence is that, 36 hours after the review submission deadline, only 73% of papers had at least 3 reviews, and around 90% of all reviews had been submitted.

Understandably, this period is particularly busy for ACs, SACs, PCs and emergency reviewers.

Dual submissions and slicing too thin

As discussed in our Call for Papers, NeurIPS coordinates with other conferences to identify dual submissions. These are conferences that overlap in subject areas and review period with NeurIPS. We want the content of the NeurIPS conference to be original and novel. As such, we want to actively discourage “double dipping,” where essentially the same content appears at two different conferences. Additionally, we want to avoid authors trying to game the system by giving a single submission multiple chances of being accepted to a conference. Identifying such submissions adds significant workload for the PCs.

For this, we have worked with the program chairs of BMVC, ECML-PKDD, EMNLP-IJCNLP and ICCV. Our workflow managers ran a script to compare the text of NeurIPS submissions with that of submissions with overlapping authors at these conferences. Then, we inspected the most similar pairs to determine if they violated the dual submission policy.

Through this exercise, we found 19 submissions to NeurIPS that were determined to significantly overlap in claimed contributions with other submissions from the same authors to different conferences. These submissions have now been desk rejected. We also identified a few borderline cases and asked the assigned AC to take a close look to help us determine a fair course of action.

Additionally, we ran our submission similarity script on all NeurIPS submissions with overlapping authors, in an attempt to highlight cases with significantly overlapping contributions. Here, instead of inspecting these cases ourselves, we assigned these submissions to the same reviewers and ACs. We then informed them that these submissions should be evaluated together to determine if accepting one would render the other too incremental to be accepted.

For more information on author responses, dual submissions and slicing too thinly, see our Call for Papers.

Alina Beygelzimer, Emily Fox, Florence d’Alché-Buc, Hugo Larochelle
NeurIPS 2019 Program Chairs