NeurIPS 2020 Online Experiments: Gather Town Poster Sessions and Mementor

Y-Lan Boureau, Facebook AI Research
Hendrik Strobelt, MIT-IBM Watson AI Lab at IBM Research

NeurIPS 2020 Online Experience Chairs

NeurIPS 2020 was held online in December 2020. Attendees had reported mixed experiences with poster sessions from previous online conferences, and found mentor-mentee matching very valuable. We aimed to craft a smoother experience by re-designing poster sessions and making mentorship connections easier through a new matching platform. We wanted to lean into the unique opportunities afforded by the virtual format, which allows for blindingly fast browsing compared to tired feet, while preserving the serendipity of unplanned discovery of many colleagues wandering in a shared space.

Deciding on Gather Town for poster sessions

ICML and ICLR had held their poster sessions as Zoom calls for each poster. Feedback from attendees and presenters was mixed: presenters were often alone for two hours in zoom calls that attendees were wary of joining for fear that they might be alone with the presenter, attendees had no general snapshot of what other people were seeing at any given time.

The affinity workshops at ICML had experimented with Gather Town for poster sessions. Some of the feedback was very promising: having posters in a shared virtual space where people could walk their avatars around was well-received and solved some of the issues of Zoom calls. However, the set-up of poster rooms had been somewhat chaotic, and disconnection and attendance control issues disrupted the sessions. The scale of NeurIPS was massively larger than any poster session Gather Town had ever hosted, with nearly 2000 posters and registration projected to be above 20,000 people.

Early talks with Gather Town seemed promising: the set-up could be automated by using the API to populate poster rooms, access could be controlled through SSO (Single Sign On) so that only registered, logged in NeurIPS attendees could enter the towns, and new towns could be designed to solve many of the interaction problems that had surfaced from previous Gather Town uses, such as people not knowing how to find a given poster, or not seeing the boundaries for interaction zones for each poster. Scalability concerns could be addressed by setting up multiple poster rooms with each room having its own dedicated server space, to guarantee smooth running for a given capacity.

We decided on a hybrid solution using both Gather Town and Zoom calls. Each poster would get both a presentation spot in Gather Town and a dedicated Zoom call accessible from both the NeurIPS website and the poster town. Zoom was a battle-tested, scalable solution with a robust record that could handle heavy attendance in case of a crowded poster. If Gather Town was plagued with unforeseen issues on the day of the session, the Zoom calls could act as a familiar back-up.

Setting up the poster sessions before the conference

The first challenge was setting up the towns in advance of the conference. We worked from a target of smoothly accommodating up to 10,000 simultaneous attendees for the poster sessions. We did not expect the simultaneous attendance to ever be that high, but we wanted to be prepared in case it was (it turns out simultaneous attendance did not go over 2,000). The towns in Gather can support 2,000 people, but Gather engineers recommended that we aim for 400 people per town for the smoothest interactions with heavy video and poster use. For each of 7 poster sessions, Program Chairs painstakingly clustered the posters into 11 or 25 thematic clusters of up to 20 posters — the morning (in US timezones) sessions were preferred by roughly twice as many people as the evening ones.

Gather Town worked with us to produce a new custom poster room template that would address feedback from previous conferences, and a “hyperroom” that would allow attendees to go from a poster room to another. We can’t stress enough how valuable early usability tests were, allowing us to iterate and arrive to a better final custom poster room template: posters were clearly separated by markings on the carpet of the poster room, allowing to see which attendees were listening to a poster or simply wandering; each poster had a clearly marked presenter spot to easily spot the presenter; people could teleport directly to the poster of their choice from the NeurIPS website, and a coordinate systems allowed people to locate a poster of interest once they were in a room.

The final custom poster-room template

We then wrote the code to automatically create and populate the poster sessions. The rarefied documentation for the Gather Town API and the fact that most of us are not web programmers made this step far from frictionless, even though Gather Town engineers got us started by sharing a sample script for populating a poster room. Thankfully, this was alleviated by their constant availability to answer our questions on Slack.

We hope to make this stage a lot easier for organizers in the future by releasing our poster session set up code within the MiniConf repo, at https://github.com/Mini-Conf/Mini-Conf/tree/master/gather. This code creates poster sessions as sets of poster towns organized in a garden.

Once the poster sessions were set up, we reached out to poster presenters to give them a chance to check how their posters looked in advance of the conference. The advance checks turned out to be very important: about 10% of posters got re-uploaded in the first iteration, and some posters needed to be re-uploaded several times before authors were satisfied with the result.

During and after the conference

Attendees who went to poster sessions gave overall positive feedback: according to a post-conference survey, 75% of people who attended poster sessions enjoyed the experience, while 8% didn’t, Attendees were having lively interactions, reported being satisfied with the ability to instantly see what posters other attendees were visiting, and enjoyed randomly bumping into colleagues. Interactions were mostly smooth thanks to the generous capacity we had budgeted for. Gather Town engineers were available throughout the poster sessions to assist attendees with any problems, in a dedicated support booth we had set up for them in the poster session garden.

Garden from which all poster rooms could be accessed, with the Gather Town support booth

There were many fires that had us working down to the wire, sometimes because of our own mistakes trying to ship last-minute improvements, sometimes due to people’s documented aversion to reading instructions on how to check and update their posters (even a one-liner with bolded words in capital letters), or watching videos in advance about how to interact with the towns, with a stark preference for the wonder of just-in-time exploration and trial and error. But we discovered that communication plastered on the floor close to where people are going to need the help was an effective megaphone.

Empirically, over 10% of people who receive these instructions will not follow them and will email to ask why the update didn’t work.

Trying to inform people as quickly as possible when we messed up by pasting it on the floor

We also used Gather Town to set up a social space for attendees to hang in. We were somewhat worried that the towns could be overwhelmed by the larger-than-expected registration numbers at NeurIPS, so didn’t overly advertise the social space, or schedule specific events there. We split up the social space into several separate areas to ensure that it could support a large capacity, with a hanging space for smaller group discussions, a lobby, and a wellness garden with relaxation, workout, yoga and cute animal videos as well as access to the poster sessions. However, the splitting into separate spaces meant that attendees who were in the social space were too spread out and couldn’t see each other if they were in separate spaces. In the future, it might work better to build incrementally: start from a small, more compact space, and dynamically add more rooms if the first space is at capacity.

Map of the social space: Gather Cafe

Overall, we are happy with our decision to use Gather Town, and have learned a lot from this attempt. Attendees have given us precious feedback and suggestions, for example expressing an appetite for slightly more structure in the unstructured interactions: scheduled social time in the social space so that people can be expected to be there at the same time, dedicated space for people who are welcoming random chats with strangers. We think it could support even richer unplanned interactions in future conferences and will keep exploring how to balance the uncertainty of who is going to show up and how they will behave, with the excitement of experimenting with new ways to help people connect.

Creating a mentor platform

This was a joint effort of our team and Marc Deisenroth, Emtiyaz Khan, Cheng Soon Ong, Adam White, and Olga Isupova to enable mentorship opportunities for researchers in machine learning, both as mentors and mentees, with a special focus on underrepresented minorities. We think that these mentor sessions are a beneficial outcome of the need to go online. In physical conferences, it was harder to find a small group setting for meeting mid-career or senior members of research communities — in Mementor it is much easier.

Scheduling a mentor session is easy. To limit misuse, the video link is only available 30 min before start.

After experiences with spontaneous mentorship sessions at ICLR and ICML, we thought of creating our own platform specifically supporting this idea. We wanted to create a platform that could not only be reused from conference to conference but also allow these gatherings throughout the year. For mentors the platform should be an easy way to disseminate the information about a mentor session and for mentees to stay informed. Scheduling a meeting requires some essential information and a topic — which can be specific, like “ML in Health Care”, or general, like “Ph.D. advice”. The scheduling is announced via email to all subscribers.

The web portal was released and tested during NeurIPS 2020. It is available at https://mementor.net .

Mentor sessions are announced via email and are shown on the landing page. Events can be exported to iCal.