A platform for physical and virtual collaboration

Ask any student or remote worker and they’ll tell you, Zoom fatigue is real. However, researchers at Carnegie Mellon University think they’ve found the answer. The Augmented Reality Edge Networking Architecture (ARENA) is a unique new networking platform designed by the CONIX Research Center.

The ARENA aims to change the way physical and digital entities interact, combining Augmented Reality (AR) with Virtual Reality (VR) into a single platform. Inside the ARENA, users can attend a virtual conference, hold meetings in a digital replica of their office, or even swing by a coffee shop to chit chat with colleagues—just a few of the many social interactions that have been lost to the remote work environment. These innovations are possible thanks to collaborative mixed reality technologies that link headsets and smart devices in a way that seamlessly blends the physical with the virtual.

While AR uses computing to enhance the real world, VR replaces the real world altogether with a virtual one. ARENA combines both AR and VR to allow remote participants in VR to interact with AR users in a physical space. For example, if a surgeon in the ER needed to consult a specialist half-way around the world about a procedure, the specialist could be digitally teleported using VR into a live capture of the operating room where the surgeon is using enhanced AR glasses. The surgeon could see 3D scan information overlaid on the patient and look up to see a hologram of the specialist interacting with them as if they were in the same room.

While the foundation of the ARENA was in place before the Coronavirus pandemic landed, the transition to remote work only accelerated the need for a virtual platform for collaboration. “Though our main goal was creating AR applications,” said Nuno Pereira, executive director of CONIX, “the current social distancing requirements and inability to share physical space have made the ARENA and its VR features a perfect virtual space for conducting our own meetings and events.”

With the help of a 3D scanning camera, researchers created a centimeter accurate model of the Tepper building on campus, which was then used to hold a poster session for the last CONIX annual review.  Previously, such level of detail could only be achieved by a 3D artist, performing intensive hand-drawn modeling.

The team is experimenting with several features using dynamic audio and visual content to convey a more realistic sense of presence and proximity in VR. Objects like a virtual presentation screen reliably mimic their real-world counterpart, allowing members to sit in an “audience” to watch a presentation or lecture, even conveying a sense of distance by adjusting its volume based on an individual’s proximity to the screen. The open-source, highly customizable nature of the platform means that users will be able to create almost any feature they would imagine as a runtime loadable application. They can easily share their programs with other users.

“It’s easy to imagine numerous practical applications such as modeling how equipment may function within a factory space or a wayfinding application to guide users through spaces like a campus or airport,” said Anthony Rowe, professor of electrical and computer engineering and co-director of CONIX.

In a way, it feels like the pioneering days of the Internet.

Anthony Rowe, Professor, Electrical and Computer Engineering

The team anticipates most immediate interest will be in VR applications; however, these developments are also crucial steps on the path to building a new architecture for AR/VR interaction. Some features are still in progress, such as face tracking software, and tools to create more realistic avatars and interactive virtual worlds. Other aspects, such as those involved in augmented reality and indoor localization, will play a much greater role in the fully realized ARENA.

The team is currently launching a network of ARENA servers across the seven universities that are part of the CONIX Research Center. The CONIX researchers are working on transplanting the same virtual objects they’re creating in different ARENAs in order to create a massive shared AR/VR environment. In Rowe’s words, “In a way, it feels like the pioneering days of the Internet, as we roll out a platform that we believe has explosive potential well beyond what is obvious to us today.”

You can experiment with ARENA yourself at https://arenaxr.org/.