Collaborating remotely on physical objects has always been a challenge, but a new system called SharedNeRF is changing the game. Developed by Mose Sakashita, a doctoral student in information science, this system allows users to manipulate a 3D view of the scene in real-time, making complex tasks like hardware debugging much easier. By combining slow, photorealistic rendering with instantaneous but less precise techniques, SharedNeRF offers a unique way for remote users to interact with physical spaces.
Sakashita worked on this remote conferencing tool as an intern at Microsoft in 2023, collaborating with Andrew Wilson, a former Cornell computer science major. The system, which will be presented at the ACM CHI conference, leverages photorealistic rendering and view-dependent techniques to enhance real-time remote collaboration. Wilson pointed out that traditional video conferencing systems struggle when it comes to tasks involving physical objects, and SharedNeRF is at the forefront of utilizing advanced computer graphics and rendering techniques to address this issue.
Sakashita’s research, conducted at the lab of François Guimbretière, focuses on developing new technologies for remote collaboration. SharedNeRF stands out by utilizing a neural radiance field (NeRF) rendering method, which constructs a 3D representation of a scene using 2D images. This approach creates highly realistic depictions with reflections, transparent objects, and accurate textures that can be viewed from any angle. By combining detailed visuals from NeRF with point cloud rendering, the system enables remote users to experience the scene in high-quality while also seeing real-time movements through point clouds.
Seven volunteers participated in testing SharedNeRF by engaging in a collaborative flower-arranging project. The majority preferred SharedNeRF over standard video conferencing tools or point cloud rendering alone, citing the system’s ability to provide detailed views and enhanced control over what they were seeing. Users appreciated the independence to change viewpoints, zoom in on details, and interact seamlessly with the scene. While SharedNeRF currently supports one-on-one collaboration, researchers are exploring possibilities to extend it to multiple users in the future. Improving image quality and incorporating virtual reality or augmented reality techniques are on the agenda for further enhancing the system.
SharedNeRF represents a significant advancement in remote collaboration tools, offering a unique and immersive experience for users working on complex physical tasks. With its innovative approach to rendering and real-time interaction, SharedNeRF has the potential to revolutionize the way people collaborate across distances. As technology continues to evolve, we can expect to see even more exciting developments in the field of remote collaboration, making it easier than ever for individuals to work together seamlessly, regardless of their physical locations.
Leave a Reply