SIGGRAPH is just around the corner so that means research groups like Facebook Reality Lab are starting to reveal some of their cutting edge work. The latest from FRL (formerly Oculus Research) demonstrates a method to capture the appearance of mirrors from the real world.
The new research opens up the door to capturing the look of complex real world environments which feature mirrors and reflective surfaces. Mirrors are the enemy of a number of computer vision applications and if Facebook’s research could be used to help solve that problem it might ultimately lead to a number of useful applications.
“Mirror and glass surfaces are essential components of our daily environment yet notoriously hard to scan. Starting from the simple idea of robustly detecting a reflected planar target, we demonstrate a complete system for robust and accurate reconstruction of scenes with mirrors and glass surfaces,” the report reads. “Given the ease of capture, our system could also be used to collect training data for learning-based approaches to detect reflective surfaces. Besides our core application of scanning indoor scenes, we envision multiple extensions and applications.”
The system finds mirrors by looking for a target that is on the camera rig, then refines the shape of the mirror by analyzing various features of the image.
“Our key idea is to add a tag to the capture rig that can only be observed when the camera faces a mirror or glass surface,” the report reads.
The most obvious application is easier to capture environments that are more realistic to experience in VR. The research could enable representations of real world locales to more realistically mix with digital elements, like your avatar, even if there are a number of mirrors and reflective surfaces.
“Mirrors are typically skirted around in 3D reconstruction, and most earlier work just ignores them by pretending they don’t exist,” Research Scientist Thomas Whelan said in an Oculus blog post. “But in the real world, they exist everywhere and ruin the majority of reconstruction approaches. So in a way, we broke the mold and tackled one of the oldest problems in 3D reconstruction head-on.”
Source: VentureBeat
