Center For Visualization & Virtual Environments
Research Project:
Volumetric Reconstruction for Scenes with Textureless Regions and Specular Highlights
Investigators:
Ruigang Yang
Sifang Li
Collaborators:
Greg Welch (University of North Carolina),
Marc Pollefeys (University of North Carolina)
Sponsors:
UK Office of Research, National Science Foundation
There has been a considerable amount of work on volumetric scene reconstruction from multiple views. Most of this work can be considered variations of the Space Carving framework by Kutulakos and Seitz. Under this framework, an initial bounding volume is divided into a regular 3D voxel grid, then inconsistent voxels are removed until the remaining voxels are photo-consistent with a set of input images. That is, rendered images of the resulting voxels from each input viewpoint should reproduce the actual image as closely as possible. Because of the flexibility of the volumetric representation and the elegant treatment of visibility, space carving approaches have been used to achieve strong results on a variety of both natural and artificial scenes. However such approaches typically run into difficulty when applied to scenes with textureless or specular surfaces.
We present two extensions to the Space Carving framework. The first is a progressive scheme to better reconstruct surfaces lacking sufficient textures. The second is a novel photo-consistency measure that is valid for both specular and diffuse surfaces, under unknown lighting conditions.
Our techniques are particularly effective for scenes in which textureless regions and specular highlights are the norm instead of the exception, for example, in a surgical environment. Such a scene poses significant challenges for traditional reconstruction techniques. However, results from our techniques are encouraging.
Currently we are investigating better representation schemes for large-scale reconstruction and more general photo-consistency measures that can be applied to other types of materials such as translucent or reflective ones.
