Single view modeling: a new technique creates realistic 3D free-form surfaces from photographs and paintings. (techwatch).

The human visual system is incredible not only because it allows us to see, but also because it allows us to perceive more in what we see than is actually there. When we look at a photograph or a painting, for example, not only do we see the 2D representation, but we also perceive the "hidden" 3D shape information by incorporating a variety of cues, including shading, texture, and focus.

Realizing that a digital incarnation of this impressive capability could have a significant impact on the creation of realistic computer-generated imagery, many graphics researchers have focused their efforts on developing algorithms to somehow mimic this aspect of the human perceptual system. The result has been the development of various computer vision-based techniques for single-view reconstruction of 3D models from a photographic scene. A particular hurdle, however, has been the re-creation of free-form surfaces with general reflectance properties.

"Some single-view methods are applicable only if the shape is planar or consists of primitive shapes, such as spheres and cylinders," says computer graphics researcher Li Zhang at the University of Washington in Seattle. "Other methods work by requiring users to manually specify the depth for every pixel, and yet others work only if the statistics of a target shape are known a priori."

To address this deficiency, Zhang and University of Washington colleague Steven Seitz, along with Guallaume Dugas-Phocion, and Jean-Sebastien Samson of the Ecole Polytechnique in France, have developed a novel system for reconstructing free-form 3D scene models with arbitrary reflectance properties from a single painting or photograph with no prior knowledge about the shape. The interactive technique updates the model in real time as constraints are added, allowing fast reconstruction of photorealistic scene models.

Most methods for free-form surface modeling focus on generating smooth artificial surfaces and thus yield images with an artificial appearance. The new technique takes as input a sparse set of user-specified constraints, including surface positions, normals, silhouettes, and creases, and generates a "well behaved" 3D surface satisfying the parameters. As each constraint is specified, the system recalculates and displays the reconstruction in real time.

A Modeling Hierarchy

The new system builds on previous work in hierarchical surface modeling. Basically, a scene is modeled as a piecemeal continuous surface represented on an adaptive grid, and it is computed using a wavelet-based hierarchical transformation technique. Wavelets let a function be represented at a lower resolution by maintaining values, called detail coefficients, from the original dataset. These enable the original function to be regenerated without any loss of information, and the process can be repeated any number of times.