Avatar Advances.(Technology Information)

A new image-based technique helps get people into virtual worlds

The most critical component to a successful virtual reality application is the ability to make users believe they are actually in the digital environment. A number of factors contribute to this sense of immersion, including real-time interaction and believable (though not necessarily photorealistic) 3D graphics.

A new system under development at the University of North Carolina (UNC) promises to move virtual worlds one step closer to this ideal by enabling real-time 3D reconstructions of the user and other real objects in an immersive virtual environment. When the user steps into or places a body part such as an arm into the virtual field of view, the image-based system captures the body part and builds a 3D graphical representation of it, which the user sees in the virtual world in the same way it would appear in the real world. In addition, the user can interact with other real objects that are also contained within the system's field of view. For example, if the user reaches an arm into the viewing space to grab a physical object off of a desk, he or she will see an accurately lit, pigmented, and clothed graphical representation of the arm and a similarly accurate representation of the object.

To achieve this, the system combines multiple camera views of a real scene taken from various perspectives and analyzes the collected data to extrude object forms. It then renders the defined objects in real time using graphics hardware.

To collect the image information, the novel system builds a reconstruction volume based on the combined projection boundaries of the cameras. For example, the researchers have implemented the technique using a six camera setup, whereby five of the cameras are wall-mounted and the sixth is attached to a headmounted display worn by the user. With this configuration, the system reconstructs objects within an 8- by 6-by 6-foot volume.

What makes the system unique is its reliance on camera images rather than explicit 3D models. Also, it does not use tracking sensors to model user interaction (optical tracking is used to enable navigation through the scene), nor does it require prior object knowledge. Instead, it interprets the collected image data using a "visual-hull" technique.

A visual hull is a geometric shape obtained by integrating silhouettes of an object as seen from a number of views, extruding volumes that contain the objects from each silhouette, and representing the intersection of these volumes as an image in 3D space.