Riding the wavelets: a new compression and display technique gives researchers a window into some of the world's biggest science problems. (techwatch).

High-performance computing has given scientists an unprecedented ability to study complex theoretical and experimental problems. It's also given them an unprecedented number of new questions about how to archive, retrieve, transmit, visualize, and analyze the resulting massive datasets. "Terascale physics simulations are now producing tens of terabytes of output for a several-day run on the largest computer systems," says Mark Duchaineau, a visualization researcher at Lawrence Livermore National Laboratory. Such simulations produce surface datasets comprising hundreds of millions of polygons--crippling numbers for today's highest-end commercial storage and graphics hardware.

To enable researchers to actually see and interact with what the high-performance systems let them compute, Duchaineau and colleagues Martin Bertram of the University of Kaiserslautern, along with Serban Porumbescu, Bernd Hamann, and Kenneth Joy of the University of California at Davis, have developed a system that reduces the size and improves the manageability of the huge datasets for interactive display.

At the heart of the system is a subdivision-surface wavelet compression technique, complemented by a view-dependent optimization scheme. Together, these serve to minimize data storage requirements and drive the graphics hardware. "We want to achieve high-quality compression and display of the largest surface data in the world," says Duchaineau.

Toward this end, the researchers are testing their technique on 3D scientific simulations generated on ASCI White, the world's most powerful supercomputer. One example is a recent simulation of instability in a shock-tube experiment, which produced isosurfaces consisting of 460 million unstructured triangles. For this application, says Duchaineau, "if we use 32-bit values for coordinates, normals, and indices, then we need 16 gigabytes for the storage of a single isosurface, and several terabytes for a single surface tracking through all 274 time steps of the simulation." With the gigabyte-per-second read rates of current RAID storage, he says, "it would take 16 seconds to read a single surface." The numbers also strangle high-performance graphics hardware. "Today, the fastest commercial systems can effectively draw 20 million triangles per second," says Duchaineau. To achieve interactive rates, the triangle count of such a dataset would have to be reduced almost one-thousand fold.

Using existing surface-compression techniques, such a reduction would surely compromise the quality of the dataset and consequently its scientific integrity. To avoid this, the researchers set their sights on high-quality wavelet-based compression, a common technique used to reduce the size of image data, such as photographs and video.

Wavelets let a function be represented at a lower resolution by maintaining values, called detail coefficients, from the original dataset. These enable the original function to be regenerated without any ...