Digital archiving in the twenty-first century: practice at the national library of the Netherlands.

ABSTRACT

Research journals are increasingly being published digitally. The advantage of digital publishing is obvious: immediate accessibility anywhere. Gradually a disadvantage is also becoming clear: digital publishing endangers the continuity of research information. As a consequence of the obsolescence of formats, hardware, software, and carriers, digital information will be lost unless we act. Digital publishing is also causing a shift in the roles and responsibilities of publishers and libraries concerned with archiving digital publications for future use. Archiving digital publications requires a major turnaround in the policy and practice of national libraries. Although some actions have been taken, digital preservation research and implementation are still in their infancy. National libraries will need substantial funding for venture research activities and development of archival infrastructures. They will also have to work together more closely to successfully organize digital archiving in the twenty-first century.

**********

In 1994 the Koninklijke Bibliotheek (KB), the national library of the Netherlands, responded to the risk of losing digital information and started to include digital publications in its depository library. For this purpose the KB designed the "Safe Place Strategy," a strategy ensuring the transfer of digital publications from their publishing environment to a dedicated archiving environment. Through a series of experiments and projects the KB developed skills, procedures, organization, and infrastructure for digital archiving. The digital archive of the KB, the e-Depot, has been fully operational since early 2003. The technical core of the e-Depot is the Digital Information Archiving System (DIAS), a system jointly developed by the KB and IBM. DIAS is the first archiving system available on the market that is compliant with the Open Archival Information System (OAIS) reference model standard. Elsevier and several other international scientific publishers have entered into a formal archiving relationship with the KB to guarantee future availability of their digital publications. The KB is researching and testing digital preservation methods and solutions that will guarantee permanent access to the archived material.

ARCHIVING IN THE DIGITAL ERA

The Impact of Digital Publishing

In the past decade journal publishing has increasingly turned digital, a change led by publishers in the field of Science, Technology, and Medicine (STM). Generally, all STM publishers now publish their journals digitally with printed derivatives only produced as a secondary format. Without doubt the pioneer of digital publishing is Elsevier (Elsevier, n.d.). Early work on digital publishing began in 1979 with project ADONIS on CDROM and, in 1991, with project TULIP for desktop delivery of e-journals. In 1995 the TULIP project was followed by ScienceDirect OnSite and in 1997 by the Web-based service ScienceDirect. New publishing models are under development, such as SPARC and BioMed Central. (1) Even though the final outcome of these experiments is still uncertain (Elsevier, 2004), it is obvious that digital publishing is a success and here to stay. This is evidenced by the increasing number of scientists who use the electronic services offering digital publications.

For publishers, digital publishing has shifted the emphasis from the production and selling of publications toward creating and providing enhanced services for searching, linking, and retrieving digital information. Despite the advantages, however, digital publishing poses a serious threat to the continuity of the "Record of Science." In the past, libraries had assumed the responsibility for archiving the printed journals they bought and collected. Now, with digital resources, libraries do not own the publications but license them from the publishers. Therefore, libraries no longer compile a collection, and the digital publications remain with the publisher. As a consequence the archiving of printed publications implicitly offered by library collections no longer exists for e-journals.

Changing Roles Subsequently, archiving needs to be organized explicitly for e-journals. To accomplish this, some institutions have taken the initiative to investigate and develop the archiving of digital publications. There are several significant examples of such initiatives, (2) and recently several national programs (3) have been set up to encourage the development of digital archiving.

Digital publishing not only affects the archiving role of libraries but also the role of publishers. To cover their share of responsibility to maintain the Record of Science, it is no longer sufficient for publishers to only produce, market, and sell the publications. An active role is required to plan and organize the archiving of the published information. Scientific publishers have become aware of this change and are rising to the challenge by implementing archiving policies. (4)

Archiving Digital Information Libraries need to encourage the publishers to deposit their digital publications for archiving. However, this is only the first part of the digital archiving story. To understand why archiving of digital information is more complicated than archiving printed information, it is helpful to compare printed publications with digital publications.

A printed publication is a physical object. A digital publication, on the contrary, is not a physical but a logical object stored on a physical medium. Another difference is that a printed publication presents the information so that it is immediately accessible to the human eye and can be read directly. To read or view the information in a digital publication, specific functionality enabled by software and hardware is required. Journalist Richard Poynder described this difference as follows:

 
   unlike paper or microfilm where the meaning is transparently 
   inscribed on the surface of the medium--digital documents are 
   opaque bit streams only understandable to humans when interpreted by 
   a machine. The hardware and software to do this interpretation, 
   however, is constantly superseded. There have, for instance, been 
   more than 200 digital storage formats alone deployed since the 
   1960s, with none lasting more than 10 years. (Poynder, 2003) 

Another difference is that publications on paper or vellum can survive "by accident" and remain readable for a very long time without any specific actions. Visualize, for instance, a manuscript such as the Book of Hours from the fifteenth century, once owned by the princes of Trivulzio living in Milan. The KB received this beautiful illuminated manuscript (5) as a valuable gift, worth at least six million Euro, from an anonymous donor. It was assumed to be lost until the family of the donor acquired the manuscript through an auction some time ago. Would a digital publication survive left sitting on a shelf for hundreds of years? Considering the threats to digital publications, the answer is definitely negative.

Threats to Digital Information One threat to digital information is that the physical carrier of a digital publication will deteriorate much faster than paper or vellum. The format of the digital object can be damaged or lost and may no longer be intact or retrievable. But even before that happens, the technology used to store the publication is likely to become obsolete. Another threat is the loss of the functionality needed to interpret, display, and use the information contained in the digital object. Without this functionality, provided by specific hardware and software, the information will not be available even if the bitstream of the digital object has been preserved. When we are able to address all the threats to a digital publication, we can successfully keep it for future use. If we cannot, sooner or later we will lose the digital publication itself or the access to the information it contains.

Permanent Access Solutions To guarantee permanent access, a variety of solutions will be required (migration, normalization, emulation, or others). The choice of which techniques to use will depend on the nature of the digital publication (for example, what is the format, is it static or executable, etc.) and also on the requirements of the user (will they want to simply view or also process the information).

Migration (sometimes referred to as conversion) is a commonly applied technique in computer science. However, migration in essence makes changes to the original and implies a risk of damaging the information, a risk that increases if a sequence of migrations is applied. This could potentially mean the loss of the information. Normalization is migration of the information to a specifically chosen format before it is accepted into the archive. This approach explicitly accepts the possibility of loss of specific characteristics…