Exploring variety in digital collections and the implications for digital preservation.

ABSTRACT

The amount of digital content produced at academic research institutions is large, and libraries and archives at these institutions have a responsibility to bring this digital material under curatorial control in order to manage and preserve it over time. But this is a daunting task with few proven models, requiring new technology, policies, procedures, core staff competencies, and cost models. The MIT Libraries are working with the DSpace[TM] open-source digital repository platform to explore the problem of capturing research and teaching material in any digital format and preserving it over time. By collaborating on this problem with other research institutions using the DSpace platform in the United States, the United Kingdom, Europe, and other parts of the world, as well as with other important efforts in the digital preservation arena, we are beginning to see ways of managing arbitrary digital content that might make digital preservation an achievable goal.

INTRODUCTION

The modes of scholarship--research, teaching, and communication-continue to evolve toward online digital content that supports critical innovations. Creating digital content and making it available on the Web as a part of the research process is not only getting easier, it is becoming routine. As an example, the U.S. National Institutes of Health (NIH) research proposals now require a description of how data produced with their funds will be made available to share with other researchers. And electronic journals with the ability to provide full-text searching and hyperlinks continue to become the accepted, and expected, norm. The amount of digital information produced annually is, by some estimates, more than thirty-five thousand times the complete contents of the Library of Congress and growing fast (Varian & Lyman, 2003).

But who will ensure that this digital scholarly record continues to exist in an era when the lifespan of digital content is normally measured by a few years? Libraries and archives are just beginning to grapple with the problem of capturing, managing, distributing, and preserving the digital material that their constituents are producing, and to effectively deal with this content requires not only new technological infrastructure but new policies and procedures, new core competencies of staff, and new business lines and cost models--in other words, significant transformation of the current models of institutional scholarly content management.

Preserving this digital material is one of the most challenging components. The digital formats of the content are various and are dictated by the short-term needs of faculty and researchers who have innovation as their driving force; thus, their motivation to use only "good," standard digital formats is very low. Libraries and archives will have to deal with this material whether or not there are well-understood ways to keep it usable over time. Thinking about how to establish the infrastructure and business practices to accomplish this, and keep the costs manageable, is a formidable task.

The MIT Libraries are working with the open-source digital repository platform called DSpace[TM] to explore the problem of capturing digital research, education material, and publications in any format and preserve them over time; to conduct research and experimentation to learn more about the issues; and to identify what larger, community-based infrastructure is needed by research institutions in order to make digital preservation a practical reality. Working together with researchers from Hewlett Packard and from other research institutions that use the DSpace platform in the United States, the United Kingdom, Europe, and elsewhere, we are beginning to see new ways of managing arbitrary digital content that might make digital preservation an achievable goal. And the emergence of a digital preservation community is helping to educate digital content software producers and authors and the governments that fund research to be more aware of the consequences of current policies and decisions.

THE CONTENT ENVIRONMENT

MIT is experiencing what…