From measurement to management: using data wisely for planning and decision-making.

ABSTRACT

The wise use of data, information, and knowledge in planning, decision-making, and management can improve library performance. While libraries have long collected data, however, it is only recently that they have begun to use it effectively in library management. This article provides an overview of data use in libraries, organizational barriers, and support issues, as well as examples of libraries that have successfully integrated data acquisition, analysis, and application into management.

INTRODUCTION

Data can be defined and used in a number of different ways. Our simple definition is that data are records of observations, facts, or information collected for reference or analysis. Data may take a number of forms, such as transactions, observations, surveys, or interviews. All of these provide data, that is, observations, both quantitative and qualitative, from which inferences may be drawn by means of analysis.

Libraries have long collected data about library operations, primarily inputs such as the size of collections and staff or expenditures. However, the degree to which they were used, or could be used, for decision-making in library management varied widely. Recently, there has been a voluminous increase in library-related data, not only in transactional information from online library systems and electronic resources usage but also from efforts to gain more direct user input through surveys, focus groups, and other methods. Funding and accrediting bodies are also asking libraries to demonstrate their impact on the user community through performance measurements that are based on outcomes and data. While many libraries recognize the value of using data for planning and decision-making, they are unsure how to collect, analyze, and apply the data effectively in library management.

This concern is not new. Libraries have struggled for years with how to utilize statistics and other data to enhance library effectiveness. Nearly two decades ago, Allen posed these questions at an international conference:

 
   The failure of library statistics to solve all the problems that 
   library management would have them solve may not, however, be 
   entirely the fault of the statistics. A number of questions may 
   be reasonably asked. Do librarians collect the appropriate 
   statistics? Are the statistics collected either accurate or 
   comparable among similar libraries? Do we ask valid questions 
   of the data? And above all, do we know how to manipulate and 
   interpret statistical information? All too often the answer to 
   these questions is "no." (Allen, 1985, p. 212) 

Although many libraries have measured aspects of library activity or operations, why have the majority failed to use data effectively in management? What are the obstacles and barriers? Are there strategies and programs that have worked well, providing models from which we can learn? This article will review both the problems and successes involved in using data wisely in library management and decision-making.

TRADITIONAL USES OF DATA IN LIBRARIES

Libraries have generated and collected data related to their operations for many years. Statistical data in such areas as expenditures, number of books purchased, and staff size were gathered and reported to appropriate administrative bodies or groups. Gerould was among the first to discuss the practical value of comparative data:

 
   No questions arise more frequently in the mind of the progressive 
   librarian than these: Is this method the best? Is our practice, 
   in this particular, adapted to secure the most effective 
   administration? Are we up to the standard set by similar 
   institutions of our class? These questions are of the most 
   fundamental type, and upon the success with which we answer them 
   depends much of the success of our administration. (Gerould, 
   1906, p. 761) 

Gerould further elaborated on the statistical categories that would prove helpful in library administration and management. These included facilities, collections, finances, staff, salaries, ordering and processing, cataloging, collection use, reference transactions, and departmental libraries. He began collecting and publishing data in 1907-8 from a select group of academic research libraries, and the practice continued (after his retirement) until 1962, when the Association of Research Libraries (ARL) took over the collection, compilation, analysis, and distribution of statistics. While these early statistics provide an invaluable record documenting the historical development of American academic research libraries, there is little evidence on how they were actually used to improve library management and decision-making. While it is likely that comparisons with other libraries may have added fuel to budget requests for increased funding, local statistics were more likely to be used for library planning. For example, the best data for projecting collection growth and the need for expanded facilities "are found in the individual library's statistical history" (Metcalfe, 1986, p. 155). In his work on the Gerould statistics, Molyneux (1986) included library collection growth as the only example of how this data set could be used.

Comparative statistics were also used to develop standards, especially by library organizations. Such standards might specify the minimum number of volumes, staff, user seating, and other library measures. Efforts were also made to incorporate these standards or other statistical data into budget allocation, both at the institutional level and within the library. Library funding models or formulas such as Clapp-Jordan in the 1960s (and subsequent variants) endeavored to tie a recommended collection size to measures such as number of faculty, undergraduate students and majors, and graduate students at the masters and doctoral levels. Internal allocation models for collection development by subject area also used faculty and student numbers correlated to academic departments, as well as data related to publishing output, costs, type of materials, loans, and other use measures. While these were clearly efforts to use standards and data in library management, they were based on assumed linkages rather than research. Because these data were input centered, the link to outcomes were, at best, difficult to measure. As Clapp and Jordan admitted:

 
   The formulas described in this article have been developed in an 
   attempt to find a method for estimating the size for minimal 
   adequacy of academic library collections more convincingly than 
   can be done with existing criteria. It may be validly objected 
   that little more has been accomplished than to transfer the locus 
   of conviction from an unknown whole to the unknown parts, of which 
   the whole is composed. (Clapp & Jordan, 1965, p. 380) 

Of greater utility to libraries were local research studies that examined specific library services and processes undertaken in order to improve library performance. These included evaluating such activities as cataloging efficiency, card catalog use, reference services, collection use, interlibrary loan and document delivery, facilities and access, library systems, budgeting, and personnel. F. W. Lancaster's influential 1977 book, The Measurement and Evaluation of Library Services, provided the first systematic review of studies designed to measure and assess library performance. Lancaster also covered the different methods that could be used for evaluation. He made the important distinction between broad-based input/output data ("macroevaluation") and more focused analysis and interpretation of system processes ("microevaluation"):

 
   Macroevaluation measures how well a system operates, and the 
   results usually can be expressed in quantitative terms (e.g., 
   percentage of success in satisfying requests for interlibrary 
   loans). It reveals that a particular system operates at a 
   particular level, but it does not, in itself, indicate why the 
   system operates at this level or what might be done to improve 
   performance in the future. Microevaluation, on the other hand, 
   investigates how a system operates and why it operates at a 
   particular level. Because it deals with factors affecting the 
   performance of the system, microevaluation is necessary if the 
   results of the investigation will, in some way, be used to 
   improve performance. (Lancaster, 1977, p. 2) 

In a subsequent paper Lancaster and McCutcheon went on to state,

 
   Many of the studies conducted in the last ten years that can be 
   grouped under the general heading of quantitative methods, are 
   pure macroevaluation because they rarely go beyond producing 
   data. In order to improve the service, we need microevaluation.... 
   This type of analysis, although we use figures in our analysis, is 
   more or less non-quantitative. It is interpretative. The 
   investigator is very much concerned with using the figures 
   acquired through quantitative procedures, to make reasonable 
   decisions on what needs to be done to raise the level of 
   performance. (Lancaster & McCutcheon, 1978, pp. 13-14) 

Library Automation and Data Generation

The development and implementation of library-related systems for information retrieval, cataloging, and circulation coupled with the increased use of computers for quantitative analysis in social sciences helped move library education to a more systems-based approach in the late 1960s and 1970s. A new generation of library educators and librarians emerged who were equipped with quantitative skills and a structured social science approach to problem-solving that resembled Lancaster's microevaluation. Swisher and McClure addressed the need for "developing a research plan and analyzing data in such a way that practicing librarians can make better decisions and improve the overall effectiveness of their libraries" (Swisher & McClure, 1984, p. xiv). They called this type of applied activity "action-research" and defined it as the "ability to formulate questions about library services and operations, collect empirical data that appropriately describe factors related to those questions, and analyze those data in such a manner that summary descriptive information will be produced to answer the original question and implement actions/decisions to increase library effectiveness" (Swisher & McClure, 1984, p. 2).

By the early 1980s automated library systems could generate copious amounts of data and reports on circulation, cataloging volume, and use of catalogs and bibliographic databases. It was envisioned that these systems would form the core data elements of the emerging Management Information Systems (MIS) and Decision Support Systems (DSS) that would underpin good library management and decision-making in the future. Heim defined an MIS as "A system that provides management with information to make decisions, evaluate alternatives, measure performance, and detect situations requiring corrective action" (Heim, 1983, p. 59).

Decision support systems were seen as supporting broader administrative and management decisions. Dowlin and McGrath envisioned this scenario in the not too distant future:

 
   The goal for the DSS is for the library director or manager to use 
   a terminal to ask the DSS: How is the library today? The system 
   would respond with such comments as: "terrible," "lousy," "fair," 
   "good," "not bad," or "great." The questioner could then ask why. 
   The system would respond with a summary report of all of the 
   indicators using predefined criteria that would indicate 
   exceptions. (Dowlin & McGrath, 1983, p. 58) 

Yet at the same conference in 1982 where Dowlin and McGrath presented their view of how systems data would be used in management (Library Automation as a Source of Management Information), Shank expressed his doubts:

 
   The whole system seems to be put into place as a perpetual motion 
   machine all too often installed without there being any analysis 
   of what to do with the data ... It is not clear, what, if 
   anything, can be done about whatever the data purports to show 
   ... Data rejection occurs because there is a lack of …