Quality in an age of measurement. (Law).(performance indicators in courts and universities)

THE BALANCE between quantitative and qualitative assessment of the performance of governmental functions affects many areas of public activity. Should museums be judged by attendances? Should university researchers be judged by numbers of publications ? Should educational institutions be judged by the earnings of their graduates? Should the ABC or SBS be judged by television ratings? Should courts be judged by the time it takes to dispose of proceedings?

The answer to each of these questions is "Yes, in part". Matters that are capable of measurement are relevant to a process of assessment both for internal purposes and in terms of public accountability. The issue is the extent to which crucial decisions are made only or primarily on the basis of what can be measured.

My central proposition is a simple one. Not everything that counts can be counted. Some matters can only be judged--that is to say they can only be assessed in a qualitative way. It is of primary significance to recognise that there are major differences between one area of government activity and another as to the importance of those matters that are capable of being reduced to quantitative terms. In some spheres of governmental decision-making the things that can be measured are the important things. In other spheres the things that are important are not measurable.

I became interested in these matters by reason of my responsibilities for the administration of the court. I recognise that in the administration of justice certain matters, such as delay, are capable of quantification. The compilation and publication of statistics relating to the measurement of delay is a perfectly appropriate activity. Nevertheless, the most important functions performed by a court are not capable of measurement. In particular, the fundamental issue of whether or not the system produces fair outcomes arrived at by fair procedures is not capable of quantification at all.

It quickly became clear to me that advocates of quantification in the context of the administration of justice do not necessarily share my opinion that quantification has limited significance for the determination of important decisions affecting the courts. In particular they seemed to expect that the use of quantification would inevitably expand to influence and, even determine, more and more decisions. These include the proposals for national benchmarks and the publication of comparative data which would in some way lead to competition between courts, perhaps directly by indicating role models, or indirectly, through an incentive to improve performance occasioned by the publication of invidious comparisons. Furthermore, some expect that, in the future, matters of quantification would become of central significance in determining the allocation of resources, for example expanding one court in a judicial hierarchy at the expense of another or transferring jurisdiction to non-judicial tribunals. Finally, at least two of the statutory tribunals responsible for the determination of judicial salaries, those of the Commonwealth and Queensland, proposed to take into account some unspecified quantified measure of performance in the course of deciding changes in judicial salaries.

OVER THE COURSE of the last two decades the organisation of governmental activity has undergone a radical transformation. It has been referred to as a "global public management revolution". In Australia, New Zealand, England and some European countries, the change has been called "the new public management". The courts are an arm of government. They have not been and cannot be insulated from changes in attitude about the proper role of government and the appropriate means to conduct governmental activity. The courts have responded and will continue to do so.

My reading of the literature of the new public management indicated that in some areas of public decision-making in which quality was of great significance, such as education and health, the imperatives of measurement had originally been applied in much the same kind of low-key way as is presently applicable or proposed in the context of the administration of justice. There are perfectly acceptable and legitimate purposes served by quantitative indicators. However, in the areas of health and education they did not stop there. Performance indicators and funding formulae have come to be applied in a mechanical and rigid way to determine budgets, remuneration and tenure with the most dramatic effects on the delivery of services and the determination of curricula and research. I was, and remain, concerned that nothing of this character should occur in the context of the administration of justice.