|
COPYRIGHT 2005 University of Santa Clara, School of Law
I. INTRODUCTION
This is not a tale of robots gone wild or other stuff of science fiction. Rather, this is about real-life situations that have occurred or are likely to occur. It is about software failure when that failure leads, not to system crashes or botched tax returns, but to serious physical injury to persons. The power of software can be seen everywhere: It flies airplanes, monitors medical patients and nuclear power plants, and even helps us drive our cars. Indeed, software is no longer confined to the domain of business systems that control inventory, issue payroll checks, and keep track of accounts receivable and payable. It extends beyond the desktop computer with its word processing and data management capabilities and now routinely interfaces with human beings in their daily lives and in unseen ways. The consequence, however, is that some software can cause physical injury if it is defective. While many early commentators have speculated about the liability regime when such a condition occurs, (1) it is now time to take stock of how the law is developing and should develop when software foreseeably causes physical injury.
The discussion is timely for a number of reasons. First, there have been sufficient numbers of instances of software failure that have caused physical injury (2) to cause serious concern, and the number can only grow, given the pervasiveness of software in our daily lives. Second, the software industry is no longer in its infancy. Its development has moved out of garages and into corporate offices. It has matured to become a dominant sector of the economy. Consequently, it is appropriate to consider liability for defective software in the same light as liability for defective automobiles, pharmaceuticals, and other products.
This article will first examine the characteristics of software and its evolutionary creep into our lives in Part II. In Parts III and IV, we will review the literature about software litigation and the eras through which it has progressed. Part V reviews the origins of strict product liability and the policies underlying it to determine whether software, which has hitherto enjoyed immunity from strict liability, fits into the strict liability context. Lastly, in Part VI we argue for the adoption of a strict liability regime for software failure that produces physical injury and offer supporting arguments for why such a move is both necessary and sensible.
II. COMPUTER HISTORY 101
While many think that computers, software, and computing are relatively new developments, this is not really the case. A timeline showing the major events in the history of the development of the computer should really start with Napier's bones in the 17th Century and include such developments as Pascal's adding machine, Jacquard's loom, and Herman Hollerith's tabulating code developed for the American census of 1890. (3)
Although the history of computers and computing becomes clouded by the security surrounding both the Second World War and the Cold War that followed, two major developments in 1943 are generally regarded as the origination of the modern stored program electronic computer: J. Prosper Eckert and Dr. John W. Mauchly's (United States) ENIAC (4) design and the design at Bletchley Park (England) (5) of Colossus, the computer that would decode the German enigma messages. (6)
Alan Turing, sometimes called the father of computer science, tied together these two developments, and it was he who considered the fundamentals of what is now called software fault tolerance. (7) Turing concluded that it was not possible to write a program that could determine if another program will compute successfully and halt. (8) In other words, the proof of correctness of any program is not computable, and software fault tolerance does not have a theoretical basis.
Neither ENIAC nor Colossus were true programmable computers in the modern sense. The development of magnetic core computer memories (patented in 1947) led to the move away from programming done by wiring a physical connection between the parts of the hardware, towards programming done by storing an easily altered program in the machine's memory.
The development of the transistor at Bell labs in the late 1940s as a replacement for unreliable relays (9) started the next phase of expansion of computers and software into our lives. Before then, electronics mainly relied upon valves (10) that were used in the logic circuits of the early computers. Valves require a lot of power, generate a lot of heat when working correctly, and are fragile. Transistors, (11) on the other hand, require less power, generate less heat and are more reliable.
It was the work of Jack Kilby (12) in the late 1950s when working for Texas Instruments that led to the idea of creating other components out of the materials used to construct transistors and making them into a single package. Thus was born the concept of the integrated circuit from which Intel in 1969 created the first microprocessor. (13)
Since the design of the first microprocessor, subsequent designs have primarily increased system speeds and system capability while decreasing costs. It was this growth in power and decrease in costs that led directly to the microprocessor becoming ubiquitous, powering an estimated 1,000,000,000 home computers in 2002. (14) Microprocessors are also found in washing machines and dishwashers (replacing the previously common clockwork timing mechanisms) and are found controlling phase-locked loops tuning portable radios, replacing the previous analogue designs of capacitors and inductors. Microprocessors with allied circuits have replaced the mechanical tape mechanism in the answering machine and the mechanical dial of the telephone. Microprocessors control the engine management, braking system, airbags, navigation system, radio, cruise control, four-wheel drive, and even the wiring of cars from all over the world. (15)
The use of a generic microprocessor and special software instead of discrete components wired for a specific purpose has led to a "throw away" attitude with respect to modern electronics. When it ceased to be viable to repair the electronic components within modern equipment, it became less expensive to replace the whole item, which led manufacturers to adopt the attitude that "it will be all right in the next revision." This attitude has now spread from hardware to software and might be one of the root causes of failures of some systems.
Because programs can be written to do things that conventional components cannot, microprocessors have helped in the creation of previously impossible new products such as digital cameras, mobile telephones, pocket computers, portable printers, small GPS receivers, and CD/DVD players. However, while all these products seem to be "cutting edge," the processor inside, at the heart of these devices, is still based upon concepts devised over fifty years ago. Thus, the technology contained within the product should be considered mature, not something new.
A. Software Versus Hardware
This history of the computer has concentrated upon the hardware that makes up the machines. Hardware is the "nuts and bolts"--the parts of a computer system that you can see, touch, and feel. (16) Typically hardware either works or it does not; the power supply gives out twelve volts or it does not, the disk drive spins or it does not. Hardware can fail for one of two reasons, faulty design or failed components. (17)
The programs that make the hardware do what the user wants the hardware to do are collectively called the software. (18) Software, supplied to the user by the hardware (disk) on which the software is recorded, is the computer instructions and data that are stored electronically within the computer; when the electricity is turned off, the software ceases to exist.
B. What Is a Software Failure and Why Does Software Fail?
Software can only fail for one reason: faulty design. (19) While it is common to call any failure of software to perform as the user expected a bug, (20) the real bug is the failure of the software to do what the programmer who created it thought it was going to do, which is not quite the same thing. Many reported "bugs" are, in fact, the correct and expected operation of the hardware and software combination, but they are not what the end-user expected it to do, so they may be considered faults, but not failures.
Software systems are products of the human mind and are vulnerable to programming mistakes. While software can only fail due to faulty design, these mistakes occur in various forms, including design inconsistencies, syntax errors, and semantic errors.
In everyday desktop systems, these glitches can result in inconvenience and frustration, but are not usually particularly serious. However, even as small an error as a punctuation mistake can cause major economic loss and mission failure as was seen in the North American Space Agency's ("NASA") Mariner 1 Venus probe that launched in 1962, where a missing hyphen caused the spacecraft to travel off-course. (21)
Newer programming techniques and compilers can mitigate many semantic and syntactic errors by increasing the amount of checking done by the compiler over the programming code entered into it. However, as Turing's proof demonstrated, they cannot test the complete correctness of any specific program. Unfortunately, tests cannot prove that there are absolutely no bugs in a program. All they can do is highlight the existence of the problems they find.
A researcher at NASA, Ames M. Lowry, argued in a report that the size of software used in space missions has been increasing exponentially. (22) He noted that there are many examples where bugs in contemporary systems have resulted in mission failure and suggests that the number of software errors in future systems may similarly increase, thereby increasing the likelihood of critical faults, which may "incapacitate safety-critical systems." (23) In the context of the report, a safety-critical system is defined as one in which a malfunction could result in death, injury or illness, major economic loss, mission failure, environmental damage, or property damage. (24) In the IEE Review, Les Hatton asserts that the average number of errors per line of computer code remains fairly constant for different source languages. (25) This is interpreted to mean that the brain of the programmer has a constant probability of introducing faults. In other words, doubling the number of lines of code will produce double the number of programming faults.
C. How Can Software Failure Be Mitigated?
Modern software engineering techniques (26) for creating and maintaining software applications combine techniques from computer science, project management, domain knowledge, and other skills and technologies, including common sense. The practices have evolved steadily, but it was the supposed software crisis (27) during the period from 1960 to 1980 that identified many problems of software development. During that time, projects were delivered late, went over budget, and some caused property damage or loss of life.
To combat the problems, a number of tools, including new ways of working, new processes, and new languages, were all suggested as solutions. However, in his article "No Silver Bullet," (28) Fred Brooks argues that there can be no easy answers to software engineering problems, describing two different types of complexity. Essential complexity is inherent and nothing can remove it. (29) If a word processor's spell-checker has to work in 50 different languages, then it has to work in 50 different languages. In contrast, accidental complexity is created by programmers and can be dealt with. (30)
The accidental complexity of writing and optimizing machine code can be dealt with by programming in high-level languages that require fewer lines of code and have very strong checking routines that test the operation of module interfaces and help to minimize syntax and semantic errors. Similarly, using object-oriented programming methods helps to minimize the number of design inconsistencies. Note, however, that these changes only deal with the accidental complexity, not the essential complexity of the problems that we are trying to solve. (31)
Testing for errors is another strategy in preventing erroneous software from being released to an unsuspecting user.
D. How Is Software Tested?
Typically, during initial testing, software engineers exercise the instructions to see that they perform as expected. For example, if the first instruction is "copy the value 2 into register A," does register A contain the value 2 after the instruction's execution? Or, if you type a word into the word processor, do the letters appear on the screen? And, does the spell-checker highlight any misspelling?
Of course, programmers tend to test that their creation does what they expect it to do. If the original specification was wrong from a user's perspective, software can pass all its tests and still not meet the user's needs. Worse, engineers will often test in their environment, not acting as non-expert users. For example, knowing that the escape key will exit the debugger, they often do not test what happens when the end user presses the escape key.
During testing, some instruction paths may never be tested if the decision choosing that path is never exercised. In addition, within a single computer, different packages of software are likely to interact. Many a user has complained, "Everything worked until I installed xxxxx." Part of the problem is that these are very complex systems. Teams of individuals who may not see the whole picture write parts according to their interpretation of a specification they have received. All these parts must then be integrated into a whole containing hundreds of different parts created by hundreds of people, all the parts coming together in one word processor or database.
There are literally hundreds of papers about software testing, ranging from the simplistic to the mathematically challenging. Most include the inherent assumption that it is not possible to remove all the errors from software.
E. Software Incidents--Deadly and Potentially So
The Association for Computing Machinery, (32) founded in 1947, maintains a "forum on the risks to the public in computers and related systems" called the Risks Digest. (33) It is a forum that provides a useful perspective on the risks associated with computers and software. Although not a complete database of known problems, a recent digest, Volume 23 with 45 issues covering the period from 7th November 2003 to 10th July 2004, contains just fewer than 680 entries of risk to the public. Items of interest range from the Nuclear Plant shut down by a lightning strike (34) (including a very short description of a computer failure causing the steam isolation valves to close), the driver of a Honda CRV trapped in a flood when his electrically driven windows would not wind down, (35) to the two accidents caused when the braking system of commuter buses was disabled by electromagnetic interference. (36)
There are other, deadly or potentially deadly examples. In February 2000, Ferrari issued a recall for its 360 Modena cars. (37) The reason for the recall was that "[t]he anti-lock braking system electronic control, under certain heavy braking conditions, may fail, resulting in the braking...
Read the full article for free courtesy of your local library.
|
Bookmark this article
Print this article
Link to this article
Email this article
Digg It!
Add to del.icio.us
RSS
