The impact of judgmental biases on the validation of simulation models.

KEYWORDS: simulation; validation, judgmental biases.

The validation of simulation models is a process that requires both time and resources. The validation process indicates the extent of model validity rather than validity or nonvalidity per se (Shannon, 1975).

The literature review on the validation of simulation models by Law and Kelton (1982) indicates that most validation procedures are subjective. Human judgment affects the outcome of a validation process because subjectivity is involved both in the application of most validation procedures and the decision regarding when to stop the validation process (Kleindorfer & Ganeshan, 1993). In a simulation project, there are many individuals who have a vested interest in developing a model that represents that system accurately (e.g., the analyst, programmer, client, decision makers, etc.). These parties, however, often have different views as to how accurate a simulation model needs to be (Velayas & Levary, 1987). They also are likely to have different opinions regarding when to stop the validation process, given the time and resources allocated to it.

It is easy to see how each individual involved with the simulation project may have different concerns and perspectives. For example, analysts and model designers might somehow become attached to their own creations and inadvertently resist criticism of diem. Programmers, on the other hand, might be concerned with a smooth-running program and may give scant attention to the model. The head of the department involved with the simulation project might be primarily concerned with the project schedule and might resist deviations. Redesign activities would affect the schedule and would interfere with other department commitments. A client might be primarily concerned with cost overruns and therefore might be uncomfortable with redesign activities. In addition, a client might be very concerned with the user friendliness of the model and unintentionally might note that aspect more than model accuracy.

Because human judgment affects the outcome of simulation models, judgmental biases must be identified both in the model designer and in the validation team (Balci & Nance, 1985; Gass, 1977; Gruhl, 1982; Kleindorfer & Ganeshan, 1993). A bias can be defined as a highly personal and unreasoned opinion regarding a situation, decision, or value. A judgmental bias can be defined as a highly personal opinion formed by a cognitive process based on experience, self-confidence, and authority. Weber and Coskunoglu (1990) emphasized the need for identifying judgmental biases in any decision-making process or tool.

Judgmental bias may be introduced in the design of any simulation model. Whereas every simulation model must be evaluated for the validity of its design, a simulation game must also be evaluated to determine its effectiveness for given objectives and an intended environment (Crookall, 1995). A game that was intended for a teaching environment, for example, would have to be evaluated accordingly and not just evaluated in general for an unspecified situation. Because designers and users of a simulation game might have different opinions regarding the way in which games should be evaluated, however, judgmental biases may be inadvertently introduced in the evaluation process as well as in the design process.

The judgmental biases found in the literature can be classified as those related to the data and those related to the decision maker(s) involved in developing the simulation model or game. Judgmental biases related to data can be subdivided into those related to availability and those related to how the data are used. Judgmental biases related to decision maker(s) can be subdivided into three groups. They are classified as

* decision maker prejudice,

* decision maker carelessness, and

* decision maker use of the data.

Note that data use is found both in the data availability category and the decision maker category. The classification of judgmental biases is provided in Table 1.

 
TABLE 1: Classification of Judgmental Biases 
 
Judgmental Biases Related to Data 
 
Data availability 
  Adjusting and anchoring 
  Availability 
  Conservatism 
  Data …